Directional vehicle notification using pre-rendered multi-dimensional sound - Patents.com

Abstract

A method for generating a multidimensional audio notification in a vehicle includes registering an event for which a directional notification is to be generated in a vehicle including an audio system including a plurality of channels; obtaining a first pre-rendered multidimensional sound assigned to the event from among a plurality of pre-rendered multidimensional sounds in a storage device, the first pre-rendered multidimensional sound including audio for more than one channel of the plurality of channels; and playing the obtained first pre-rendered multidimensional sound using the audio system.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. patent application Ser. No. 63 / 265,448, filed Dec. 15, 2021, entitled “DIRECTIONAL VEHICLE NOTIFICATIONS USING PRE-RENDERED MULTIDIMENSIONAL SOUNDS,” the disclosure of which is incorporated by reference herein in its entirety.

[0002] The present disclosure relates to directional vehicular notifications generated using pre-rendered multi-dimensional sound. [Background technology]

[0003] Vehicles have long had one or more ways to inform the driver of an acknowledgement, status, or occurrence. The increasing complexity of vehicles, coupled with the increased incorporation of computing resources, has provided new possibilities for more sophisticated forms of communication, including in the area of ​​vehicle audio systems. However, each use of computing power within a vehicle reduces the amount of resources available for other processes and functions. Summary of the Invention

[0004] In one aspect, a method for generating a multidimensional audio notification in a vehicle includes, in a vehicle including an audio system including a plurality of channels, registering an event for which a directional notification is to be generated; retrieving a first pre-rendered multidimensional sound assigned to the event from among a plurality of pre-rendered multidimensional sounds in a storage device, the first pre-rendered multidimensional sound including audio for more than one channel of the plurality of channels; and playing the retrieved first pre-rendered multidimensional sound using the audio system.

[0005] Implementations may include any or all of the following features: The pre-rendered first multidimensional sound includes a first sound for all of the plurality of channels and a second sound for less than all of the plurality of channels, the first sound and the second sound being played separately in time from one another; Playing the acquired pre-rendered first multidimensional sound includes playing the first sound before the second sound; The event reflects a situation within a passenger cabin of the vehicle; The acquired pre-rendered first multidimensional sound includes a seat belt reminder, and the pre-rendered multidimensional sounds in the storage device include a plurality of seat belt reminders each corresponding to a different location within the passenger cabin; The acquired pre-rendered first multidimensional sound includes one of a front left directivity or a front right directivity, and a pre-rendered second multidimensional sound of the pre-rendered multidimensional sounds includes another of the front left directivity or the front right directivity. The first pre-rendered multidimensional sound and the second pre-rendered multidimensional sound include a turn signal notification. The first pre-rendered multidimensional sound obtained includes a seat belt reminder. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 illustrates a schematic diagram of an example of a vehicle capable of playing pre-rendered multi-dimensional sounds for directional notification.

[0007] [Diagram 2] FIG. 1 illustrates an example of a vehicle audio system with pre-rendered multi-dimensional sound.

[0008] [Diagram 3] FIG. 1 illustrates a schematic diagram of an example of a multi-channel audio system and pre-rendered multi-dimensional sound.

[0009] [Figure 4]FIG. 1 illustrates an exemplary architecture of a computer system.

[0010] Like reference characters or numbers in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure provides examples of systems and techniques that can play pre-rendered multi-dimensional sounds for directional notifications in a vehicle. The pre-rendered multi-dimensional sounds can provide an immersive experience for the occupants for one or more types of notifications (including, but not limited to, confirmations, announcements, notices, reminders, attention, warnings, and / or alarms). Furthermore, the immersive experience can be provided while subjecting the central processing unit (CPU) to only a relatively low load. For example, this can ensure that CPU cycles are not spent rendering the multi-dimensional sounds in real time, thereby providing a more efficient use of the vehicle's computer system resources. In some implementations, the pre-rendered multi-dimensional sounds can be used to generate turn signal sounds, seat belt reminders, and / or advanced driver assistance system (ADAS) notifications.

[0012] Examples herein refer to vehicles. A vehicle is a machine that transports passengers or cargo, or both. A vehicle may have one or more motors that use at least one type of fuel, or other energy source (e.g., electricity). Examples of vehicles include, but are not limited to, cars, trucks, and buses. The number of wheels may vary among multiple types of vehicles, and one or more (e.g., all) of the wheels may be used to propel the vehicle. A vehicle may include a passenger compartment that accommodates one or more people. At least one vehicle occupant may be considered to be the driver; in this case, various tools, implements, or other devices may be provided to the driver. In examples herein, any person carried by a vehicle may be referred to as the "driver" or "passenger" of the vehicle, regardless of whether or to what extent the person drives the vehicle, whether the person has access to all or only some of the controls to drive the vehicle, or whether the person lacks the controls to drive the vehicle.

[0013] 1 illustrates generally one example of a vehicle 100 capable of playing pre-rendered multi-dimensional sound for directional notification. The vehicle 100 is shown here in a top view. The vehicle 100 includes a multi-channel audio system (e.g., as illustrated below) capable of providing an immersive audio experience to the occupants. In some implementations, the vehicle 100 can register an event and generate a corresponding notification to one or more occupants using pre-rendered multi-dimensional sound.

[0014] The immersive experience may be created by playing at least one pre-rendered multi-dimensional sound that appears to originate from one or more locations. That is, the sound is generated by one or more physical speakers of the vehicle 100, but the audio output of one or more speakers (e.g. two or more speakers playing simultaneously) may make the occupant believe that the sound is coming from any arbitrary direction relative to the occupant's position in the vehicle 100. Any direction may be used. The apparent sources 102-112 of any pre-rendered multi-dimensional sound are shown here diagrammatically for illustrative purposes. That is, the apparent sources 102-112 are here located outside the vehicle 100 and do not represent the location of physical speakers within the vehicle 100. Rather, the apparent sources 102-112 show examples of directions from which the occupant may perceive the sound to be coming. For example, apparent source 102 may be characterized as a front left location, apparent source 104 may be characterized as a front right location, and apparent source 106 may be characterized as a front center location. As another example, apparent source 108 may be characterized as a rear left location, apparent source 110 may be characterized as a rear right location, and apparent source 112 may be characterized as a rear center location. Other apparent directions may be used.

[0015] The immersive experience based on pre-rendered multi-dimensional sounds may be provided in response to registration of any of a number of events for which a directional notification will be generated. That is, at least one pre-rendered multi-dimensional sound may be assigned to such an event. The event may reflect a situation inside or outside the passenger cabin of the vehicle 100. The situation inside the cabin may include a seat belt not being fastened in a currently occupied seat. In this case, a directional notification for a seat belt reminder may be generated at least partially from the direction of the unbelted seat. For example, if a passenger sitting in the left position of the back seat has not fastened his / her seat belt, a pre-rendered multi-dimensional sound may be played that appears to come at least partially from an apparent source 108. The vehicle 100 may be provided with a number of pre-rendered multi-dimensional sounds including seat belt reminders corresponding to various locations in the passenger cabin. Thus, when a particular seat is registered as being occupied as not having its seat belt fastened, the vehicle 100 may obtain a pre-rendered multi-dimensional sound corresponding to a seat belt reminder at that particular location in the vehicle.

[0016] The directionality of the immersive experience can be useful in several ways. First, pre-rendered multi-dimensional sound, due to its apparent origin, can be more immediately noticeable to an unbelted occupant than to other occupants, thereby more immediately catching the attention of the unbelted occupant. Second, pre-rendered multi-dimensional sound can indicate to someone other than the unbelted occupant (e.g., the driver) that the unbelted occupant (e.g., a child) may need help fastening their seat belt. Third, pre-rendered multi-dimensional sound can indicate to a person in the front seat that a person in the back seat has not fastened their seat belt.

[0017] The dimension of the immersive experience may vary during playback of the pre-rendered multi-dimensional sound. In some implementations, the pre-rendered multi-dimensional sound may include both global and directional notifications. The global and directional notifications may be played separately from each other in time. The global notification may be heard equally throughout the passenger cabin and may not appear to originate only in a particular location (e.g., it may be associated with most or all of the apparent sources 102-112). Thus, the global notification may include sounds for all of the multiple channels of the audio system. Additionally, the directional notification may appear to originate in a particular location (e.g., one of the apparent sources 102-112). Thus, the directional notification may include sounds for fewer than all of the multiple channels. The global and directional notifications may occur in any order. In some implementations, the global notification is played before the directional notification. For example, a global notification may serve to alert all occupants that an event has been detected (including, but not limited to, an unfastened seat belt), while a directional notification may identify a location of interest.

[0018] In some implementations, the event that triggers the immersive audio experience may include activation of a turn signal of the vehicle 100. In the case of a turn signal, the directional notification may primarily indicate to the occupant whether it is the left or right turn signal that is flashing. Activation of the turn signal may be performed manually by the driver or automatically by the vehicle 100 (e.g., by an ADAS that may be partially or completely controlling the movement).

[0019] The vehicle 100 may be provided with at least two pre-rendered multidimensional sounds for turn indicator notification, one for the left indicator and one for the right indicator. That is, one of these pre-rendered multidimensional sounds has a forward left directivity (e.g., associated with apparent source 102) and the other of these pre-rendered multidimensional sounds has a forward right directivity (e.g., associated with apparent source 104). When a turn indicator activation is registered, the vehicle 100 may retrieve the corresponding pre-rendered multidimensional sound (e.g., turn left or turn right) from storage and play it on the multi-channel audio system. That is, the pre-rendered multidimensional sound that the vehicle 100 retrieves in response to a turn indicator activation event has one of a forward left directivity or a forward right directivity; the other of these pre-rendered multidimensional sounds has another of a forward left directivity or a forward right directivity.

[0020] Pre-rendering may include biasing the sound to the left or right. For example, the main sound may come from the nearest channel (e.g., the front left or front right channel, respectively) and a reverb tail may appear in the channel other than where the main sound is being played. Other techniques may be used.

[0021] The event that triggers the immersive audio experience may relate to an occurrence outside the vehicle 100 itself. In some implementations, the ADAS may detect one or more events from which pre-rendered multi-dimensional sound may be generated, where the sound may appear to originate from the direction of the event. For example, the ADAS may issue a warning in response to the vehicle 100 traveling toward an obstacle and / or another vehicle traveling toward the vehicle 100.

[0022] 2 shows an example of a vehicle audio system 200 with pre-rendered multi-dimensional sound. The vehicle audio system 200 may be used with one or more other examples described elsewhere herein. The vehicle audio system 200 may be implemented using some or all of the examples described below with reference to FIG.

[0023] Vehicle audio system 200 includes at least one audio source 202. Audio source 202 may be local to the vehicle (e.g., a local hard drive, memory, or other audio storage device) and includes at least a number of pre-rendered multi-dimensional sounds. Here, audio content 204 from audio source 202 represents one or more of the pre-rendered multi-dimensional sounds and is shown generally as including channels 206 of audio information. Any number of channels may be used.

[0024] The vehicle audio system 200 may include an audio processor 208 that may receive or obtain audio content 204 having channels 206 from an audio source 202. The audio processor 208 includes a renderer 210 that may be used when playing audio content that has not yet been rendered. Thus, the renderer 210 may mix sounds into one or more of the available channels. For example, the renderer may pan sounds between different channels. Using the renderer 210, sounds to be stored in the audio source 202 may be pre-rendered, or the sounds of the audio source 202 may be pre-rendered using another renderer.

[0025] The audio processor 208 includes a player 212 that can provide multiple channels of audio content to one or more speakers to be played. The player 212 can receive the rendered content from the renderer 210 when the renderer 210 is rendering the content. However, pre-rendered multi-dimensional sounds of the sound source 202 can bypass the renderer 210. In some implementations, the pre-rendered multi-dimensional sounds are pre-panned and pre-mixed among the available channels of the audio system. For example, in this case, the player 212 can obtain the pre-rendered multi-dimensional sounds directly from the sound source 202 without using the renderer 210. This can avoid using CPU resources for rendering (e.g., mixing and panning) these sounds every time they are to be played. Rather, sounds that are already properly spatialized can be called and played without dedicating computer resources to rendering them.

[0026] As shown diagrammatically, at least the audio processor 208 may be implemented in a vehicle 214. The vehicle 214 includes speakers for playing audio, including but not limited to pre-rendered directional announcements. One or more types of speaker types may be used, including but not limited to tweeter speakers, mid-range speakers, full-range speakers, and / or woofers. Each speaker (type) may include one or more transducers (e.g., voice coils) for converting electrical input into sound waves. The vehicle 214 may include a number n of tweeter speakers 216 that may have any of a number of arrangements within the vehicle 214. The vehicle 214 may include a number m of mid-range speakers 218 that may have any of a number of arrangements within the vehicle 214. The vehicle 214 may include a number p of full-range speakers 220 (sometimes referred to as twiddler speakers) that may have any of a number of arrangements within the vehicle 214. The vehicle 214 may include a number q of woofers 222 (eg, subwoofers) that may have any of a number of arrangements within the vehicle 214. Other approaches may be used.

[0027] 3 shows a schematic diagram of an example of a multi-channel audio system 300 and pre-rendered multi-dimensional sound 302. The multi-channel audio system 300 and / or pre-rendered multi-dimensional sound 302 may be used in conjunction with one or more other examples described elsewhere herein. Here, the multi-channel audio system 300 includes a number r of channels 3041, 3042, 3043, ... 3044. r-1 , 304 r where r>1. During playback, channels 3041-304 r may be provided to one or more of the speakers (not shown) of the multi-channel audio system 300.

[0028] The pre-rendered multi-dimensional sound 302 is made up of a number r of components 3061-306 rmay include. Components 3061 to 306 r When any of them is non-zero, the pre-rendered multi-dimensional vowel 302 generates a sound in one of the channels 3041 to 304 r corresponding to that component. The length of the reproduced sound and / or its timing relative to another sound of the pre-rendered multi-dimensional vowel 302 may vary among components 3061 to 306 r Here, among components 3061 to 306 r the two outermost non-zero components are components 306i and 306j, where 1 ≤ i < j and i < j ≤ r.

[0029] The pre-rendered multi-dimensional vowel 302 may include zero or more other non-zero components between components 306i and 306j. Thus, since at least two of components 3061 to 306 r are non-zero, the pre-rendered multi-dimensional vowel 302 is multi-dimensional.

[0030] FIG. 4 shows an exemplary architecture of a computing device 400 that can be used to implement aspects of the present disclosure, including any of the systems, apparatuses, and / or techniques described herein, or any other systems, apparatuses, and / or techniques that can be utilized in various possible embodiments.

[0031] Using the computing device shown in FIG. 4, the operating systems, application programs, and / or software modules (including software engines) described herein can be executed.

[0032] In some embodiments, the computing device 400 includes at least one processing device 402 (e.g., processor), such as a central processing unit (CPU). A variety of processing devices are available from a variety of manufacturers, such as Intel or Advanced Micro Devices. In this example, the computing device 400 also includes a system memory 404 and a system bus 406 that couples various system components including the system memory 404 to the processing device 402. The system bus 406 is one of any number of types of bus structures that may be used, including, but not limited to, a memory bus or memory controller; a peripheral bus; and a local bus using any of a variety of bus architectures.

[0033] Examples of computing devices that may be implemented using computing device 400 include a desktop computer, a laptop computer, a tablet computer, a mobile computing device (such as a smartphone, touchpad mobile digital device or other mobile device), or other device configured to process digital instructions.

[0034] The system memory 404 includes a read-only memory 408 and a random access memory 410. A basic input / output system 412, containing the basic routines that function to transfer information within the computing device 400, such as during start-up, may be stored in the read-only memory 408.

[0035] In some embodiments, computing device 400 also includes a secondary storage device 414, such as a hard disk drive for storing digital data. The secondary storage device 414 is connected to system bus 406 by a secondary storage interface 416. The secondary storage device 414 and its associated computer-readable media provide non-volatile and non-transitory storage of computer-readable instructions (including application programs and program modules), data structures, and other data for computing device 400.

[0036] Although the exemplary environment described herein utilizes a hard disk drive as a secondary storage device, other embodiments use other types of computer-readable storage media. Examples of these other types of computer-readable storage media include magnetic cassettes, flash memory cards, solid-state drives (SSDs), digital video disks, Bernoulli cartridges, compact disk read-only memories, digital versatile disk read-only memories, random access memories, or read-only memories. Some embodiments include non-transitory media. For example, a computer program product may be tangibly embodied in a non-transitory storage medium. Additionally, such computer-readable storage media may include local storage or cloud-based storage.

[0037] A number of program modules may be stored on the secondary storage device 414 and / or in the system memory 404, including an operating system 418, one or more application programs 420, other program modules 422 (such as the software engines described herein) and program data 424. The computing device 400 may utilize any suitable operating system.

[0038] In some embodiments, a user provides input to the computing device 400 through one or more input devices 426. Examples of input devices 426 include a keyboard 428, a mouse 430, a microphone 432 (e.g., for voice and / or other audio input), a touch sensor 434 (such as a touchpad or touch-sensitive display), and a gesture sensor 435 (e.g., for gesture input). In some implementations, the input devices 426 provide presence, proximity, and / or motion-based detection. Other embodiments include other input devices 426. The input devices may be connected to the processing device 402 through an input / output interface 436 coupled to the system bus 406. These input devices 426 may be connected by any number of input / output interfaces, such as a parallel port, a serial port, a game port, or a universal serial bus. In some possible embodiments, wireless communication between the input device 426 and the input / output interface 436 is also possible, including infrared, BLUETOOTH® wireless technology, 802.11a / b / g / n, cellular, ultra-wideband (UWB), ZigBee®, or other radio frequency communication systems, to name just a few.

[0039] In this exemplary embodiment, a display device 438, such as a monitor, liquid crystal display device, light emitting diode display device, projector, or touch-sensitive display device, is also connected to system bus 406 via an interface, such as a video adapter 440. In addition to the display device 438, computing device 400 may also include various other peripheral devices (not shown), such as speakers or printers.

[0040] Computing device 400 may be connected to one or more networks through network interface 442. Network interface 442 may provide wired and / or wireless communication. In some implementations, network interface 442 may include one or more antennas for transmitting and / or receiving wireless signals. When used in a local area networking environment or a wide area networking environment (such as the Internet), network interface 442 may include an Ethernet interface. Other possible embodiments use other communication devices. For example, some embodiments of computing device 400 include a modem for communicating across a network.

[0041] Computing device 400 may include at least some form of computer-readable media. Computer-readable media includes any available media that can be accessed by computing device 400. By way of example, computer-readable media include computer-readable storage media and computer-readable communication media.

[0042] Computer readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any device configured to store information, such as computer readable instructions, data structures, program modules, or other data. Computer readable storage media includes, but is not limited to, random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technology, compact disc read-only memory, digital versatile disk or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 400.

[0043] Computer-readable communication media typically embodied computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and include any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, computer-readable communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

[0044] The computing device shown in FIG. 4 is also one example of a programmable electronic device that may include one or more such computing devices, and when multiple computing devices are included, such computing devices may be coupled together to a suitable data communications network and collectively perform various functions, methods or operations disclosed herein.

[0045] In some implementations, the computing device 400 may be characterized as an ADAS computer. For example, the computing device 400 may include one or more components that may be used to process tasks arising in the field of artificial intelligence (AI). In this case, the computing device 400 generally includes sufficient processing power and necessary supporting architecture for the demands of ADAS or AI. For example, the processing device 402 may include a multi-core architecture. As another example, the computing device 400 may include one or more co-processors in addition to or as part of the processing device 402. In some implementations, at least one hardware accelerator may be coupled to the system bus 406. For example, a graphics processing unit may be used. In some implementations, the computing device 400 may implement neural network specific hardware to process one or more ADAS tasks.

[0046] The terms "substantially" and "about" used throughout this specification are used to describe and report small variations, such as those due to processing variations. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. Also, as used herein, indefinite articles such as "a" or "an" mean "at least one".

[0047] It should be understood that all combinations of the foregoing concepts, and additional concepts discussed in more detail below, are contemplated as being part of the inventive subject matter disclosed herein (provided such concepts are not mutually inconsistent). In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0048] Several implementations have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the specification.

[0049] Additionally, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Additionally, other processes may be provided in or removed from the described flows, and other components may be added to or removed from the described systems. Accordingly, other implementations are within the scope of the following claims.

[0050] While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes that fall within the scope of the implementations. It is to be understood that they have been presented merely as examples and not as limitations, and that various changes in form and details may be made. Any part of the apparatus and / or methods described herein may be combined in any combination, except in mutually exclusive combinations. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described.

Claims

1. 1. A method for generating multi-dimensional audio notifications in a vehicle, comprising: registering an event for which a directional notification is generated in a vehicle including an audio system including a plurality of channels; obtaining a first pre-rendered multi-dimensional sound assigned to the event from among a plurality of pre-rendered multi-dimensional sounds in a storage device, the first pre-rendered multi-dimensional sound including audio for more than one of the plurality of channels; and playing the acquired pre-rendered first multi-dimensional sound using the audio system. A method comprising:

2. 2. The method of claim 1, wherein the pre-rendered first multidimensional sound includes a first sound for all of the plurality of channels and a second sound for fewer than all of the plurality of channels, the first sound and the second sound being played in time separately from one another.

3. The method of claim 2 , wherein playing the captured pre-rendered first multi-dimensional sound comprises playing the first sound before the second sound.

4. The method of claim 2 , wherein the events reflect conditions within a passenger cabin of the vehicle.

5. 5. The method of claim 4, wherein the acquired first pre-rendered multi-dimensional sound includes a seat belt reminder, and the plurality of pre-rendered multi-dimensional sounds in the storage device include a plurality of seat belt reminders each corresponding to a different location within the passenger cabin.

6. 5. The method of claim 1, wherein the acquired first pre-rendered multidimensional sound comprises one of a front-left directivity or a front-right directivity, and a second pre-rendered multidimensional sound of the pre-rendered plurality of multidimensional sounds comprises another of the front-left directivity or the front-right directivity.

7. The method of claim 6 , wherein the first pre-rendered multi-dimensional sound and the second pre-rendered multi-dimensional sound include a turn signal notification.

8. The method of claim 1 , wherein the obtained first pre-rendered multi-dimensional sound comprises a seat belt reminder.

Images