Dynamic chunking for sounds from car input

Abstract

A computer system for manipulating, combining, or composing dynamic sound accesses a package comprising one or more music tracks. The computer system then receives an input variable from one or more vehicle sensors. The one or more vehicle sensors measure an aspect of a driving parameter of a vehicle. In response to the input variable, the computer system generates a particular audio effect using the one or more music tracks.

Description

Cross-reference to related applications

[0001] This application claims the benefit and priority of the following applications: 1) U.S. Patent Application Serial No. 19 / 008,329, filed January 2, 2025, entitled "DYNAMIC CHUNKING FOR SOUNDS FROM AUTOMOTIVE INPUTS"; 2) U.S. Provisional Patent Application Serial No. 63 / 733,180, filed December 12, 2024, entitled "DYNAMIC SOUNDS FROM AUTOMOTIVE INPUTS"; 3) U.S. Provisional Patent Application Serial No. 63 / 632,464, filed April 10, 2024, entitled "DYNAMIC SOUNDS FROM AUTOMOTIVE INPUTS"; and 4) U.S. Patent Application Serial No. 19 / 008,329, filed January 4, 2024, entitled "DYNAMIC SOUNDS FROM AUTOMOTIVE INPUTS". U.S. Provisional Patent Application No. 63 / 617,716, entitled "INPUTS (Dynamic Sounds from Automotive Inputs)," is incorporated herein by reference in its entirety.

[0002] This application is also a continuation-into-part of U.S. Application No. 18 / 337,017, filed June 18, 2023, entitled "DYNAMIC SOUNDS FROM AUTOMOTIVE INPUTS," which claims the benefit and priority of the following applications: 1) U.S. Provisional Patent Application No. 63 / 447,265, filed February 21, 2023, entitled "DYNAMIC SOUNDS FROM AUTOMOTIVE INPUTS"; 2) U.S. Provisional Patent Application No. 63 / 440,879, filed January 24, 2023, entitled "AI GENERATED SOUNDS FROM AUTOMOTIVE INPUTS"; and 3) U.S. Provisional Patent Application No. 63 / 440,879, filed November 28, 2022, entitled "AI GENERATED SOUNDS FROM AUTOMOTIVE INPUTS." U.S. Provisional Patent Application Serial No. 63 / 428,376 entitled "AI GENERATED SOUNDS FROM AUTOMOTIVE INPUTS" filed June 21, 2022, entitled "AI GENERATED SOUNDS FROM AUTOMOTIVE INPUTS" Serial No. 63 / 354,174. The entire contents of each of the foregoing applications and / or patents are incorporated herein by reference in their entirety. Background Technology

[0003] In recent years, the popularity of music streaming services has grown significantly. Users now have access to vast music libraries, allowing them to stream their favorite songs anytime, anywhere. Most music services now offer users customized music selections. Users can request fully customized playlists, or they can request music of specific genres or types. The music service can then generate a playlist, either wholly or partially, based on the user's previously identified preferences.

[0004] Therefore, modern music services can offer users customized music experiences. These experiences are driven by a suite of software, including artificial intelligence, that carefully collects information about a user's musical tastes and curates the music experience based on that information. The ability to map music and sound to a user using technology provides significant benefits to the user's music experience.

[0005] The subject matter claimed herein is not limited to embodiments that address any shortcomings, nor is it limited to embodiments that operate only in environments such as those described above. Rather, this background art is provided merely to illustrate an exemplary technical field in which some of the embodiments described herein can be practiced. Summary of the Invention

[0006] The disclosed embodiments include a computer system for manipulating, combining, or synthesizing dynamic sounds. The computer system accesses a package of one or more music stems. The computer system then receives input variables from one or more vehicle sensors. The one or more vehicle sensors measure one aspect of vehicle driving parameters. In response to the input variables, the computer system uses the one or more music stems to generate specific audio effects.

[0007] In some aspects, the technology described herein relates to a computer system for dynamically segmenting sound input from a vehicle at a vehicle location, comprising: one or more processors; and one or more computer-readable media storing executable instructions that, when executed by the one or more processors, configure the computer system to: receive a digital audio file of a specific song; process the digital audio file into multiple tracks and segments, wherein: each track represents a separate component or element of the song, and each segment comprises a continuous portion of the song; number the segments in priority order; associate the tracks from the segments with input variables from one or more vehicle sensors; and generate digital data packets comprising the one or more tracks, one or more segments, and the associations with the input variables from the one or more vehicle sensors.

[0008] In some aspects, the technology described herein relates to a computer-implemented method for dynamically segmenting sound input from a vehicle at a vehicle location, the computer implementation method being executed by one or more processors, the computer implementation method comprising: receiving a digital audio file of a specific song; processing the digital audio file into multiple tracks and segments, wherein: each track represents a separate component or element of the song, and each segment comprises a continuous portion of the song; numbering the segments in priority order; associating the tracks from the segments with input variables from one or more vehicle sensors; and generating a digital data packet comprising the one or more tracks, one or more segments, and the association with the input variables from the one or more vehicle sensors.

[0009] This summary is provided to introduce some concepts in a simplified form, which will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to serve as an aid in determining the scope of the claimed subject matter.

[0010] Additional features and advantages will be set forth in the following description and will be apparent in part from the description, or may be learned by practicing the teachings herein. The features and advantages of the invention can be realized and obtained by the tools and combinations particularly pointed out in the appended claims. The features of the invention will become more apparent from the following description and the appended claims, or may be learned by practicing the invention as described below. Attached Figure Description

[0011] To describe how the aforementioned and other advantages and features are obtained, the subject matter briefly described above will be described in more detail with reference to specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only typical embodiments and are therefore not intended to limit the scope; the embodiments will be described and explained in a more specific and detailed manner using the drawings described below.

[0012] Figure 1 A schematic diagram of a computer system for generating voices using AI based on input from a car is shown.

[0013] Figure 2 A schematic diagram of roads and vehicles is shown.

[0014] Figure 3 A flowchart is shown for a method to generate AI-generated sounds from input from a car.

[0015] Figure 4 The user interface for generating AI-generated sounds from car input is shown.

[0016] Figure 5 Another user interface is shown for generating AI-generated sounds from car input.

[0017] Figure 6 An example audio track is shown that has been divided into sections.

[0018] Figure 7 A schematic diagram of a sample audio engine for managing chunks of digital audio files is shown.

[0019] Figure 8 A schematic diagram of an example audio graph is shown.

[0020] Figure 9A A schematic diagram of an example update process for segmenting digital audio files is shown.

[0021] Figure 9B A schematic diagram of another example update process for segmenting digital audio files is shown.

[0022] Figure 10 A schematic diagram of an example audio block view table buffer is shown.

[0023] Figure 11 A flowchart is shown for a method of dynamically segmenting sound input from a vehicle at a vehicle location. Detailed Implementation

[0024] The disclosed embodiments include a computer system for combining, manipulating, or synthesizing dynamic sounds. The computer system receives input variables from vehicle sensors. In response to these input variables, the system generates, combines, and / or manipulates specific sounds mapped to those input variables. For example, when a driver is traveling along a road or highway, the user's speed, braking, steering, reversing, and other vehicle actions can be used to create a unique soundtrack that matches the driving experience.

[0025] The disclosed embodiments allow drivers to create a customized soundscape that connects the driver, the vehicle, and the driving environment. As used herein, a “soundscape” includes any recording of audio synthesis or soundtrack that is dynamically adjusted to suit the driving of the vehicle. Drivers can create customized soundscapes wholly or partially based at least in part on sensor readings (i.e., input variables) from vehicle sensors. As used herein, vehicle sensors may include, but are not limited to, the following sensors: steering sensors, suspension sensors, IMU sensors, gyroscopes, accelerometers, speed sensors, acceleration sensors, gear position sensors, brake sensors, GPS sensors, temperature sensors, clocks, rain sensors, odometers, weather data, and any other common vehicle sensors. In at least one embodiment, vehicle sensors may include sensors not integrated into the vehicle itself, such as a GPS sensor within a mobile phone that transmits the GPS coordinates of the mobile phone, and therefore the GPS coordinates of the vehicle, to a computer system. A combination of one or more sensors can be used to create customized soundscapes that respond to the driver and the area the vehicle is traversing. I. Computer system for dynamically generating sound from automotive input

[0026] The disclosed embodiments include an artificial intelligence system that utilizes input variables (such as navigation routes, speed, time, location, etc.) from vehicle sensors to generate or manipulate audio synthesis in real time. Thus, in some embodiments, the driver performs the functions of a DJ or even a composer of unique music or soundscapes based on how, where, and when the driver is driving the vehicle. Similarly, the vehicle becomes an ecosystem for new creative experiences. In at least one embodiment, the driver is able to publish the soundscapes they create on other platforms for others to consume and / or purchase.

[0027] The driver can select a specific base song, multiple tracks, or a single track, which the user can control through their driving. For example, when the user accelerates, the base song or track can be sped up or its volume increased. Conversely, when the user applies the brakes, the speed or volume of the base song or track can be decreased. Similarly, in at least one embodiment, when the user accelerates or decelerates, the computer can apply one or more filters to the base song or track, or remove one or more filters from the base song or track. Thus, the user can select a popular song, such as Will.i.am's OMG. The user can then "customize" or otherwise manipulate the song in real time based on their driving. Specifically, the song or track can be manipulated to reflect the user's feelings based on how they drive the vehicle or other relevant information.

[0028] Figure 1 A schematic diagram of a computer system 100 for generating AI voice from automotive input is shown. The computer system 100 includes one or more processors 110 and a computer storage medium 120. The computer storage medium 120 stores computer-executable instructions that, when executed, cause the computer system 100 to perform various actions. For example, these computer-executable instructions may include instructions for use with dynamic sound generation software 130.

[0029] The dynamic sound generation software 130 application includes a sound generation engine 140, a sensor API 150, and a music library storage 160. In at least one embodiment, the disclosed system can intelligently generate any aspect of a soundscape, including melody, rhythm, tempo, and various audio effects. Audio during a driving journey can be generated in real time and played on the vehicle's speakers in real time, becoming an integral part of the driving experience. The audio can also be saved as a file for later playback. The audio can also be uploaded to social media platforms and / or marketplaces for other users to consume and experience.

[0030] As used herein, "engine" includes computer-executable code and / or computer hardware that performs a particular function. Those skilled in the art will understand that the distinction between different engines is at least partly arbitrary, and engines can be otherwise combined and divided, while still remaining within the scope of this disclosure. Therefore, the description of components as "engines" is provided merely for clarity and explanation purposes and should not be construed as indicating any particular structure requiring computer-executable code and / or computer hardware unless expressly stated otherwise. Similarly, the terms "component," "agent," "manager," "service," "module," "virtual machine," etc., may be used in this specification.

[0031] In at least one embodiment, the sound generation engine 140 includes artificial intelligence algorithms, machine learning algorithms, neural networks, or some other suitable algorithm that can be used to synthesize sound tracks based on various sensor inputs received from the driver's vehicle 170. The sound generation engine 140 can receive sensor input from a sensor API 150. The sensor API 150 can be configured to receive sensor data from vehicle sensors.

[0032] When the sound generation engine 140 generates a soundscape, it can utilize information from both the sensor API 150 and the music library storage 160. For example, the music library storage 160 may include acoustic profiles of different types of instruments, different genres, different songs, different song samples, different individual tracks, and / or different groups of tracks. Furthermore, the dynamic sound generation software 130 application can store custom-created tracks in the music library storage 160 when the driver is creating the soundscape.

[0033] In at least one embodiment, one or more portions of the dynamic sound generation software 130 may be distributed among multiple different processors located in multiple different locations. For example, in at least one embodiment, a portion of the dynamic sound generation software 130 is hosted in the cloud, enabling multiple different vehicles to communicate with the cloud-hosted portion. Similarly, portions of the dynamic sound generation software 130 may be hosted by each of the multiple different vehicles.

[0034] In at least one embodiment, the dynamic sound generation software 130 may include a music store that allows users to download and / or upload songs, audio tracks, and other soundscape components. For example, users may be allowed to purchase multiple different audio tracks, from which a driver can select and / or mix them together to create a desired soundscape. Once purchased, these audio tracks, songs, or other soundscape components may be stored locally at the vehicle. Alternatively, in some embodiments, the audio tracks, songs, or other soundscape components are stored in the cloud and downloaded by the vehicle 170 as needed.

[0035] In at least one embodiment, certain audio tracks, songs, or other soundscape components are available only at a specific destination, at a specific time, after a user performs a specific action, and / or under certain other specific circumstances. For example, a specific audio track may only be available to the driver if the driver passes a specific restaurant at midday. Similarly, a specific soundscape may only be available to the driver once the driver has driven more than 100,000 miles in vehicle 170. As another example, a specific song may only be available if the driver is driving in snow. Once available, these audio tracks, songs, or other soundscape components may be automatically downloaded to vehicle 170, automatically downloaded to the driver's cloud storage device, and / or presented to the user as an optional download.

[0036] In some embodiments, a user may be able to purchase audio tracks, songs, or other soundscape components from their mobile phone or computer. Alternatively, a user may be able to purchase audio tracks, songs, or other soundscape components from a user interface integrated into the vehicle's entertainment system. In either case, once purchased, the user may be able to control and use the purchased audio tracks, songs, or other soundscape components. II. Sensors and data used as input to the sound generation engine

[0037] Any movement or parameters received from vehicle sensors can affect the synthesis of the soundscape. Sensor API 150 can receive data from vehicle sensors, including but not limited to turning, speed, acceleration and deceleration, route taken, braking actions, gear shifting and gear changes, forward and reverse motion, sonar, radar, energy recovery, and any other mechanical movement of the vehicle. Sensor API 150 can also receive any data from vehicle sensors related to vehicle 170 or the driving experience, which may include, but is not limited to, journey time and duration, weather conditions, environmental conditions, traffic conditions, journey location, origin and destination, characteristics of the driver and passengers, the brand and model of the vehicle itself, and other vehicle-specific characteristics. In at least one embodiment, Sensor API 150 can receive input variables from gyroscopes and accelerometers integrated within vehicle 170. For example, input variables from a gyroscope can provide a better soundscape experience than input variables from a steering wheel, because the driver turning the steering wheel to stop may not reflect the actual physical feel of steering within vehicle 170. In this case, the gyroscope can provide a better input variable.

[0038] In at least one embodiment, the soundscape synthesis can be customized for a specific driver based on information about the driver. The driver can create a user profile, and / or can create user profiles for driving over time. For example, a driver may prefer specific types of music, music intensity, instrument type, specific performers, music loudness, music era, and various other categories and characteristics of music. In some embodiments, the user's voice and music preferences can be collected from pre-existing data about the user, such as the user's playlists or music listening history. Furthermore, the server can be able to track an individual's music preferences over time and location. For example, the server can indicate that the user prefers different music / sounds when driving in a forest compared to driving in a desert, or when driving in the rain compared to driving in a sunny day. The resulting music / sound synthesis can be adjusted to reflect these differences.

[0039] In at least one embodiment, the sensor API 150 may receive data from vehicle sensors, including one or more of the following example sources: acceleration, GPS, AI voice, front sonar / radar, rear sonar / radar, and / or vibrations in the seat. In additional or alternative embodiments, the sound generation engine 140 may map specific sensors to specific audio parameters. For example, acceleration in the vehicle 170 may be mapped to an audio / patch / preset sound. The brake pedal may be mapped to an audio release / attenuation / timbre / preset sound. The steering wheel may be mapped to an envelope / filter / timbre / preset sound. The suspension may be mapped to an LFO / timbre / preset sound. The speedometer may be mapped to an arpeggio / timbre / preset sound activated after a specific speed is reached.

[0040] In at least one embodiment, each track, track group, or audio effect can be mapped to a specific input variable via metadata stored along with that track, track group, or audio effect. Users can use a software interface to map specific tracks, track groups, or audio effects to desired input variables. For example, a user can map a percussion track group to an input variable from an accelerometer. Furthermore, users can define the mapping by indicating the relationship between the accelerometer input variable and the percussion track group. For example, a user can define that pressing the accelerometer causes the percussion track group to play at a faster speed and a higher volume. Additionally, users can define that a specific filter is applied to the percussion track group when the vehicle is below a specified speed, and another filter is applied when the vehicle is above another specified speed. Each of these parameters can be stored along with the percussion track group, enabling the vehicle to correctly map the track group to the correct input variable. A system for associating tracks, track groups, or audio effects with input variables will be further described below.

[0041] Similar to the mappings described above, the sound generation engine 140 may also include default mappings if no other mappings are provided. For example, input variables of the vehicle 170 suspension may be mapped to percussion instruments, while input variables of the accelerator may be mapped to guitars. It should be understood that the listed mappings are provided for illustrative purposes only. Various additional or alternative mappings may also be used without departing from this disclosure. Additionally or alternatively, the sound generation engine 140 may also utilize the non-speaker characteristics of the vehicle to create a more complete audio experience. For example, the sound generation engine 140 may vibrate the driver's seat or steering wheel based on sensor data. Such features can allow for a more immersive audio experience. In at least one embodiment, parameter configurations on all channels can be set to provide the vehicle with ultra-flexible, ultra-dynamic audio / perceptual intelligence.

[0042] Furthermore, in at least one embodiment, one or more input variables can be dynamically scaled by the sensor API 150. For example, input variables related to speed or acceleration can be dynamically scaled based on the speed limit of the road on which the vehicle 170 is traveling. For example, input variables from vehicle sensors may include a vision system that reads speed limit signs, or a location / map system (e.g., GPS) that provides speed limits from a map or database based on the location detected by the vehicle. The sensor API 150 can scale the input variables such that a full range of audio effects can be applied within the speed limits. For example, metadata associated with a particular audio track can indicate lower and higher speeds for which different audio effects are applied. In at least one embodiment, the sensor API 150 scales the lower and higher speeds such that both are adapted to the speed limit of the road on which the vehicle is traveling. Therefore, the sensor API 150 is configured to encourage safe driving by ensuring that audio effects are scaled to be applied within the speed limits. III. Dynamic Generation and Customization of Music

[0043] In at least one embodiment, the user's vehicle 170 is capable of functioning as a soundscape synthesis system as the user drives from point A to point B. Such a system transforms the vehicle 170 into an ecosystem for new creative experiences. The disclosed embodiments open the door for creative communities to create soundscapes for drivers, adding new color to the world of musical and audio journeys. Users can then sell, license, or otherwise share their soundscape compositions through streaming services or other downloadable services.

[0044] For example, in at least one embodiment, sensor API 150 feeds data to sound generation engine 140, which in turn generates a customized dynamic soundscape for the driver and passengers of the vehicle. In some embodiments, sound generation engine 140 includes an artificial intelligence algorithm that processes the data received from sensor API 150 as well as driver-specific data. The artificial intelligence algorithm creates a soundscape that is generated in real time and is also personalized for the driver.

[0045] Furthermore, in at least one embodiment, one or more modes may be associated with the playback of a soundscape. For example, the audio system of a vehicle may include various modes, such as aggressive mode, relaxed mode, cheerful mode, etc. The sound generation engine 140 may adjust the soundscape based on the audio system mode. Additionally or alternatively, the sound generation engine 140 may adjust the soundscape based on the driving mode of the vehicle 170. For example, many vehicles have an economy driving mode, a sport driving mode, a normal driving mode, and various other driving modes. Each driving mode may include unique scaling parameters, limits, and / or AI responses. For example, putting the car in sport mode may cause the sound generation engine 140 to play faster audio effects and at a higher volume, while economy mode may result in slower audio effects and at a lower volume.

[0046] The following discussion relates to non-limiting examples of the dynamic sound generation software 130. These examples are provided for illustrative and clarity purposes only and should not be construed as limiting or otherwise restricting key or essential features of the invention.

[0047] The sound of a vehicle 170 traveling along a highway can generate a natural rhythm based on road noise from road joints or on the driver's movement over speed bumps at the road edge. In at least one embodiment, sensors within the suspension of the vehicle 170 can identify this rhythm and transmit it to the computer system 100 via sensor API 150. A sound generation engine 140 can generate an acoustic fingerprint based on the recorded rhythm. The acoustic fingerprint can be created using a spectrogram or any other acoustic fingerprinting method used in the art.

[0048] The sound generation engine 140 can then map the acoustic fingerprint to a pre-stored acoustic fingerprint in a music library storage 160. The music library storage 160 may include a database of beats, rhythms, hooks, melodies, etc., associated with the pre-stored acoustic fingerprints. After identifying a match or the closest match to the generated road noise acoustic fingerprint, the sound generation engine 140 can insert the identified match or the closest match music from the music library storage 160 into the soundscape.

[0049] Additionally or alternatively, the sound generation engine 140 can access track bundles from the music library storage 160, each track bundle being mapped to an input variable. The input variables associated with the suspension sensors can be used to manipulate the track bundles associated with the suspension. For example, the sound generation engine 140 can apply filters, adjust filters, adjust speed, adjust volume, or perform any number of other audio adjustments to the track bundles. For example, the sound generation engine 140 can speed up the audio of the track bundles until their tempo matches the tempo of the suspension vibrations (or a factor of that tempo).

[0050] Furthermore, in at least one embodiment, when searching for a matching acoustic fingerprint in the music library storage 160, the sound generation engine 140 can limit the search scope based on the driver's profile. For example, the driver profile may indicate a preference for country music. Therefore, the sound generation engine 140 can limit the search in the music library storage 160 to include only tracks, track groups, beats, rhythms, catchy snippets, melodies, etc., belonging to the country music genre.

[0051] In additional or alternative embodiments, the sound generation engine 140 may utilize GPS and map data when generating soundscapes. For example, Figure 2 A schematic diagram of road 200 and vehicle 170 is shown. Sound generation engine 140 can use GPS data to identify the type of area the vehicle 170 is traveling through. For example, if the vehicle is traveling along a Pacific Coast Highway, sound generation engine 140 can generate a beach-style song. Furthermore, sound generation engine 140 can rely on tracks, track groups, beats, rhythms, catchy snippets, melodies, etc., from music library storage 160, based on songs from bands such as the Beach Boys, Jack Johnson, Colbie Caillat, and other musicians with a distinctly "beach-style" aesthetic.

[0052] As another example of the sound generation engine 140 utilizing input variables in the form of location data (such as GPS data), if the sound generation engine 140 detects that a vehicle is entering New York City, it can generate a soundscape with a stronger influence of urban music styles. For example, the sound generation engine 140 can generate a soundscape based on recent music samples created by music groups located in New York City. Similarly, the sound generation engine 140 can utilize GPS data to identify popular songs in the local market, or songs that reference or are related to the vehicle's location. Therefore, the sound generation engine 140 can generate a soundscape at least partially based on a list of currently popular songs in New York City, or based on songs that famously reference or are related to New York City. For example, in Figure 2 In this embodiment, digital data packet 240 may represent a location-specific audio track or audio file, which vehicle 170 can access when entering a general geographic area already associated with digital data packet 240. It should be understood that the visual representations of digital data packets 240 and 230 are provided for illustrative purposes only. In practice, digital data packet 240 is not necessarily physically located at a geographic location. Instead, digital data packet 240 may be hosted on a server and provided to vehicle 170 when vehicle 170 reaches within a threshold distance of the location of digital data packet 240. Additionally or alternatively, in at least one embodiment, digital data packet 240 may be hosted by a server located at that geographic location, such that the audio track or audio file is provided to vehicle 170 via a local area network when vehicle 170 enters network range.

[0053] In at least one embodiment, the sound generation engine 140 generates only soundscapes that conform to a user profile. For example, a user may indicate a preference for hip-hop and rock music, and may also indicate a dislike for jazz music. In response, the sound generation engine 140 may generate only soundscapes consistent with hip-hop and rock music, while avoiding soundscapes that utilize jazz music elements.

[0054] In additional or alternative embodiments, the sound generation engine 140 may also utilize location data to identify current weather conditions in the area where the vehicle 170 is located. For example, the sound generation engine 140 may use an online weather service to determine that it is currently snowing in the area where the vehicle is traveling. In response to identifying that it is snowing, the sound generation engine 140 may create a soundscape based on the weather information. In this case, the soundscape may include warmer and softer tones, and / or may include sound elements based on winter-related music (e.g., Christmas music). Conversely, if the sound generation engine 140 determines that the weather is sunny based on external weather conditions, the sound generation engine 140 may generate a cheerful and faster-paced soundscape. It should be understood that different weather patterns can be associated with many different attributes. Therefore, the examples above are provided only for clarity and discussion purposes. Any number of different soundscape features can be generated based on data received from weather reports.

[0055] The sound generation engine 140 can also receive map data related to the driver's trip plan. For example, the sound generation engine 140 can receive the origin and destination of the vehicle. In response, the sound generation engine 140 can generate a soundscape that considers the entire planned journey of the vehicle. For example, the sound generation engine 140 can consider the time of day, expected weather, expected traffic patterns, and other trip-related data when creating the soundscape. At the start of the trip, the sound generation engine 140 can generate a cheerful and energetic soundscape to motivate the driver during the journey. As the driver approaches expected congestion later in the journey, the sound generation engine 140 can generate a calming soundscape to help the driver better cope with traffic. If the trip ends late, the sound generation engine 140 can generate a loud and exciting soundscape to help the driver stay alert and focused.

[0056] Furthermore, in at least one embodiment, the voice assistant can also be integrated into a soundscape. For example, if a driver is receiving driving instructions from a voice assistant, the voice generation engine 140 can manipulate the voice assistant to speak with a volume, rhythm, beat, effect, etc., that matches the soundscape. For instance, the voice generation engine 140 can make the voice assistant speak with an echo that matches the rhythm of the soundscape. As another example, the voice generation engine 140 can make the voice assistant sing in a style that matches the soundscape.

[0057] In at least one embodiment, at least a portion of the soundscape created by the driver is stored in a music library storage. The music library storage may be located locally on the vehicle, in the cloud, in local storage at a specific location, or a combination of local and cloud storage. The driver may subsequently access and listen to the soundscape, share it with others, sell or license it, or otherwise manipulate it as desired by the driver. For example, a renowned music artist may create a specific soundscape based on a drive from Los Angeles, California to Santa Barbara, California. Other drivers may purchase or otherwise listen to the same soundscape.

[0058] For example, another driver might plan the same journey from Los Angeles, California to Santa Barbara, California. This driver might want to experience the same soundscape created by the same musical artist. Therefore, the user could purchase or license the soundscape created by the musical artist for the same journey. In at least one embodiment, the sound generation engine 140 can adjust and modify the original soundscape in real time based on the driver's location along the journey. For example, the driver might leave at a different time than the original soundscape composer did. Due to traffic differences, the driver's location might not be synchronized with the composer's location. Therefore, the sound generation engine 140 can lengthen or shorten specific portions of the original soundscape to ensure the driver's location is synchronized with the geographic location in the original soundscape. Thus, when the driver passes through a specific location between Los Angeles and Santa Barbara, California, the driver is experiencing a soundscape as created by the original composer.

[0059] In additional or alternative embodiments, music artists may also create custom audio layers or tracks, which they geolocate to specific locations on a map. The music library storage 160 may store multiple custom audio layers from different artists, and at least some of these audio layers may be geolocated to specific locations. For example, an artist may create an audio layer and associate it with a specific location on Park Avenue in New York City. When a vehicle passes that specific location, the sound generation engine 140 may access the audio layer and add it to the current soundscape. In at least one embodiment, the driver may have the option to automatically incorporate audio layers from artists into their driving.

[0060] In some embodiments, acquiring audio layers from musical artists can employ gamification mechanisms. For example, when a driver passes a location designated by the artist, the sound generation engine 140 can unlock or download the audio layer. For instance, if vehicle 170 passes building 220, vehicle 170 can gain access to digital data packets 230. Digital data packets 230 may include audio tracks and / or audio files, which the user can now use to create soundscapes. In at least one embodiment, once a driver acquires an audio layer, the driver can autonomously use the audio layer at any location. Therefore, as the driver acquires more and more audio layers, the driver is able to create increasingly complex and interesting soundscapes by utilizing these audio layers.

[0061] In additional or alternative embodiments, advertising creatives can also be incorporated into the soundscape. For example, in some configurations, as a driver drives a vehicle along a road, the sound generation engine 140 can identify nearby locations where advertising creatives have been prepared for the system. For instance, before the vehicle passes a fast-food restaurant (e.g., building 220), an advertising audio layer (e.g., digital data packet 230) prepared by the fast-food company can be added to the soundscape. The fast-food company can prepare a series of different audio layers tailored to different users' music genre preferences, allowing different drivers to load different audio layers at the same location based on their individual profiles. Thus, when a driver passes a specific point, customized advertising creatives can be provided to the driver, which are then overlaid into their customized soundscape.

[0062] In at least one embodiment, the driver can pay a subscription fee to avoid advertisements. Furthermore, the user can select advertisements that interest them personally. Additionally, the sound generation engine 140 can "intelligently" identify advertisements of interest. For example, the sound generation engine 140 might play fast food advertisements near lunchtime but not at 3 PM. Similarly, the sound generation engine 140 might play gas station advertisements when the fuel sensor indicates a low fuel level, but not when the fuel sensor indicates a high fuel level.

[0063] Therefore, the sound generation engine 140 is able to create custom soundscapes in a variety of ways that respond to the driver, the location of the vehicle, the loaded content (e.g., an audio layer from a music artist), the weather, and / or various other inputs. IV. Example method for AI-generated sound from automotive input.

[0064] The following discussion now involves multiple methods and method actions that can be performed. Although method actions may be discussed in a specific order, or shown in a flowchart as occurring in a specific order, no particular order is required unless explicitly stated, or because one action depends on another action that must be completed before that action is performed.

[0065] Figure 3 A flowchart is shown for a method 300 for generating AI-generated sound from automotive input. Method 300 includes an action 310 of accessing music tracks. Action 310 includes a package for accessing one or more music tracks. For example, computer system 100 may access music tracks stored in music library storage 160.

[0066] Furthermore, method 300 includes an action 320 of receiving input variables. Action 320 includes receiving input variables from a vehicle sensor. The vehicle sensor measures aspects of vehicle driving parameters. For example, sensor API 150 may receive data from a sensor connected to an accelerometer.

[0067] Method 300 may also include action 330 for generating sound. Action 330 includes generating a specific sound mapped to the input variable in response to the input variable. For example, sound generation engine 140 may create a custom soundscape within a vehicle based on a driver pressing and releasing the accelerator.

[0068] In at least one embodiment, the sound generation engine 140 can also be configured to create an external soundscape for the vehicle 170. The external soundscape may match the internal soundscape, or may include different sounds. For example, the sound generation engine 140 may generate an external soundscape as a safety feature of an electric vehicle. In many cases, electric vehicles are so quiet during normal operation that pedestrians may not hear the vehicle reversing or approaching from behind. Therefore, in at least one embodiment, the sound generation engine 140 may utilize input variables to create an external soundscape to provide a warning to others about the approach of a vehicle. For example, in at least one embodiment, placing the vehicle 170 in reverse will cause external speakers on the vehicle to play the internal soundscape. Additionally or alternatively, different soundscapes may be played on external speakers. The external soundscape may utilize audio tracks or groups of audio tracks to create a customized soundscape based on input variables from the vehicle 170. V. Systems used for creating soundscape packs

[0069] In at least one embodiment, the computer system may provide a user with an interface for creating soundscape packs. Figure 4A user interface 400 for generating AI-generated sounds from vehicle input is shown. As used herein, a soundscape package includes audio components, such as soundtracks, which have been associated with input variables received from vehicle sensors. For example, the interface may display visual representations of audio tracks 412, 414, 416, and 418 in a selection of soundtrack group 410. Soundtrack group 410 may be provided to the computer system by a music artist who has uploaded the soundtrack group to the computer system. The user can then associate one or more of the soundtracks 412, 414, 416, and 418 with visual representations of specific input variables 420 (e.g., accelerator 422, brakes 424, suspension 426, GPS 428, etc.) and / or visual representations of specific filters and / or audio effects 430, 432, 434, 436, and 438. For example, the user can associate bass track 412 with the acceleration 422 of vehicle 170. Such a connection can be achieved by dragging the bass track 412 onto the visual indicator of the accelerometer 422.

[0070] Figure 5 Another user interface 500 is shown for generating AI-generated sound from vehicle input. Once the user has associated the bass track 412 with the accelerometer 422, the user interface 500 allows the user to further customize the interaction between specific input variables 420 and track groups 410. For example, in the user interface 500, a scaled line 510 for the accelerometer is shown. As further described herein, the scaled line 510 can be used to allow the computer system 100 to scale the soundscape to the speed limit of a given road, and / or to allow the computer system 100 to scale the soundscape to a specific model of vehicle.

[0071] In user interface 500, the user has already associated filter A 432 with an accelerator input variable from a scaling range of 2 to 4. Once accelerator 422 reaches level 4 on scaling line 510, bass track 412 is associated with accelerator 422. Furthermore, once accelerator 422 reaches level 4 on scaling line 510, bass track 412 becomes associated with the input variable from the accelerator. Furthermore, once accelerator 422 reaches level 7 on scaling line 510, effect B 438 becomes associated with bass track 412 and accelerator 422. Furthermore, once accelerator 422 reaches level 4 on scaling line 510... Therefore, when initially accelerating, the user can first hear filter A 432 applied to the soundscape. Filter A 432 will transition to bass track 412 at some point during the acceleration process. Finally, effect B 438 will be applied to bass track 412. It should be understood that the user can add additional audio effects, such as specific filters that should be applied at different times based on one or more input variables. For example, a user can instruct that a distortion filter should be applied to a percussion track during the initial acceleration of a vehicle.

[0072] Once the user has completed their desired association between the track groups, input variables, and audio effects, the computer system can create a soundscape package formatted for reading by a computer system within the vehicle. In at least one embodiment, the soundscape package includes metadata associated with the package and / or the track groups within it. For example, each individual track group may be stored along with metadata that associates the track group with one or more input variables from one or more vehicle sensors and one or more audio effects.

[0073] In at least one embodiment, the computer system can set limits on various input variables. For example, the computer system can set an upper limit of 100 MPH for any speed input variable. Additionally or alternatively, the computer system can set a dynamic upper limit on the speed input variable based on the vehicle's real-time speed limit. In at least one embodiment, when associating a track group with an input variable (e.g., speed), the user specifies the relationship in terms of scale rather than specifying actual values. For example, the user can instruct that a specific filter should be added at level 8 in an exemplary scale of 1-10. The scale information can be encoded into the metadata associated with the track group. In use, the vehicle can recognize the current speed limit and normalize the scale to the speed limit such that a specific filter is added when 80% of the speed limit is reached. It should be understood that the examples of the 1-10 scale, speed input variable, and filter are merely exemplary, and the process can be applied to any number of different input variables.

[0074] In at least one embodiment, the computer system can set limits on various audio effects. For example, the computer system can set limits on the volume levels allowed for a specific track group and / or soundscape pack. For example, a user can indicate that volume level 8 should be used (level 8 in example scale 1-10). Scale information can be encoded into metadata associated with the track group. In use, the system can identify the current volume level of the audio system and scale the audio of the track group accordingly. It should be understood that the example of the 1-10 scale and volume levels is merely exemplary, and this process can be applied to any number of different audio effects.

[0075] In additional or alternative embodiments, the computer system may also correlate various input variables. For example, in response to a first input variable, such as the acceleration of a vehicle, the sound generation engine 140 may apply a specific audio effect (e.g., increase volume) to one or more music tracks. The sensor API 150 may then determine that the first input variable exceeds a threshold. For example, the sensor API 150 may determine that the driver has released the accelerator more than a threshold amount. Based on the first input variable exceeding the threshold, the sound generation engine 140 may apply a specific audio effect (e.g., volume) to one or more music tracks in response to a second input variable.

[0076] For example, immediately cutting off a specific audio effect from the soundscape based on the driver releasing the accelerator could produce an undesirable soundscape. To create a more seamless soundscape experience, when the sensor API 150 detects that the accelerator has been released to a threshold amount, the sensor API 150 can switch to associating vehicle speed with a specific music track and / or audio effect. Therefore, when the user releases the accelerator, the music track and audio effect associated with vehicle acceleration can seamlessly switch to being associated with vehicle speed. Such a switch should provide a more continuous and subtle drop-off in audio effects and music tracks. It should be understood that the examples provided are not limiting. Any number of different audio effects, input variables, and tracks can be associated with thresholds that cause the sensor API 150 and / or sound generation engine 140 to switch associations between input variables, audio effects, and tracks to create a better soundscape experience.

[0077] Furthermore, in at least one embodiment, it may be desirable to adjust the soundscape based on the characteristics of the vehicle playing and creating the soundscape. For example, different vehicles include different audio systems, different haptic systems, and different performance characteristics. A sports car with a high-end stereo system will provide a very different soundscape experience than a large SUV with a low-end stereo system.

[0078] In at least one embodiment, the soundscape package can be customized to operate with a specific type and configuration of vehicle. For example, each type and configuration of vehicle can download slightly different soundscape packages with metadata optimized to work with a specific vehicle type and configuration. Additionally or alternatively, the sound generation engine 140 can be configured to apply a transfer function and / or predetermined scaling to the soundscape package to optimize the soundscape experience for the vehicle. For example, each type and configuration of vehicle can be acoustically characterized such that each audio effect is associated with a scaling or transfer function that optimizes a specific audio effect for a given vehicle. Furthermore, the performance of each type and configuration of vehicle can be characterized such that each input variable from the vehicle's sensors is associated with a scaling or transfer function that optimizes the input variable for a given vehicle. As used herein, "scaling" can include linear scaling or nonlinear scaling. In at least one embodiment, "scaling" can include an equation or a step function.

[0079] For example, scaling or transfer functions can have a greater impact on acceleration-related input variables from a sports car than on those from an SUV. Similarly, scaling or transfer functions can have a smaller impact on bass performance in a sports car's better audio system compared to the impact on bass performance in a poorer audio system in an SUV. Through these optimizations, the acceleration of the two different types of vehicles can be better mapped into the soundscape, and the bass performance of the two vehicles can be more easily distinguished between different types of stereo systems.

[0080] In at least one embodiment, music creators can also associate specific tracks in a song with specific geographical locations. For example, a musician might want to hold a secret concert. Directions to the concert can be hidden within a soundscape created by the musician. For instance, the musician can associate one or more sets of tracks with the location of the concert, causing one or more tracks to play at a higher volume and / or faster speed as the driver approaches the concert. Thus, fans of the musician can find the secret concert by following the soundscape created within the fan's vehicle as they get closer to the concert location.

[0081] In additional or alternative embodiments, music creators can use geolocation-related information for a variety of different purposes. For example, a music creator can create a "treasure hunt" for listeners. Metadata associated with an audio track can guide users to a specific physical location via volume, beat, or some other audio metric. Once a user reaches the specific location, they can be offered a specific physical item, such as a coupon, meal, product, or any other physical item. Similarly, music creators can create metadata associated with a specific physical location to unlock a new album or track for a user to listen to at that location. Similar geolocation features can also be used for navigation. For example, metadata can guide users along a navigation path to multiple different locations. At specific locations, the soundscape can change to incorporate audio descriptions of those locations or otherwise accompany the navigation. Furthermore, in at least one embodiment, the metadata associated with an audio track can be time-limited, making a specific geolocation valid only during a specified time period.

[0082] In at least one embodiment, the computer system for creating the soundscape package may also provide the ability to add digital rights management (DRM) features to the resulting soundscape package. For example, each soundscape package may be signed and / or encrypted using a unique token. The token used to decrypt the encryption may be provided only to approved users, allowing only approved users to decrypt the soundscape package within their vehicles. Furthermore, DRM features can prevent the soundscape package from being played by unauthorized systems. For example, only systems that have been specifically authorized to play the soundscape package may be provided with the tools required to satisfy the DRM features.

[0083] Furthermore, in at least one embodiment, when a driver creates their own work using a soundscape package, the same digital rights management features can also be applied to any recordings of that driver's work. Such features can restrict the driver's ability to share licensed soundscape package content with unauthorized individuals or systems. Therefore, soundscapes and related licensing rights can be managed within the system to prevent unauthorized sharing and / or unauthorized playback of artist content. VI. Track and section identification in digital audio files

[0084] The disclosed embodiments include a computer system and a computer-implemented method for dynamically segmenting sound input from a vehicle. The system may include one or more processors and one or more computer-readable media storing executable instructions that, when executed by the one or more processors, configure the computer system to dynamically segment a digital audio file of sound input from a vehicle within the vehicle. For example, as... Figure 1As shown, computer system 100 can receive a digital audio file 102 of a specific song. The digital audio file 102 can be stored in music library storage 160. As used herein, the digital audio file 102 can include various file types, including but not limited to WAV, AIFF, AUI, MP3, AAC, FLAC, OGG, and WMA.

[0085] Upon receiving the digital audio file 102, the dynamic sound generation software 130 processes the digital audio file 102 into multiple audio tracks and segments. For example, Figure 6 An example audio track 610(ae) is shown, divided into segments 620(ac). In at least one embodiment, in order to generate the audio track 610(ae) and segments 620(ac), the sound generation engine 140 can generate one or more waveforms 600 from the digital audio file 102 and analyze the one or more waveforms to identify segments 620(ac) based on changes in the components of the one or more waveforms 600.

[0086] Those skilled in the art will understand that, for clarity and explanation, Figure 6 The sine wave depicted is greatly simplified and shortened. Each track 610 (ae) represents a separate component or element of the song. Tracks are individual audio files that break down a complete track into individual elements such as vocals, instruments, bass, and drums, allowing for greater control and customization in music production. These tracks can be mixed and manipulated independently, making them suitable for remixing, mastering, and creating dynamic soundscapes. Each section 620 (ac) comprises consecutive sections of the song.

[0087] In at least one embodiment, the process of identifying segment 620(ac) in digital audio file 102 begins in sound generation engine 140 by creating audio track 610(ae) from digital audio file 102 (e.g., stereo audio file). This involves splitting the audio into its individual components, such as different instrument or vocal tracks. In at least one embodiment, sound generation engine 140 may receive a pre-existing audio track 610(ae) or generate audio track 610(ae) from a stereo audio file using a track separation algorithm. Exemplary track separation algorithms include open-source algorithms such as Spleeter and Demucs.

[0088] Before or after acquiring track 610(ae), the sound generation engine 140 analyzes the waveform to identify where new instruments or components are introduced, as well as where other content is attenuated. The sound generation engine 140 is able to identify significant shifts in the digital audio file. The sound generation engine 140 examines tempo information and maps it to a grid, while simultaneously examining frequency information to determine significant changes in the digital audio file. This dual approach ensures that the system can accurately identify transitions and changes in the structure of the digital audio file.

[0089] The identified 620(ac) segments are typically 4, 8, or 16 bars long, depending on the song's characteristics. These 620(ac) segments are then used to create chunks that can be downloaded and played seamlessly. This chunking process allows for a more efficient and responsive playback experience because it avoids the need to download the entire track at once. Instead, the system can dynamically load and play segments based on user interaction and song structure, creating a smooth and adaptive listening experience.

[0090] For example, in Figure 6 Waveform 600 shows the waveform of each audio track 610(ae). The sound generation engine 140 can analyze audio tracks 610(ae) and identify changes in the waveform. For example, in this example, the sound generation engine 140 identifies three segments 620a, 620b, and 620c. These segments are visually distinguishable by the introduction of content in audio track 610b, changes in content in audio track 610d, and changes in content in audio track 610e. Segment 620(ac) can represent different parts of the song, such as the chorus, intro, bridge, catchy section, interlude, etc. Alternatively, segment 610(ac) can represent only the segments where the song's characteristics change.

[0091] In at least one embodiment, a machine learning algorithm trained to identify segments 620(ac) within a digital audio file can be used to identify segment 620(ac). For example, the machine learning algorithm can be trained on a labeled dataset of digital audio files manually segmented by professional users. In various embodiments, a variety of different machine learning algorithms can be used. For example, machine learning algorithms for processing audio include artificial intelligence algorithms, neural networks, principal component analysis (PCA), particle swarm optimization (PSO), convolutional neural networks (CNN), Mel-frequency cepstral coefficients (MFCC), gamma-tonal frequency cepstral coefficients (GFCC), linear predictive cepstral coefficients (LFCC), and deep neural networks (DNN).

[0092] Once segment 620(ac) is identified, the sound generation engine 140 numbers segments 620(ac) according to priority. When numbering segments 620(ac), the sound generation engine 140 typically loads the intro segment 620a first and designates it as segment 1. Subsequently, segments 620b, 620c are numbered according to a sequence reflecting their order in the song. The sound generation engine 140 can then use energy scores to help determine the priority of each segment, where higher-energy segments are typically prioritized during periods of high activity, such as acceleration.

[0093] In at least one embodiment, the sound generation engine 140 calculates an energy score by analyzing the waveform of the digital audio file 102. The sound generation engine 140 examines the amount of frequency information transmitted and the chokes associated with the music tempo. Peaks in the waveform are identified to determine high-tempo and high-energy sections. In at least one embodiment, the sound generation engine 140 can generate energy scores for the entire digital audio file, each section 620 (ac), and each track 610 (ae). In at least one embodiment, the desired energy score can be generated using the integral of the Fourier transform of the frequency information. The sound generation engine 140 can normalize the energy score by assigning an energy score within 100 to each section 620 (ac) or track 610 (ae). This score helps prioritize sections for playback during periods of high activity.

[0094] The sound generation engine 140 can associate a track 610(ae) in segment 620(ac) with input variables received via sensor API 150 from one or more vehicle sensors. For example, an accelerometer sensor can be associated with a bass track (e.g., track 610b). Furthermore, left and right gyroscope sensors can be associated with a synthesizer track (e.g., 610c), which is subsequently introduced when the vehicle turns left or right. This association is part of the system logic used to create a dynamic and responsive audio experience.

[0095] The sound generation engine 140 can then generate digital data packets (e.g., 230, 240) that include one or more audio tracks 610(ae), one or more segments 620(ac), and associations with input variables from one or more vehicle sensors. As will be explained more fully below, the digital data packets may include a single file containing the entire contents of a song, or the digital data packets may include “chunks” of the song that are sent in small segments to meet memory and bandwidth limitations. Therefore, the digital data packets may not include the entire segment 620(ac) and audio track 610(ae) from the digital audio file. For example, sending the entire song to vehicle 170 may result in undue latency or slower-than-acceptable operation of the processing unit in the vehicle. Therefore, in some embodiments, the digital data packets include chunks of the song that are small enough to be cached within the processing unit of vehicle 170. Therefore, in some embodiments, it may be desirable to transmit one or more audio tracks 610(ae) and one or more segments 620(ac) as dynamic chunks.

[0096] Furthermore, digital data packets can include one or more segments with specific priorities. For example, the specific priorities included in a digital data packet can be based on input variables received from vehicle sensors. If the vehicle is accelerating or engaging in more aggressive driving, the digital data packet can include music segments with higher energy scores. Conversely, if the vehicle is decelerating, the digital data packet can include music segments with lower energy scores. VII. Dynamically segment the sound input from the car.

[0097] In at least one embodiment, processing digital audio files in a chunked manner enhances the user experience by making song playback more dynamic and responsive to real-time input from various sensors in the vehicle. By breaking songs down into smaller, manageable segments or “chunks,” the system can quickly and seamlessly adapt the music to changes in driving conditions, such as acceleration, braking, and cornering. This method avoids the need to download the entire song at once, reduces waiting time, and ensures a smooth and immediate playback experience.

[0098] At least one advantage of this segmentation method is that it allows for a more interactive and immersive audio experience. For example, a specific track 610 (ae) or section 620 (ac) in a song can be triggered by a specific driving action, such as introducing a brass section when the car turns left or right. This not only makes the music feel more integrated into the driving experience but also provides unique and personalized tracks that evolve with the journey. Furthermore, the system's ability to identify and prioritize high-energy sections of a song ensures that the intensity of the music matches the driving conditions, thereby enhancing the user's overall enjoyment and engagement.

[0099] Figure 7 A schematic diagram of an example sound generation engine 140 for managing chunks of a digital audio file 102 is shown. To manage these music chunks, the sound generation engine 140 may include various components, including an audio transmission function 710, a source processor 712, and audio buffer systems 720a and 720b. Figure 7 Two different buffers 720a and 720b are shown, but those skilled in the art will understand that a single buffer or multiple buffers may be used as needed to meet hardware and software considerations. These components work together to enable real-time audio rendering and synchronization with vehicle driving actions.

[0100] The sound generation engine 140 can be used as a control and synchronization audio graph 800 (such as...). Figure 8 The highest-level classes are audio transmission function 710 and audio buffer systems 720a and 720b (shown in the diagram). The sound generation engine 140 ensures all audio components work seamlessly together to provide a consistent audio experience. In at least one embodiment, audio transmission function 710 maintains the current playback position, loop start and end positions, loop state, and playback state (playing, stopped, stopped, paused). For example, audio transmission function 710 can manage data structures to ensure accurate synchronization between audio playback and vehicle driving actions. Audio transmission function 710 is responsible for advancing the playback position and loop state, ensuring that audio data is preloaded before rendering.

[0101] Source processor 712 can be embedded within audio graph 800 and is responsible for rendering the loaded audio data. Source processor 712 retrieves audio data from audio buffer system 720b and writes the correct data to its output, where it is subsequently processed by other processors in audio graph 800. Source processor 712 is configured to read audio blocks from audio transfer buffer 720a, which is loaded from audio block server 740. Source processor 712 then renders any audio blocks that overlap with the currently processed block.

[0102] Audio buffer systems 720a and 720b are configured to load short segments of audio data from storage devices (e.g., music library storage 160) only in a timely manner for playback. The audio buffer systems 720a and 720b may utilize at least three components: an audio track sequencing function 730, an audio transfer function 710, and an audio block server 740. The audio track sequencing function 730 maintains a record of which audio files are being used, where each audio file is placed on the timeline, and which mixing channel in the audio graph 800 each audio file should be played through. The audio transfer function 710 tracks the current playback position and loop state, ensuring that audio data is pre-loaded before rendering. The audio block server 740 runs on a background timer thread, loads the required audio data from storage devices, and provides a view of that audio data to the source processor 712 in the audio block view table buffer 720b. The audio block view table buffer 720b may include double buffering with lock-free read behavior.

[0103] In at least one embodiment, the audio track sequence function 730 acts as a loop around std::vector. <sequencedaudiostem>A very thin mutex lock protects the wrapper. Those skilled in the art will understand that `std::vector` is a dynamic array provided by the Standard Template Library (STL) in C++ programming. It allows elements to be stored in contiguous memory locations and provides the ability to dynamically resize the array as needed. "SequencedAudioStem" is a custom data type representing an audio track, which is a reference to an audio file placed at a specific position in the timeline. The audio track sequence function 730 acts as a wrapper around `std::vector`. <sequencedaudiostem>The thin mutex lock protection wrapper means that the audio track sequence function 730 manages the collection of SequencedAudioStem objects, thus ensuring thread safety when updating track group 610(ae). The audio block server 740 can update the audio block view table buffer 720b when needed, and the rest of the sound generation engine 140 will respond accordingly.

[0104] In at least one embodiment, the audio transport function 710 includes at least a single audio transport data structure instance (i.e., an "AudioTransport struct" or "audio transport structure") that tracks the current playback position and loop state. The audio transport function 710 can load the audio transport data structure into an audio transport buffer 720a for viewing and / or copying by the audio block server 740. Furthermore, the audio block server 740 can run on a background timer thread that iterates every 50ms. (Based on std::vector...) <sequencedaudiostem>With the latest data from the audio transmission function 710, the audio block server 740 can load the required audio data from the storage device into short audio "blocks" and present a view of the audio data to the source processor 712 through the audio block view table buffer 720b.

[0105] Figure 9A and Figure 9B A schematic diagram illustrates different update processes for chunking digital audio files. The update iterations within the audio buffer systems 720a and 720b are specifically managed by the audio block server 740. This process ensures that the necessary audio data is loaded just in time for playback, thus maintaining a seamless and uninterrupted audio experience. An update iteration may include at least three steps. The first step in an update iteration may be to remove any "required" regions that are no longer needed from the audio block pool 742. The audio block pool 742 provides audio blocks to the audio block view table buffer 720b. Removing these blocks ensures that the sound generation engine 140 does not retain audio data that is no longer relevant, thereby optimizing memory usage.

[0106] The next step is to load the audio data that is in the "required" region but is not yet in the audio block pool 742. This involves reading the necessary audio blocks from the storage device and preparing the audio blocks for playback. As used herein, unless otherwise stated, "block" and "chunk" may be used interchangeably. Figure 9A and Figure 9B An example of identifying the desired region is shown. For example, in Figure 9A In the diagram, the playback head 900a is shown at approximately the midpoint between the loop start point 910a and the loop end point 912a. Therefore, the desired audio block 920a includes the upcoming audio block for each track within the current playback segment. In at least one embodiment, the audio portion between the loop start point 910a and the loop end point 912a comprises a single segment of music (e.g., 620c).

[0107] and Figure 9A compared to, Figure 9B The playback head 900b is shown to be almost at the loop end point 912b. In this case, the required blocks 920b and 922b include blocks from at least two different positions within the music. One set of required blocks 920b includes blocks that transition from the current music segment 620b to the next music segment 620c. Furthermore, required blocks 922b also include blocks from the beginning of the current music segment 620b. By accessing both sets of required blocks 920b and 922b, the sound generation engine 140 is able to consider input variables from a vehicle sensor that indicate the loop should continue (in this case, required block 922b would be used), or input variables that indicate the next loop should begin (in this case, required block 920a would be used).

[0108] The third step in updating the audio block view table buffer 720b on the audio block server 740 is for each mix channel (e.g., Figure 8 (As shown) A set of audio block views is constructed and written to the audio block view table buffer 720b. These views are then ready to be retrieved by the source processor 712, thus ensuring that the correct audio data is available for real-time processing.

[0109] The update iteration is configured to handle dynamic playback and loop states, which are determined by the source processor 712. The audio block server 740 can respond to changes in these states, including activation or deactivation at the last moment of a loop. By knowing the playback position and the start and end points of the loop, if it is approaching the end of the loop, the sound generation engine 140 can load audio data before the current playback position and at the beginning of the loop.

[0110] Figure 10 A schematic diagram of an example of an audio block view table buffer 720b is shown. Each row in array 1000 is associated with a music block, and each column is associated with a specific mix channel (e.g., track 610(ae)). The source processor 712 can independently address any given row or column within the audio block view table buffer 720b. Therefore, each music block or chunk is provided to the source processor based on each mix channel for transmission to the audio graph 800.

[0111] In at least one embodiment, the output of the audio block view table buffer 720b includes digital data packets as described above. The digital data packets include at least a portion of one or more audio tracks and at least a portion of one or more segments. In at least one embodiment, the digital data packets may also include associations between specific input variables from vehicle sensors and specific audio tracks and / or segments within the digital data packets. The audio block server 740 can create an array of potential audio blocks for playback at vehicle 170. Array 1000 includes a first dimension (e.g., columns) defined by a mixing channel; and a second dimension defined by different audio tracks (e.g., rows) created from one or more segments and one or more audio tracks.

[0112] In at least one embodiment, when in use, the sound generation engine 140 simultaneously plays a first audio track in a first segment of a particular song and a second audio track in a second segment of the same song. This can occur when the second audio track is associated with a particular vehicle sensor that is not associated with any audio track in the first segment of the song. For example, the sound generation engine 140 may associate a drum track (e.g., 610b) with an accelerometer sensor. However, one or more segments (e.g., 620a and 620c) may not have any information from track 610b. Therefore, in these segments 620a and 620c, if the audio graph 800 plays the actual audio track 610b in these segments 620a and 620c, no music will be played within the vehicle 170. Thus, in some cases, the sound generation engine 140 will cause an audio block obtained from track 610b in segment 620b to be played simultaneously with an audio block from segment 620a or 620c.

[0113] Figure 11 This is a flowchart of an example method for dynamically segmenting sound input from a vehicle at a vehicle location. In step 1110, the computer system receives a digital audio file of a specific song. In step 1120, the computer system processes the digital audio file into multiple tracks and segments, wherein: each track represents a separate component or element of the song, and each segment comprises a continuous portion of the song. In step 1130, the computer system numbers the segments according to priority. In step 1140, the computer system associates the tracks from the segments with input variables from one or more vehicle sensors. In step 1150, the computer system generates a digital data packet comprising one or more tracks, one or more segments, and the associations with the input variables from one or more vehicle sensors. VIII. Aspects of AI-Generated Voice from Automotive Input

[0114] Clause 1. A computer system for dynamically segmenting sound input from a vehicle at a vehicle location, comprising: one or more processors; and one or more computer-readable media storing executable instructions that, when executed by the one or more processors, configure the computer system to: receive a digital audio file of a specific song; process the digital audio file into a plurality of tracks and segments, wherein: each track represents a separate component or element of the song, and each segment comprises a continuous portion of the song; number the segments in order of priority; associate the tracks from the segments with input variables from one or more vehicle sensors; and generate a digital data packet comprising the one or more tracks, the one or more segments, and the association with the input variables from the one or more vehicle sensors.

[0115] Clause 2. A computer system as described in Clause 1, wherein processing the digital audio file into multiple tracks and segments includes instructions executable to configure the computer system to perform the following operations: generating a waveform from the digital audio file; and analyzing the waveform to identify segments based on changes in the components of the waveform.

[0116] Clause 3. A computer system as described in Clause 1, wherein the executable instructions include instructions executable to configure the computer system to perform the following operation: assigning energy fractions to each segment and each track based on frequency information within each segment and each track.

[0117] Clause 4. A computer system as described in Clause 3, wherein the numbering of the segments in order of priority is based on the energy fraction.

[0118] Clause 5. A computer system as described in Clause 1, wherein the executable instructions include instructions executable to configure the computer system to perform the following operation: playing a first audio track from a first section of the particular song while playing a second audio track from a second section of the particular song.

[0119] Clause 6. A computer system as described in Clause 5, wherein playing the first audio track from the first segment of the particular song while playing the second audio track from the second segment of the particular song is based on the second audio track being associated with a particular vehicle sensor, and the particular vehicle sensor not being associated with any audio track within the first segment of the particular song.

[0120] Clause 7. A computer system as described in Clause 1, wherein the digital data packet does not include all of the segments and tracks from the digital audio file.

[0121] Clause 8. A computer system as described in Clause 7, wherein the digital data packets are cached at the vehicle.

[0122] Clause 9. A computer system as described in Clause 7, wherein the digital data packet comprises one or more segments having a specific priority.

[0123] Clause 10. A computer system as described in Clause 1, wherein generating the digital data packet comprising one or more audio tracks, one or more segments, and associated with input variables from one or more vehicle sensors includes instructions executable to configure the computer system to perform the following operations: creating an array of potential audio chunks for playback at the vehicle, wherein the array includes a first dimension and a second dimension, the first dimension being defined by a mixing channel, and the second dimension being defined by different audio tracks created from the one or more segments and the one or more audio tracks; and selecting a specific audio track from the different audio tracks based on specific input variables from the one or more vehicle sensors.

[0124] Clause 11. A computer-implemented method for dynamically segmenting sound input from a vehicle at a vehicle, executed on one or more processors, the computer-implemented method comprising: receiving a digital audio file of a specific song; processing the digital audio file into a plurality of tracks and segments, wherein: each track represents a separate component or element of the song, and each segment comprises a continuous portion of the song; numbering the segments in order of priority; associating the tracks from the segments with input variables from one or more vehicle sensors; and generating a digital data packet comprising the one or more tracks, the one or more segments, and the association with the input variables from the one or more vehicle sensors.

[0125] Clause 12. The computer implementation method as described in Clause 11, wherein processing the digital audio file into the plurality of audio tracks and segments further comprises: generating a waveform from the digital audio file; and analyzing the waveform to identify segments based on changes in the components of the waveform.

[0126] Clause 13, the computer implementation method as described in Clause 11, further includes: assigning energy fractions to each segment and each track based on frequency information within each segment and each track.

[0127] Clause 14. The computer implementation method as described in Clause 13, wherein: the numbering of the segments according to the priority order is based on the energy fraction.

[0128] Clause 15, the computer implementation method as described in Clause 11, further includes: playing a first audio track from a first segment of the particular song while playing a second audio track from a second segment of the particular song.

[0129] Clause 16. A computer implementation method as described in Clause 15, wherein playing the first audio track from the first segment of the particular song while playing the second audio track from the second segment of the particular song is based on the second audio track being associated with a particular vehicle sensor, and the particular vehicle sensor not being associated with any audio track within the first segment of the particular song.

[0130] Clause 17. The computer implementation method as described in Clause 11, wherein the digital data packet does not include all of the said segments and tracks from the digital audio file.

[0131] Clause 18. A computer implementation method as described in Clause 11, wherein the digital data packets are cached in the vehicle.

[0132] Clause 19. A computer-implemented method as described in Clause 11, wherein the digital data packet comprises one or more segments having a specific priority.

[0133] Clause 20, the computer implementation method as described in Clause 11, wherein generating the digital data packet comprising one or more audio tracks, one or more segments, and associated with input variables from one or more vehicle sensors further comprises: creating an array of potential audio chunks for playback at the vehicle, wherein the array comprises a first dimension and a second dimension, the first dimension being defined by a mixing channel, and the second dimension being defined by different audio tracks created from one or more segments and one or more audio tracks; and selecting a specific audio track from the different audio tracks based on specific input variables from one or more vehicle sensors. IX. Example Architecture and Computer Hardware

[0134] In at least one embodiment, the computer hardware for the system described above can be integrated into an original equipment manufacturer (OEM) multimedia center supplied with the vehicle. Alternatively or additionally, the described system can be added to the vehicle after purchase via a brand-new aftermarket multimedia center and / or via a plug-in device. For example, in at least one embodiment, a user can plug a standalone device into their vehicle to obtain the aforementioned functionality. For example, the standalone device can be plugged into a USB port within the vehicle. Alternatively or additionally, the standalone device can be plugged into an on-board diagnostic system (e.g., OBDII) to collect data about the vehicle's sensors, thereby feeding said data to Sensor API 150 within the standalone device.

[0135] Furthermore, in at least one embodiment, the in-vehicle device may include an internal inertial measurement unit (IMU) capable of inferring at least a portion of the vehicle's sensor readings. For example, the IMU may detect the vehicle's steering and acceleration. Similarly, the IMU may detect vibrations via the suspension. The IMU may be fed into Sensor API 150 as if its readings were received from the vehicle's sensors. The in-vehicle device can then provide audio-visual content to the multimedia system within the vehicle via a USB port or some other means, such as Bluetooth. Therefore, the disclosed embodiments include both built-in systems and stand-alone devices capable of retrofitting a vehicle to include the described functionality.

[0136] Furthermore, the method can be executed by a computer system including one or more processors and computer-readable media such as computer memory. Specifically, the computer memory may store computer-executable instructions that, when executed by one or more processors, cause various functions to be performed, such as the actions described in the embodiments.

[0137] The functionality of a computing system can be enhanced by its ability to interconnect with other computing systems via network connections. Network connections can include, but are not limited to, wired or wireless Ethernet, cellular connections, or even computer-to-computer connections via serial, parallel, USB, or other connections. These connections allow the computing system to access services on other computing systems and to receive application data from other computing systems quickly and efficiently.

[0138] The interconnection of computing systems has facilitated distributed computing systems, such as so-called "cloud" computing systems. In this specification, "cloud computing" can refer to a system or resource used to enable ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, services, etc.), which can be provided and published with reduced administrative effort or service provider interaction. Cloud models can consist of various features (e.g., on-demand self-service, widespread network access, resource pooling, rapid elasticity, measurement services, etc.), service models (e.g., Software as a Service ("SaaS"), Platform as a Service ("PaaS"), Infrastructure as a Service ("IaaS"), and deployment models (e.g., private cloud, community cloud, public cloud, hybrid cloud, etc.).

[0139] Cloud-based and remote-based service applications are ubiquitous. These applications are hosted on public and private remote systems such as the cloud and typically provide a set of web-based services for bidirectional communication with clients.

[0140] Many computers are designed to be used through direct interaction between the user and the computer. Therefore, computers have input hardware and software user interfaces to facilitate user interaction. For example, modern general-purpose computers may include a keyboard, mouse, touchpad, camera, etc., to allow users to input data into the computer. In addition, various software user interfaces may be provided.

[0141] Examples of software user interfaces include graphical user interfaces, text-based command-line user interfaces, function key or hotkey user interfaces, etc.

[0142] The disclosed embodiments may include or utilize a special-purpose computer or a general-purpose computer, including computer hardware discussed in more detail below. The disclosed embodiments also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available medium accessible by a general-purpose computer system or a special-purpose computer system. A computer-readable medium storing computer-executable instructions is a physical storage medium. A computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of the invention may include at least two distinctly different types of computer-readable media: physical computer-readable storage media and transmission computer-readable media. In all cases, both computer-readable storage media and transmission computer-readable media are directed to non-transitory media.

[0143] Physical computer-readable storage media include RAM, ROM, EEPROM, CD-ROM or other optical disc storage (e.g., CD, DVD, etc.), disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose computer or a special-purpose computer.

[0144] A "network" is defined as one or more data links that enable the transmission of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer via a network or another communication connection (whether hardwired, wireless, or a combination of hardwired and wireless), the computer appropriately considers that connection as a transmission medium. The transmission medium may include a network and / or a data link that can be used to carry program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose computer or a special-purpose computer. The above combinations are also included within the scope of computer-readable media.

[0145] Furthermore, upon arrival at various computer system components, program code in the form of computer-executable instructions or data structures can be automatically transferred from a transmission computer-readable medium to a physical computer-readable storage medium (and vice versa). For example, computer-executable instructions or data structures received via a network or data link can be buffered in RAM within a network interface module (e.g., "NIC") and then ultimately transferred to the computer system RAM and / or a less volatile computer-readable physical storage medium at the computer system. Thus, computer-readable physical storage media can be included in computer system components that also (or even primarily) utilize the transmission medium.

[0146] Computer-executable instructions include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a particular function or group of functions. Computer-executable instructions can be, for example, binary files, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological actions, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary forms for implementing the claims.

[0147] Those skilled in the art will understand that this invention can be practiced in networked computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframes, mobile phones, PDAs, pagers, routers, switches, etc. This invention can also be practiced in distributed system environments, where both local and remote computer systems, linked via a network (via a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links), perform tasks. In a distributed system environment, program modules can reside in both local and remote memory storage devices.

[0148] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0149] The invention may be practiced in other specific forms without departing from the spirit or characteristics thereof. The described embodiments should be considered in all respects as illustrative rather than restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than the foregoing description. All variations within the meaning and scope of equivalents of the claims will be included within their scope.< / sequencedaudiostem> < / sequencedaudiostem> < / sequencedaudiostem>

Claims

1. A computer system for dynamically segmenting sound input from a vehicle at a vehicle location, comprising: One or more processors; as well as One or more computer-readable media, wherein executable instructions are stored on the one or more computer-readable media, the executable instructions, when executed by the one or more processors, configure the computer system to: Receive digital audio files for a specific song; The digital audio file is processed into multiple tracks and multiple segments, wherein: each track represents a separate component or element of the song, and each segment includes a continuous portion of the song; The segments are numbered in order of priority; Associate one or more audio tracks from one or more segments with input variables from one or more vehicle sensors; and A digital data packet is generated, the digital data packet comprising at least a portion of one or more audio tracks and at least a portion of one or more segments.

2. The computer system as described in claim 1, wherein, Processing the digital audio file into multiple tracks and multiple segments includes executable instructions to configure the computer system to perform the following operations: Generate a waveform from the digital audio file; and The waveform is analyzed to identify segments based on changes in the components of the waveform.

3. The computer system as described in claim 1, wherein, The executable instructions include instructions that are executable to configure the computer system to perform the following operations: Energy scores are assigned to each segment and each track based on frequency information within each segment and each track.

4. The computer system as described in claim 3, wherein: The numbering of the segments according to the priority order is based on the energy fraction.

5. The computer system as claimed in claim 1, wherein, The executable instructions include instructions that are executable to configure the computer system to perform the following operations: While playing the second audio track from the second section of the specific song, the first audio track from the first section of the specific song is played.

6. The computer system as described in claim 5, wherein, Playing the first audio track from the first section of the specific song while playing the second audio track from the second section of the specific song is based on the fact that the second audio track is associated with a specific vehicle sensor, while the specific vehicle sensor is not associated with any audio track within the first section of the specific song.

7. The computer system as claimed in claim 1, wherein, The digital data packet does not include all the sections and tracks from the digital audio file.

8. The computer system as claimed in claim 7, wherein, The digital data packets are cached on the vehicle.

9. The computer system as claimed in claim 7, wherein, The digital data packet includes one or more segments with a specific priority.

10. The computer system as claimed in claim 1, wherein, Generating the digital data packet, which includes at least a portion of the one or more audio tracks and at least a portion of the one or more segments, includes instructions executable to configure the computer system to perform the following operations: Create an array of potential audio chunks for playback at the vehicle, wherein the array includes a first dimension and a second dimension, the first dimension being defined by a mixing channel and the second dimension being defined by different audio tracks created from the one or more segments and the one or more audio tracks.

11. A computer-implemented method for dynamically segmenting sound input from a vehicle at a vehicle, the computer-implemented method being executed on one or more processors, the computer-implemented method comprising: Receive digital audio files for a specific song; The digital audio file is processed into multiple tracks and multiple segments, wherein: each track represents a separate component or element of the song, and each segment includes a continuous portion of the song; The segments are numbered in order of priority; Associate one or more audio tracks from one or more segments with input variables from one or more vehicle sensors; and A digital data packet is generated, the digital data packet comprising at least a portion of one or more audio tracks and at least a portion of one or more segments.

12. The computer implementation method as described in claim 11, wherein, Processing the digital audio file into the multiple audio tracks and the multiple segments also includes: Generate a waveform from the digital audio file; and The waveform is analyzed to identify segments based on changes in the components of the waveform.

13. The computer implementation method as described in claim 11, further comprising: Energy scores are assigned to each segment and each track based on frequency information within each segment and each track.

14. The computer implementation method as described in claim 13, wherein: The numbering of the segments according to the priority order is based on the energy fraction.

15. The computer implementation method as described in claim 11, further comprising: While playing the second audio track from the second section of the specific song, the first audio track from the first section of the specific song is played.

16. The computer implementation method as described in claim 15, wherein, Playing a first track from the first section of the specific song while playing a second track from the second section of the specific song is based on the fact that the second track is associated with a specific vehicle sensor, while the specific vehicle sensor is not associated with any track within the first section of the specific song.

17. The computer implementation method as described in claim 11, wherein, The digital data packet does not include all the sections and tracks from the digital audio file.

18. The computer implementation method as described in claim 11, wherein, The digital data packets are cached on the vehicle.

19. The computer implementation method as described in claim 11, wherein, The digital data packet includes one or more segments with a specific priority.

20. The computer implementation method as described in claim 11, wherein, Generating the digital data packet, which includes at least a portion of the one or more audio tracks and at least a portion of the one or more segments, further includes: Create an array of potential audio chunks for playback on a vehicle, wherein the array includes a first dimension and a second dimension, the first dimension being defined by a mixing channel, and the second dimension being defined by different audio tracks created from the one or more segments and the one or more audio tracks.

Images