Method, device and equipment for generating music, vehicle and product

Abstract

The invention relates to a method, device and equipment for generating music, a vehicle and a product. The method includes generating lyric text content corresponding to human voice content based on the human voice content in a cabin of a vehicle. The method further comprises the step of generating music based on the audio features and the lyric text content corresponding to at least one of the vehicle body audio data and the outside audio data of the vehicle. Through the mode, the conversation content of the user and the background sound of the environment where the vehicle is located can be combined, the music which the user may be interested in can be generated for the user, in addition, the repeatability of music generation can be avoided, and therefore the personalized requirement of the user is met.

Description

Technical Field

[0001] This disclosure relates to the field of audio technology, and more specifically, to methods, apparatus, devices, vehicles, and products for generating music. Background Technology

[0002] With the rapid development of vehicle and network technologies, cars have become an indispensable part of people's daily lives. At the same time, people's demands for vehicles are constantly upgrading; they are no longer satisfied with the conventional driving functions of vehicles, but are pursuing a more intelligent and user-friendly driving experience.

[0003] Music has always been a primary form of entertainment and use in vehicles. Drivers and passengers alike desire a superior sensory experience, and pleasant music can significantly enhance this. As a crucial component of this enhanced experience, music also needs continuous improvement. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, device, vehicle, and product for generating music.

[0005] In a first aspect of this disclosure, a method for generating music is provided. The method includes generating lyrics text content corresponding to human voice content within the vehicle's cabin. The method also includes generating music based on audio features of at least one of the following audio data: vehicle body audio data and exterior audio data, along with the lyrics text content.

[0006] In a second aspect of this disclosure, an apparatus for generating music is provided. The apparatus includes a lyrics text content generation unit configured to generate lyrics text content corresponding to human voice content within a vehicle's cabin. The apparatus also includes a music generation unit configured to generate music based on audio features of at least one of the following audio data: vehicle body audio data and exterior audio data, and the lyrics text content.

[0007] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method provided according to a first aspect of this disclosure.

[0008] In a fourth aspect of this disclosure, a vehicle is provided. The vehicle includes a plurality of sound sensors and electronic equipment provided according to a third aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions, which are executed by a processor to implement the method provided according to a first aspect of this disclosure.

[0010] In a sixth aspect of this disclosure, a computer program product is provided, which is tangibly stored on a computer-readable medium and includes machine-executable instructions that, when executed, implement the method described in the first aspect of this disclosure.

[0011] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1 A schematic diagram of an example environment in which several embodiments of the present disclosure may be implemented is shown;

[0014] Figure 2 A schematic diagram of a process for generating music according to some embodiments of the present disclosure is shown;

[0015] Figure 3 A flowchart of a method for generating music according to some embodiments of the present disclosure is shown;

[0016] Figure 4 A schematic diagram of a user interface according to some embodiments of the present disclosure is shown;

[0017] Figure 5 A schematic diagram illustrating driver / passenger selection of tone according to some embodiments of the present disclosure is shown;

[0018] Figure 6 A block diagram of an apparatus for generating music according to some embodiments of the present disclosure is shown; and

[0019] Figure 7 A block diagram of a device that can implement several embodiments of the present disclosure is shown. Detailed Implementation

[0020] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0021] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0022] As mentioned above, whether the music is pleasant to listen to and meets the user's preferences is a crucial aspect of enhancing the driving experience. Users desire more personalized features in their vehicles, such as the ability to generate personalized music or participate in the music creation process, to express their individuality or improve their driving experience. This, in turn, can enhance their overall driving experience to a certain extent.

[0023] Personalized music solutions are limited to adjusting timbre, volume, and other aspects. This limited scope results in limited musical expressiveness and offers little benefit to enhancing the user's driving experience. Users generally have no way to participate in music generation, creating music that expresses their personality or preferences.

[0024] Therefore, embodiments of this disclosure propose a method for generating music. In embodiments of this disclosure, the method includes generating lyrics text content corresponding to human voice content within the vehicle's cabin. The method also includes determining corresponding audio features based on at least one of vehicle body audio data and external audio data. Furthermore, the method includes generating music based on the lyrics text content and the audio features.

[0025] This method combines user conversations with background noise from the vehicle's environment to generate different types of music for different users, avoiding repetitive music and thus meeting personalized needs. Furthermore, the information used to generate music is richer and more diverse, making the music more relevant to the current context and possessing greater richness and depth, thereby enhancing the user's driving experience.

[0026] Figure 1 A schematic diagram of an example environment 100 in which various embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, in example environment 100, a vehicle 102, an in-vehicle sensor 104, a body sensor 106, and an external sensor 108 are included. Example environment 100 also includes a processor 110 communicatively connected to the vehicle 102. According to embodiments of this disclosure, vehicle 102 refers to any type of motorized or non-motorized vehicle capable of carrying people and / or goods and being movable. Figure 1 As shown, vehicle 102 is illustrated as a car. It should be understood that although vehicle 102 is... Figure 1 The image depicts a car, but this is merely exemplary and far from limited to that; examples may also include buses, trucks, motorcycles, and electric vehicles. It should also be understood that embodiments of this disclosure are applicable to other means of transportation besides vehicles, such as boats, trains, and airplanes. The processor 110 can be implemented as various user terminals or service terminals with computing capabilities. Service terminals may be servers provided by various service providers, large computing devices, etc. User terminals may be any type of mobile terminal, fixed terminal, or portable terminal.

[0027] In example environment 100, the in-vehicle sensor 104, the body sensor 106, and the external sensor 108 can be sound sensors used to collect sound signals from different locations in different environments. Different types of sensors may employ different technical principles to collect sound signals, such as capacitive, piezoelectric, and electromechanical sensors. The installation locations of the three sound sensors are as follows: for example, the in-vehicle sensor 104 can be installed in key locations inside the cabin of the vehicle 102, such as the steering wheel, the Human Machine Interface (HMI) dashboard, and the doors. The body sensor 106 can be installed on the vehicle body, such as the hood and rear, primarily for capturing sounds emitted by the vehicle body. In some embodiments of this disclosure, the external sensor 108 can be located on the exterior of the vehicle 102, such as on the front and rear bumpers of the vehicle 102, or beside the Advanced Driving Assistance System (ADAS) camera, primarily for collecting ambient sounds outside the vehicle 102.

[0028] In some embodiments of this disclosure, the in-vehicle sensor 104 is primarily used to collect conversations and other internal sounds from the driver and passengers. For example, the in-vehicle sensor 104 is used to detect and collect sound signals within the vehicle 102 (such as inside the cabin) to obtain audio data within the cabin. Examples include conversations of the driver and passengers, their commands, signal prompts, and multimedia sounds within the vehicle. The body sensor 106 is used to detect and collect all or part of the sound signals from the vehicle body structure and engine of the vehicle 102 to obtain body audio data. In one example, the audio signals collected by the body sensor 106 may include, but are not limited to, sound interactions between the vehicle body and the external environment (e.g., the sound of doors opening or closing, the sound of tires rubbing against the ground, wind noise), and sounds caused or generated by the body structure (e.g., engine sounds, horn sounds).

[0029] In some embodiments of this disclosure, the external sensor 108 can be used to detect and collect sound signals from the surrounding environment of the vehicle 102 to obtain external audio data. The sound signals from the surrounding environment of the vehicle 102 may include sounds from other vehicles (such as horns), wind, rain, traffic noise, etc. Different sound sensors collect different types of audio signals, allowing for more accurate capture of the vehicle 102's state, its environment, and the user's state across multiple dimensions and in various ways. This provides more comprehensive and richer information for generating music that matches the user's context.

[0030] In some embodiments of this disclosure, after collecting various types of audio data, preprocessing can be performed on these data to provide higher-quality audio data for subsequent music generation. For example, echo cancellation can be used to eliminate or reduce echoes generated during audio signal transmission, making the audio data clearer and purer. It is understood that the preprocessed audio signals can be transmitted in real time to the vehicle-side storage database or to the processor 110 via a bus (e.g., Universal Serial Bus (USB), Controller Area Network (CAN)). The processor 110 then processes these audio signals.

[0031] In some embodiments of this disclosure, the processor 110 may include an audio signal decomposition model 114, a lyrics generation model 116, an audio feature extraction model 118, and a music generation model 120. The audio signal decomposition model 114 can decompose in-cabin audio data, vehicle body audio data, and external audio data to obtain various types of audio signals. For example, decomposing in-cabin audio data yields human voice content and non-human voice content (e.g., multimedia sounds, information prompts, etc.). The human voice content refers to the voices of the driver and passengers inside the vehicle, which can reflect their current psychological state to some extent. Based on this, the processor 110 can generate lyrics text content based on the human voice content. Figure 1 (Not shown in the image). The style or theme of the generated lyrics text is consistent with the psychological state of the driver and passengers. The audio signal decomposition model 118 can also decompose the vehicle audio data and / or external audio data to obtain non-human voice content (such as ambient sounds and engine sounds).

[0032] like Figure 1 As shown, the audio feature extraction model 118 can be a feature extraction model for non-human voice audio data. The audio feature extraction model 118 can extract features from at least one or more of the non-human voice content in the external audio data, the vehicle body audio data, and the cabin audio data to determine the non-human voice audio features. The audio features and lyrics text can be input into the music generation model 120, which generates music 112 that meets the requirements. The music 112 can be categorized into various types. According to style, music 112 can include pop music, rock music, electronic music, classical music, etc. According to whether there is vocal singing, music 112 can include instrumental music, accompaniment, a cappella, etc.

[0033] In this way, based on various types of audio data collected from the vehicle, dynamically changing and stylistically diverse music can be generated for the user. The generated music is no longer confined to traditional, fixed genres, but rather is generated based on the conversations between the driver and passengers, expressing their personality or preferences, thus enhancing the user's driving experience.

[0034] Figure 2 A schematic diagram of a process for generating music according to some embodiments of the present disclosure is shown. For example... Figure 2As shown, multiple audio sensors located at various locations within the vehicle can detect and collect different audio data. For example, the in-vehicle audio sensor 202 can collect audio data inside the vehicle or cabin, obtaining corresponding cabin audio data 208. The vehicle body audio sensor 204 can collect audio data from the vehicle body, obtaining corresponding vehicle body audio data 210. The external audio sensor 206 can collect audio data from the vehicle's surrounding environment, obtaining corresponding external audio data 212. It is understandable that users can choose whether to collect different types of audio data according to their needs. For example, the vehicle can provide a one-click audio recording function; after the user triggers this function, multiple audio sensors can collect audio data from different locations within the vehicle.

[0035] It is understandable that in order to effectively process and decompose the waveform data of the original audio data, facilitating subsequent human voice content analysis and background audio feature extraction, the audio data can be decomposed. In some embodiments of this disclosure, the same audio signal decomposition model can be used to decompose the cabin audio data 208, the vehicle body audio data 210, or the exterior audio data 212. To improve the accuracy of audio decomposition, different audio signal decomposition models can also be used to decompose the three types of audio data. For example, audio signal decomposition model 224 can be used to decompose the cabin audio data 208, resulting in decomposed audio signals 230 containing different categories (e.g., including but not limited to human voice content, media resources, and signal prompts). Among them, media resources can be multimedia resources played by the driver and passengers in the cabin using the vehicle, terminal devices, or other devices, such as movies, videos, and music.

[0036] In some embodiments of this disclosure, the audio signal decomposition model 226 can be used to decompose the vehicle audio data 210 into audio signals 232 containing different categories (e.g., including but not limited to engine sounds, horn sounds, suspension system noise, tire noise, wind noise, etc.). In some embodiments of this disclosure, the audio signal decomposition model 228 can be used to decompose the vehicle exterior audio data 212 into audio signals 234 containing different categories (e.g., including but not limited to animal sounds, natural ambient sounds). Natural ambient sounds can be sounds from the vehicle's surrounding environment, such as rain, the sound of leaves rustling, and the sound of flowing water.

[0037] It is understandable that audio signal decomposition models 224, 226, and 228 can be models with the same structure but different network parameters. For example, audio signal decomposition models 224, 226, and 228 can be the same type of deep learning model (such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), etc.), multiple audio signal decomposition models with different network parameters obtained by training with different training data.

[0038] In some embodiments of this disclosure, the state of the vehicle itself and the state of the vehicle control unit are used as "feedforward signals" to assist in the decomposition of audio signals, which can improve the efficiency and accuracy of audio signal decomposition. Specifically, the vehicle data and vehicle control state feedback module 214 can synchronously collect relevant vehicle signals during the audio reception time (within the time period of collecting different audio data) according to user instructions. These include, for example, vehicle data 216, driving operations 218, the vehicle control unit 220, and system resources 222. Vehicle data 216 may include, but is not limited to, vehicle speed, engine status, and battery level. Driving operations 218 may include, but is not limited to, acceleration, steering, and braking. The vehicle control unit 220 may include, but is not limited to, doors and windows, suspension system, and air conditioning. System resources 222 may include, but is not limited to, entertainment applications, navigation, and communication.

[0039] In some embodiments of this disclosure, after the human voice content is decomposed, the lyrics generation model 248 can be used to generate corresponding lyrics text content 250 based on the human voice content. The lyrics generation model 248 can be a language model (LM). A language model is an artificial intelligence algorithm based on deep learning technology, whose goal is to enable computers to understand and generate natural language. After training on a large-scale dataset, it can understand and generate human natural language, completing various natural language processing tasks (such as lyrics generation). In one example, the lyrics generation model 248 can be an autoregressive text generation model, or a decoder (Transformer-Decoder) based on a deep self-attention network, a Long Short-Term Memory (LSTM) model, a Generative Adversarial Networks (GAN) model, etc.

[0040] In some embodiments of this disclosure, since there may be multiple drivers and passengers in the vehicle, different users will output different voice content. Based on this, the voice content can be distinguished by timbre to determine the independent audio of different drivers and passengers. For example, in one example, based on timbre features, the voices of people speaking in different locations in the cabin can be identified, and the voice content can be distinguished by timbre to obtain the independent audio of different drivers and passengers, such as A's independent audio "The weather looks great this weekend, how about we go camping at the beach?" and B's independent audio "Great, I can't wait to go out and enjoy the sunshine and sea breeze."

[0041] In some embodiments of this disclosure, the input information to the lyrics generation model 248 can be multimodal (e.g., audio, images, text). Based on this, independent audio can be directly input to the lyrics generation model 248, which then generates corresponding lyrics text content 250 based on the independent audio from different drivers and passengers. In some embodiments of this disclosure, to improve the efficiency and accuracy of lyrics text content generation, the vocal content can first be converted into text to obtain corresponding text content, and then the lyrics text model 248 can generate the corresponding lyrics text content 250 based on this text content. For example, speech transcription technology can be used to convert the separated independent audio into text content. The text content includes dialogue content from different drivers and passengers.

[0042] In some embodiments of this disclosure, the emotions contained in the lyrics are diverse, and different lyrics evoke different feelings in people. To ensure that the emotions contained in the generated lyrics text content 250 better match the current feelings and emotions of the driver and passengers, emotion analysis can be performed on the vocal content to obtain the emotions of the driver and passengers contained in the vocal content. This emotion 246 is used as auxiliary information for one dimension of the generated lyrics text content 250, and is input as a prompt word into the lyrics generation model 248, so that the lyrics generation model 248 generates the corresponding lyrics text content 250 based on the prompt word.

[0043] In some embodiments of this disclosure, a text sentiment analysis model 244 can be used to perform sentiment analysis on the text content to determine the emotions 246 contained in the voice content. The types of emotions 246 can be various, including but not limited to happiness, sadness, anger, surprise, fear, and neutrality. It is understood that in some embodiments of this disclosure, after determining the category of emotion 246, the level of emotion 246 can be further determined. For example, in one example, the identified emotion 246 and its corresponding level can be "Level 1 Happy". After identifying the corresponding emotion 246 (or the level of emotion 246), the emotion 246 can be input into a lyrics generation model 248, which can generate corresponding lyrics text content 250 based on the identified emotion 246 or the level of emotion 246.

[0044] In practical applications, lyrics texts come in various styles and / or themes. Different styles or themes of lyrics texts have different structures and content. It's understandable that different drivers and passengers have different preferred lyrics or want to hear different lyrics at any given moment. For example, user A might prefer lyrics with a love theme, while user B might prefer lyrics with a youth theme. In this case, to make the generated lyrics text content 250 more appealing to drivers and passengers and further enhance their driving experience, a function to specify the music style 238 can be provided.

[0045] In some embodiments of this disclosure, a user's preferred music style 238 can be determined based on the user-input music style 236. The user can specify the music style 238 or other attributes of the lyrics (e.g., lyric format, rhyme scheme, theme, length, etc.). In some embodiments of this disclosure, the user can customize the attributes of the lyrics on their terminal, such as inputting the music style 236 or the theme of the lyrics. For example, the user-input music style could be "pop," and the theme could be "campus." In some embodiments of this disclosure, the music style 238 can also serve as the basis for generating the lyrics generation model 248. For example, after determining the music style 238, it can be input into the lyrics generation model 248, which can then generate corresponding lyrics text content 250 based on the music style 238. In one example, the generated rock-style lyrics text 250 could be: "The wind is howling, the tires are beating like drumbeats. The sun is burning, our dreams are soaring. Raise your hands, shout together, this wild coast, release your perfect vacation..."; the generated rap-style lyrics text 250 could be: "Weekend is here, mood is booming, freedom we're going to find you today, the sun is shining, let's go treasure hunting, turn the music up, laughter never stops. Worries are behind us, music will lead the way..."

[0046] In some embodiments of this disclosure, users have different preferred timbres. For example, user A prefers the timbre of singer x, while user B prefers the timbre of singer y. Based on this, to make the final generated music 260 more in line with user preferences, the function of specifying or customizing timbres can be provided to the user. For example, the user can input a timbre 262 according to their needs. This timbre 264 can be a timbre from a preset timbre library, or it can be a user-defined timbre (e.g., a timbre determined based on an audio clip recorded by the user).

[0047] It's understandable that a song consists of both lyrics and melody. To provide users with an immersive driving experience, one or both of the vehicle's surrounding sounds or the vehicle's own sounds can be used as the basis for generating the melody. For example... Figure 2 As shown, the non-human voice audio feature extraction model 252 can extract features from the audio signal 232 obtained by decomposing the vehicle audio data 210 and / or the audio signal 234 obtained by decomposing the external audio data 212, to obtain the corresponding audio features 254. It can be understood that the audio features 254 can include time-domain and frequency-domain features of various types of audio signals. In some embodiments of this disclosure, the audio features can be Mel-Frequency Cepstral Coefficients (MFCCs) and features processed by a shallow neural network, such as peak points and smoothness of the spectrum.

[0048] like Figure 2 As shown, the information input to the music generation model 258 includes various types, such as lyrics text content 250, music style 238, emotion 246, timbre 264, and audio features 254. This information can be input to the music generation model 258 individually, or it can be fused to obtain fused information 256, which is then input to the music generation model 258. There are various fusion methods; fusion can be performed through simple concatenation, or more complex fusion algorithms (such as attention mechanisms, feature transformations, etc.) can be used to fuse multiple pieces of information. The music generation model 258 can generate corresponding music 260 based on the fused information 256. In some embodiments of this disclosure, the music generation model can be a basic machine learning model or a deep learning model. For example, the music generation model includes recurrent neural networks, Hidden Markov Models (HMMs), Support Vector Machines (SVMs), or deep learning models such as convolutional neural networks and long short-term memory networks.

[0049] In some embodiments of this disclosure, the music generation model 258, when generating music 260, can consider all input information to generate music that matches these inputs. For example, if the input lyrics 250 describe a cheerful scene, the music style 238 is pop music, the driver's mood 246 is happy, and the audio features include a melodic characteristic, then the model may generate a cheerful pop song whose melody matches the audio features. It can be understood that the music generation model 258 can generate virtual human voice audio content based on the input lyrics. For example, text-to-speech conversion technology can be used to convert the lyrics into audio. During the generation of audio, the intensity, pitch, and other parameters of the audio can be adjusted according to the music style or the theme or emotional tone contained in the lyrics to obtain the final, compliant human voice audio signal.

[0050] In this way, the music generation process can combine user preferences, state, and audio data from the vehicle's surroundings or the vehicle itself to generate music of various styles. Furthermore, it avoids repetitive music generation, thus meeting users' personalized needs. On the other hand, because the information used to generate music is richer and more diverse, the generated music is more relevant to the current context, possessing greater richness and depth, thereby enhancing the user's driving experience.

[0051] Figure 3 A flowchart of a method 300 for generating music according to some embodiments of the present disclosure is shown. Method 300 can be... Figure 1 The method 300 is executed at the processor 110. Furthermore, with the continuous development and innovation of smart vehicles and smartphones, more and more vehicles and mobile devices possess computing power (such as smart vehicles and smartphones). That is to say, the method 300 according to embodiments of this disclosure is not limited to execution at the processor 110, but may also be executed at the vehicle 102, and this disclosure does not impose any limitation thereon.

[0052] like Figure 3 As shown in block 302, method 300 includes determining lyrics text content corresponding to the human voice content within the vehicle's cabin. The human voice content within the cabin can be obtained by decomposing the acquired audio data within the cabin. The audio data within the cabin can be obtained from sound sensors installed within the vehicle (e.g., [insert sensor name here]). Figure 1 The in-vehicle sensor 104 shown collects sound signals from inside the vehicle. The type of sound sensor installed inside the vehicle can be various, including microphone sensors (e.g., microphone arrays), capacitive sensors, etc.

[0053] Because the audio data within the cockpit contains various types of sounds, such as vehicle information prompts, multimedia sounds (e.g., sounds being played on the vehicle or the user's terminal device, such as music or movies), conversations between the user and other types of sounds, to determine the human voice content within the cockpit audio data, the human voice content can be separated from the audio data by performing audio decomposition based on the characteristics of the audio data (frequency, amplitude, phase, etc.). In some embodiments of this disclosure, audio signal decomposition algorithms (e.g., Fourier transform algorithms, wavelet transform algorithms) or audio signal decomposition models (e.g., deep learning models) can be used to perform audio signal decomposition on the cockpit audio data.

[0054] It is understandable that the conversation between drivers and passengers can usually express their current state and thoughts. For example, in a conversation where "A: I really want to go to the beach to watch the sunset; B: Yes, I really want the sea breeze to blow away these unhappy feelings," it can be seen that the drivers and passengers' current goal is "the beach" and what they want to see is "the sunset." Based on this, corresponding lyrics can be generated based on the drivers' and passengers' voice content. For example, the generated lyrics could be themed around "sunset at the beach." In some embodiments of this disclosure, a generative pre-trained model can be used to generate corresponding lyrics text content based on the voice content (e.g., Figure 2 The lyrics text shown is 250).

[0055] In box 304, method 300 includes generating music based on audio features of at least one of the following audio data: vehicle body audio data, vehicle exterior audio data, and lyrics text content. The vehicle body audio data refers to audio features generated by sound sensors (e.g., those installed on the vehicle body) Figure 1 The vehicle body sensor 106 shown collects data representing the sound of the vehicle body itself. For example, vehicle body audio data may include the vehicle's engine sound, horn sound, tire friction sound, etc. External audio data refers to the sounds of the vehicle's surrounding environment, such as, but not limited to, the sound of flowing water, rain, thunder, etc.

[0056] It's understandable that when a user (e.g., a driver) is driving, the vehicle's state and their surroundings are constantly changing. As the environment changes, the collected vehicle audio data or external audio data will also change accordingly. This audio data not only contains rich audio features but also data related to the vehicle's state and the user's environment, providing more contextual information for music generation. Based on this, to ensure that the final generated music complements the vehicle's state or the state of its surrounding environment, providing a more immersive driving experience, corresponding audio features (e.g., ...) can be extracted from the vehicle audio data and / or external audio data. Figure 2 The audio features shown (254) provide rich data for music generation, thus making the generated music more in line with user needs.

[0057] In some embodiments of this disclosure, audio features are the result of numerically representing a certain attribute of an audio signal. Audio features can be divided into physical features and perceptual features. Physical features, depending on the feature space, can be further divided into time domain (short-time energy, zero-crossing rate, and linear prediction coefficients), frequency domain (linear prediction cepstral coefficients), and time-frequency domain (short-time Fourier transform and wavelet coefficients). Perceptual features can be features such as rhythm and pitch. In some embodiments of this disclosure, corresponding audio features can be extracted using an audio feature extraction model. It can be understood that audio features can be features of vehicle body audio data, features of external vehicle audio data, or features of both external vehicle audio data and vehicle body audio data. It can be understood that, to increase the richness and environmental adaptability of the generated music, audio features in multiple dimensions can be extracted. These multiple dimensions of audio features can include pitch features, spectral features, rhythm features, acoustic features, etc.

[0058] For example, in one instance, when a vehicle is traveling on a highway, external sound sensors (e.g., Figure 2 The external audio sensor 206 shown can capture sounds such as wind and tire friction outside the vehicle to generate external audio data. Based on the audio characteristics of the external audio data, a dynamic and energetic piece of music can be generated; while when the vehicle is parked in a quiet park, the external sound sensor can capture sounds such as birdsong and water sounds to generate a calm and relaxing piece of music.

[0059] In some embodiments of this disclosure, the generated music can be music in the traditional sense, containing vocal audio and background audio. The vocal audio refers to the audio corresponding to the lyrics. The background audio can be generated based on the lyrics using audio features. It is understood that the background audio is the same length as the vocal audio and has a consistent style, ensuring the harmony and unity of the entire musical work. In other embodiments, the generated music can also be instrumental, for example, instrumental music can be generated based on the themes and characteristics contained in the lyrics using audio features. The themes and characteristics of the instrumental music are consistent with the lyrics. In some embodiments of this disclosure, the type of music can be determined based on the weight of the vocal audio and the weight of the background audio. For example, if a user prefers instrumental music, the weight of the vocal audio can be set below a preset value, reducing the presence of the vocal audio in the music.

[0060] In this way, based on various types of audio data, dynamic and stylistically diverse music can be generated for users. The generated music is no longer confined to traditional, fixed genres, but rather creates music that expresses the driver's or passenger's personality or preferences based on their vocal content, thus enhancing the user's driving experience.

[0061] In some embodiments of this disclosure, to determine the human voice content in the cabin audio data, an audio signal decomposition model can be used to decompose the cabin audio data to obtain human voice content and non-human voice content. To improve the efficiency of audio feature extraction, different audio signal decomposition models can also be used to decompose the vehicle body audio data and the exterior audio data separately. Different audio signal decomposition models have the same network structure but different network parameters and are trained using different training data. After decomposing different categories of audio signals, sub-audio features of multiple types of audio signals can be extracted to determine the corresponding audio features.

[0062] In some embodiments of this disclosure, there may be multiple drivers and passengers in the vehicle, who can converse, resulting in the audio content potentially including the speech of each user. It is understood that distinguishing the speech of each driver and passenger provides data support for generating music that caters to the preferences of different users. For example, to generate music that user A likes, the weight of user A's speech can be increased while the weight of user B's speech can be decreased, making the generated lyrics more aligned with user A's needs. Alternatively, when a user wants music containing their own timbre, the generated music can be tailored to meet that user's requirements. In some embodiments of this disclosure, timbre features can be used to identify the voices of people speaking in different locations within the cabin. These timbre features may include spectral envelope, formants, fundamental frequency (F0), etc. For example, machine learning or deep learning algorithms can be used to cluster or classify the extracted timbre features to obtain independent audio for each driver and passenger.

[0063] In some embodiments of this disclosure, human voice content can be converted from speech to text. For example, Automatic Speech Recognition (ASR) technology can be used to convert human voice content into corresponding text content. It is understood that for each driver's individual audio, ASR technology can be used to convert the audio into text content. The converted text content is then associated with the user in the original audio to ensure that each driver's text corresponds to their audio content. Based on the converted text content, a lyrics generation model can generate corresponding lyrics text content. For example, prompts (e.g., "generate lyrics with the theme of a sunset at the beach") can be generated based on the text content and its keywords, and the lyrics generation model can then generate the corresponding lyrics text content based on these prompts.

[0064] In practical applications, drivers and passengers often prefer different music depending on their emotional state. For example, a user in a high-spirited and happy mood might want to listen to upbeat rock music, while a user in a low and unhappy mood might want to listen to soothing music to alleviate their emotions. The content of a driver's or passenger's speech can reflect their emotions to some extent, and these emotions express the emotional tone of the music. Based on this, the emotions contained in the vocal content can be identified and used as a reference for generating music. In some embodiments of this disclosure, user emotions can be of various types, including but not limited to happiness, excitement, sadness, surprise, fear, and neutrality.

[0065] In some embodiments of this disclosure, to more accurately identify the emotions of drivers and passengers, visual sensors can be used to acquire facial or body images of the drivers and passengers. Based on the user's facial expressions or body movements, the judgment of the driver's or passenger's emotions can be aided. By recognizing facial images to determine the driver's or passenger's facial expressions, such as smiling, frowning, or glaring, the driver's or passenger's emotional state, such as happiness, sadness, or anger, can be inferred. Similarly, body movements can also convey the driver's or passenger's emotional information. Analyzing the driver's or passenger's posture, gestures, and other actions based on body images can further assist in judging the driver's or passenger's emotions. For example, when a driver or passenger exhibits a tense posture or gesture, it can be determined to some extent that the driver or passenger may be in a tense or anxious emotional state.

[0066] In some embodiments of this disclosure, after determining the final emotion, the emotion can be input as a cue word into the lyrics generation model. The lyrics generation model then generates corresponding lyrics text content based on the text content and the cue word. The lyrics generation model is a deep learning model for text generation that can be trained with data. For example, the lyrics generation model can be an autoregressive text generation model, or it can be a decoder based on a deep self-attention network, a long short-term memory model, a generative adversarial network model, etc. In some embodiments of this disclosure, the cue word refers to the text input to the lyrics generation model, which can be used to prompt or guide the lyrics generation model to give the expected output result. By inputting the cue word and text content into the lyrics generation model, the lyrics generation model can generate corresponding lyrics text content based on the learned linguistic features (rhythm, grammar, etc.) and contextual relationships of the lyrics.

[0067] In some embodiments of this disclosure, after the final music is generated, it can be played using the vehicle's speakers or other sound devices. During music playback, to provide a more immersive music experience for the user, other components or elements within the cabin (such as air conditioning, seats, ambient lighting, etc.) can be used to enhance the music playback effect. In some embodiments of this disclosure, the parameters of the components used to assist playback can be determined based on the characteristics of the music. For example, the parameters of the components can be determined based on the style, melody, and theme of the music. In one example, when the music is rock music, the ambient lighting parameters can be set to "blinking, alternating red and blue colors, and 80% brightness." In this way, different components within the cabin can be combined to provide the user with an immersive entertainment experience, improving the user's driving experience.

[0068] In some embodiments of this disclosure, different drivers and passengers may have different preferred music styles while traveling. To ensure that the generated music better suits their preferences, drivers and passengers can select a corresponding music style based on their own tastes and needs, using this as a reference for music generation. For example, drivers and passengers can select the corresponding music style through touch operation, gestures, voice input, brainwave input, or other input methods. In some embodiments of this disclosure, a terminal device that communicates with the vehicle or with the processor can provide a user interface for human-computer interaction for drivers and passengers.

[0069] Figure 4 A schematic diagram of a user interface according to some embodiments of the present disclosure is shown. Figure 4 As shown, the interactive interface 402 may include multiple display components 404, selection controls 406 corresponding to the display components 404, and confirmation controls 408, etc. Display components 404 are used for different types of music styles, such as pop, rock, classical, folk, etc. Drivers and passengers can freely select one or more music styles according to their preferences. After selecting a music style, it can be determined whether to trigger the selection control 406. The selection control 406 can be triggered and respond through a trigger event. Trigger events may include mouse clicks, finger clicks, etc. The confirmation control 504 can be in the form of a button control, which may include text box buttons, plain icon buttons, etc. In response to the user's trigger operation, the music style selected by the driver or passenger can be sent to the lyrics generation model for subsequent processing. In some embodiments of this disclosure, drivers and passengers can also conveniently and quickly obtain their favorite music styles by manually searching in the style market according to their needs.

[0070] It is understandable that, in order to allow drivers and passengers to participate more in the music creation process, users can not only specify the style of the music to be generated, but also set the duration, theme, and instruments to be played. The user input information is sent to the music generation model as a reference for music generation, and the music generation model generates music that meets the requirements. It is also understandable that user input can affect the generation of lyrics; therefore, user input information can also be sent to the lyrics generation model, which generates lyrics that meet the requirements. In some embodiments of this disclosure, the music generation model can be a pre-trained machine learning model or a deep learning model for generating music.

[0071] In this way, the data on which the music is generated becomes more diverse, enabling personalized experiences for each user and avoiding repetition and regularity, so that users feel fresh and unique every time they experience it.

[0072] In some embodiments of this disclosure, the music generation model can generate music by integrating all inputs. For example, the inputs to the music generation model may include user input (such as a user-specified melody, a user-specified music style, etc.), identified user emotions, multi-dimensional audio features, and lyrics text content. In some embodiments of this disclosure, the music generation model can generate corresponding virtual human voice audio signals based on the lyrics text content and user input. For example, the music generation model can utilize Text-to-Speech (TTS) technology to convert the lyrics text content into speech audio. Based on the user input or the theme or emotional tone contained in the lyrics text content, the corresponding timbre information is determined. The timbre information may include pitch, intensity, etc. Based on this timbre information, the pitch, intensity, and other parameters of the generated audio can be adjusted to meet the user's expectations and requirements. In this way, a virtual human voice audio signal that matches the lyrics text content and has a suitable timbre can be generated.

[0073] In some embodiments of this disclosure, a corresponding melody can be generated based on the lyrics text content and audio features. For example, the music generation model can determine the length of the melody based on the length of the lyrics text, and then generate a melody of that length based on the audio features. In some embodiments of this disclosure, a corresponding musical melody can be generated based on various dimensions of information contained in the audio features.

[0074] In some embodiments of this disclosure, the generated vocal audio signal and musical melody are mixed to produce a complete musical work. During the mixing process, the harmony between the lyrics or vocal audio signal and the melody needs to be considered; for example, their styles, themes, and expressed emotions need to match. It is understood that weight values ​​can be assigned to both the vocal audio signal and the musical melody during the fusion process. These weight values ​​characterize the degree of influence of the musical melody or vocal audio signal on the final generated music. Different types of music can be generated based on different weight value settings. For example, a lower weight value for the vocal audio signal can generate instrumental music containing a melody. It is understood that when fusing the vocal audio signal and the musical melody, attention must be paid to the volume balance and time synchronization of both to ensure that the generated musical work is audibly harmonious and unified.

[0075] In some embodiments of this disclosure, after generating music, it can be determined whether the style of the vocal audio signal and the melody are consistent. For example, it can be determined whether the styles of the two are consistent based on the stylistic characteristics of the vocal audio signal and the musical melody. If the difference between the two exceeds a preset threshold, it can be determined that the styles do not match. Based on the melody and the keywords corresponding to the melody, the lyrics text content can be regenerated. For example, the structure of the new lyrics text content can be planned according to the rhythm of the melody (e.g., determining the number of paragraphs, the number of lines in each paragraph, and the number of words in each line). In other embodiments of this disclosure, new lyrics text content is created based on the generated keywords and the theme or emotion of the vocal content. During the creation process, it is necessary to ensure that the content of the lyrics matches the style of the musical melody and is consistent with the theme or emotion of the original vocal content. Alternatively, the generated lyrics text content can be optimized, including modifying inappropriate words, adjusting sentence structure, adding rhymes, etc., to improve the attractiveness and harmony of the generated music.

[0076] In some embodiments of this disclosure, during the music generation process, users can adjust aspects of the music they are dissatisfied with, such as modifying lyrics, adjusting the style and rhythm, etc. The user's adjustment methods reflect their preferences to some extent. Therefore, based on this, the user's historical adjustments to music can be used to determine their preferred or desired styles. A prompt word is generated based on the user's desired adjustments. For example, if a user dislikes particularly loud music, the generated prompt word would be "the music style is soothing." It can be understood that the generated prompt word can be input into either the lyrics generation model or the music generation model. In this way, the music generated based on the prompt word, whether in terms of lyrics or melody, is more in line with the user's preferences. This saves adjustment steps, improves generation efficiency, and conserves processing resources.

[0077] In some embodiments of this disclosure, to improve the decomposition efficiency and accuracy of the audio signal decomposition model, relevant vehicle information can be input into the audio signal decomposition model to assist in signal decomposition. For example, vehicle dynamics and vehicle control status can be sent to the audio signal decomposition model as a "feedforward signal enhancer." Vehicle dynamics can include relevant vehicle data (e.g., engine status, battery level, or fuel level), vehicle driving status (vehicle speed, acceleration, whether steering, steering angle, whether braking, braking parameters, etc.), and user driving operations (the force applied to the accelerator or brake pedal, the angle of the steering wheel).

[0078] In some embodiments of this disclosure, the vehicle control state can be the state of the vehicle's control unit. For example, the vehicle control state may include whether doors and windows are open and their opening degree, the state of the suspension system, and the state of the air conditioning (e.g., whether the air conditioning is on, the air conditioning mode, the air conditioning temperature, etc.). Based on vehicle dynamics and vehicle control state, the audio signal decomposition model can more accurately identify and understand the different components in the audio signal. For example, when a vehicle is traveling at high speed, the engine speed increases, and the frequency of the engine sound also increases accordingly. At this time, if the audio signal decomposition model can receive vehicle dynamic information such as vehicle speed and engine state, it can more accurately identify the engine sound and decompose it from the complex audio signal.

[0079] For example, when vehicle windows or other components are open, they alter the acoustic environment inside the vehicle, causing audio signals to undergo different reflections, refractions, and attenuations during propagation. This information helps audio signal decomposition models more accurately identify and separate the different components of the audio signal. In other examples, the status of the air conditioning also affects the in-vehicle sound environment. When the air conditioning is on, the rotation of the fan and the flow of cold air generate noise, which can be superimposed on other vehicle sounds, affecting the quality of the audio signal. By using the air conditioning status as input, audio signal decomposition models can better identify and process this noise.

[0080] By using vehicle dynamics and vehicle control status as feedforward signals to the audio signal decomposition model in this way, not only can the decomposition efficiency of the model be improved, but also the accuracy of audio signal decomposition can be improved.

[0081] In some embodiments of this disclosure, drivers and passengers may have their own preferred musical atmosphere. For example, driver A might like songs by singer S and want the generated music to have the timbre of singer S. Alternatively, in some embodiments of this disclosure, the user can be provided with the ability to specify a particular timbre. For example, the user can input their preferred timbre or customize their preferred timbre. The user can set the timbre on their terminal; for example, the user can input a timbre on their terminal or select the timbre of the music to be generated from a preset timbre library. In this way, users can directly discover their own voice content in the generated music, resulting in higher engagement and better interactive fun, further enhancing the user's driving experience.

[0082] Figure 5 A schematic diagram illustrating driver / passenger sound selection according to some embodiments of the present disclosure is shown. Figure 5As shown, the interactive interface 502 may include a selectable voice library 504 (e.g., a male voice library 506 and a female voice library 508) and a custom voice 510. It is understood that drivers and passengers can select a voice according to their needs (e.g., selecting voice A from the male voice library and voice B from the female voice library). The voice library may contain various voice samples, which can be recordings of real singers or virtual voices generated through voice synthesis technology. In some embodiments of this disclosure, drivers and passengers can also add custom voices, such as their own voice resources or those of friends. This allows drivers and passengers to participate more in the music creation process, providing them with a more personalized and customized music experience.

[0083] Figure 6 A block diagram of an apparatus 600 for generating music according to some embodiments of the present disclosure is shown. Figure 6 As shown, the device 600 includes a lyrics text content generation unit 602, configured to generate lyrics text content corresponding to human voice content within the vehicle's cabin. The device 600 also includes a music generation unit 604, configured to generate music based on audio features of at least one of the vehicle's body audio data and exterior audio data, and based on the lyrics text content.

[0084] In some embodiments, the device 600 further includes an audio decomposition unit configured to: determine human voice content in the audio data within the cabin by performing audio decomposition on the audio data within the cabin, the audio data within the cabin being collected by the vehicle through in-vehicle sensors; determine multiple types of audio by performing audio decomposition on at least one of the vehicle body audio data and the external audio data; and determine corresponding audio features based on the multiple types of audio.

[0085] In some embodiments, the audio decomposition unit is further configured to: decompose the audio data in the cabin using a first audio signal decomposition model to determine the human voice content; and decompose the vehicle body audio data and the external audio data using a second audio signal decomposition model and a third audio signal decomposition model respectively to determine multiple types of audio, wherein the first audio signal decomposition model, the second audio signal decomposition model, and the third audio signal decomposition model have the same structure but different weights.

[0086] In some embodiments, the lyrics text content generation unit 602 is further configured to: determine the text corresponding to the human voice content by performing text conversion on the human voice content; determine the emotional cue words contained in the human voice content by performing emotion recognition on the human voice content; and generate lyrics text content corresponding to the human voice content based on the text and the emotional cue words.

[0087] In some embodiments, the lyrics text content generation unit 602 is further configured to: determine the music style specified by the user based on the information input by the user; and generate lyrics text content corresponding to the vocal content based on the text, emotional prompts, and music style.

[0088] In some embodiments, the lyrics text content generation unit 602 is further configured to: determine multiple users and multiple independent audios corresponding to multiple users by performing timbre feature recognition on the human voice content; and determine multiple texts corresponding to multiple users by performing text conversion on the multiple independent audios.

[0089] In some embodiments, the music generation unit 604 is further configured to: generate a virtual human voice audio signal based on the lyrics text content; generate a corresponding musical melody based on audio features; and generate music by fusing the human voice audio signal with the musical melody.

[0090] In some embodiments, the music generation unit 604 is further configured to: determine corresponding timbre information based on the lyrics text content and the user's input; and generate corresponding virtual human voice audio signals based on the lyrics text content and the timbre information.

[0091] In some embodiments, the device 600 further includes a lyrics regeneration unit configured to: generate keywords based on the music melody in response to a style mismatch between the human voice audio signal and the music melody; and regenerate new lyrics text content based on the keywords and the human voice content.

[0092] In some embodiments, the music generation unit 604 is further configured to generate a corresponding musical melody based on the lyrics text content, the emotions contained in the vocal content, and audio features, wherein the length and structure of the musical melody are the same as the length and structure of the lyrics text content.

[0093] In some embodiments, the audio decomposition unit is further configured to: acquire vehicle dynamics and vehicle control status; and, based on the vehicle dynamics and vehicle control status, determine the human voice content in the audio data within the cabin by performing audio decomposition on the audio data within the cabin.

[0094] It is understood that by utilizing the apparatus 600 of this disclosure, at least one of the many advantages that can be achieved by the methods or processes described above can be realized.

[0095] Figure 7A schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure is shown. As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 702 or loaded from storage unit 708 into random access memory (RAM) 703. Various programs and data required for the operation of device 700 may also be stored in RAM 703. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0096] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as method 300. For example, in some embodiments, method 300 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of method 300 described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform method 300 by any other suitable means (e.g., by means of firmware).

[0098] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload programmable logic devices (CPLDs), and so on.

[0099] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0101] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating music, comprising: Based on the human voice content inside the vehicle's cabin, generate lyrics text content corresponding to the human voice content; as well as Music is generated based on the audio features of at least one of the vehicle's body audio data and exterior audio data, as well as the lyrics text content.

2. The method according to claim 1, further comprising: The audio content within the cabin is determined by performing audio decomposition on the audio data within the cabin, which is collected by the vehicle through in-vehicle sensors. Multiple types of audio are determined by performing audio decomposition on at least one of the vehicle body audio data and the vehicle exterior audio data. as well as Based on the various types of audio, the corresponding audio features are determined.

3. The method according to claim 2, further comprising: The audio data in the cockpit is decomposed using a first audio signal decomposition model to determine the human voice content. as well as The vehicle audio data and the external audio data are decomposed using a second audio signal decomposition model and a third audio signal decomposition model, respectively, to determine the various types of audio. The first audio signal decomposition model, the second audio signal decomposition model, and the third audio signal decomposition model have the same structure but different weights.

4. The method according to claim 2, wherein determining the human voice content by decomposing the audio data collected by the in-vehicle sensors using a first audio signal decomposition model includes: Obtain the vehicle dynamics and vehicle control status of the vehicle; as well as Based on the vehicle dynamics and vehicle control status, the human voice content in the audio data within the cabin is determined by performing audio decomposition on the audio data within the cabin.

5. The method according to claim 1, wherein generating lyrics text content corresponding to the vocal content includes: The text corresponding to the human voice content is determined by performing text conversion on the human voice content. By performing emotion recognition on the human voice content, the emotional cue words contained in the human voice content are determined; as well as Based on the text and the emotion prompts, generate lyrics text content corresponding to the human voice content.

6. The method according to claim 5, wherein generating lyrics text content corresponding to the human voice content further includes: Based on the information input by the user, the music style specified by the user is determined; as well as Based on the text, the emotional cue words, and the music style, generate lyrics text content corresponding to the vocal content.

7. The method according to claim 5, wherein determining the text corresponding to the human voice content includes: By performing timbre feature recognition on the human voice content, multiple users and multiple independent audio recordings corresponding to the multiple users are identified in the human voice content. as well as By performing text conversion on the multiple independent audio files, multiple texts corresponding to the multiple users are determined.

8. The method of claim 1, wherein generating music comprises: Based on the lyrics text content, a virtual human voice audio signal is generated; as well as Based on the aforementioned audio features, a corresponding musical melody is generated; as well as Music is generated by fusing the human voice audio signal with the musical melody.

9. The method of claim 8, wherein generating the virtual human voice audio signal comprises: Based on the lyrics text content and the user's input, the corresponding timbre information is determined; as well as Based on the lyrics text content and timbre information, a corresponding virtual human voice audio signal is generated.

10. The method of claim 8, further comprising: In response to a mismatch between the style of the human voice audio signal and the musical melody, keywords are generated based on the musical melody; as well as Based on the keywords and the vocal content, new lyrics text is regenerated.

11. The method according to claim 8, wherein generating the corresponding musical melody comprises: Based on the lyrics text content, the emotions contained in the vocal content, and the audio features, a corresponding musical melody is generated, the length and structure of which are the same as the length and structure of the lyrics text content.

12. The method of claim 1, wherein generating music comprises: Based on the audio features of at least one of the vehicle's body audio data and exterior audio data, and the lyrics text content, a music generation model generates corresponding music.

13. The method of claim 12, wherein generating the corresponding music by the music generation model comprises: Based on the audio features and the lyrics text content, the fusion information is determined; as well as Based on the fused information, the music generation model generates the corresponding music.

14. An apparatus for generating music, comprising: The lyrics text generation unit is configured to generate corresponding lyrics text based on the human voice content inside the vehicle's cabin. as well as The music generation unit is configured to generate music based on the audio features of at least one of the vehicle's body audio data and exterior audio data, as well as the lyrics text content and the audio features.

15. An electronic device comprising: At least one processor; as well as A memory coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the device to perform the method according to any one of claims 1-13.

16. A vehicle comprising a plurality of sound sensors and an electronic device according to claim 15.

17. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions that, when executed, implement the method according to any one of claims 1 to 13.

Images