Intelligent music creation method and system
Through the deep learning model training and processing of the intelligent creative system, combined with the data of the karaoke system and the dial system, the problem that the existing system cannot create personalized music is solved, and efficient and personalized music creation and mixing effects are achieved.
Patent Information
- Application Number
- CN202411861316.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-12-31
- Publication Date
- 2025-03-21
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing karaoke system and the DJ system cannot combine to form a new working mode, and cannot temporarily create DJ and music works that meet users' preferences and are personalized.
Through an intelligent creation system, audio and video materials and music databases are input, music data and user historical data are collected, music data and user historical data are preprocessed and fusion, deep learning models are trained, creative instructions are received, and replicas are recognized and processed, and mixing songs and songs that meet user intentions.
It realizes the integration of the karaoke system and the dial system, provides rich and personalized music materials, reduces the threshold and learning costs of users, and can independently realize music mixing and song creation, and generates customized music works that meet users' intentions.
Smart Images
Figure CN119673133A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of audio and video creation, and in particular relates to an intelligent music creation method and system. Background Art
[0002] Karaoke system is an accompaniment system, singers can sing with pre-recorded music accompaniment, often used in KTV. DJ system, referred to as the DJ system, refers to a system that uses professional machines to edit and process electronic music, and is used to create a live atmosphere in nightclubs, bars and other places.
[0003] Nowadays, music creation is becoming more and more intelligent, including lyrics writing and song writing, especially live creation. With the development of the times, people's requirements for the original live sound effects have increased. Users hope to have a more dynamic place with stronger music and more gorgeous lights to sing karaoke. If this effect is to be achieved, the scene needs to be equipped with karaoke equipment and DJ equipment at the same time. However, since the two devices are independent of each other, they cannot be combined to form a new working mode, and it is even more impossible to temporarily create personalized DJ works and music works that meet the user's preferences. Summary of the invention
[0004] The present invention provides an intelligent music creation method and system, and in particular proposes a new solution for the integrated creation of a karaoke system and a disc making system.
[0005] The present invention is implemented by providing an intelligent music creation method based on an intelligent creation system, characterized in that: Input audio and video materials and music database into the intelligent creation system, including collecting graphic materials online; Collect a large amount of representative music data and retrieve user history data from its own database or music platform, the music data includes music materials, songs, lyrics data and audio data, and the user history data includes the user's historical singing data and music preference data; Preprocessing the music data and the user history data; Merging the music data with the user history data; Training a deep learning model according to the fused music data and the user history data to obtain a target model; Receive a creation instruction, and according to the creation instruction issued by the user, call and combine the target model to identify the creation instruction and interactively obtain the target range of the creation instruction confirmed by the user; the target range includes text creation output and audio and video creation output; The creative output results can be output separately and / or merged.
[0006] Receive a target song, and according to a DJ instruction issued by a user, call and combine the target model to mix the target song to generate a mixed song that meets the user's intention and transmit it to the DJ system, wherein the target song includes a recorded song uploaded by the user, an existing song, or a song generated by the system; The target lyrics are received, and according to the song creation instruction issued by the user, the target model is called and combined to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored original song and transmit it to the karaoke system. The target song includes self-created lyrics uploaded by the user, existing lyrics, or lyrics generated by the system.
[0007] Furthermore, before the step of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system, the following steps are included: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The steps of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system include: The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.
[0008] Furthermore, before the step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system, the step includes: Performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention and transmitting it to the karaoke system includes: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening out suitable music materials and audio adjustment methods from the music data according to the lyrics features and the user history data, wherein the lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; Synthesizing the music material after audio processing with the target lyrics to generate a song that meets the user's intention; The generated songs are transmitted to the karaoke system for users to use.
[0009] Furthermore, before the step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system, the step also includes: Determine whether the target lyrics are existing lyrics; If the target lyrics are existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The steps of calling and combining the target model to create a song based on the target lyrics to generate a highly restored imitation song and transmitting it to the karaoke system include: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening corresponding music materials and audio adjustment methods from the music data according to the lyrics features, wherein the lyrics features include emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; The music material after audio processing is synthesized with the target lyrics to generate a highly restored imitation song; The generated imitation original song is transmitted to the karaoke system for user use.
[0010] Furthermore, after the step of preprocessing the music data and the user history data, the step further includes: Performing spectrum analysis and feature extraction on the pre-processed audio data to obtain audio features and generate an audio feature vector, wherein the audio features include pitch, rhythm, volume and channel; The pre-processed lyrics data is subjected to word segmentation, sentiment analysis, topic modeling and feature extraction to obtain lyrics features and generate lyrics feature vectors. The lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics.
[0011] Furthermore, after the step of preprocessing the music data and the user history data, the step further includes: Performing speech signal processing and feature extraction on the pre-processed singing data to obtain singing features and generate a singing feature vector, wherein the singing features include the user's pitch, rhythm and singing skills; The preprocessed music preference data is subjected to data normalization and feature extraction to obtain music preference features, and a music preference feature vector is generated. The music preference features include the user's emotional inclination, style preference, and music material preference.
[0012] Furthermore, the step of fusing the music data with the user history data includes: splicing the audio feature vector, the lyrics feature vector, the singing feature vector and the music preference vector together to obtain a complete input feature vector; The step of training the deep learning model according to the fused music data and the user history data to obtain the target model comprises: The input feature vector is input into a deep learning model for training to obtain a target model.
[0013] Furthermore, after receiving the target lyrics and, according to the song creation instruction issued by the user, calling and combining the target model to create a song for the target lyrics to produce a song that meets the user's intention, the step also includes: The present invention also provides an intelligent music creation system, which is used to execute the intelligent music creation method according to claim 1, and is characterized by comprising: Collection module: used to collect a large amount of representative music data from the music database and retrieve user history data from its own database or music platform. The music data includes music materials, songs, lyrics data and audio data. The user history data includes the user's historical singing data and music preference data. The above collection module also includes online collection of graphic materials, especially real-time popular materials; Preprocessing module: used to preprocess the music data and the user history data, Fusion module: used for fusing the music data with the user history data; Training module: used for training the deep learning model according to the fused music data and the user history data to obtain a target model; Mixed song creation module: used to receive the target song, and according to the DJ instruction issued by the user, call and combine the target model to mix the target song to generate a mixed song that meets the user's intention and transmit it to the DJ system. The target song includes a recorded song uploaded by the user or an existing song; Song creation module: used to receive target lyrics, and according to the song creation instructions issued by the user, call and combine the target model to create a song for the target lyrics to produce a song that meets the user's intention or a highly restored imitation song and transmit it to the system. The target song includes self-created lyrics uploaded by the user or existing lyrics.
[0014] Furthermore, the remix song creation module is also used for: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.
[0015] The beneficial effect of the present invention lies in that, compared with the prior art, the intelligent music creation method of the present invention, especially the system and system applied to on-site creation are combined together, the data of the system and the data of the DJ system can be used mutually, thereby providing rich and user-personalized materials for music mixing and song creation, and the present invention can autonomously realize the process of music mixing and song creation according to instructions, which greatly reduces the user's music creation threshold and learning cost, and at the same time can also produce personalized customized music works for users that meet the user's intentions, greatly stimulating the user's interest in music.
[0016] Specifically, a large amount of representative music data is collected from the Internet or from a specific music database, and user historical data and other materials are retrieved from its own database or music platform, and the music data and user historical data are preprocessed, and then the music data and user historical data are merged, and the deep learning model is trained according to the merged music data and user historical data to obtain the target model; the creation instruction is interactively received according to the target model, and according to the creation instruction issued by the user, the target range of the creation instruction is identified by calling and combining the target model and interactively obtaining the user's confirmation input creation instruction; the target range includes text creation output and audio and video creation output; the creation output result is output separately and / or fused output. After receiving the instruction, the system will call and combine the target model to generate lyrics or mixed songs that meet the user's intentions according to the instruction, and then transmit it to the system for the user to use. The user can not only make personalized songs in real time and efficiently, but also imitate the original song according to the existing lyrics to make an imitation of the original song, which improves the user's creative fun and increases the creation of works. The quality of the works created by this system and method is guaranteed and can be used free of charge. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flowchart of the intelligent music creation method provided by the present invention; Figure 2 It is a system block diagram of the intelligent music creation system provided by the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0019] An intelligent music creation method, inputting audio and video materials and a music database into an intelligent creation system, including collecting graphic materials online; collecting a large amount of representative music data and retrieving user history data from its own database or music platform, the music data including music materials, songs, lyrics data and audio data, the user history data including the user's historical singing data and music preference data; preprocessing the music data and the user history data; fusing the music data and the user history data; training a deep learning model according to the fused music data and the user history data to obtain a target model; receiving a creation instruction, and according to the creation instruction issued by the user, calling and combining the target model to identify the creation instruction and interactively obtaining the target range of the creation instruction confirmed by the user; the target range includes text creation output and audio and video creation output; outputting the creation output results separately and / or fusion output. The user's creation instructions may be very diverse, ranging from simple style requirements to complex music structure requirements. The system can accurately identify and parse these diverse instructions. According to the user's instructions, the system can dynamically call different models or model combinations for creation. Obtaining the target range of the user's input creation instructions through interactive means is the key to ensuring that the final output meets the user's expectations. The system provides an intuitive and easy-to-use interactive interface and feedback mechanism. The system supports multiple output formats, including text, audio, video, etc., and can achieve separate or fusion output of these formats. Embodiment 1
[0020] refer to Figure 1 Embodiment 1 provides an intelligent music creation method, comprising steps S101 to S106: S101, collect materials online or collect a large amount of representative music data from a music database and retrieve user history data from its own database or music platform. The music data includes music materials, songs, lyrics data and audio data. The user history data includes the user's historical singing data and music preference data.
[0021] The music database may be any music platform, and users may also import their own singing data and music preference data into the system to expand user historical data.
[0022] S102, pre-processing the music data and the user history data.
[0023] Among them, preprocessing can include operations such as removing noise, standardizing and segmenting the data.
[0024] After the step of preprocessing the music data and the user history data, the following steps are further included: Performing spectrum analysis and feature extraction on the pre-processed audio data to obtain audio features and generate an audio feature vector, wherein the audio features include pitch, rhythm, volume and channel; The pre-processed lyrics data is subjected to word segmentation, sentiment analysis, topic modeling and feature extraction to obtain lyrics features and generate lyrics feature vectors. The lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics.
[0025] Spectral analysis of the audio data in the music data, as well as word segmentation, sentiment analysis, and topic modeling of the lyrics data in the music data, are specific analyses based on the different feature requirements of the music data. They can more accurately and reliably reflect the audio characteristics and lyrics characteristics of all the music collected in the music database, and finally generate feature vectors for model training and learning.
[0026] After the step of preprocessing the music data and the user history data, the following steps are further included: Performing speech signal processing and feature extraction on the pre-processed singing data to obtain singing features and generate a singing feature vector, wherein the singing features include the user's pitch, rhythm and singing skills; The preprocessed music preference data is subjected to data normalization and feature extraction to obtain music preference features, and a music preference feature vector is generated. The music preference features include the user's emotional inclination, style preference, and music material preference.
[0027] Performing speech signal processing on singing data in user historical data and data normalization on music preference data in music data is based on specific analysis of different feature requirements of user historical data, so as to more accurately and reliably reflect the singing characteristics and music preference characteristics of the user, and finally generate feature vectors for model training and learning. The more times a user uses it, the more user historical data will be obtained, and the trained model will be able to better understand the user's needs, so that the generated mixed songs or songs will be more popular with users.
[0028] S103, integrating the music data and the user history data.
[0029] Wherein, the step of fusing the music data with the user history data comprises: The audio feature vector, the lyrics feature vector, the singing feature vector and the music preference vector are concatenated together to obtain a complete input feature vector.
[0030] S104: training a deep learning model according to the fused music data and the user history data to obtain a target model.
[0031] Among them, the deep learning model of the present invention can be a convolutional neural network (CNN). In the process of training the convolutional neural network (CNN), multimodal deep learning is adopted. If the feature vectors are not fused in advance, the embedding layer can also be used to convert the audio feature vector, lyrics feature vector, singing feature vector and music preference vector into vectors of the same length, and then use the weighted sum method to fuse these vectors to obtain the input feature vector. Finally, the input feature vector is input into the convolutional neural network (CNN), and the features are learned in the convolution layer and the pooling layer.
[0032] The step of training the deep learning model according to the fused music data and the user history data to obtain the target model comprises: The input feature vector is input into a deep learning model for training to obtain a target model.
[0033] S106, receiving the target song, and according to the DJ instruction issued by the user, calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmit it to the DJ system. The target song includes a recorded song uploaded by the user or an existing song.
[0034] The step of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system includes: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The steps of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system include: The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.
[0035] First, the target song is segmented, feature analyzed, and matched through the trained target model, so as to obtain music materials and mixing methods that are suitable for the target song and the user. Then, the target song is mixed according to the screened music materials and mixing methods to obtain a mixed song for DJing.
[0036] S106, receiving the target lyrics, and according to the song creation instruction issued by the user, calling and combining the target model to create a song for the target lyrics to produce a song that meets the user's intention or a highly restored imitation song and transmit it to the karaoke system, wherein the target song includes self-created lyrics uploaded by the user or existing lyrics.
[0037] The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system includes: Performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention and transmitting it to the karaoke system includes: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening out suitable music materials and audio adjustment methods from the music data according to the lyrics features and the user history data, wherein the lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; Synthesizing the music material after audio processing with the target lyrics to generate a song that meets the user's intention; The generated songs are transmitted to the karaoke system for users to use.
[0038] It should be noted that if the target lyrics are not existing lyrics (that is, they are original lyrics), the target lyrics are first subjected to feature analysis through the target model and matched and screened according to the user's historical data of the lyrics features, so as to obtain music materials and audio adjustment methods that are suitable for the target lyrics and the user's personal needs. The screened music materials are then audio-processed according to the audio adjustment method to make them conform to the target lyrics and the user's personalized requirements. Finally, they are synthesized with the target lyrics to obtain a new song that meets the user's intentions for use in karaoke or subsequent mixing and DJing.
[0039] The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system also includes: Determine whether the target lyrics are existing lyrics; If the target lyrics are existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The steps of calling and combining the target model to create a song based on the target lyrics to generate a highly restored imitation song and transmitting it to the karaoke system include: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening corresponding music materials and audio adjustment methods from the music data according to the lyrics features, wherein the lyrics features include emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; The music material after audio processing is synthesized with the target lyrics to generate a highly restored imitation song; The generated imitation original song is transmitted to the karaoke system for user use.
[0040] It should be noted that if the target lyrics are existing lyrics, the target lyrics are feature analyzed and screened through the target model, so as to screen out the music materials and audio adjustment methods that match the target lyrics. Since the target lyrics are existing lyrics, the music materials and audio adjustment methods of the corresponding original songs should be the most matching with the target lyrics. Then, according to the audio adjustment methods of the screened original songs, the music materials are subjected to corresponding audio processing, so that a re-produced imitation song with a high degree of restoration to the original song can be obtained for use in K songs, which effectively solves the problem of insufficient original song resources in the K song system.
[0041] After receiving the target lyrics and, according to the song creation instruction issued by the user, calling and combining the target model to create a song for the target lyrics to produce a song that meets the user's intention, the step further includes: According to the social sharing instructions issued by the user, the completed music works will be shared to the corresponding social platform, and according to the characteristics of the music works, suitable music circles or communication circles will be screened for the user to make music recommendations or communication recommendations to meet the user's sharing needs and music dissemination and communication.
[0042] The intelligent music creation method of the present invention, in particular, combines the karaoke system and the DJ system together. The data of the karaoke system and the DJ system can be used interchangeably, thereby providing rich and user-personalized materials for music mixing and song creation. The present invention can autonomously realize the process of music mixing and song creation according to instructions, which greatly reduces the user's music creation threshold and learning cost. At the same time, it can also produce personalized customized music works for users that meet the user's intentions, greatly stimulating the user's interest in music.
[0043] Specifically, a large amount of representative music data is first collected from the music database and user historical data is retrieved from its own database or music platform, and the music data and user historical data are pre-processed, and then the music data and user historical data are fused, and the deep learning model is trained based on the fused music data and user historical data to obtain a target model; after receiving the target song, if the user clicks on the touch screen to issue a DJing command, the system will call and combine the target model to mix the target song according to the DJing command to generate a mixed song that meets the user's intention, and then transmit it to the DJing system for It is for users to use for DJing, and they can use self-composed songs for remix DJing, or they can use existing songs for remix DJing; after receiving the target lyrics, if the user clicks on the touch screen to issue a DJing command, the system will call and combine the target model to compose the target lyrics according to the song creation command to generate a song that meets the user's intention or a highly restored imitation of the original song and transmit it to the karaoke system. It can not only produce personalized original songs for users, but also imitate the original songs according to the existing lyrics to solve the problem that a large number of original songs cannot be used for free and for a long time by the karaoke system. Embodiment 2
[0044] refer to Figure 2 Embodiment 2 provides an intelligent music creation system, including: Collection module: used to collect a large amount of representative music data from the Internet or music database and retrieve user history data from its own database or music platform. The music data includes music materials, songs, lyrics data and audio data. The user history data includes the user's historical singing data and music preference data; Preprocessing module: used for preprocessing the music data and the user history data.
[0045] Fusion module: used to fuse the music data and the user history data.
[0046] Training module: used to train the deep learning model according to the fused music data and the user history data to obtain a target model.
[0047] Mixed song creation module: used to receive the target song, and according to the DJ instruction issued by the user, call and combine the target model to mix the target song to produce a mixed song that meets the user's intention and transmit it to the DJ system. The target song includes a recorded song uploaded by the user or an existing song.
[0048] Song creation module: used to receive target lyrics, and according to the song creation instructions issued by the user, call and combine the target model to create a song for the target lyrics to produce a song that meets the user's intention or a highly restored imitation song and transmit it to the K song system. The target song includes self-created lyrics uploaded by the user or existing lyrics.
[0049] This includes lyrics creation and audio-visual creation, and based on the creation instructions issued by the user, the target model is called and combined to identify the creation instructions and interactively obtain the target parameter range input by the user for confirmation; the creation output includes lyrics output and audio-visual output; the user's creation instructions may be very diverse, ranging from simple style requirements to complex music structure requirements.
[0050] Specifically, the remix song creation module is also used for: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.
[0051] The song creation module is also used to: Performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening out suitable music materials and audio adjustment methods from the music data according to the lyrics features and the user history data, wherein the lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; Synthesizing the music material after audio processing with the target lyrics to generate a song that meets the user's intention; The generated songs are transmitted to the karaoke system for users to use.
[0052] The song creation module is also used to: Determine whether the target lyrics are existing lyrics; If the target lyrics are existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening corresponding music materials and audio adjustment methods from the music data according to the lyrics features, wherein the lyrics features include emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; The music material after audio processing is synthesized with the target lyrics to generate a highly restored imitation song; The generated imitation original song is transmitted to the karaoke system for user use.
[0053] The preprocessing module is also used for: Performing spectrum analysis and feature extraction on the pre-processed audio data to obtain audio features and generate an audio feature vector, wherein the audio features include pitch, rhythm, volume and channel; The pre-processed lyrics data is subjected to word segmentation, sentiment analysis, topic modeling and feature extraction to obtain lyrics features and generate lyrics feature vectors. The lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics.
[0054] The preprocessing module is also used for: Performing speech signal processing and feature extraction on the pre-processed singing data to obtain singing features and generate a singing feature vector, wherein the singing features include the user's pitch, rhythm and singing skills; The preprocessed music preference data is subjected to data normalization and feature extraction to obtain music preference features, and a music preference feature vector is generated. The music preference features include the user's emotional inclination, style preference, and music material preference.
[0055] The fusion module is also used for: The audio feature vector, the lyrics feature vector, the singing feature vector and the music preference vector are concatenated together to obtain a complete input feature vector.
[0056] The training module is also used to: input the input feature vector into a deep learning model for training to obtain a target model.
[0057] The intelligent music creation system also includes a sharing module: According to the social sharing instructions issued by the user, the completed music works will be shared to the corresponding social platforms, and according to the characteristics of the music works, suitable music circles or communication circles will be screened for the users to make music recommendations or communication recommendations.
[0058] Upon receiving the instruction, the system will call and combine the target model to generate lyrics or remixed songs that meet the user's intentions according to the instruction, and then transmit it to the system for the user to use. The user can use self-composed songs for remixing and DJing, or existing songs for remixing and DJing. After receiving the target lyrics, if the user clicks on the touch screen to issue a DJing instruction, the system will call and combine the target model to create songs for the target lyrics according to the song creation instruction to generate songs that meet the user's intentions or highly restored original songs and transmit them to the K song system. It can not only produce personalized original songs for users, but also imitate the original songs according to the existing lyrics to produce imitation songs, so as to solve the problem that a large number of original songs cannot be used for free and for a long time by the K song system.
[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An intelligent music creation method, characterized in that the steps include: Collect a large amount of representative music data from the music database and retrieve user history data from its own database or music platform, the music data includes music materials, songs, lyrics data and audio data, and the user history data includes the user's historical singing data and music preference data; Preprocessing the music data and the user history data; Merging the music data with the user history data; Training a deep learning model according to the fused music data and the user history data to obtain a target model; Receive a target song, and according to a DJ instruction issued by a user, call and combine the target model to mix the target song to generate a mixed song that meets the user's intention and transmit it to the DJ system, wherein the target song includes a recorded song uploaded by the user or an existing song; The target lyrics are received, and according to the song creation instruction issued by the user, the target model is called and combined to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored original song and transmit it to the karaoke system. The target song includes self-created lyrics uploaded by the user or existing lyrics.
2. The intelligent music creation method according to claim 1, characterized in that: The step of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system includes: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The steps of calling and combining the target model to mix the target song to generate a mixed song that meets the user's intention and transmitting it to the DJ system include: The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.
3. The intelligent music creation method according to claim 1, characterized in that: The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system includes: Determine whether the target lyrics are existing lyrics; If the target lyrics are not existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention and transmitting it to the karaoke system includes: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening out suitable music materials and audio adjustment methods from the music data according to the lyrics features and the user history data, wherein the lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; Synthesizing the music material after audio processing with the target lyrics to generate a song that meets the user's intention; The generated songs are transmitted to the karaoke system for users to use.
4. An intelligent music creation method according to any one of claims 1 to 3, based on an intelligent creation system, characterized in that: Before collecting a large amount of representative music data from the music database and retrieving user historical data from its own database or music platform, Input audio and video materials and music database into the intelligent creation system, including collecting graphic materials online; At the same time, it can also receive creation instructions, and according to the creation instructions issued by the user, call and combine the target model to identify the creation instructions and interactively obtain the target range of the creation instructions input by the user; the target range includes text creation output and audio and video creation output; The creative output results can be output separately and / or merged.
5. The intelligent music creation method according to claim 4, characterized in that: The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system includes: Determine whether the target lyrics are existing lyrics; If the target lyrics are not existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention and transmitting it to the karaoke system includes: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening out suitable music materials and audio adjustment methods from the music data according to the lyrics features and the user history data, wherein the lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; Synthesizing the music material after audio processing with the target lyrics to generate a song that meets the user's intention; The generated songs are transmitted to the karaoke system for users to use.
6. The intelligent music creation method according to claim 4, characterized in that: The step of calling and combining the target model to create a song for the target lyrics to generate a song that meets the user's intention or a highly restored imitation song and transmitting it to the karaoke system also includes: Determine whether the target lyrics are existing lyrics; If the target lyrics are existing lyrics, performing a second preprocessing on the target lyrics, wherein the second preprocessing includes removing stop words, word segmentation, and part-of-speech tagging; The steps of calling and combining the target model to create a song based on the target lyrics to generate a highly restored imitation song and transmitting it to the karaoke system include: and calling the target model to perform feature analysis on the target lyrics after the second preprocessing to obtain lyrics features, and matching and screening corresponding music materials and audio adjustment methods from the music data according to the lyrics features, wherein the lyrics features include emotional tendency, emotional intensity, theme and topic of the lyrics; Performing corresponding audio processing on the selected music material according to the selected audio adjustment method, wherein the audio processing includes cutting, merging, adding effects, adjusting volume, changing pitch, changing speed or extracting tracks of the music material; The music material after audio processing is synthesized with the target lyrics to generate a highly restored imitation song; The generated imitation original song is transmitted to the karaoke system for user use.
7. The intelligent music creation method according to claim 4, characterized in that: After the step of preprocessing the music data and the user history data, the following steps are further included: Performing spectrum analysis and feature extraction on the pre-processed audio data to obtain audio features and generate an audio feature vector, wherein the audio features include pitch, rhythm, volume and channel; The pre-processed lyrics data is subjected to word segmentation, sentiment analysis, topic modeling and feature extraction to obtain lyrics features and generate lyrics feature vectors. The lyrics features include the emotional tendency, emotional intensity, theme and topic of the lyrics.
8. The intelligent music creation method according to claim 4, characterized in that: After the step of preprocessing the music data and the user history data, the following steps are further included: Performing speech signal processing and feature extraction on the pre-processed singing data to obtain singing features and generate a singing feature vector, wherein the singing features include the user's pitch, rhythm and singing skills; The preprocessed music preference data is subjected to data normalization and feature extraction to obtain music preference features, and a music preference feature vector is generated. The music preference features include the user's emotional inclination, style preference, and music material preference.
9. The intelligent music creation method according to claim 6 or 7, characterized in that: The step of fusing the music data with the user history data comprises: splicing the audio feature vector, the lyrics feature vector, the singing feature vector and the music preference vector together to obtain a complete input feature vector; The step of training the deep learning model according to the fused music data and the user history data to obtain the target model comprises: The input feature vector is input into a deep learning model for training to obtain a target model.
10. An intelligent music creation system, used to execute the intelligent music creation method according to claim 1, characterized in that: include: Collection module: used to collect a large amount of representative music data from the music database and retrieve user history data from its own database or music platform. The music data includes music materials, songs, lyrics data and audio data. The user history data includes the user's historical singing data and music preference data; it also includes the online collection of pictures, texts, audio and video materials; Preprocessing module: used to preprocess the music data and the user history data, Fusion module: used for fusing the music data with the user history data; Training module: used for training the deep learning model according to the fused music data and the user history data to obtain a target model; Mixed song creation module: used to receive a target song, and according to the instruction issued by the user, call and combine the target model to mix the target song to generate a mixed song that meets the user's intention and transmit it to the system, wherein the target song includes a recorded song uploaded by the user or an existing song; Song creation module: used for receiving target lyrics, and according to the song creation instruction issued by the user, calling and combining the target model to create songs for the target lyrics to generate songs that meet the user's intention or highly restored original songs and transmit them to the system, wherein the target songs include self-created lyrics uploaded by the user or existing lyrics; The remix song creation module is also used to: Performing a first preprocessing on the uploaded recorded song, wherein the first preprocessing includes denoising, format conversion and editing; The target model is called to segment the target song after the first preprocessing, and feature analysis is performed on the target song after the segmentation process to obtain target song features, and suitable music materials and mixing methods are matched and screened from the music data according to the target song features and the user history data, wherein the target song features include melody, rhythm, harmony and emotion; Mixing the target song according to the selected music materials and mixing methods to generate a mixed song that meets the user's intention; The generated mixed song is transmitted to the DJ system for the user to use in DJing.