Tuning method and device, electronic equipment and medium

Abstract

The invention provides a tone tuning method and apparatus, an electronic device and a medium. The method comprises the steps of obtaining a global semantic feature of a target audio and a paragraph signal feature of each paragraph in the target audio; generating a target prompt text for the target audio based on the global semantic feature of the target audio and the paragraph signal feature of each paragraph; the target prompt text comprises overall style description of the target audio and paragraph feature description of each paragraph; inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and the paragraph feature description in the target prompt text, and determining paragraph sound effect parameters for each paragraph, thereby generating target sound effect parameters for the target audio, and finely adjusting the sound effect according to music semantics and music structures. And a sound effect scheme fitting the music style is produced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a tuning method and device, electronic equipment and medium. BACKGROUND

[0002] There are some sound effect processing schemes on the market at present, for example, some preset sound effect parameter configurations are provided according to music genres, like pop music may enhance the bass part to make the rhythm more prominent, and classical music may highlight the mid-high frequency to enable the user to hear the tone of the musical instrument more clearly. Some other schemes automatically adjust the sound effect by using simple audio feature analysis, such as volume size and frequency distribution, like automatically reducing the background noise in a relatively quiet music section. However, these existing intelligent sound effect processing technologies are relatively single, and most of them can only adjust the sound effect based on the type of music or simple audio features, lacking a deeper and more comprehensive understanding of music. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a tuning method, device, electronic equipment and medium to generate a more refined sound effect scheme that fits the music style and different sections.

[0004] In a first aspect, an embodiment of the present application provides a tuning method applied to a server, comprising: obtaining global semantic features of a target audio and section signal features of each section in the target audio; generating a target prompt text for the target audio based on the global semantic features of the target audio and the section signal features of each section; the target prompt text includes an overall style description of the target audio and a section feature description of each section; inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and the section feature description in the target prompt text by the sound effect parameter generation model, and determining section sound effect parameters for each section; processing the section sound effect parameters of each section by the sound effect parameter generation model to generate target sound effect parameters for the target audio, which are used for sound effect generation of the target audio.

[0005] In some embodiments of the tuning method, the global semantic features are used to determine target reference information for sound effect generation of the target audio. inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and the section feature description in the target prompt text by the sound effect parameter generation model, and determining section sound effect parameters for each section, comprising: inputting the target prompt text and the target reference information into an audio effect parameter generation model, adjusting the target reference information based on overall style description in the target prompt text by the audio effect parameter generation model, and obtaining audio effect parameters matching overall style of the target audio; adjusting, by the audio effect parameter generation model, the audio effect parameters corresponding to each paragraph in the audio effect parameters based on the paragraph feature description, and generating paragraph audio effect parameters for each paragraph.

[0006] In some embodiments, the tuning method further includes, before inputting the target prompt text into the audio effect parameter generation model: determining, based on the global semantic features of the target audio, a target basic audio effect template matching the target audio from a plurality of preset basic audio effect templates as the target reference information; the target basic audio effect template includes configuration parameters of a plurality of effectors.

[0007] In some embodiments, the tuning method further includes, before inputting the target prompt text into the audio effect parameter generation model: obtaining audio data and lyrics text data of the target audio; inputting the audio data and the lyrics text data into a trained multi-modal large language model; processing the input audio data and lyrics text data by the multi-modal large language model to generate global semantic features of the target audio; the global semantic features include at least one of music style, emotional expression, theme content, and main musical instruments.

[0008] In some embodiments, the tuning method further includes, before inputting the target prompt text into the audio effect parameter generation model: processing the input audio data and lyrics text data by the multi-modal large language model to generate global semantic features of the target audio based on pre-constructed first prompt information; The first prompt information is used to set an analysis role for the multi-modal large language model and set a first output format of the global semantic features.

[0009] In some embodiments, the tuning method further includes, before inputting the target prompt text into the audio effect parameter generation model: extracting music signal features and vocal signal features of each paragraph in the target audio; generating text-formatted paragraph signal features based on the music signal features and the vocal signal features of each paragraph in the target audio; The music signal features include at least one of a music spectrum feature, a dynamic feature, and a spatial sound field feature. The vocal signal features include at least one of a vocal spectrum feature, a vocal dynamic feature, and a vocal high pitch feature.

[0010] In some embodiments, in the tuning method, the spectrum feature includes at least one of a spectrum centroid, a spectrum spread, and a harmonic proportion. The dynamic feature includes at least one of a dynamic range, a peak level, and an average level. The spatial sound field feature includes at least one of a stereo width, a phase correlation, and a sound image position.

[0011] In some embodiments, in the tuning method, before obtaining the paragraph signal features of each paragraph in the target audio, the method further includes: inputting audio data of the target audio into a trained audio segmentation model; processing the audio data of the target audio by the audio segmentation model, analyzing the analysis structure of the target audio, dividing different paragraphs of the target audio, and generating paragraph information of each paragraph.

[0012] In some embodiments, in the tuning method, before obtaining the paragraph signal features of each paragraph in the target audio, the method further includes: inputting audio data of the target audio into a trained audio segmentation model; processing the audio data of the target audio by the audio segmentation model, generating a plurality of independent audio track data of the target audio; The music signal features of each paragraph include music signal features of each independent audio track data of each paragraph.

[0013] In some embodiments, in the tuning method, the global semantic features and the paragraph signal features are in a text format. The generating of the target prompt text for the target audio based on the global semantic features and the paragraph signal features of each paragraph of the target audio includes: filling the global semantic features and the paragraph signal features of each paragraph into a predefined prompt text template to generate the target prompt text for the target audio.

[0014] In some embodiments, the tuning method further includes: in response to receiving the individualized adjustment text for the target audio, inputting the individualized adjustment text into an audio effect parameter generation model; generate a sound effect adjustment parameter matching the personalized adjustment text through the sound effect parameter generation model; the sound effect adjustment parameter is used to be superimposed with a target sound effect parameter of the target audio to obtain an adjusted target sound effect parameter.

[0015] In some embodiments, in the sound tuning method, after the target sound effect parameter for the target audio is generated, the method further includes: sending the audio file of the target audio and the target sound effect parameter to a terminal device, so that the terminal device applies a paragraph sound effect parameter of a paragraph in the target sound effect parameter to different paragraphs in the audio file when playing the audio file of the target audio.

[0016] In a second aspect, the embodiments of the present application further provide another sound tuning method applied to a terminal device, and the method includes: receiving an audio file of the target audio and a target sound effect parameter, and applying a paragraph sound effect parameter of a paragraph in the target sound effect parameter to different paragraphs in the audio file when playing the audio file of the target audio; wherein the target sound effect parameter is generated based on the following steps: obtaining a global semantic feature of the target audio and a paragraph signal feature of each paragraph in the target audio; generating a target prompt text for the target audio based on the global semantic feature of the target audio and the paragraph signal feature of each paragraph; the target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph; inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and the paragraph feature description in the target prompt text through the sound effect parameter generation model, and determining a paragraph sound effect parameter for each paragraph; generating a target sound effect parameter for the target audio by processing the paragraph sound effect parameter of each paragraph through the sound effect parameter generation model.

[0017] In some embodiments, the sound tuning method further includes: in response to receiving a personalized adjustment text for the target audio, sending the personalized adjustment text to a server, so that the server generates a sound effect adjustment parameter matching the personalized adjustment text through a sound effect parameter generation model, and sends the sound effect adjustment parameter to the terminal device; in response to receiving the sound effect adjustment parameter, superimposing the sound effect adjustment parameter with a target sound effect parameter to determine an adjusted target sound effect parameter; applying a paragraph sound effect parameter of a paragraph in the adjusted target sound effect parameter to different paragraphs in the audio file, and playing the audio file.

[0018] In some embodiments, the method further comprises: in response to receiving the sound effect adjustment parameter, and superimposing the target sound effect parameter, determining the superimposed target sound effect parameter, comprising: In response to receiving the sound effect adjustment parameter, superimposing the sound effect adjustment parameter and the target sound effect parameter based on a pre-set sound effect parameter range, and determining the superimposed target sound effect parameter.

[0019] In a third aspect, the embodiments of the present application further provide a sound tuning device, applied to a server, comprising: The acquisition module is configured to acquire global semantic features of the target audio and paragraph signal features of each paragraph in the target audio. The first generation module is configured to generate a target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph; the target prompt text comprises an overall style description of the target audio and a paragraph feature description of each paragraph. The determination module is configured to input the target prompt text into a sound effect parameter generation model, process the overall style description and the paragraph feature description in the target prompt text by the sound effect parameter generation model, and determine a paragraph sound effect parameter for each paragraph. The second generation module is configured to generate a target sound effect parameter for the target audio by processing the paragraph sound effect parameter of each paragraph by the sound effect parameter generation model; the target sound effect parameter is used for sound effect generation of the target audio.

[0020] In a fourth aspect, the embodiments of the present application further provide a sound tuning device, applied to a terminal device, comprising: The receiving module is configured to receive an audio file of the target audio and a target sound effect parameter, and apply a paragraph sound effect parameter of each paragraph in the target sound effect parameter to different paragraphs in the audio file when playing the audio file of the target audio; wherein the target sound effect parameter is generated based on the following steps: Acquiring global semantic features of the target audio and paragraph signal features of each paragraph in the target audio. Generating a target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph; the target prompt text comprises an overall style description of the target audio and a paragraph feature description of each paragraph. Inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and the paragraph feature description in the target prompt text by the sound effect parameter generation model, and determining a paragraph sound effect parameter for each paragraph. Generating a target sound effect parameter for the target audio by processing the paragraph sound effect parameter of each paragraph by the sound effect parameter generation model.

[0021] In a fifth aspect, the embodiments of the present application further provide an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor communicates with the memory through the bus, and the machine readable instructions are executed by the processor to perform the steps of the tuning method.

[0022] In a sixth aspect, the embodiments of the present application further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the steps of the tuning method.

[0023] In the embodiments of the present application, a tuning method, device, electronic device and medium are provided, the method acquires global semantic features of a target audio and paragraph signal features of each paragraph in the target audio; based on the global semantic features of the target audio and the paragraph signal features of each paragraph, a target prompt text for the target audio is generated; the target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph; the target prompt text is input into an audio effect parameter generation model, the overall style description and the paragraph feature description in the target prompt text are processed by the audio effect parameter generation model to determine paragraph audio effect parameters for each paragraph; the paragraph audio effect parameters of each paragraph are processed by the audio effect parameter generation model to generate target audio effect parameters for the target audio, and the target audio effect parameters are used for audio effect generation of the target audio; the method cooperatively extracts global semantic features and paragraph signal features of music, generates comprehensive music feature descriptions to deeply understand music semantics, adjusts audio effects according to music semantics and music structure based on large model technology, and outputs audio effect schemes that fit music styles. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0025] Figure 1 A flowchart of the tuning method according to the embodiments of the present application is shown; Figure 2 A flowchart of the effecter processing audio according to the embodiments of the present application is shown; Figure 3 A flowchart of the method for acquiring global semantic features of a target audio according to the embodiments of the present application is shown; Figure 4A method flowchart for determining the paragraph sound effect parameter for each paragraph is shown; Figure 5 A structural schematic diagram of the tuning device is shown. Figure 6 A structural schematic diagram of the electronic device is shown. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application by those skilled in the art.

[0027] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0028] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0029] There are some sound effect processing solutions in the market at present. For example, some music playing software provides some preset sound effect parameter configurations according to music genres. Different music genres apply different sound effect parameter configurations to achieve different music playing effects. For example, pop music may enhance the bass part to make the rhythm more prominent; classical music may highlight the mid-high frequency to enable users to hear the tone of the musical instruments more clearly. Some other solutions automatically adjust the sound effect by using simple audio feature analysis, such as volume size and frequency distribution. For example, the sound effect of background noise is automatically reduced in relatively quiet music passages. However, these existing intelligent sound effect processing technologies are relatively single. Most of them can only adjust the sound effect based on the type of music or simple audio features, and lack a deeper and more comprehensive understanding of music.

[0030] However, the existing intelligent sound effect processing technology has many deficiencies. First, most of them only adjust the sound effect according to the music genre or simple audio features, and are difficult to deeply understand the semantic information such as the emotion and theme of the music, resulting in a lack of pertinence in sound effect processing and a failure to truly match the inner emotion and style characteristics conveyed by the music, which greatly limits the expressiveness and appeal of the music work. Second, the current sound effect processing process is relatively fixed, usually using uniform preset parameters for processing, ignoring the diversity and complexity of the music structure, and failing to achieve dynamic adjustment and smooth transition of the sound effect, resulting in insufficient coherence and naturalness of the sound effect of the entire song.

[0031] Therefore, in the embodiments of the present application, a tuning method, device, electronic equipment and medium are provided. The method obtains the global semantic features of a target audio and the paragraph signal features of each paragraph in the target audio. Based on the global semantic features of the target audio and the paragraph signal features of each paragraph, a target prompt text for the target audio is generated. The target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph. The target prompt text is input into a sound effect parameter generation model. The overall style description and the paragraph feature description in the target prompt text are processed by the sound effect parameter generation model to determine the paragraph sound effect parameters for each paragraph. The paragraph sound effect parameters of each paragraph are processed by the sound effect parameter generation model to generate target sound effect parameters for the target audio. The target sound effect parameters are used for sound effect generation of the target audio. The method cooperatively extracts the global semantic features and the paragraph signal features of the music, generates a comprehensive music feature description to deeply understand the music semantics, and adjusts the sound effect according to the music semantics and the music structure based on the large model technology to output a sound effect solution that matches the music style.

[0032] Please refer to Figure 1 , Figure 1 The flowchart of the tuning method described in the embodiments of the present application is shown. The tuning method is applied to a server, such as Figure 1As shown, the method comprises the following steps S101-S104: S101, obtaining global semantic features of the target audio and paragraph signal features of each paragraph in the target audio; S102, generating a target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph; the target prompt text comprises an overall style description of the target audio and a paragraph feature description of each paragraph; S103, inputting the target prompt text into an audio effect parameter generation model, processing the overall style description and the paragraph feature description in the target prompt text through the audio effect parameter generation model, and determining a paragraph audio effect parameter for each paragraph; S104, processing the paragraph audio effect parameter of each paragraph through the audio effect parameter generation model to generate a target audio effect parameter for the target audio, wherein the target audio effect parameter is used for audio effect generation of the target audio.

[0033] The tuning method can run on a terminal device or a server; wherein the terminal device can be a local terminal device, and when the tuning method runs on the server, the tuning method can be implemented based on the server; the tuning method can also be implemented and executed based on an interactive system, wherein the interactive system at least comprises a server and a client device (i.e. a terminal device).

[0034] When the tuning method is implemented and executed based on the interactive system, the interactive system can be referred to as an intelligent music audio effect processing system. The operation of the intelligent music audio effect processing system is divided into two parts: offline generation in the background and synchronous adjustment in the client. In the offline generation stage in the background, the system comprehensively analyzes the target audio to generate personalized target audio effect parameters; in the synchronous adjustment stage in the client, the user listens to the song, and the background downloads the audio file and the pre-generated personalized target audio effect parameters to the client, so that the client processes and plays the target audio based on the audio file and the personalized target audio effect parameters.

[0035] Based on this, in some embodiments, the system architecture of the intelligent music audio effect processing system is as follows: Basic template preset module: provides a complete audio processing tool chain, covering multiple effectors from graphic equalizers to compressors; by adopting modular design, independent configuration and flexible adjustment are supported, and basic template parameters are preset for different music genres.

[0036] Music structure analysis module: using deep learning technology, the song is analyzed for musical structure, and five basic music paragraphs such as prelude, main song, chorus, etc. are accurately identified and divided.

[0037] Global semantic feature extraction module: with the help of multi-modal large language model, fuse audio and text information, extract the global semantic features of the song, including style, emotion, theme and musical instruments and other key information.

[0038] Paragraph signal feature extraction module: multi-dimensional feature analysis is performed on each music paragraph, and features are extracted from audio, vocals and music structure to comprehensively analyze the characteristics of the music paragraph.

[0039] Sound effect Agent module: based on large model technology, responsible for generating personalized sound effect configuration according to music characteristics and user needs.

[0040] Sound effect interaction module: use large language model to realize natural language interaction between user and system, users can describe the desired sound effect effect through natural language, the system analyzes and adjusts the sound effect parameters in real time to meet the personalized needs of users.

[0041] In the step S101, the global semantic features of the target audio and the paragraph signal features of each paragraph in the target audio are obtained.

[0042] The target audio, that is, the audio to be generated sound effect, can be a song, an audio novel, etc.

[0043] The paragraph of the target audio usually refers to a continuous segment with consistent semantics or structure in the audio, and its division method varies with the type of audio.

[0044] For example, the paragraph types of music audio, such as songs, include intro, verse, pre-chorus, chorus, bridge, interlude, and outro.

[0045] It should be noted that in some embodiments, the paragraphs of the target audio do not need to distinguish all paragraph types, but can divide some paragraph types, such as only dividing five basic music paragraphs of intro, verse, chorus, etc.

[0046] The global semantic information includes popular, rock, classical, electronic, jazz and other genres, as well as sadness, happiness and other emotions.

[0047] In some embodiments, the global semantic features are used to determine the target reference information for sound effect generation of the target audio, and the target reference information is the parameters of some pre-configured effects.

[0048] For example, the target reference information is a target basic sound effect template matching the target audio, and the target basic sound effect template includes configuration parameters of multiple effects.

[0049] In some embodiments, the target reference information can also be a null value.

[0050] In some embodiments, before inputting the target prompt text into the sound effect parameter generation model, the method further comprises: determining, based on the global semantic features of the target audio, a target basic sound effect template matching the target audio from a plurality of preset basic sound effect templates as the target reference information; the target basic sound effect template includes configuration parameters of a plurality of effectors.

[0051] In some embodiments, the determination of the target basic sound effect template matching the target audio from the plurality of preset basic sound effect templates is implemented based on a basic template preset module in the intelligent music sound effect processing system.

[0052] The basic template preset module is the starting point of the entire intelligent music sound effect processing system, which provides an initial framework and parameter basis for subsequent sound effect processing. The basic sound effect template is a combination of a group of effectors, including the parameters of each effector. A possible combination of effectors includes graphic equalizers, reverb effectors, delay effectors, stereo enhancers, and compression limiters, etc.

[0053] In addition to the necessary parameters, each effector also has a parameter to control whether to turn on or not. Therefore, the effectors turned on in the basic sound effect template are not fixed, but depend on the selection of professional tuning. That is, the effectors turned on in different basic sound effect templates can be the same or different.

[0054] For different mainstream music genres such as pop, rock, classical, electronic, and jazz, the basic template preset module has a series of sound effect templates designed by professional sound engineers. These templates fully consider the characteristics of each genre of music and common auditory needs. For example, the template for rock music usually emphasizes strong bass, powerful mid-high frequencies, and appropriate reverb effects to highlight the intensity of the guitar and drum points; while the template for classical music pays more attention to the purity of tone, balance between instruments, and natural reverb space to restore the atmosphere of live performance.

[0055] Please refer to Figure 2 , Figure 2The flowchart of the effect processor processing audio according to the embodiments of the present application is shown. The audio signal first enters the graphic equalizer, which adjusts the gain or attenuation of the audio signal of different frequencies, thereby changing the frequency response characteristics of the audio, making the sound in certain frequency range more prominent or softer. The signal processed by the graphic equalizer is partially mixed in the reverb effect processor, which simulates different space environments and adds reverb effects to the audio, making the sound like it is produced in a specific space (such as a hall, a room, etc.), increasing the spatial sense and depth of the sound. The signal from the reverb effect processor then enters the delay effect processor, which delays the input audio signal for a certain time and then mixes it with the original signal to output, creating echo, repetition, and other effects, increasing the level and rhythm of the sound. The stereo enhancement effect processor: the audio signal processed by the delay effect processor enters the stereo enhancement effect processor, which adjusts the left and right channel characteristics of the audio signal, enhances the stereo field effect of the sound, and makes the listener feel a wider sound field and more obvious sound source positioning. The audio signal processed by the stereo enhancement effect processor enters the compressor, which is mainly used to control the dynamic range of the audio signal to prevent clipping distortion caused by signal overload, and can also improve the overall loudness and clarity of the audio. The audio signal processed by the compressor is finally output as audio, providing subsequent playback devices or other processing links.

[0056] Here, through the combined use of multiple effect processors and processors, the audio signal can be comprehensively processed and optimized.

[0057] In some embodiments, before obtaining the paragraph signal features of each paragraph in the target audio, the method further comprises: inputting the audio data of the target audio into the trained audio segmentation model; processing the audio data of the target audio by the audio segmentation model, analyzing the analysis structure of the target audio, dividing different paragraphs of the target audio, and generating paragraph information of each paragraph.

[0058] For example, the audio segmentation model can use a pre-trained Demucs model and an open-source All-In-One tool.

[0059] The paragraph information of the paragraph includes the start time, end time and paragraph label of each paragraph; and the paragraph information is used to identify different paragraphs of the target audio.

[0060] In some embodiments, before obtaining the paragraph signal features of each paragraph in the target audio, the method further comprises: inputting the audio data of the target audio into the trained audio segmentation model; The audio data of the target audio is processed by the audio track separation model to generate multiple independent audio track data of the target audio.

[0061] In some embodiments, for the intelligent music sound effect processing system, the steps of generating the paragraph information of each paragraph and generating the multiple independent audio track data of the target audio are implemented based on a music structure analysis module in the system.

[0062] Specifically, the music structure analysis module undertakes the key preliminary analysis task in the intelligent music sound effect processing system, and mainly completes two objectives: one is to realize audio track separation processing, accurately separating a complex mixed audio signal into multiple tracks, such as four tracks of drums, bass, vocals, and other instruments, to provide a basis for subsequent targeted sound effect feature extraction; the other is to analyze the form structure of music, identify different paragraphs such as the prelude, main song, chorus, interlude, and tail song in the audio, and provide structural basis for dynamic sound effect adjustment.

[0063] The specific implementation process of the music structure analysis module to divide different paragraphs of the target audio and separate different audio tracks is as follows: A pre-trained Demucs model is used as the audio track separation model, and its powerful audio source separation capability is used to decompose the input mixed audio file into four independent audio tracks of drums, bass, vocals, and other instruments. Demucs is based on deep learning technology and can learn the feature patterns of different instruments and sounds in audio, thereby achieving accurate separation. In actual operation, the audio file is input into the Demucs model, and the model can automatically identify and separate the four main audio tracks after learning a large amount of training data, and output independent audio track files.

[0064] An open-source All-In-One tool is used: this tool provides efficient music structure analysis functions and can accurately identify paragraphs such as preludes, main songs, choruses, interludes, and tail songs in audio. It is based on a deep learning model and trained with a large amount of music data, and can understand the characteristics and patterns of different music paragraphs. The main audio or other comprehensive audio tracks after track separation are input into the All-In-One tool, which outputs detailed analysis results about the music structure, including the start time, end time, and paragraph label of each paragraph.

[0065] In some embodiments, please refer to Figure 3 , the steps S301-S303 of obtaining the global semantic features of the target audio include: S301, obtaining audio data and lyrics text data of the target audio; S302, inputting the audio data and lyrics text data into a trained multi-modal large language model; S303, processing the input audio data and lyrics text data through the multi-modal large language model to generate global semantic features of the target audio; the global semantic features include at least one of music style, emotional expression, theme content and main musical instruments.

[0066] In some embodiments, processing the input audio data and lyrics text data through the multi-modal large language model to generate global semantic features of the target audio includes: Processing the input audio data and lyrics text data through the multi-modal large language model, the multi-modal large language model generates global semantic features of the target audio based on pre-constructed first prompt information; The first prompt information is used to set the analysis role for the multi-modal large language model and set the first output format of the global semantic features.

[0067] That is, the global semantic features output here conform to the first output format; In some embodiments, extracting global semantic features is implemented based on a global semantic feature extraction module in the intelligent music sound effect processing system.

[0068] The global semantic feature extraction module focuses on in-depth analysis of music audio and its lyrics information, and accurately extracts the global semantic features of music. These features cover music style (such as pop, rock, classical, etc.), emotional expression (such as happy, sad, passionate, etc.), theme content (such as love, nature, inspiration, etc.), and main musical instruments (such as guitar, piano, drum, etc.) and other key elements. Through the extraction of these semantic features, the module can provide detailed semantic guidance for subsequent sound effect processing. This enables sound effect optimization to closely match the content and emotional expression of music, significantly enhancing the infectivity and expressiveness of music, and providing users with a more immersive auditory experience.

[0069] Model selection: Given the rapid development of multi-modal large models, the multi-modal large language model here uses QwenAudio to achieve the required music semantic feature extraction function; QwenAudio can process both audio and text modal input data, and has learned the complex correlation between audio and text data through large-scale pre-training, with excellent semantic understanding ability, and can provide deep and accurate semantic feature extraction services for music analysis.

[0070] Constructing the first prompt information, i.e., role setting and prompt construction, is used to set a clear role and task for the QwenAudio model, i.e., "a professional music analysis assistant", and to construct a special prompt to guide the model to output music semantic feature analysis results that meet the requirements.

[0071] In some embodiments, the first prompt information includes a role description and an output format requirement.

[0072] For example, the specific role description and output format requirement are as follows: Role description: "You are a professional music analysis assistant who is good at extracting key semantic features from music audio and lyrics. Your task is to analyze the key information of the music genre, emotion, rhythm, and main instruments according to the input music information, and present it in a clear and concise manner." Output format requirement: "Please output your analysis results in the following format: Music style: [style type, such as pop, rock, classical, etc.] Emotional expression: [emotional description, such as happy, sad, passionate, etc.] Rhythm content: [rhythm overview, such as fast, slow, etc.] Main instruments: [list of instruments, such as guitar, piano, drum, etc.]

[0073] The audio (denoted as audio) and lyrics text (denoted as lyrics) of the song to be analyzed are taken as input data, and the dialogue prompt for the dialogue is constructed as follows: "Analyze this song {audio}, whose lyrics are {lyrics}". Through this prompt method, the audio and lyrics information are integrated and input into the QwenAudio model to start the music semantic feature analysis process.

[0074] The multi-modal large language model generates description results about the global semantic features of the song according to the constructed dialogue prompt and the input audio data and lyrics text data; these results are presented in a preset first output format, directly reflecting the core semantic information of the song. Subsequent processing steps will use these semantic features for targeted sound effect parameter adjustment and optimization.

[0075] In some embodiments, the paragraph signal features of each paragraph in the target audio are obtained, including: Generating text format paragraph signal features based on the music signal features and the vocal signal features of each paragraph in the target audio; The music signal features include at least one of music spectrum features, dynamic features, and spatial sound field features; The vocal signal features include at least one of vocal spectrum features, vocal dynamic features, and vocal sound height features.

[0076] In some embodiments, the music signal features of each paragraph include the music signal features of each independent audio track data of each paragraph.

[0077] In some embodiments, the spectral features include at least one of: spectral centroid, spectral spread, harmonic ratio; The dynamic features include at least one of: dynamic range, peak level, average level; The spatial sound field features include at least one of: stereo width, phase correlation, sound image location.

[0078] In some embodiments, extracting the paragraph signal features is implemented based on a paragraph signal feature extraction module in the intelligent music sound effect processing system.

[0079] The main function of the paragraph signal feature extraction module is to conduct detailed and in-depth multi-dimensional feature analysis on each music segment divided by the music structure analysis module. In the embodiments of the present application, a series of key features are extracted from the two dimensions of music and vocals, aiming to comprehensively analyze the characteristics of the music paragraph, provide accurate data support for subsequent sound effect optimization, make the sound effect processing more in line with the actual content and emotional expression of the music, and improve the infectivity and auditory experience of the music.

[0080] Specifically, the paragraph signal features include music signal features and vocal signal features.

[0081] The music signal features are specifically as follows: Music spectral features: the spectrum of the audio signal is calculated by Fast Fourier Transform (FFT), and the energy distribution of different frequency bands is obtained. Further analysis of spectral centroid (representing the center position of the spectrum, reflecting the light and dark degree of timbre), spectral spread (describing the distribution range of spectral energy, measuring the richness of timbre), harmonic ratio (evaluating the proportion of harmonic components in the audio, related to the purity and pleasantness of the sound), etc.

[0082] Dynamic features: the short-time energy of the audio signal is calculated, and the frequency band dynamic range (the difference between the highest energy and the lowest energy, reflecting the amplitude of the strength change of the music), the peak level (the maximum energy point in the music, determining the peak volume of the audio), the average level (reflecting the overall loudness level of the audio), etc. These features can reflect the intensity change and energy distribution of the music.

[0083] Spatial features: using the left and right channel information of the audio, the stereo width (reflecting the spatial distribution range of the audio in the left and right channels, too narrow width will make the sound thin, too wide width may cause too strong sound separation), phase correlation (measuring the similarity of left and right channel signals, affecting the fusion degree and positioning of the sound), sound image location (determining the position of the sound in the stereo sound field, such as left, right or center, etc.), etc. to provide basis for spatial sound effect processing.

[0084] The vocal signal features are specifically as follows: Spectrum features: Adopting spectrum analysis techniques, calculate the proportion of human voice (the energy proportion of human voice signal in the overall audio, reflecting the prominence of human voice in music), harmonic-to-noise ratio (measuring the proportion of harmonic components and noise components in human voice, related to the clarity and purity of sound), spectral centroid (reflecting the light and dark degree of human voice color), spectral roll-off (describing the trend of spectral energy attenuation with increasing frequency, reflecting the richness of high-frequency components of sound), spectral flatness (evaluating the flatness of spectral energy, related to the timbre characteristics of sound), etc.

[0085] Dynamic features: Analyze the dynamic range of human voice part (reflecting the amplitude of human voice, affecting the emotional expression of sound), envelope average (reflecting the average trend of human voice energy), envelope standard deviation (measuring the dispersion of human voice energy changes, reflecting the stability of sound), envelope maximum (the maximum energy point in human voice, determining the peak intensity of sound), envelope minimum (the minimum energy point in human voice, affecting the delicacy of sound), etc., capturing the dynamics and emotional changes of human voice.

[0086] Pitch features: Using pitch detection algorithms, determine the pitch range (the lowest pitch to the highest pitch of human voice, reflecting the width of the voice range), pitch stability (measuring the stability of pitch changes, related to the accuracy and stability of sound), vibrato depth (reflecting the amplitude of vibrato, embodying the emotional color of sound), vibrato frequency (measuring the frequency of vibrato, related to the expressiveness of sound), etc. Precise description of the pitch characteristics of human voice.

[0087] In the step S102, based on the global semantic features of the target audio and the paragraph signal features of each paragraph, a target prompt text for the target audio is generated; the target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph.

[0088] Here, the feature information is summarized, and the global semantic features and paragraph signal features are integrated. The global semantic features cover the key information of the style, emotion, theme and main instruments of the music; the paragraph signal features include the audio features (such as spectrum, dynamics, spatial features) and voice features (such as spectrum, dynamics, pitch features) of each paragraph, as well as the paragraph division information obtained from the music structure analysis module.

[0089] In some embodiments, the global semantic features and the paragraph signal features are in text format; Based on the global semantic features of the target audio and the paragraph signal features of each paragraph, the target prompt text for the target audio is generated, including: Fill the global semantic features and the paragraph signal features of each paragraph into the pre-defined prompt text template to generate the target prompt text for the target audio.

[0090] According to the summarized feature information, a standardized target prompt text (Prompt) is constructed. For example: "This is a {global feature} song, containing n paragraphs, each paragraph feature is {paragraph 1: feature}, {paragraph 2: feature}…{paragraph N: feature}. The basic sound effect template is set as: {sound effect template}. Considering the above features, please fine-tune the sound effect for each paragraph based on the basic sound effect template to generate sound effect configuration that accurately matches the song style and emotion.

[0091] Here, the target prompt text presents the audio features and sound effect generation requirements in a clear format, with high readability.

[0092] In steps S103 and S104, the target prompt text is input into the sound effect parameter generation model, and the overall style description and paragraph feature description in the target prompt text are processed by the sound effect parameter generation model to determine the paragraph sound effect parameters for each paragraph. The paragraph sound effect parameters for each paragraph are processed by the sound effect parameter generation model to generate target sound effect parameters for the target audio. The target sound effect parameters are used for sound effect generation of the target audio.

[0093] The process of generating sound effect configuration is implemented based on the sound effect Agent module in the intelligent music sound effect processing system.

[0094] Please refer to Figure 4 The target prompt text is input into the sound effect parameter generation model, and the overall style description and paragraph feature description in the target prompt text are processed by the sound effect parameter generation model to determine the paragraph sound effect parameters for each paragraph, including steps S401-S402: S401, input the target prompt text and the target reference information into the sound effect parameter generation model, and adjust the target reference information based on the overall style description in the target prompt text by the sound effect parameter generation model to obtain audio sound effect parameters that match the overall style of the target audio; S402, adjust the sound effect parameters corresponding to each paragraph in the audio sound effect parameters by the sound effect parameter generation model based on the paragraph feature description, to generate paragraph sound effect parameters for each paragraph.

[0095] After receiving the prompt, the sound effect Agent module uses its internal sound effect generation logic and rules to adjust and optimize the basic sound effect template, generating exclusive sound effect configuration for each paragraph of the music, thereby realizing personalized customization of sound effects, making sound effects deeply integrated with music content, and enhancing the infectivity and expressiveness of music.

[0096] The definition of the sound effect expert Agent determines how the large model thinks and reasons based on the input to generate reasonable sound effects. It needs to be designed based on specific needs. An example of the definition is as follows: Role definition, as follows: The sound effect expert Agent is an intelligent sound effect design assistant that can automatically generate or adjust appropriate sound effect configurations based on input audio characteristics and user needs. It simulates the thinking mode of professional audio engineers, combines music theory and audio processing knowledge, and provides users with more personalized and high-quality sound effect experiences.

[0097] Supported effects, which define the process of sound effect processing and the types of available effects. Each effect has a clear function, parameters, and their value ranges.

[0098] For example: (1) The parameters of the Graphic Equalizer include Eqs, which control the volume gain of each frequency band, with a typical range of -12 dB to +12 dB.

[0099] (2) The parameters of Reverb include: 1) Time: Reverb decay time, range 0.1 seconds to 60 seconds, default 2 seconds.

[0100] 2) Size: Simulate space size, range 0 to 100, default 50.

[0101] 3) wl: Reverb signal intensity, range 0% to 100%, default 50%.

[0102] 4) mix: Dry signal to wet signal ratio, range 0% to 100%, default 30%.

[0103] Sound effect input and output format: Specifically, the input and output sound effect formats are in json format, including a set of parameters for the used effects, for example: “Graph EQ” :{“eqs”: [0,3,0,-0.8,1.5,4.8,5.8,3.2,-0.5,2]}, “Reverb”: {“time”: 2, “Size”:40, “wl”:30%, “mix”:10%}。

[0104] Effect combination rules, including: (1) The sequence order needs to be guaranteed: Graphic Equalizer → Dynamic Range Compressor → Reverb → Delay → Stereo Enhancer → Limiter. Follow the professional audio processing flow to ensure sound quality optimization.

[0105] (2) Linkage rule: when reverb is enabled, appropriately lower the compressor threshold to enhance the reverb effect.

[0106] (3) Detect high-energy audio and automatically optimize effecter parameters.

[0107] Sound effect generation logic, specifically, (1) when generating based on audio features, first adjust the sound effect template based on global audio features such as music style, emotion, etc., and then adjust each segment based on segment features to ensure consistency in overall style while considering segment transitions; (2) when adjusting based on user natural language description, generate sound effect adjustment values instead of directly outputting sound effects.

[0108] The target sound effect parameters are a series of numerical values or setting options for generating sound effects, which accurately define various aspects of sound effects, such as equalizer frequency response curve parameters, reverb effect time and decay parameters, compressor threshold and compression ratio parameters, etc., directly determining the characteristics of the final generated sound effects.

[0109] Target sound effect parameters for sound effect generation of the target audio, and the sound effect generation process can be performed in the server or in the client (terminal device).

[0110] For example, the server processes the target audio based on the target sound effect parameters, such as calling high-performance digital signal processing libraries to perform real-time filtering, reverb addition, dynamic range compression, etc. on the audio signal; after processing, the server returns the generated audio file with sound effects to the client, or directly pushes the sound effect data stream to the client for playback.

[0111] After the client obtains the target sound effect parameters locally, it uses its own computing power to perform real-time processing on the target audio through the built-in sound effect generation module or plug-in according to the target sound effect parameters, and the processed audio can be directly played on the client. In addition to the lack of fine sound effect generation in existing intelligent sound effect processing solutions, there are also the following defects: poor user interaction experience, mostly only providing limited preset options for users to choose from, users cannot intuitively express their expectations for sound effects according to their preferences and needs, and it is difficult to obtain music experience that truly meets individual aesthetic and emotional needs.

[0112] Based on this, in some embodiments, the tuning method further includes: In response to receiving the individualized adjustment text for the target audio, inputting the individualized adjustment text into the sound effect parameter generation model; The sound effect adjustment parameter is used to be superimposed with a target sound effect parameter of the target audio to obtain an adjusted target sound effect parameter.

[0113] Here, the personalized adjustment text, i.e., the expectation of the user for the sound effect intuitively expressed according to the user's own preferences and needs, is described in natural language, such as "I want softer background sound" and "make the spatial sense of sound stronger". This expression method conforms to people's daily communication habits, and the user does not need to have professional audio knowledge, which greatly reduces the operation threshold and enables more users to easily participate in sound effect adjustment.

[0114] Moreover, when the user adjusts the parameters one by one through the effectors, it is often local and fine, and the user may need to adjust the parameters of multiple effectors to achieve a certain overall sound effect style. However, in this process, it is difficult to comprehensively grasp the mutual influence between the parameters, and it is easy to adjust one parameter and destroy the effects of other aspects. For example, when the user adjusts the reverb parameter to increase the spatial sense, the user may accidentally affect the clarity of the sound.

[0115] The personalized adjustment described in the embodiments of the present application is based on the personalized adjustment text, and the model can comprehensively consider the coordinated adjustment of multiple sound effect parameters to ensure that the final generated sound effect conforms to the overall style expected by the user, avoiding the uncoordinated problems that may be caused by local adjustment.

[0116] The sound effect adjustment parameter is used to be superimposed with a target sound effect parameter of the target audio. Here, the target sound effect parameter of the target audio can be the target sound effect parameter generated in step S104, and the sound effect adjustment parameter is adjusted again. Or it can be the original target sound effect parameter of the target audio to be adjusted.

[0117] In some embodiments, in the sound tuning method, after the target sound effect parameter for the target audio is generated, the method further includes: The audio file of the target audio and the target sound effect parameter are sent to a terminal device, so that the terminal device applies the paragraph sound effect parameter of the paragraph in the target sound effect parameter to different paragraphs in the audio file when playing the audio file of the target audio.

[0118] On the client side, when a user plays a song, the system will simultaneously issue the audio file of the song and the corresponding target sound effect parameter file, which includes the paragraph sound effect parameter file. During normal playback, the client will automatically apply different sound effect configuration files according to the time points divided by the song paragraphs, and ensure linear connection when the sound effect switches. When the user has a personalized sound effect adjustment requirement, natural language descriptions such as "vocal stronger" "accompaniment stronger" "drums stronger" can be input. These descriptions will be passed to the sound effect expert Agent as input. Agent generates corresponding sound effect adjustment templates according to the user's instructions. The sound effect adjustment template will be superimposed with the paragraph sound effect configuration file issued to obtain the real-time adjusted sound effect configuration. During the superimposition process, the system will ensure that the sound effect parameters are within a reasonable range to avoid sound distortion or unnaturalness. These adjusted sound effect configurations will be applied to the song played by the client in real time, allowing users to obtain personalized sound effect experience.

[0119] The above embodiments mainly describe the implementation of the sound effect processing method on the server side. It should be understood that the method described in the present application can also be deployed on the terminal device side. Another implementation embodiment of the method on the terminal device is introduced below.

[0120] In some embodiments, a sound tuning method is also provided, applied to a terminal device, the method comprising: receiving an audio file of the target audio and target sound effect parameters, and applying the paragraph sound effect parameters of the target sound effect parameters corresponding to different paragraphs in the audio file when playing the audio file of the target audio; wherein the target sound effect parameters are generated based on the following steps: obtaining the global semantic features of the target audio and the paragraph signal features of each paragraph in the target audio; generating a target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph; the target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph; inputting the target prompt text into a sound effect parameter generation model, processing the overall style description and paragraph feature description in the target prompt text through the sound effect parameter generation model, and determining the paragraph sound effect parameters for each paragraph; generating target sound effect parameters for the target audio by processing the paragraph sound effect parameters of each paragraph through the sound effect parameter generation model.

[0121] In some embodiments, the sound tuning method further comprises: In response to receiving the personalized adjustment text for the target audio, the personalized adjustment text is sent to a server to enable the server to generate sound effect adjustment parameters matching the personalized adjustment text through a sound effect parameter generation model, and the sound effect adjustment parameters are sent to the terminal device; In response to receiving the sound effect adjustment parameters, the target sound effect parameters are superimposed to determine superimposed target sound effect parameters. The superimposed target sound effect parameters are applied to different paragraphs in the audio file to play the audio file.

[0122] In some embodiments, the method for tuning the sound includes: In response to receiving the sound effect adjustment parameters, the target sound effect parameters are superimposed based on a pre-set sound effect parameter range to determine superimposed target sound effect parameters.

[0123] Based on the same inventive concept, the embodiments of the present application also provide a tuning device corresponding to the tuning method. Since the tuning device in the embodiments of the present application solves the problem by a similar principle to the above-mentioned tuning method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0124] Please refer to Figure 5 , Figure 5 The result schematic diagram of the tuning device according to the embodiments of the present application is shown, which is applied to a server, as shown in Figure 5 The device includes: The acquisition module 501 is configured to acquire global semantic features of target audio and paragraph signal features of each paragraph in the target audio. The first generation module 502 is configured to generate target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph. The target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph. The determination module 503 is configured to input the target prompt text into a sound effect parameter generation model, process the overall style description and the paragraph feature description in the target prompt text through the sound effect parameter generation model, and determine paragraph sound effect parameters for each paragraph. The second generation module 504 is configured to process the paragraph sound effect parameters of each paragraph through the sound effect parameter generation model to generate target sound effect parameters for the target audio. The target sound effect parameters are used for sound effect generation of the target audio.

[0125] In some embodiments, a tuning device is also provided, which is applied to a terminal device, and the method includes: receive an audio file of the target audio and target sound effect parameters, and apply a paragraph sound effect parameter of the target sound effect parameters corresponding to a different paragraph in the audio file to the paragraph when playing the audio file of the target audio; wherein the target sound effect parameters are generated based on the following steps: obtain a global semantic feature of the target audio and a paragraph signal feature of each paragraph in the target audio; generate a target prompt text for the target audio based on the global semantic feature of the target audio and the paragraph signal feature of each paragraph; the target prompt text includes an overall style description of the target audio and a paragraph feature description of each paragraph; input the target prompt text into a sound effect parameter generation model, process the overall style description and the paragraph feature description in the target prompt text through the sound effect parameter generation model, and determine a paragraph sound effect parameter for each paragraph; generate the target sound effect parameters for the target audio by processing the paragraph sound effect parameter of each paragraph through the sound effect parameter generation model.

[0126] Based on the same inventive concept, the embodiments of the present application also provide an electronic device corresponding to the sound tuning method. Since the principle of solving the problem in the electronic device of the embodiments of the present application is similar to the sound tuning method described above, the implementation of the electronic device can be referred to the implementation of the method, and the repeated parts will not be described again.

[0127] Please refer to Figure 6 , Figure 6 The structure schematic diagram of the electronic device described in the embodiments of the present application is shown; the electronic device comprises a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to execute the steps of the sound tuning method.

[0128] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium corresponding to the sound tuning method. Since the principle of solving the problem in the computer readable storage medium of the embodiments of the present application is similar to the sound tuning method described above, the implementation of the computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described again.

[0129] A computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to execute the steps of the sound tuning method.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the system and the device described above can refer to the corresponding process in the method embodiment, and will not be repeated in the present application. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other means. The above-described device embodiments are only schematic, for example, the division of the modules is only a logical function division, and the actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual elements can be indirect coupling or communication connection through some communication interface, device or module, which can be electrical, mechanical or other forms.

[0131] The modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical units, i.e. can be located in one place or distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0132] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0133] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk and various program code storage media.

[0134] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A tuning method characterized by, The method applied to a server comprises: obtaining global semantic features of target audio and paragraph signal features of each paragraph in the target audio; generating target prompt text for the target audio based on the global semantic features of the target audio and the paragraph signal features of each paragraph; the target prompt text comprises overall style description of the target audio and paragraph feature description of each paragraph; inputting the target prompt text into an audio effect parameter generation model, processing the overall style description and the paragraph feature description in the target prompt text by the audio effect parameter generation model, and determining paragraph audio effect parameters for each paragraph; processing the paragraph audio effect parameters of each paragraph by the audio effect parameter generation model to generate target audio effect parameters for the target audio, which are used for audio effect generation of the target audio.

2. The tuning method of claim 1, wherein, The global semantic features are used to determine target reference information for audio effect generation of the target audio; inputting the target prompt text and the target reference information into the audio effect parameter generation model, adjusting the target reference information based on the overall style description in the target prompt text by the audio effect parameter generation model, and obtaining audio effect parameters matching the overall style of the target audio; adjusting the audio effect parameters corresponding to each paragraph in the audio effect parameters by the audio effect parameter generation model based on the paragraph feature description to generate paragraph audio effect parameters for each paragraph. Before inputting the target prompt text into the audio effect parameter generation model, the method further comprises:

3. The tuning method of claim 2, wherein, determining a target basic audio effect template matching the target audio as target reference information from a plurality of preset basic audio effect templates based on the global semantic features of the target audio; the target basic audio effect template comprises configuration parameters of a plurality of effectors. Obtaining global semantic features of target audio comprises:

4. The tuning method of claim 1, wherein, obtaining audio data and lyrics text data of the target audio; inputting the audio data and the lyrics text data into a trained multi-modal large language model; processing the input audio data and lyrics text data by the multi-modal large language model to generate global semantic features of the target audio; the global semantic features comprise at least one of music style, emotional expression, theme content and main musical instruments. Processing the input audio data and lyrics text data by the multi-modal large language model to generate global semantic features of the target audio comprises:

5. The tuning method of claim 4, wherein, processing the input audio data and lyrics text data by the multi-modal large language model, and the multi-modal large language model generates the global semantic features of the target audio based on pre-constructed first prompt information; the first prompt information is used to set an analysis role for the multi-modal large language model and set a first output format of the global semantic features. Obtaining paragraph signal features of each paragraph in the target audio comprises:

6. The tuning method of claim 1, wherein, ​ extracting a music signal feature and a vocal signal feature of each paragraph in the target audio; generating a paragraph signal feature in a text format based on the music signal feature and the vocal signal feature of each paragraph in the target audio; the music signal feature comprises at least one of a music spectrum feature, a dynamic feature and a spatial sound field feature; the vocal signal feature comprises at least one of a vocal spectrum feature, a vocal dynamic feature and a vocal high pitch feature.

7. The tuning method of claim 6, wherein, the spectrum feature comprises at least one of a spectrum centroid, a spectrum spread, and a harmonic proportion; the dynamic feature comprises at least one of a dynamic range, a peak level, and an average level; the spatial sound field feature comprises at least one of a stereo width, a phase correlation, and a sound image position.

8. A tuning device, characterized by The apparatus is applied to a server and comprises: an acquisition module configured to acquire a global semantic feature of a target audio and a paragraph signal feature of each paragraph in the target audio; a first generation module configured to generate a target prompt text for the target audio based on the global semantic feature of the target audio and the paragraph signal feature of each paragraph; the target prompt text comprises an overall style description of the target audio and a paragraph feature description of each paragraph; a determination module configured to input the target prompt text into an audio effect parameter generation model, process the overall style description and the paragraph feature description in the target prompt text by the audio effect parameter generation model, and determine a paragraph audio effect parameter for each paragraph; a second generation module configured to process the paragraph audio effect parameter of each paragraph by the audio effect parameter generation model, and generate a target audio effect parameter for the target audio; the target audio effect parameter is used for audio effect generation of the target audio.

9. An electronic device, comprising: comprise: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to perform the steps of the tuning method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to perform the steps of the tuning method of any one of claims 1 to 7.

Images