Sound effect determination method, apparatus, device, and storage medium

Abstract

The application provides an audio effect determination method, device and equipment and a storage medium. The method comprises the following steps: determining at least one music feature vector of each music in a music library; determining an audio effect recommendation template of each music according to an audio effect template library and the at least one music feature vector of each music; determining a mapping relationship between a music identifier, an audio file and the audio effect recommendation template of each music according to each music and the audio effect recommendation template of each music; in response to a music data acquisition request sent by a client, determining a target audio file and a target audio effect recommendation template based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template, and sending the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template. The application can intelligently and accurately recommend an audio effect template to a user, thereby meeting the personalized processing demand of the user for music.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a sound effect determination method and device, equipment and storage medium. BACKGROUND

[0002] At present, different sound effect processors (also referred to as effectors) such as distortion effectors, delay effectors, phase shift effectors and compression effectors are set in a music player to change the playing effect of music, so that the same music can be played with different effects and different auditory experiences are brought to users. Since each effector has a set of effector parameters such as equalizer parameters, reverb parameters and harmony parameters, professional musicians such as sound engineers can form an effector group by combining different effectors according to the desired music playing effect, and combine the effector parameters of each effector in each effector group to obtain various sound effect templates. Then, users can add different sound effect templates to music to perform different sound effect processing on music.

[0003] The implementation manners of the user adding a sound effect template to music include the following manners: first, the user manually selects a target sound effect template from a plurality of sound effect templates provided by the music player. Second, the music player obtains the most commonly used sound effect template as the target sound effect template according to historical data of the public user using the sound effect template. Third, the target template is determined from a mapping relationship between music types and sound effect templates according to the music type of the currently played music by the user.

[0004] However, the above-mentioned manners have the following problems: for the first manner, when the user's perception of music is not strong enough, it is difficult to accurately select a suitable target sound effect template from a large number of sound effect templates, and the user needs to constantly switch sound effect templates, which is very cumbersome, thereby affecting the user experience. For the second manner, when the music played by the user is some niche music or unpopular music, the processing effect of adding the most commonly used sound effect template as a general template to the music is poor, and the personalized processing demand of the user for music cannot be met. For the third manner, because the mapping relationship between music types and sound effect templates is artificially customized, the mapping granularity is usually large, and it cannot be guaranteed that each music under the same type is suitable for the same sound effect template, resulting in poor sound effect processing effect on music. SUMMARY

[0005] The present application provides a sound effect determination method, device, equipment and storage medium, which can intelligently recommend a sound effect template to a user, improve the recommendation accuracy of the sound effect template, save the time spent in determining the sound effect template, simplify the operation of determining the sound effect template, meet the personalized processing demand of the user for music, and improve the user's listening experience.

[0006] In a first aspect, the present application provides a sound effect determination method applied to a server, the method comprising:

[0007] determining at least one music feature vector of each music in a music library;

[0008] determining a sound effect recommendation template of each music according to the sound effect template library and the at least one music feature vector of each music;

[0009] determining a mapping relationship between a music identifier, an audio file and a sound effect recommendation template according to each music and the sound effect recommendation template of each music;

[0010] in response to a music data acquisition request sent by a client, determining a target audio file and a target sound effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file and the sound effect recommendation template, and sending the target audio file and the target sound effect recommendation template to the client to instruct the client to determine a final sound effect template of the target audio file according to the target sound effect recommendation template.

[0011] In some optional implementations, the determining of the at least one music feature vector of each music in the music library comprises:

[0012] inputting an audio file of each music in the music library into a feature extraction model, performing segmentation processing on each audio file according to a preset segmentation rule through the feature extraction model, and obtaining at least one audio segment;

[0013] performing feature extraction on each audio segment of each audio file through the feature extraction model, and obtaining the at least one music feature vector of each music in the music library.

[0014] In some optional implementations, before the determining of the at least one music feature vector of each music in the music library, the method further comprises:

[0015] obtaining a training data set, the training data set comprising at least two training audio files;

[0016] selecting a first training audio file, and performing segmentation processing on the first training audio file according to the preset segmentation rule to obtain a target audio segment and a positive sample, the positive sample and the target audio segment being different audio segments in the first training audio file;

[0017] selecting a second training audio file, and performing segmentation processing on the second training audio file according to the preset segmentation rule to obtain a negative sample;

[0018] According to the target audio segment, the positive sample and the negative sample, a deep learning network model is trained to obtain the feature extraction model.

[0019] In some optional implementations, the sound effect template library includes at least two seed music feature vectors, and each seed music feature vector corresponds to a sound effect template.

[0020] Correspondingly, the determining of the sound effect recommendation template of each music according to the sound effect template library and the at least one music feature vector of each music includes:

[0021] determining the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library;

[0022] determining the sound effect recommendation template of each music according to the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library.

[0023] In some optional implementations, the determining of the sound effect recommendation template of each music according to the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library includes:

[0024] if the number of music feature vectors of each music is one, determining the sound effect recommendation template of each music according to the seed music feature vector corresponding to the highest similarity in all similarities;

[0025] if the number of music feature vectors of each music is at least two, selecting a preset number of target similarities from all similarities in descending order of similarity, and determining the sound effect recommendation template of each music according to the seed music feature vector corresponding to the target similarity.

[0026] In some optional implementations, the determining of the sound effect recommendation template of each music according to the seed music feature vector corresponding to the highest similarity in all similarities includes:

[0027] determining the sound effect template corresponding to the seed music feature vector as the sound effect recommendation template of each music, wherein the seed music feature vector is the seed music feature vector corresponding to the highest similarity.

[0028] In some optional implementations, the determining of the sound effect recommendation template of each music according to the seed music feature vector corresponding to the target similarity includes:

[0029] determine an audio effect template corresponding to each seed music feature vector, the seed music feature vector being a seed music feature vector corresponding to the target similarity;

[0030] fuse each audio effect template and the target similarity to obtain an audio effect recommendation template of each music.

[0031] In some optional implementations, the audio effect template library is constructed by the following steps, comprising:

[0032] determine a seed music set of each audio effect template, wherein each seed music set comprises at least one seed music;

[0033] extract a music feature vector of each audio file in each seed music set by the feature extraction model to obtain a music feature vector set of each audio effect template, wherein each audio file corresponds to a seed music, and the number of music feature vectors of each audio file is at least one;

[0034] cluster the music feature vectors in the music feature vector set of each audio effect template to obtain a seed music feature set, wherein the seed music feature set comprises at least one seed music feature vector;

[0035] construct the audio effect template library according to the seed music feature set of each audio effect template.

[0036] In some optional implementations, the determination of the seed music set of each audio effect template comprises:

[0037] determine at least one effective audio effect template according to a playback time length proportion of each audio effect template in a first music, the first music being a historical playback music added with the audio effect template;

[0038] determine a seed music of each effective audio effect template according to a number of times a second music is effectively played by a user, the second music being a historical playback music added with the effective audio effect template;

[0039] obtain the seed music set of each audio effect template according to all seed musics of each effective audio effect template.

[0040] In some optional implementations, the determination of the at least one effective audio effect template according to the playback time length proportion of each audio effect template in the first music comprises:

[0041] in response to a playback time length proportion of any audio effect template in the first music being greater than a proportion threshold value, determine the audio effect template as an effective audio effect template.

[0042] In some optional implementations, the determining of the seed music for each of the effective sound effect templates according to the number of times that the second music is effectively played by the user comprises:

[0043] In response to the number of times that any of the second music is played by the user being greater than the number threshold, determining the second music as the seed music for each of the effective sound effect templates.

[0044] In a second aspect, the present application provides a sound effect determination method applied to a client, the method comprising:

[0045] In response to a music playing request, obtaining a to-be-played music identifier corresponding to the music playing request;

[0046] generating a music data acquisition request according to the to-be-played music identifier, and sending the music data acquisition request to a server to instruct the server to return a target sound effect recommendation template and a target audio file corresponding to the to-be-played music identifier;

[0047] determining a final sound effect template of the target audio file according to the target sound effect recommendation template and a user sound effect template, the user sound effect template being a most commonly used sound effect template determined based on user historical use data.

[0048] In some optional implementations, the determining of the final sound effect template of the target audio file according to the target sound effect recommendation template and the user sound effect template comprises:

[0049] fusing the target sound effect recommendation template and the user sound effect template to obtain the final sound effect template of the target audio file.

[0050] In a third aspect, the present application provides a sound effect recommendation device configured in a server, comprising:

[0051] a vector determination module configured to determine at least one music representation vector of each music in a music library;

[0052] a template determination module configured to determine a sound effect recommendation template of each of the music according to a sound effect template library and the at least one music representation vector of each of the music;

[0053] a relationship determination module configured to determine a mapping relationship among a music identifier, an audio file and a sound effect recommendation template according to each of the music and the sound effect recommendation template of each of the music;

[0054] The request response module is configured to, in response to a music data acquisition request sent by a client, determine a target audio file and a target sound effect recommendation template corresponding to the music data acquisition request based on a mapping relationship between the music identifier, the audio file, and the sound effect recommendation template, and send the target audio file and the target sound effect recommendation template to the client to instruct the client to determine a final sound effect template of the target audio file according to the target sound effect recommendation template.

[0055] In some optional implementations, the vector determination module comprises:

[0056] The segmentation unit is configured to input an audio file of each music in the music library into a feature extraction model, perform segmentation processing on each audio file according to a preset segmentation rule through the feature extraction model, and obtain at least one audio segment.

[0057] The feature extraction unit is configured to perform feature extraction on each audio segment of each audio file through the feature extraction model, and obtain at least one music feature vector of each music in the music library.

[0058] In some optional implementations, the method further comprises:

[0059] The data acquisition module is configured to acquire a training data set, wherein the training data set comprises at least two training audio files.

[0060] The first processing module is configured to select a first training audio file, perform segmentation processing on the first training audio file according to the preset segmentation rule, and obtain a target audio segment and a positive sample, wherein the positive sample and the target audio segment are different audio segments in the first training audio file.

[0061] The second processing module is configured to select a second training audio file, perform segmentation processing on the second training audio file according to the preset segmentation rule, and obtain a negative sample.

[0062] The model training module is configured to train a deep learning network model according to the target audio segment, the positive sample, and the negative sample, and obtain the feature extraction model.

[0063] In some optional implementations, the sound effect template library comprises at least two seed music feature vectors, and each seed music feature vector corresponds to a sound effect template.

[0064] Correspondingly, the template determination module comprises:

[0065] The first determination unit is configured to determine a similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library.

[0066] a second determining unit, configured to determine a sound effect recommendation template of each of the musics according to similarities between each of music feature vectors of each of the musics and each of the seed music feature vectors in the sound effect template library.

[0067] In some optional implementations, the second determining unit is specifically configured to:

[0068] if the number of the music feature vectors of each of the musics is one, determine the sound effect recommendation template of each of the musics according to a seed music feature vector corresponding to a highest similarity in all the similarities;

[0069] if the number of the music feature vectors of each of the musics is at least two, select a preset number of target similarities from all the similarities in a descending order of the similarities, and determine the sound effect recommendation template of each of the musics according to seed music feature vectors corresponding to the target similarities.

[0070] In some optional implementations, the second determining unit is further configured to:

[0071] determine a sound effect template corresponding to the seed music feature vector corresponding to the highest similarity as the sound effect recommendation template of each of the musics.

[0072] In some optional implementations, the second determining unit is further configured to:

[0073] determine a sound effect template corresponding to each of the seed music feature vectors corresponding to the target similarities;

[0074] fuse each of the sound effect templates and the target similarities to obtain the sound effect recommendation template of each of the musics.

[0075] In some optional implementations, the method further includes:

[0076] a template library construction module, configured to determine a seed music set of each sound effect template, wherein each seed music set comprises at least one seed music; extract a music feature vector of each audio file in each seed music set by using the feature extraction model, to obtain a music feature vector set of each sound effect template, wherein each audio file corresponds to one seed music, and the number of music feature vectors of each audio file is at least one; cluster the music feature vectors in the music feature vector set of each sound effect template to obtain a seed music feature set, wherein the seed music feature set comprises at least one seed music feature vector; and construct the sound effect template library according to the seed music feature set of each sound effect template.

[0077] In some optional implementations, the template library construction module comprises:

[0078] a template determination unit, configured to determine at least one valid sound effect template according to a playback time proportion of each sound effect template in first music, wherein the first music is historical playback music to which the sound effect template is added;

[0079] a music determination unit, configured to determine a seed music of each valid sound effect template according to a number of times that second music is effectively played by a user, wherein the second music is historical playback music to which the valid sound effect template is added;

[0080] a set acquisition unit, configured to obtain a seed music set of each sound effect template according to all seed musics of each valid sound effect template.

[0081] In some optional implementations, the template determination unit is specifically configured to:

[0082] in response to a playback time proportion of any sound effect template in the first music being greater than a proportion threshold, determine the sound effect template as a valid sound effect template.

[0083] In some optional implementations, the music determination unit is specifically configured to:

[0084] in response to a number of times that any second music is played by a user being greater than a number threshold, determine the second music as a seed music of each valid sound effect template.

[0085] In a fourth aspect, the present application provides an audio effect recommendation device configured in a client, comprising:

[0086] an identifier acquisition module, configured to acquire a to-be-played music identifier corresponding to a music playing request in response to the music playing request;

[0087] The request sending module is configured to generate a music data acquisition request according to the to-be-played music identifier, and send the music data acquisition request to a server to instruct the server to return a target sound effect recommendation template and a target audio file corresponding to the to-be-played music identifier.

[0088] The sound effect determination module is configured to determine a final sound effect template of the target audio file according to the target sound effect recommendation template and a user sound effect template, the user sound effect template being a most frequently used sound effect template determined based on user historical use data.

[0089] In some optional implementations, the sound effect determination module is specifically configured to:

[0090] fuse the target sound effect recommendation template and the user sound effect template to obtain the final sound effect template of the target audio file.

[0091] In a fifth aspect, the present application provides an electronic device, comprising:

[0092] a processor and a memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program stored in the memory to execute the sound effect determination method according to the first aspect or the second aspect.

[0093] In a sixth aspect, the present application provides a computer readable storage medium configured to store a computer program, the computer program causing a computer to execute the sound effect determination method according to the first aspect or the second aspect.

[0094] In a seventh aspect, the present application provides a computer program product containing program instructions, when the program instructions are run on an electronic device, causing the electronic device to execute the sound effect determination method according to the first aspect or the second aspect.

[0095] The technical solutions disclosed in the embodiments of the present application have at least the following beneficial effects:

[0096] The present application determines the sound effect recommendation template corresponding to each music according to the music characteristic vector of each music, so that the user can determine the corresponding sound effect recommendation template according to the music characteristic vector of the music when playing the music, and then can determine the final sound effect template according to the sound effect recommendation template. Therefore, the sound effect template can be recommended intelligently, and the recommendation accuracy of the sound effect template can be improved, so as to save the time spent on determining the sound effect template, simplify the operation of determining the sound effect template, meet the personalized processing needs of the user for the music, and improve the user's listening experience. BRIEF DESCRIPTION OF DRAWINGS

[0097] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0098] Figure 1 is a flowchart of an audio effect determination method provided by an embodiment of the present application;

[0099] Figure 2 is a flowchart of a representation extraction model training method provided by an embodiment of the present application;

[0100] Figure 3 is a schematic diagram of obtaining a target audio segment and a positive sample from a first training audio file provided by an embodiment of the present application;

[0101] Figure 4 is a schematic diagram of obtaining a negative sample from a second training audio file provided by an embodiment of the present application;

[0102] Figure 5 is a schematic diagram of a representation extraction model training method provided by an embodiment of the present application;

[0103] Figure 6 is a flowchart of another audio effect determination method provided by an embodiment of the present application;

[0104] Figure 7 is a schematic diagram of constructing an audio effect template library provided by an embodiment of the present application;

[0105] Figure 8 is a flowchart of still another audio effect determination method provided by an embodiment of the present application;

[0106] Figure 9 is a flowchart of yet another audio effect determination method provided by an embodiment of the present application;

[0107] Figure 10 is a flowchart of still another audio effect determination method provided by an embodiment of the present application;

[0108] Figure 11 is a schematic block diagram of an audio effect determination apparatus provided by an embodiment of the present application;

[0109] Figure 12 is a schematic block diagram of another audio effect determination apparatus provided by an embodiment of the present application;

[0110] Figure 13 is a schematic block diagram of an electronic device provided by an embodiment of the present application;

[0111] Figure 14is a schematic block diagram of a computer readable storage medium provided by an embodiment of the present application. DETAILED DESCRIPTION

[0112] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0113] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0114] At present, when a user performs sound effect processing on music through a sound effect template provided by a music player, the user usually manually selects a target sound effect template from multiple sound effect templates, or uses a sound effect template most commonly used by the public as the target sound effect template, or the music player determines the target sound effect template from a mapping relationship between music types and sound effect templates according to the type of music currently played by the user. However, for the manual selection of the target sound effect template, the user needs to constantly switch the sound effect templates, which is tedious and time-consuming, and affects the user experience; for the use of the sound effect template most commonly used by the public as the target sound effect template, when the played music is a niche music or a cold music, using the most commonly used sound effect template to process the above music will have the problem of poor processing effect, which cannot meet the user's personalized processing demand for music; for the determination of the target sound effect template based on the mapping relationship between the music type and the sound effect template, because the mapping granularity is large, it cannot guarantee that each music under the same type is suitable for the same sound effect template, resulting in the problem of poor sound effect processing effect on the music. Therefore, the present application provides a sound effect determination scheme to realize intelligent and accurate recommendation of a sound effect template to a user through the scheme, so as to simplify the determination operation of the sound effect template, meet the user's personalized processing demand for music, and thus improve the user's listening experience.

[0115] Before the embodiments of the present application are described in detail, first, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations:

[0116] Effectors: are a series of measures to change the waveform of the original sound, modulate or delay the phase of the sound wave, enhance the harmonic components of the sound wave, and produce various special sound effects. In the present application, effectors refer to relevant effectors simulated by algorithms, and the audio files are processed according to the algorithm input parameters to obtain the audio files after adding effectors.

[0117] Sound effect templates: are obtained by combining the effect parameters corresponding to each effector in the defined effector group, and are used for different sound effect processing of audio.

[0118] After introducing some concepts involved in the embodiments of the present application, the sound effect determination method, device, equipment and storage medium provided by the embodiments of the present application will be described in detail in combination with the drawings.

[0119] Figure 1 is a flowchart of a sound effect determination method provided by the embodiments of the present application. The embodiments of the present application can be applied to the scene of intelligently recommending sound effect templates to users, and the sound effect determination method can be executed by a sound effect determination device. Moreover, the sound effect determination device can be composed of hardware and / or software, and can be integrated in an electronic device. Preferably, in the embodiments of the present application, the electronic device is a server.

[0120] As shown in Figure 1 , the method can include the following steps:

[0121] S101, determining at least one music representation vector of each music in a music library.

[0122] In the embodiments of the present application, the music library can include various types of music resources to meet the music listening needs of different users.

[0123] The types of the above-mentioned music resources can include but are not limited to rock, pop, jazz, folk, original music, cover and light music, etc.

[0124] Among them, the music representation vector is used to represent the basic characteristics of the music.

[0125] It should be understood that the at least one music representation vector represents one or more music representation vectors.

[0126] Considering that each music has unique basic characteristics, the application can determine the sound effect template corresponding to each music according to the basic characteristics of each music, so as to ensure that a matching sound effect recommendation template can be screened for each music, thereby providing favorable conditions for improving the user's listening experience.

[0127] Optionally, the application determines at least one music feature vector of each music in the music library, which can include but is not limited to the following implementation manners:

[0128] The first implementation manner

[0129] The audio file of each music in the music library is input as input data into the feature extraction model for extracting the music feature vector, so as to process each input audio file through the feature extraction model to obtain at least one music feature vector of the corresponding music of each audio file.

[0130] It should be understood that the above-mentioned feature extraction model specifically converts the input audio signal into a fixed dimension vector that can represent the music feature. Wherein, the fixed dimension can also be understood as the fixed length.

[0131] The above-mentioned feature extraction model can be an existing network model such as VGGish model; or it can also be a specially trained network model, which can be a deep learning model or a ResNet residual network model, etc., and the application does not make any limitation on this.

[0132] Wherein, the specific training process of the specially trained feature extraction model will be specifically described in the following embodiments, and no more details will be described here.

[0133] The second implementation manner

[0134] Using a sound feature extraction method, at least one music feature vector of each music is extracted from the audio file of each music in the music library.

[0135] The above-mentioned sound feature extraction method usually extracts a multi-dimensional feature vector for each sound signal.

[0136] Considering that the essence of music is composed of multiple sounds, from a certain angle, music includes sound, so at least one music feature vector of the audio file corresponding to each music in the music library can be extracted through the sound feature extraction method.

[0137] Specifically, before extracting the at least one music representation vector of each music, the audio file of each music can be optionally pre-processed, such as analog-digital conversion, pre-emphasis and windowing. Then, the music representation vector of each music is extracted by using the sound feature extraction method. In this way, the pre-processed audio file can better meet the actual processing needs of the sound feature extraction method, laying a foundation for improving the processing accuracy.

[0138] It should be understood that, considering that sound is an analog signal, analog-digital conversion is to convert an analog signal into a digital signal, including two steps: sampling and quantization, i.e. converting a continuous sound waveform into discrete data points with a certain sampling rate and sampling bit number.

[0139] For the spectrum of a sound signal, the energy of the low-frequency part is usually higher than that of the high-frequency part, and the spectral energy decays by 20 decibels (dB) every 10 hertz (Hz), and the influence of the circuit background noise when the microphone collects the sound signal also increases the energy of the low-frequency part. Therefore, in order to make the energy of the high-frequency part and the energy of the low-frequency part have similar amplitudes, it is necessary to pre-emphasize the energy of the high-frequency part, i.e. pre-emphasize the energy of the high-frequency part, so that the acoustic model can better utilize the high-frequency formant, thereby improving the recognition accuracy. Specifically, the pre-emphasis processing can be realized by a high-pass filter, such as a FIR filter, etc. The FIR filter can usually be a first-order finite impulse response filter.

[0140] In addition, since sound is generally a non-stationary signal, its statistical performance is not fixed, but within a relatively short period of time, the signal can be considered to be stationary, which is windowing. The window is described by three parameters: window length (in milliseconds), offset and shape. Each windowed sound signal is called a frame, the number of milliseconds of each frame is called the frame length, and the distance between the left boundaries of adjacent two frames is called the frame shift.

[0141] In some optional implementations, the sound feature extraction method can be, but is not limited to: Mel-Frequency Cepstral Coefficients (MFCC), Linear Prediction Cepstrum Coefficient (LPCC), Multimedia Content Description Interface MPEG7, etc.

[0142] Among them, because the specific implementation process of extracting the music representation vector of each music by using the sound feature extraction method is the same as or similar to extracting the sound feature from the sound, the specific process of extracting the music representation vector of music by using the sound feature extraction method can refer to the existing sound feature extraction process, and the present application will not make too much repetition.

[0143] S102, determining an audio effect recommendation template of each music according to the audio effect template library and the at least one music feature vector of each music.

[0144] S103, determining a mapping relationship between the music identifier, the audio file and the audio effect recommendation template according to each music and the audio effect recommendation template of each music.

[0145] The audio effect template library is used to store various audio effect templates modulated by professional musicians and a seed music feature set corresponding to each audio effect template. It should be understood that each seed music feature set includes at least one seed music feature vector.

[0146] Considering that the audio effect template library includes multiple audio effect templates and at least one seed music feature vector corresponding to each audio effect template, the application can optionally determine the matching degree between each music feature vector of each music and each seed music feature vector, and then determine the audio effect recommendation template corresponding to each music from the audio effect template library according to the matching degree.

[0147] The above determination of the matching degree between each music feature vector of each music and each seed music feature vector can be understood as the determination of the similarity between each music feature vector of each music and each seed music feature vector. When the determined similarity is higher, it means that the music feature vector of the music is more matched with the seed music feature vector, and vice versa.

[0148] In some optional implementations, the similarity calculation method such as the cosine distance calculation method, the Euclidean distance calculation method or other methods can be used to calculate the similarity between each music feature vector of each music and each seed music feature vector.

[0149] For example, assuming that the similarity calculation method is the cosine distance calculation method, when the cosine similarity between the pth music feature vector of the kth music and the qth seed music feature vector is calculated according to the cosine distance calculation method, it can be shown as the following formula (1):

[0150]

[0151] Wherein, cos(θ) represents the cosine similarity, represents the pth music feature vector of the kth music, represents the qth seed music feature vector, represents the component of the pth music feature vector, and q rrepresents a component of the qth seed music representation vector, r represents the rth length in the pth music representation vector or the qth seed music representation vector, n represents the nth length in the pth music representation vector or the qth seed music representation vector, and n is an integer greater than or equal to 1.

[0152] After calculating the similarity between each music representation vector of each music and each seed music representation vector, in some optional implementations, the application can filter the highest similarity from all the similarities corresponding to each music, and determine the seed music representation vector corresponding to the highest similarity, and then determine the sound effect template corresponding to the determined seed music representation vector as the sound effect recommendation template of each music.

[0153] In some optional implementations, the application can also filter a preset number of highest similarities from all the similarities corresponding to each music, and determine the seed music representation vector corresponding to each highest similarity, and the sound effect template corresponding to each seed music representation vector. Then, all the determined sound effect templates are fused, and the fused sound effect template is determined as the sound effect recommendation template of each music.

[0154] After determining the sound effect recommendation template corresponding to each music in the music library, the optional application can establish a mapping relationship between the music identifier of each music and the sound effect recommendation template, so that subsequent music data acquisition requests sent by the client can quickly determine the corresponding sound effect recommendation template according to the above mapping relationship.

[0155] Considering that the playing of music depends on the audio file, after the application determines the sound effect recommendation template corresponding to each music in the music library, the optional application can establish a mapping relationship between the music identifier, the audio file and the sound effect recommendation template, so that the server can determine the audio file corresponding to the to-be-played music identifier and the sound effect recommendation template through the mapping relationship between the music identifier, the audio file and the sound effect recommendation template based on the to-be-played music identifier sent by the client, thereby providing a basis for further improving the music resource acquisition speed. The music identifier is used to uniquely identify the corresponding music, such as the music name.

[0156] In some optional implementations, since an index is a separate, physical storage structure for sorting the values of one or more columns in a database table in order to speed up the retrieval of physical data. Therefore, after the application determines the sound effect recommendation template corresponding to each music in the music library, the optional application can also create an index based on the music identifier, the audio file and the sound effect recommendation template, so as to improve the retrieval speed of the client when acquiring music resources subsequently through the index.

[0157] S103, in response to the music data acquisition request sent by the client, determining the target audio file and the target sound effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file and the sound effect recommendation template, and sending the target audio file and the target sound effect recommendation template to the client to instruct the client to determine the final sound effect template of the target audio file according to the target sound effect recommendation template.

[0158] In the embodiment of the application, the client refers to a device used by a user when playing music. The music player for playing music can be installed in the client, so that the user can play music through the music player when the user needs to play music.

[0159] Optionally, when the client detects a music playing operation triggered by the user, such as detecting that the user clicks a play control associated with any song, the client first acquires a to-be-played music identifier corresponding to the music playing operation. The to-be-played music identifier is used to uniquely identify the corresponding to-be-played music, such as the to-be-played music name. Then, the client generates a music data acquisition request according to the acquired to-be-played music identifier, and sends the music data acquisition request to the server with music resources, so that the server feeds back the target audio file and the target sound effect recommendation template corresponding to the music data acquisition request. The to-be-played music identifier is carried in the music data acquisition request.

[0160] In some optional implementation manners, after the server acquires the music data acquisition request sent by the client, the server first parses the music data acquisition request to acquire the to-be-played music identifier carried in the data acquisition request. Then, according to the acquired to-be-played music identifier, the target audio file and the target sound effect recommendation template corresponding to the to-be-played music identifier are determined from the mapping relationship between the music identifier, the audio file and the sound effect recommendation template established in advance, or from the index established in advance. Then, the target audio file and the target sound effect recommendation template are sent to the client, so that the client determines the final sound effect template of the target audio file according to the target sound effect recommendation template, and can perform sound effect processing on the target audio file according to the final sound effect template.

[0161] It is considered that the music in the music library in the present application can be updated, such as adding new music, etc. Therefore, the present application can periodically detect whether the music in the music library is updated, if it is determined that new music is added to the music library, at least one music feature vector of the new music is determined, and the sound effect recommendation template of the new music is determined according to the sound effect template library and the at least one music feature vector of the new music. Then, based on the new music and the sound effect recommendation template corresponding to the new music, the mapping relationship between the music identifier, the audio file and the sound effect recommendation template is updated, so that the server can always provide the required music data to the client, wherein the music data is specifically the target sound effect recommendation template and the target audio file, thereby meeting the personalized playback demand of the user for any music such as the latest music, and further improving the user experience.

[0162] The above-mentioned periodic detection of whether the music in the music library is updated can be, but is not limited to, periodic detection of whether there is new content in the music directory in the music library. If there is new content, it is determined that there is new music in the music library, otherwise it is determined that there is no new music in the music library. Wherein, the periodicity can be understood as a preset time interval, and the preset time interval can be flexibly set according to the detection requirement, for example, if it is desired to determine the existence of new music in the music library in the first time, the preset time interval can be set to be shorter, such as 24 hours detection once, or 48 hours detection once. For example, if the real-time requirement of detecting new music is not high, the preset time interval can be set to be longer, such as detection once a week, or detection once a month, etc.

[0163] The technical scheme provided in the application determines at least one music feature vector of each music in a music library, determines an audio effect recommendation template of each music according to an audio effect template library and the music feature vector of each music, determines a mapping relationship between a music identifier, an audio file and the audio effect recommendation template according to the music without the music and the audio effect recommendation template of each music, and then determines a target audio file and a target audio effect recommendation template corresponding to a music data acquisition request based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template in response to a music data acquisition request sent by a client, and sends the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template. The application determines the audio effect recommendation template corresponding to each music according to the music feature vector of each music, so that the user can determine the corresponding audio effect recommendation template according to the music feature vector of the music when playing the music, and then can determine the final audio effect template according to the audio effect recommendation template, thereby realizing intelligent recommendation of the audio effect template to the user, improving the recommendation accuracy of the audio effect template, saving the time spent on determining the audio effect template, simplifying the operation of determining the audio effect template, meeting the personalized processing needs of the user for the music and improving the user's listening experience.

[0164] The training process of the feature extraction model in the embodiment of the application will be described in detail below. Figure 2 As shown in FIG. 1, the training process of the feature extraction model includes the following steps: Figure 2

[0165] S201, obtaining a training data set, the training data set including at least two training audio files.

[0166] In order to ensure that the trained feature extraction model can fully recognize more music variations and extract reliable music feature vectors, the training data set is obtained according to the principle that the larger the scale is and the more the music types are.

[0167] Optionally, when obtaining the training data set, different audio files provided by different music libraries of different companies can be collected together to obtain the training data set, or a public music data set with a large amount of music data can be used as the training data set, and the application does not limit the obtaining of the training data set.

[0168] The public music data set with a large amount of music data can be selected as an AudioSet. The AudioSet includes about 1 million music segment data.

[0169] ​S202, Select the first training audio file and segment the first training audio file according to the preset segmentation rules to obtain the target audio segment and positive samples. The positive samples and the target audio segment are different audio segments in the first training audio file.

[0170] In this application, the preset segmentation rule can be any method or strategy that can divide the audio file into different segments, and this application does not impose any restrictions on it. For example, the preset segmentation rule can be to extract an audio segment of 10 seconds (s) from the audio file, or it can be to segment the audio file every 5 seconds (s), etc.

[0171] In some optional embodiments, any training audio file can be selected from the acquired training dataset as the first training audio file. Then, two audio segments of the same length but different positions are extracted from the first training audio file according to a preset segmentation rule, and one of the audio segments is used as the target audio segment, while the other audio segment is used as a positive sample.

[0172] For example, such as Figure 3 As shown, assuming the preset segmentation rule is to extract a 10-second audio segment from an audio file, and the first training audio file is audio file aa, then an audio segment of 0-10 seconds can be extracted from audio file aa as the target audio segment. Then, from the remaining audio frequency bands in audio file aa besides the target audio segment, another 10-second audio segment can be extracted, such as an audio segment of 15-25 seconds, as a positive sample.

[0173] S203, Select the second training audio file and segment it according to the preset segmentation rules to obtain negative samples.

[0174] It should be understood that the preset segmentation rule in step S203 is the same as the preset segmentation rule in step S202 above.

[0175] In some optional embodiments, any training audio file other than the first training audio file can be selected from the acquired training dataset as the second training audio file. Then, an audio segment is selected from the second training audio file according to a preset segmentation rule, and this audio segment is used as a negative sample.

[0176] For example, such as Figure 4 As shown, assuming the preset segmentation rule is to extract a 10-second audio segment from the audio file, and the second training audio file is audiobb, then an audio segment of 10-20 seconds can be extracted from audiobb as a negative sample.

[0177] It should be noted that the execution order of the above steps S202 and S203 can be to execute step S202 first and then execute step S203, or to execute step S203 first and then execute step S202, or to execute steps S202 and S203 in parallel, and the present application does not make specific limitations thereon.

[0178] S204, training the deep learning network model according to the target audio segment, the positive sample and the negative sample to obtain a feature extraction model.

[0179] Wherein, the deep learning network model can be selected as a basic network model (EfficientNet-B0 model) or can be other deep network model, and the present application does not make any limitation thereon.

[0180] Considering that the training principle of obtaining the feature extraction model by training the deep learning network model is the same, the following mainly takes the training of the basic network model EfficientNet-BO as the main body structure as an example for explanation and description.

[0181] For the convenience of understanding, the above EfficientNet-BO is described. Specifically, the EfficientNet-BO model includes deep separable convolutional layers, ordinary convolutional layers, batch normalization layers, activation layers and other common layers in convolutional neural networks, which are organized in order to form EfficientNet-BO. The total number of layers can be 237 layers.

[0182] After introducing the EfficientNet-BO model, the following describes in detail the training of the EfficientNet-BO to obtain the feature extraction model. Optionally, after obtaining the target audio segment, the positive sample and the negative sample, the present application can input the target audio segment, the positive sample and the negative sample as input data into the EfficientNet-BO model, so as to extract the log mel spectrum of the target audio segment, the positive sample and the negative sample through the EfficientNet-BO model, to obtain the log mel spectrum of the target audio segment, the log mel spectrum of the positive sample and the log mel spectrum of the negative sample. Then, the EfficientNet-BO model processes the log mel spectrum of the target audio segment, the log mel spectrum of the positive sample and the log mel spectrum of the negative sample respectively, and applies global maximum pooling at the last layer to obtain three music feature vectors with a preset dimension, specifically a first music feature vector corresponding to the target audio segment, a second music feature vector corresponding to the positive sample, and a third music feature vector corresponding to the negative sample. Wherein, the preset dimension can be flexibly set according to actual application requirements, such as 512 dimensions or 1024 dimensions, and the present application does not make any limitation thereon.

[0183] Further, based on the first music feature vector, the second music feature vector and the third music feature vector, a loss function is calculated. Then, according to the loss function, iterative training is performed, so as to continuously improve the ability of the trained model to extract the audio feature vector, and when the calculated loss function is minimum or basically stable, the model trained at this time is taken as the feature extraction model. For details of the training process, please refer to Figure 5 .

[0184] As an optional implementation of the present application, based on the first music feature vector, the second music feature vector and the third music feature vector, the loss function can be calculated by the following formula (2):

[0185]

[0186] Wherein, L represents the loss function, log represents the logarithm, cxp represents the exponential function with e as the base, s represents the similarity calculation function, x represents the first music feature vector, x + represents the second music feature vector, and x - represents the third music feature vector.

[0187] It should be noted that the similarity calculation function s in the present application can be a cosine distance calculation method, a Euclidean distance calculation method or other methods, which are not limited here.

[0188] Suppose the similarity calculation function s in the present application is the cosine distance calculation method, then the similarity between the first music feature vector and the second music feature vector, or the similarity between the first music feature vector and the third music feature vector can be calculated as follows:

[0189]

[0190] Wherein, S represents the similarity, F a represents the first music feature vector, F b represents the second music feature vector or the third music feature vector, f ai represents the component of the first music feature vector, f bi represents the component of the second music feature vector or the component of the third music feature vector, i represents the i-th length in F a or F b , and M represents the M-th length in F a or F b , M is an integer greater than or equal to 1. Optionally, when the dimension of F a or F b in the present application is 512, M is equal to 512. When the dimension of F a or F bWhen the dimension of the log mel-spectrogram is 1024, M is equal to 1024.

[0191] It should be understood that the above log mel-spectrogram is a common audio feature extraction method, which can convert a one-dimensional time-domain audio signal into a two-dimensional frequency-domain signal, where the horizontal axis is the time frame and the vertical axis is the frequency. In this application, the conversion window length can be selected as 25 milliseconds (ms), the step length can be selected as 10 ms, the frequency range can be selected as 60-12000 Hertz (Hz), and the mel-spectrum frequency band can be divided into 80. Then, each 480 time frame is input, corresponding to about 4800 ms of audio segment.

[0192] The present application obtains the target audio segment, the positive sample and the negative sample from different training audio files of the training data set, trains the deep learning network model through the target audio segment, the positive sample and the negative sample, and obtains the feature extraction model. Therefore, the music feature vector of the music can be obtained through the feature extraction model, so as to increase the acquisition method of the music feature vector, and the extraction effect of the music feature vector can be further improved through the feature extraction model.

[0193] The process of constructing the sound effect template library in the present application will be described in detail below in combination with Figure 6 and Figure 7 As shown in Figure 6 , the sound effect template library construction can include the following steps:

[0194] S301, determining a seed music set of each sound effect template, wherein each seed music set includes at least one seed music.

[0195] Wherein, at least one seed music can be understood as one or more seed music.

[0196] Considering that after the sound effect template goes online, different users can actively use different sound effect templates, and after each sound effect template is used for a period of time, there will be a lot of use data. Therefore, the present application can count the effective use of music for each sound effect template, and the counted music is used as seed music. Further, based on the seed music corresponding to each sound effect template, the seed music set of each sound effect template can be obtained.

[0197] Specifically, determining the seed music set of each sound effect template can include the following steps:

[0198] Step 1, determining at least one effective sound effect template according to the playback time length proportion of each sound effect template in the first music, the first music being a historical playback music with sound effect templates added.

[0199] It should be noted that the sound effect template added in the first music can be any sound effect template, and the present application does not make any limitation on this. In addition, the number of the first music is one or more than one.

[0200] Since the user may add the sound effect template to any music when actively using the sound effect template, the present application first acquires all the first music to which the sound effect template is added. Then, the playing time proportion (i.e., the playing time ratio) of each sound effect in the corresponding first music is determined. The playing time proportion is compared with the proportion threshold value. If the playing time proportion of any sound effect template in the first music is greater than the proportion threshold value, the sound effect template is determined as a valid sound effect template. If the playing time proportion of any sound effect template in the first music is less than or equal to the proportion threshold value, the sound effect template is determined as an invalid sound effect template.

[0201] In the embodiment of the present application, the proportion threshold value can be flexibly set according to the construction accuracy of the sound effect template library, such as 50% or 60%, etc., which is not limited herein.

[0202] That is, the present application determines at least one valid sound effect template, which specifically includes: in response to the playing time proportion of any sound effect template in the first music being greater than the proportion threshold value, the sound effect template is determined as a valid sound effect template.

[0203] Step 2, determining the seed music of each valid sound effect template according to the number of times that the second music is effectively played by the user, the second music being the historical playing music to which the valid sound effect template is added.

[0204] Specifically, it can be determined whether the number of times that the second music is effectively played by the user is greater than the number of times threshold value. If the number of times that any second music is played by the user is greater than the number of times threshold value, the second music is determined as the seed music of each valid sound effect template. If the number of times that any second music is played by the user is less than or equal to the number of times threshold value, the second music is determined as not being the seed music of each valid sound effect template.

[0205] In the embodiment of the present application, the number of times threshold value can be flexibly set according to the construction accuracy of the sound effect template library, such as 500 times, 1000 times or 1000 times, etc., which is not limited herein.

[0206] For example, it is assumed that the sound effect template XX is added to the historical playing music YY, and the total number of times that the historical playing music YY is played by the client user is 1290 times, which is greater than the number of times threshold value 1000 times. At this time, the historical playing music YY is determined as the seed music of the sound effect template XX.

[0207] That is, determining the seed music of each valid sound effect template in the present application can include: in response to the number of times that any second music is played by a user being greater than a number threshold, determining the second music as the seed music of each valid sound effect template.

[0208] Step 3, obtaining the seed music set of each sound effect template according to all the seed music of each valid sound effect template.

[0209] After determining all the seed music of each valid sound effect template based on the above-mentioned step 1 and step 2, all the seed music of each valid sound effect template can be placed into a set to obtain the seed music set of each valid sound effect template. When each sound effect template is a valid sound effect template, the seed music set of each valid audio template at this time is the seed music set of each sound effect template.

[0210] S302, extracting the music feature vector of each audio file in each seed music set by the feature extraction model to obtain the music feature vector set of each sound effect template, each audio file corresponding to a seed music, and the number of music feature vectors of each audio file being at least one.

[0211] In the present application, the feature extraction model can be a network model trained in the foregoing embodiments, or can also be an existing network model, which is not limited herein. The feature extraction model of the present application is preferably a specially trained network model.

[0212] In some optional embodiments, the present application can perform feature extraction on the audio file of each seed music in the seed music set of each sound effect template by the feature extraction model to obtain at least one music feature vector of each seed music in the seed music set of each sound effect template. Further, at least one music feature vector corresponding to each sound effect template is placed into a set to obtain the music feature vector set of each sound effect template.

[0213] S303, clustering the music feature vectors in the music feature vector set of each sound effect template to obtain a seed music feature set, wherein the seed music feature set includes at least one seed music feature vector.

[0214] Optionally, the present application can use any clustering method to cluster the music feature vectors in the music feature vector set of each sound effect template, which is not limited herein. For example, K-means clustering method, hierarchical clustering, density-based clustering algorithm, network-based clustering algorithm, etc.

[0215] The following takes the K-means clustering method as an example to describe the process of clustering the music feature vectors in the music feature vector set of each sound effect template. Considering that the process of clustering the music feature vectors in the music feature vector set of each sound effect template is basically the same or similar, the application mainly describes the clustering of the music feature vectors in the music feature vector set of one sound effect template in detail.

[0216] Specifically, the clustering process can include the following steps:

[0217] First, randomly select N music feature vectors from the music feature vector set as initial cluster centers. Wherein, N is an integer greater than or equal to 1.

[0218] Second, for each music feature vector in the music feature vector set, calculate the distance from each music feature vector to each initial cluster center. And each music feature vector is assigned to the initial cluster center with the smallest distance.

[0219] Wherein, calculating the distance from each music feature vector to N initial cluster centers can include various forms, such as Euclidean distance or Mahalanobis distance, etc.

[0220] Optionally, in this application, the Euclidean distance is used to calculate the distance from each music feature vector to N initial cluster centers, and the implementation process can be shown in the following formula (4):

[0221]

[0222] Wherein, D represents the Euclidean distance, K represents the total length of the music feature vector, F j represents the jth value (i.e. the value of the jth length position) in each music feature vector, U j represents the jth value in the initial cluster center.

[0223] Third, for each category A t , recalculate its cluster center. In other words, calculate the centroid of all samples belonging to this category.

[0224] Optionally, the cluster center of each category A t can be recalculated by the following formula (5):

[0225]

[0226] Wherein, B t represents the cluster center of each category A t recalculated, i.e. the new cluster center; W t represents the cluster center of each category A tThe number of center-based music representation vectors, t represents the tth value in the music representation vector, F represents the music representation vector in each category A t .

[0227] Fourthly, repeat the second step and the third step until a preset iteration number is reached or the iteration is in a stable state.

[0228] The preset iteration number can be flexibly set according to the iteration requirement, such as 500 or 600 times, etc.

[0229] The stable state can be understood as the distance between the music representation vector in each category and the cluster center does not change, or the distance becomes smaller than a preset value. The preset value can be flexibly set according to the iteration accuracy, such as 0.1 or 0.2, etc. This place does not make any limitation.

[0230] S304, constructing an audio effect template library according to the seed music representation set of each audio effect template.

[0231] After obtaining the seed music representation set of each audio effect template, all audio effect templates and the seed music representation set of each audio effect template can be put into a database to construct an audio effect template library.

[0232] The application can quickly determine the audio effect recommendation template corresponding to the client to be played music based on the audio effect template library, thereby automatically recommending the audio effect recommendation template to the user, and providing a condition for obtaining a personalized audio effect template.

[0233] On the basis of the foregoing embodiments, the following will be described with reference to Figure 8 to further explain and describe the at least one music representation vector of each music in the music library in the application. As shown in Figure 8 The method can include the following steps:

[0234] S401, inputting the audio file of each music in the music library into the representation extraction model, and segmenting each audio file according to a preset segmentation rule through the representation extraction model to obtain at least one audio segment.

[0235] In the application, the representation extraction model is preferably the network model trained as described above.

[0236] The preset segmentation rule can be any method or strategy that can divide the audio file into different segments, and the application does not make any limitation thereon. For example, the preset segmentation rule can be randomly cutting a 10-second audio segment from the audio file, or it can also be segmenting the audio file every 5 seconds.

[0237] For example, assuming that the music library includes V pieces of music, where V is an integer greater than or equal to 2, the V pieces of music can be input into the feature extraction model as input data, so that the feature extraction model extracts at least one audio segment such as audio segment F1, audio segment F2, audio segment F3, … from each piece of music according to the segmentation rule of a time length of 15 seconds.

[0238] S402, performing feature extraction on each audio segment of each audio file by using the feature extraction model to obtain at least one music feature vector of each piece of music in the music library.

[0239] Each piece of music corresponds to an audio file.

[0240] Optionally, after obtaining at least one audio segment of the audio file corresponding to each piece of music, the application can continue to perform feature extraction on each audio segment of each audio file by using the feature extraction model to obtain a music feature vector corresponding to each audio segment of each piece of music.

[0241] S403, determining an audio effect recommendation template of each piece of music according to the audio effect template library and the at least one music feature vector of each piece of music.

[0242] S404, determining a mapping relationship between the music identifier, the audio file, and the audio effect recommendation template according to each piece of music and the audio effect recommendation template of each piece of music.

[0243] S405, in response to a music data acquisition request sent by a client, determining a target audio file and a target audio effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file, and the audio effect recommendation template, and sending the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template.

[0244] The technical scheme provided in the application determines at least one music feature vector of each music in the music library, determines an audio effect recommendation template of each music according to the audio effect template library and the music feature vector of each music, determines a mapping relationship between the music identifier, the audio file and the audio effect recommendation template according to the audio effect recommendation template of each music and the music without audio effect, and then determines the target audio file and the target audio effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template in response to the music data acquisition request sent by the client, and sends the target audio file and the target audio effect recommendation template to the client to instruct the client to determine the final audio effect template of the target audio file according to the target audio effect recommendation template. The application determines the audio effect recommendation template corresponding to each music according to the music feature vector of each music, so that the user can determine the corresponding audio effect recommendation template according to the music feature vector of the music when playing the music, and then can determine the final audio effect template according to the audio effect recommendation template, thereby realizing intelligent recommendation of the audio effect template to the user, improving the recommendation accuracy of the audio effect template, saving the time spent on determining the audio effect template, simplifying the operation of determining the audio effect template, meeting the personalized processing needs of the user for the music, and improving the user's listening experience. In addition, the at least one music feature vector of each music in the music library is determined by using the feature extraction model, so that the music feature vector can be obtained in a rich manner, and the extraction effect of the music feature vector can be improved by the feature extraction model.

[0245] The following will be described in combination with Figure 9 The application further optimizes the determination of the audio effect recommendation template of each music according to the audio effect template library and the at least one music feature vector of each music. As shown in the method, the method can include the following steps: Figure 9

[0246] S501, determining at least one music feature vector of each music in the music library.

[0247] S502, determining the similarity between each music feature vector of each music and each seed music feature vector in the audio effect template library.

[0248] S503, determining the audio effect recommendation template of each music according to the similarity between each music feature vector of each music and each seed music feature vector in the audio effect template library.

[0249] The audio effect template library includes at least two seed music feature vectors, and each seed music feature vector corresponds to an audio effect template.

[0250] ​Optionally, the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library can be calculated by using a similarity calculation method, such as a cosine distance calculation method, an Euclidean distance calculation method, or other methods. For details, refer to the calculation process in S102 in the foregoing embodiments, which will not be described herein.

[0251] Considering that the number of music feature vectors of each music can be one or more, after obtaining the similarity, the sound effect recommendation template of each music can be determined according to the similarity, which can include the following cases.

[0252] Case 1

[0253] If the number of music feature vectors of each music is one, the sound effect recommendation template of each music is determined according to the seed music feature vector corresponding to the highest similarity in all similarities.

[0254] Specifically, the sound effect recommendation template of each music is determined according to the seed music feature vector corresponding to the highest similarity in all similarities, which can specifically be that the sound effect template corresponding to the seed music feature vector is determined as the sound effect recommendation template of each music, wherein the seed music feature vector is the seed music feature vector corresponding to the highest similarity.

[0255] Case 2

[0256] If the number of music feature vectors of each music is at least two, a preset number of target similarities are selected from all similarities in descending order of similarity, and the sound effect recommendation template of each music is determined according to the seed music feature vector corresponding to the target similarity.

[0257] Specifically, the sound effect recommendation template of each music is determined according to the seed music feature vector corresponding to the target similarity, which specifically includes: determining the sound effect template corresponding to each seed music feature vector, wherein the seed music feature vector is the seed music feature vector corresponding to the target similarity; and fusing each sound effect template and the target similarity to obtain the sound effect recommendation template of each music.

[0258] Optionally, the sound effect recommendation template of each music can be obtained by fusing each sound effect template and the target similarity in any fusion manner, which is not limited in the present application.

[0259] As an optional implementation manner, the sound effect recommendation template of each music can be obtained by fusing each sound effect template and the target similarity by using the following formula (6):

[0260]

[0261] wherein, S1 represents the first target similarity, S2 represents the second target similarity, S represents the yth target similarity, y is an integer greater than or equal to 2, MB1 represents the sound effect template corresponding to S1, MB2 represents the sound effect template corresponding to S2, and MB represents the sound effect template corresponding to S y S1 represents the first target similarity, S2 represents the second target similarity, S represents the yth target similarity, y is an integer greater than or equal to 2, MB1 represents the sound effect template corresponding to S1, MB2 represents the sound effect template corresponding to S2, and MB represents the sound effect template corresponding to S y S1 represents the first target similarity, S2 represents the second target similarity, S represents the yth target similarity, y is an integer greater than or equal to 2, MB1 represents the sound effect template corresponding to S1, MB2 represents the sound effect template corresponding to S2, and MB represents the sound effect template corresponding to S y S1 represents the first target similarity, S2 represents the second target similarity, S represents the yth target similarity, y is an integer greater than or equal to 2, MB1 represents the sound effect template corresponding to S1, MB2 represents the sound effect template corresponding to S2, and MB represents the sound effect template corresponding to S

[0262] It should be noted that when some specific parameters exist in any sound effect template in the above formula (6) but do not exist in other sound effect templates, the application does not fuse the above some specific parameters when performing fusion operation based on formula (6), but directly adds the above some specific parameters to the fused sound effect recommendation template, so as to realize the reuse of the above some specific parameters, so as to avoid invalid fusion caused by fusing the above some reuse parameters, thereby improving the fusion speed.

[0263] For example, assuming that music V1 has three music feature vectors F1, F2 and F3, the number of target similarities selected from all similarities in descending order of similarity is 3, and the three target similarities are S1, S2 and S3. In addition, the sound effect template corresponding to the seed music feature vector corresponding to the target similarity S1 is MB1, the sound effect template corresponding to the seed music feature vector corresponding to the target similarity S2 is MB2, and the sound effect template corresponding to the seed music feature vector corresponding to the target similarity S3 is MB3. Then, based on the above formula (6), the target similarities S1, S2 and S3, and the sound effect templates MB1, MB2 and MB3 are fused, and the sound effect recommendation template of music V1 obtained is:

[0264]

[0265] s504, according to each music and the sound effect recommendation template of each music, determining the mapping relationship between the music identifier, the audio file and the sound effect recommendation template.

[0266] S505, in response to the music data acquisition request sent by the client, based on the mapping relationship between the music identifier, the audio file and the sound effect recommendation template, determining the target audio file and the target sound effect recommendation template corresponding to the music data acquisition request, and sending the target audio file and the target sound effect recommendation template to the client to instruct the client to determine the final sound effect template of the target audio file according to the target sound effect recommendation template.

[0267] The technical scheme provided in the application determines at least one music feature vector of each music in the music library, determines an audio effect recommendation template of each music according to the audio effect template library and the music feature vector of each music, determines a mapping relationship between the music identifier, the audio file and the audio effect recommendation template according to the music without the music and the audio effect recommendation template of each music, and then determines the target audio file and the target audio effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template in response to the music data acquisition request sent by the client, and sends the target audio file and the target audio effect recommendation template to the client to instruct the client to determine the final audio effect template of the target audio file according to the target audio effect recommendation template. The application determines the audio effect recommendation template corresponding to each music according to the music feature vector of each music, so that the user can determine the corresponding audio effect recommendation template according to the music feature vector of the music when playing the music, and then can determine the final audio effect template according to the audio effect recommendation template, thereby realizing intelligent recommendation of the audio effect template to the user, improving the recommendation accuracy of the audio effect template, saving the time spent on determining the audio effect template, simplifying the operation of determining the audio effect template, meeting the personalized processing needs of the user for the music, and improving the user's listening experience. In addition, when the number of music feature vectors of the music is multiple, a preset number of target similarities are selected in descending order of similarity, and the target similarities and the audio effect templates associated with the target similarity corresponding seed music feature vectors are fused to obtain the audio effect recommendation template of the music, so that the exclusive audio effect recommendation template can be obtained, thereby further improving the auditory experience of the user when listening to the music based on the exclusive audio effect recommendation template.

[0268] The following will be described in combination with Figure 10 The application embodiment provided in the application will be described below. In the application embodiment, the client can be an electronic device supporting user music playing. Optionally, the electronic device in the application can be various terminal devices, such as a smart phone, a tablet computer, a palm computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a wearable device, etc., and the type of the electronic device is not limited in the application. The terminal device can also be a user device, a user terminal, etc., and the application does not make any limitation on this.

[0269] As Figure 10 shown, the method can include the following steps:

[0270] S601, in response to a music playing request, obtaining a music identifier corresponding to the music playing request.

[0271] Optionally, when the user plays music by using the client, various music playlists are displayed on the client display interface, so that the user can select any song from the music playlists for playing. Since the client display interface provides a search bar in addition to displaying various music playlists, the user can also input a singer name or a song name in the search bar and trigger a search control in the search bar, so that the client feeds back corresponding search results to the user according to the search information input by the user. Then, the user can select any song from the search results for playing, and the application does not limit the specific implementation of the user playing music by using the client.

[0272] When the client detects a music playing request triggered by the user, such as detecting that the user clicks a play control associated with any song, it means that the user needs to play the music. At this time, the client can respond to the music playing request to determine the music triggered by the user as the music to be played and obtain the music to be played identifier. The music to be played identifier is used to uniquely identify the corresponding music to be played, such as the music to be played name.

[0273] S602, generating a music data acquisition request according to the music to be played identifier, and sending the music data acquisition request to the server to instruct the server to return the target sound effect recommendation template and the target audio file corresponding to the music to be played identifier.

[0274] Optionally, the client can generate a music data acquisition request according to the obtained music to be played identifier, and send the music data acquisition request to the server with music resources, so that the server feeds back the corresponding target audio file and target sound effect recommendation template. The music to be played identifier is carried in the music data acquisition request.

[0275] Further, after the server obtains the music data acquisition request sent by the client, the server can analyze the music data acquisition request to obtain the music to be played identifier carried in the data acquisition request. Then, according to the obtained music to be played identifier, the target audio file and the target sound effect recommendation template corresponding to the music to be played identifier are determined from the pre-established mapping relationship between the music identifier, the audio file and the sound effect recommendation template, or the pre-established index. Then, the target audio file and the target sound effect recommendation template are sent to the client, so that the client determines the final sound effect template of the target audio file according to the target sound effect recommendation template.

[0276] S603, determining the final sound effect template of the target audio file according to the target sound effect recommendation template and the user sound effect template, which is the most commonly used sound effect template determined based on the user historical use data.

[0277] The user sound effect template can be determined by analyzing historical data of the user using the sound effect template, and determining the most frequently used sound effect template as the user sound effect template.

[0278] In view of the fact that the user's favorite sound effect template can change at different time periods, the application can determine the most frequently used sound effect template based on all historical data of the user using the sound effect template each time the user plays music. In this way, the user's favorite sound effect template at different time periods can be tracked in a timely manner, so that the user's favorite personalized sound effect template at different time periods can be determined based on the latest determined user sound effect template, further meeting the user's demand for personalized sound effect template.

[0279] In some optional implementations, determining the final sound effect template of the target audio file according to the target sound effect recommendation template and the user sound effect template can be to fuse the target sound effect recommendation template and the user sound effect template in different fusion manners to obtain the final sound effect template of the target audio file.

[0280] Optionally, the target sound effect recommendation template and the user sound effect template can be fused by using the following formula (7):

[0281]

[0282] wherein ZMB zz represents the final sound effect template, Mix music represents the target sound effect recommendation template, MB usr represents the user sound effect template.

[0283] In some optional implementations, the application can also fuse the target sound effect recommendation template and the user sound effect template by using the following formula (8):

[0284]

[0285] wherein ZMB zz , represents the final sound effect template, Mix musie represents the target sound effect recommendation template, MB usr represents the user sound effect template, a represents the weight parameter of the target sound effect recommendation template, and b represents the weight parameter of the user sound effect template.

[0286] The parameters a and b above can be flexibly set according to the user's needs for music and sound effect processing. For example, if you want to increase the weight of the user's sound effect template in the final sound effect template, you can set the weight parameter of the user's sound effect template to be relatively large, and the weight parameter of the target sound effect recommendation template to be relatively small. This will allow the final sound effect template to better reflect the user's preferred sound effect, thus meeting the user's personalized needs. Alternatively, if you want to increase the weight of the target sound effect recommendation template in the final sound effect template, you can set the weight parameter of the user's sound effect template to be relatively small, and the weight parameter of the target sound effect recommendation template to be relatively large. This will allow the final sound effect template to better reflect the music-based music representation vector, more objectively and accurately recommending sound effect templates to the user, thus ensuring a more ideal and precise music and sound effect processing effect.

[0287] After obtaining the final sound effect template of the target audio file, the client can add the final sound effect template to the target audio file. When the client plays the target audio file, it can perform real-time sound effect processing on the target audio file through the final sound effect template. This allows the user's personal preferences to be combined with the characteristics of the music itself to obtain a personalized sound effect template, so as to perform personalized sound effect processing on the music played by the user.

[0288] It should be understood that when the client has sufficient memory resources and high-performance processing capabilities, this application may optionally configure the operation of determining the target sound effect recommendation and target sound effect file corresponding to the music to be played on the client, thereby realizing the determination of the personalized sound effect template of the music to be played locally on the client.

[0289] The technical solution provided in this application, in response to a music playback request, obtains the identifier of the music to be played corresponding to the music playback request, generates a music acquisition request based on the music to be played identifier, and sends the music data acquisition request to the server. The server then returns a target sound effect recommendation template and a target audio file corresponding to the music to be played identifier. Finally, based on the target sound effect recommendation template and the user's sound effect template, the final sound effect template for the target audio file is determined. This enables intelligent recommendation of sound effect templates to users, while also improving the accuracy of sound effect template recommendations. This saves time in determining sound effect templates, simplifies the sound effect template determination process, meets users' personalized music processing needs, and enhances the user's listening experience.

[0290] The following is a reference to the appendix. Figure 11 This application describes a sound effect determination device according to an embodiment. The sound effect determination device is configured on a server. Figure 11 As shown, the sound effect determination device 700 includes: a vector determination module 710, a template determination module 720, a relationship determination module 730, and a request-response module 740.

[0291] The vector determination module 710 is configured to determine at least one music feature vector of each music in the music library.

[0292] The template determination module 720 is configured to determine an audio effect recommendation template of each music according to the audio effect template library and the at least one music feature vector of each music.

[0293] The relationship determination module 730 is configured to determine a mapping relationship among the music identifiers, the audio files and the audio effect recommendation templates according to each music and the audio effect recommendation template of each music.

[0294] The request response module 740 is configured to, in response to a music data acquisition request sent by a client, determine a target audio file and a target audio effect recommendation template corresponding to the music data acquisition request based on the mapping relationship among the music identifiers, the audio files and the audio effect recommendation templates, and send the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template.

[0295] In an optional implementation of the embodiment, the vector determination module 710 includes:

[0296] The segmentation unit is configured to input an audio file of each music in the music library into a feature extraction model, perform segmentation processing on each audio file according to a preset segmentation rule through the feature extraction model, and obtain at least one audio segment.

[0297] The feature extraction unit is configured to perform feature extraction on each audio segment of each audio file through the feature extraction model, and obtain at least one music feature vector of each music in the music library.

[0298] In an optional implementation of the embodiment, the apparatus 700 further includes:

[0299] The data acquisition module is configured to acquire a training data set, the training data set including at least two training audio files.

[0300] The first processing module is configured to select a first training audio file, perform segmentation processing on the first training audio file according to the preset segmentation rule, and obtain a target audio segment and a positive sample, the positive sample and the target audio segment being different audio segments in the first training audio file.

[0301] The second processing module is configured to select a second training audio file, perform segmentation processing on the second training audio file according to the preset segmentation rule, and obtain a negative sample.

[0302] The model training module is configured to train a deep learning network model according to the target audio segment, the positive samples and the negative samples, and obtain the representation extraction model.

[0303] In an optional implementation of the embodiment, the sound effect template library includes at least two seed music representation vectors, and each seed music representation vector corresponds to a sound effect template.

[0304] Correspondingly, the template determination module 720 includes:

[0305] The first determination unit is configured to determine a similarity between each music representation vector of each music and each seed music representation vector in the sound effect template library.

[0306] The second determination unit is configured to determine a sound effect recommendation template of each music according to the similarity between each music representation vector of each music and each seed music representation vector in the sound effect template library.

[0307] In an optional implementation of the embodiment, the second determination unit is specifically configured to:

[0308] If the number of music representation vectors of each music is one, the second determination unit is configured to determine a sound effect recommendation template of each music according to a seed music representation vector corresponding to the highest similarity in all similarities.

[0309] If the number of music representation vectors of each music is at least two, the second determination unit is configured to select a preset number of target similarities from all similarities in descending order of similarity, and determine a sound effect recommendation template of each music according to a seed music representation vector corresponding to the target similarity.

[0310] In an optional implementation of the embodiment, the second determination unit is further configured to:

[0311] The second determination unit is configured to determine a sound effect recommendation template of each music as a sound effect template corresponding to the seed music representation vector corresponding to the highest similarity.

[0312] In an optional implementation of the embodiment, the second determination unit is further configured to:

[0313] The second determination unit is configured to determine a sound effect recommendation template of each music as a sound effect template corresponding to the seed music representation vector corresponding to the target similarity.

[0314] The second determination unit is configured to fuse each sound effect template and the target similarity to obtain a sound effect recommendation template of each music.

[0315] In an optional implementation of the embodiment of the application, the apparatus 700 further includes:

[0316] The template library construction module is configured to: determine a seed music set of each sound effect template, wherein each seed music set includes at least one seed music; extract a music feature vector of each audio file in each seed music set by using the feature extraction model, to obtain a music feature vector set of each sound effect template, wherein each audio file corresponds to one seed music, and the number of music feature vectors of each audio file is at least one; cluster the music feature vectors in the music feature vector set of each sound effect template, to obtain a seed music feature set, wherein the seed music feature set includes at least one seed music feature vector; and construct the sound effect template library according to the seed music feature set of each sound effect template.

[0317] In an optional implementation of the embodiment of the application, the template library construction module includes:

[0318] The template determination unit is configured to determine at least one effective sound effect template according to a playback time proportion of each sound effect template in the first music, wherein the first music is historical playback music to which the sound effect template is added.

[0319] The music determination unit is configured to determine a seed music of each effective sound effect template according to a number of times that the second music is effectively played by a user, wherein the second music is historical playback music to which the effective sound effect template is added.

[0320] The set acquisition unit is configured to obtain a seed music set of each sound effect template according to all seed musics of each effective sound effect template.

[0321] In an optional implementation of the embodiment of the application, the template determination unit is specifically configured to:

[0322] In response to a playback time proportion of any sound effect template in the first music being greater than a proportion threshold, the sound effect template is determined as an effective sound effect template.

[0323] In an optional implementation of the embodiment of the application, the music determination unit is specifically configured to:

[0324] In response to a number of times that any second music is played by a user being greater than a number threshold, the second music is determined as a seed music of each effective sound effect template.

[0325] The application provides an audio effect determination device. The audio effect determination device determines at least one music feature vector of each music in a music library, determines an audio effect recommendation template of each music according to an audio effect template library and the music feature vector of each music, determines a mapping relationship between a music identifier, an audio file and the audio effect recommendation template of each music according to the music and the audio effect recommendation template of each music, and determines a target audio file and a target audio effect recommendation template corresponding to a music data acquisition request based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template in response to a music data acquisition request sent by a client, and sends the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template. The application determines the audio effect recommendation template corresponding to each music according to the music feature vector of each music, so that the user can determine the corresponding audio effect recommendation template according to the music feature vector of the music when playing the music, and then can determine the final audio effect template according to the audio effect recommendation template, thereby realizing intelligent recommendation of the audio effect template to the user, improving the recommendation accuracy of the audio effect template, saving the time for determining the audio effect template, simplifying the operation of determining the audio effect template, meeting the personalized processing demand of the user for the music and improving the user experience of listening to music.

[0326] The application further provides an audio effect determination device configured in a client. As shown in the figure, the audio effect determination device 800 comprises an identifier acquisition module 810, a request sending module 820 and an audio effect determination module 830. Figure 12

[0327] The identifier acquisition module 810 is configured to acquire a to-be-played music identifier corresponding to a music playing request in response to the music playing request.

[0328] The request sending module 820 is configured to generate a music data acquisition request according to the to-be-played music identifier, send the music data acquisition request to a server to instruct the server to return a target audio effect recommendation template and a target audio file corresponding to the to-be-played music identifier.

[0329] The audio effect determination module 830 is configured to determine a final audio effect template of the target audio file according to the target audio effect recommendation template and a user audio effect template, wherein the user audio effect template is a most commonly used audio effect template determined based on user historical use data.

[0330] In an optional implementation of the application, the audio effect determination module 830 is specifically configured to:

[0331] fuse the target audio effect recommendation template and the user audio effect template to obtain the final audio effect template of the target audio file.

[0332] ​The sound effect determination device provided in this application, in response to a music playback request, obtains the identifier of the music to be played corresponding to the music playback request, generates a music acquisition request based on the music to be played identifier, and sends the music data acquisition request to a server. The server then returns a target sound effect recommendation template and a target audio file corresponding to the music to be played identifier. Finally, based on the target sound effect recommendation template and the user's sound effect template, the final sound effect template for the target audio file is determined. This enables intelligent recommendation of sound effect templates to users, while also improving the accuracy of sound effect template recommendations. This saves time in determining sound effect templates, simplifies the sound effect template determination process, meets users' personalized music processing needs, and enhances the user's listening experience.

[0333] It should be understood that the above-described apparatus embodiments and corresponding method embodiments can correspond to each other, and similar descriptions can be found in the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 11 The device 700 shown can perform Figure 1 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in device 700 are respectively implemented to achieve Figure 1 For the sake of brevity, the corresponding processes in each method are not described in detail here. Figure 12 The device 800 shown can perform Figure 10 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the device 800 are respectively for implementing Figure 10 For the sake of brevity, the corresponding processes in each method are not described in detail here.

[0334] The apparatus 700 or 800 of the embodiments of this application has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0335] Figure 13 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Figure 13 As shown, the electronic device 900 may include:

[0336] a memory 910, configured to store a computer program and transmit the program code to the processor 920. In other words, the processor 920 can call and run the computer program from the memory 910 to implement the sound effect determination method in the embodiments of the present application.

[0337] For example, the processor 920 can be configured to execute the above-mentioned method embodiments according to the instructions in the computer program.

[0338] In some embodiments of the present application, the processor 920 can include but is not limited to:

[0339] a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like.

[0340] In some embodiments of the present application, the memory 910 includes but is not limited to:

[0341] The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0342] In some embodiments of the present application, the computer program can be divided into one or more modules, which are stored in the memory 910 and executed by the processor 920 to complete the method provided by the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0343] As shown in Figure 13 The electronic device can further include:

[0344] The transceiver 930 can be connected to the processor 920 or the memory 910.

[0345] The processor 920 can control the transceiver 930 to communicate with other devices, specifically, can send information or data to other devices, or receive information or data sent by other devices. The transceiver 930 can include a transmitter and a receiver. The transceiver 930 can further include an antenna, and the number of antennas can be one or more.

[0346] It should be understood that the various components within the electronic device are connected by a bus system, which includes, in addition to a data bus, a power supply bus, a control bus, and a state signal bus.

[0347] The embodiment of the present application further provides a computer readable storage medium for storing a computer program, and the computer program causes a computer to execute the sound effect determination method in the above method embodiment. Figure 14 Figure 14 The computer readable storage medium is 1000, and the computer program is 1010.

[0348] The embodiment of the present application further provides a computer program product containing program instructions, which, when running on an electronic device, causes the electronic device to execute the sound effect determination method in the above method embodiment.

[0349] When implemented by using software, the computer program product can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When loaded and executed on a computer, the computer program instructions produce the flow or function of the embodiment of the present application in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, digital video disc (DVD)), or a semiconductor medium (for example, solid state disk (SSD)) and the like.

[0350] Those skilled in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.​

[0351] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the division of the above-described device embodiment is only a logical function division, and there can be another division manner for the actual implementation. For example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or modules, and can be in electrical, mechanical or other forms.

[0352] The modules described as separated components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to the actual needs to achieve the purposes of the embodiments of the present application. For example, the functional modules in the embodiments of the present application can be integrated in one processing module, or can be physically separated, or two or more modules can be integrated in one module.

[0353] The above describes only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A sound effect determination method, characterized by, The method applied to a server comprises: determining at least one music feature vector of each music in a music library; determining an audio effect recommendation template of each music according to an audio effect template library and the at least one music feature vector of each music; determining a mapping relationship between a music identifier, an audio file and the audio effect recommendation template of each music according to each music and the audio effect recommendation template of each music; in response to a music data acquisition request sent by a client, determining a target audio file and a target audio effect recommendation template corresponding to the music data acquisition request based on the mapping relationship between the music identifier, the audio file and the audio effect recommendation template, and sending the target audio file and the target audio effect recommendation template to the client to instruct the client to determine a final audio effect template of the target audio file according to the target audio effect recommendation template; the audio effect template library is constructed by the following steps, comprising: determining a seed music set of each audio effect template, wherein each seed music set comprises at least one seed music; extracting a music feature vector of each audio file in each seed music set by a feature extraction model to obtain a music feature vector set of each audio effect template, wherein each audio file corresponds to one seed music, and the number of music feature vectors of each audio file is at least one; clustering the music feature vectors in the music feature vector set of each audio effect template to obtain a seed music feature set, wherein the seed music feature set comprises at least one seed music feature vector; constructing the audio effect template library according to the seed music feature set of each audio effect template; the determination of the seed music set of each audio effect template comprises: determining at least one effective audio effect template according to the playback time length proportion of each audio effect template in a first music, wherein the first music is a historical playback music with the audio effect template added; determining a seed music of each effective audio effect template according to the number of times a second music is effectively played by a user, wherein the second music is a historical playback music with the effective audio effect template added; obtaining the seed music set of each audio effect template according to all seed musics of each effective audio effect template.

2. The method of claim 1, wherein, the determination of at least one music feature vector of each music in the music library comprises: inputting an audio file of each music in the music library into a feature extraction model, performing segmentation processing on each audio file according to a preset segmentation rule by the feature extraction model to obtain at least one audio segment; performing feature extraction on each audio segment of each audio file by the feature extraction model to obtain at least one music feature vector of each music in the music library.

3. The method of claim 2, wherein, before the determination of at least one music feature vector of each music in the music library, further comprising: obtaining a training data set, wherein the training data set comprises at least two training audio files; selecting a first training audio file, and performing segmentation processing on the first training audio file according to the preset segmentation rule to obtain a target audio segment and a positive sample, the positive sample and the target audio segment being different audio segments in the first training audio file; selecting a second training audio file, and performing segmentation processing on the second training audio file according to the preset segmentation rule to obtain a negative sample; training a deep learning network model according to the target audio segment, the positive sample, and the negative sample to obtain the feature extraction model.

4. The method of claim 1, wherein, The sound effect template library includes at least two seed music feature vectors, and each seed music feature vector corresponds to a sound effect template. Correspondingly, the method of determining a sound effect recommendation template for each music according to the sound effect template library and at least one music feature vector of each music comprises: determining the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library; determining the sound effect recommendation template for each music according to the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library.

5. The method of claim 4, wherein, The method of determining the sound effect recommendation template for each music according to the similarity between each music feature vector of each music and each seed music feature vector in the sound effect template library comprises: if the number of music feature vectors of each music is one, determining the sound effect recommendation template for each music according to the seed music feature vector corresponding to the highest similarity among all similarities; if the number of music feature vectors of each music is at least two, selecting a preset number of target similarities from all similarities in descending order of similarity, and determining the sound effect recommendation template for each music according to the seed music feature vector corresponding to the target similarity.

6. The method of claim 5, wherein, The method of determining the sound effect recommendation template for each music according to the seed music feature vector corresponding to the highest similarity among all similarities comprises: determining the sound effect template corresponding to the seed music feature vector as the sound effect recommendation template for each music, wherein the seed music feature vector is the seed music feature vector corresponding to the highest similarity.

7. The method of claim 5, wherein, The method of determining the sound effect recommendation template for each music according to the seed music feature vector corresponding to the target similarity comprises: determining the sound effect template corresponding to each seed music feature vector, wherein the seed music feature vector is the seed music feature vector corresponding to the target similarity; fusing each sound effect template and the target similarity to obtain the sound effect recommendation template for each music.

8. The method of claim 1, wherein, The method of determining at least one effective sound effect template according to the playback time length proportion of each sound effect template in the first music comprises: in response to the playback time length proportion of any sound effect template in the first music being greater than a proportion threshold, determining the sound effect template as an effective sound effect template.

9. The method of claim 1, wherein, The method of determining a seed music for each effective sound effect template according to the number of times the second music is effectively played by a user comprises: In response to a number of times that any of the second music is played by the user being greater than a number threshold, the second music is determined as a seed music of each of the effective sound effect templates.

10. A sound effect determination method, characterized by, The method applied to a client comprises: In response to a music playing request, obtaining a to-be-played music identifier corresponding to the music playing request; generating a music data obtaining request according to the to-be-played music identifier, and sending the music data obtaining request to a server to instruct the server to return a target sound effect recommendation template and a target audio file corresponding to the to-be-played music identifier; determining a final sound effect template of the target audio file according to the target sound effect recommendation template and a user sound effect template, the user sound effect template being a most frequently used sound effect template determined based on user historical use data; determining at least one music feature vector of each music in a music library; determining a sound effect recommendation template of each music according to a sound effect template library and the at least one music feature vector of each music; determining a mapping relationship between a music identifier, an audio file and a sound effect recommendation template according to each music and the sound effect recommendation template of each music, the mapping relationship being applied to a server; The sound effect template library is constructed by the following steps, comprising: determining a seed music set of each sound effect template, wherein each seed music set comprises at least one seed music; extracting a music feature vector of each audio file in each seed music set by a feature extraction model to obtain a music feature vector set of each sound effect template, each audio file corresponding to one seed music, and the number of music feature vectors of each audio file being at least one; clustering the music feature vectors in the music feature vector set of each sound effect template to obtain a seed music feature set, wherein the seed music feature set comprises at least one seed music feature vector; constructing the sound effect template library according to the seed music feature set of each sound effect template; The determination of the seed music set of each sound effect template comprises: determining at least one effective sound effect template according to a playback time length proportion of each sound effect template in first music, the first music being a historical playback music to which the sound effect template is added; determining a seed music of each effective sound effect template according to a number of times that second music is effectively played by a user, the second music being a historical playback music to which the effective sound effect template is added; obtaining the seed music set of each sound effect template according to all seed musics of each effective sound effect template.

11. The method of claim 10, wherein, The determination of the final sound effect template of the target audio file according to the target sound effect recommendation template and the user sound effect template comprises: fusing the target sound effect recommendation template and the user sound effect template to obtain the final sound effect template of the target audio file.

12. An audio effect recommendation apparatus, characterized by comprising: The server comprises: a vector determination module configured to determine at least one music feature vector of each music in a music library; a template determination module configured to determine a sound effect recommendation template of each music according to a sound effect template library and the at least one music feature vector of each music; A relationship determining module is configured to determine a mapping relationship among music identifiers, audio files, and sound effect recommendation templates according to each music and each sound effect recommendation template of the music; A request response module is configured to, in response to a music data acquisition request sent by a client, determine a target audio file and a target sound effect recommendation template corresponding to the music data acquisition request based on the mapping relationship among the music identifiers, the audio files, and the sound effect recommendation templates, and send the target audio file and the target sound effect recommendation template to the client to instruct the client to determine a final sound effect template of the target audio file according to the target sound effect recommendation template; A template library construction module is configured to determine a seed music set of each sound effect template, wherein each seed music set includes at least one seed music; extract a music feature vector of each audio file in each seed music set by a feature extraction model to obtain a music feature vector set of each sound effect template, each audio file corresponds to a seed music, and the number of music feature vectors of each audio file is at least one; cluster the music feature vectors in the music feature vector set of each sound effect template to obtain a seed music feature set, wherein the seed music feature set includes at least one seed music feature vector; and construct the sound effect template library according to the seed music feature set of each sound effect template. A template determining unit is configured to determine at least one effective sound effect template according to a playing time proportion of each sound effect template in a first music, the first music being a historical playing music with the sound effect template added. A music determining unit is configured to determine a seed music of each effective sound effect template according to a number of times a second music is effectively played by a user, the second music being a historical playing music with the effective sound effect template added. A set obtaining unit is configured to obtain a seed music set of each sound effect template according to all seed musics of each effective sound effect template.

13. The apparatus of claim 12, wherein, The vector determining module comprises: A segmentation unit is configured to input an audio file of each music in the music library into a feature extraction model, and perform segmentation processing on each audio file according to a preset segmentation rule by the feature extraction model to obtain at least one audio segment. A feature extraction unit is configured to perform feature extraction on each audio segment of each audio file by the feature extraction model to obtain at least one music feature vector of each music in the music library.

14. The apparatus of claim 13, wherein, Further comprising: A data obtaining module is configured to obtain a training data set, the training data set including at least two training audio files; A first processing module is configured to select a first training audio file, and perform segmentation processing on the first training audio file according to the preset segmentation rule to obtain a target audio segment and a positive sample, the positive sample and the target audio segment being different audio segments in the first training audio file; A second processing module is configured to select a second training audio file, and perform segmentation processing on the second training audio file according to the preset segmentation rule to obtain a negative sample; The model training module is configured to train a deep learning network model according to the target audio segment, the positive sample and the negative sample, and obtain the representation extraction model.

15. The apparatus of claim 12, wherein, The sound effect template library includes at least two seed music representation vectors, and each seed music representation vector corresponds to a sound effect template. Correspondingly, the template determination module includes: A first determination unit configured to determine a similarity between each music representation vector of each music and each seed music representation vector in the sound effect template library. A second determination unit configured to determine a sound effect recommendation template of each music according to the similarity between each music representation vector of each music and each seed music representation vector in the sound effect template library.

16. The apparatus of claim 15, wherein, The second determination unit is specifically configured to: If the number of music representation vectors of each music is one, determine the sound effect recommendation template of each music according to the seed music representation vector corresponding to the highest similarity among all similarities. If the number of music representation vectors of each music is at least two, select a preset number of target similarities from all similarities in descending order of similarity, and determine the sound effect recommendation template of each music according to the seed music representation vector corresponding to the target similarity.

17. The apparatus of claim 16, wherein, The second determination unit is further configured to: Determine the sound effect template corresponding to the seed music representation vector as the sound effect recommendation template of each music, wherein the seed music representation vector is the seed music representation vector corresponding to the highest similarity.

18. The apparatus of claim 16, wherein, The second determination unit is further configured to: Determine the sound effect template corresponding to each seed music representation vector, wherein the seed music representation vector is the seed music representation vector corresponding to the target similarity. Fuse each sound effect template and the target similarity to obtain the sound effect recommendation template of each music.

19. The apparatus of claim 12, wherein, The template determination unit is specifically configured to: In response to a playback time ratio of any sound effect template in the first music being greater than a ratio threshold, determine the sound effect template as a valid sound effect template.

20. The apparatus of claim 12, wherein, The music determination unit is specifically configured to: In response to any second music being played by a user more than a number threshold, determine the second music as a seed music of each valid sound effect template.

21. An audio effect determination apparatus, characterized by The client includes: An identifier acquisition module configured to acquire a to-be-played music identifier corresponding to a music playback request in response to the music playback request. A request sending module configured to generate a music data acquisition request according to the to-be-played music identifier, send the music data acquisition request to a server, and instruct the server to return a target sound effect recommendation template and a target audio file corresponding to the to-be-played music identifier. A sound effect determination module configured to determine a final sound effect template of the target audio file according to the target sound effect recommendation template and a user sound effect template, wherein the user sound effect template is a most commonly used sound effect template determined based on user historical use data. A vector determination module configured to determine at least one music representation vector of each music in a music library. The template determination module determines an audio effect recommendation template of each music according to the audio effect template library and at least one music feature vector of each music. The relationship determination module determines a mapping relationship between the music identifier, the audio file, and the audio effect recommendation template according to each music and the audio effect recommendation template of each music, and the mapping relationship is applied to the server. The template library construction module determines a seed music set of each audio effect template, wherein each seed music set includes at least one seed music; extracts a music feature vector of each audio file in each seed music set by using a feature extraction model to obtain a music feature vector set of each audio effect template, each audio file corresponds to one seed music, and the number of music feature vectors of each audio file is at least one; clusters the music feature vectors in the music feature vector set of each audio effect template to obtain a seed music feature set, wherein the seed music feature set includes at least one seed music feature vector; and constructs the audio effect template library according to the seed music feature set of each audio effect template. The template determination unit determines at least one effective audio effect template according to a playing time length proportion of each audio effect template in a first music, and the first music is a historical playing music to which the audio effect template is added. The music determination unit determines a seed music of each effective audio effect template according to a number of times that a second music is effectively played by a user, and the second music is a historical playing music to which the effective audio effect template is added. The set acquisition unit obtains a seed music set of each audio effect template according to all seed musics of each effective audio effect template.

22. The apparatus of claim 21, wherein, The audio effect determination module is specifically configured to: fuse the target audio effect recommendation template and the user audio effect template to obtain a final audio effect template of the target audio file.

23. An electronic device, comprising: The audio effect determination method comprises: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the audio effect determination method in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program causes a computer to execute the audio effect determination method in any one of claims 1 to 11.

Images