Audio processing method and device, equipment, storage medium and vehicle

Abstract

The invention relates to an audio processing method and device, equipment, a storage medium and a vehicle, and the method comprises the steps: carrying out the track division processing of an original audio, obtaining a plurality of audio track data, and enabling the original audio to be an audio obtained after the mixing of a plurality of sounds; adjusting target audio track data in the multiple pieces of audio track data; and carrying out sound mixing on the adjusted target audio track data and the audio track data except the target audio track data in the plurality of pieces of audio track data to obtain a target audio. According to the method and the device, the mixed audio is subjected to track division processing, the target audio track data obtained after track division is independently adjusted, and then the adjusted target audio track data and other audio track data are subjected to audio mixing to obtain the target audio, so that the playing effect of each audio track can be independently adjusted, the finally played target audio is more diversified, and the playing efficiency is improved. Listeners can feel the timbre and details of each audio track more visually, and richer auditory experience is provided for the listeners.

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio processing, and in particular to an audio processing method, device, equipment, storage medium and vehicle. Background Art

[0002] With the popularity of electronic devices, people are increasingly inclined to use electronic devices to play music. In order to improve the user's listening experience, various music playback software will also have built-in multiple sound effect adjustment information to play the adjusted audio according to user needs. However, music playback software can only make overall adjustments to the mixed audio, and the processing effect is limited, which cannot bring users a diverse music experience. Therefore, how to improve the audio adjustment effect and thus improve the user experience when playing audio is a technical problem that needs to be solved. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides an audio processing method, device, equipment, storage medium and vehicle.

[0004] A first aspect of an embodiment of the present disclosure provides an audio processing method, the method comprising:

[0005] Performing track processing on the original audio to obtain multiple track data, wherein the original audio refers to the audio obtained by mixing multiple sounds;

[0006] Adjusting target audio track data among the plurality of audio track data;

[0007] The adjusted target audio track data and the audio track data other than the target audio track data among the multiple audio track data are mixed to obtain the target audio.

[0008] A second aspect of an embodiment of the present disclosure provides an audio processing device, the device comprising:

[0009] A track splitting module is used to process the original audio into tracks to obtain multiple track data, wherein the original audio refers to the audio obtained by mixing multiple sounds;

[0010] An adjustment module, used for adjusting target audio track data among the plurality of audio track data;

[0011] A mixing module is used to mix the adjusted target audio track data and the audio track data other than the target audio track data in the multiple audio track data to obtain target audio.

[0012] A third aspect of an embodiment of the present disclosure provides a computer device, including a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the audio processing method of the first aspect described above is implemented.

[0013] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the audio processing method of the first aspect described above is implemented.

[0014] A fifth aspect of an embodiment of the present disclosure provides a vehicle, comprising a memory and a processor, and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the audio processing method as described in the first aspect above is implemented.

[0015] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0016] In the audio processing method, apparatus, device, storage medium and vehicle provided in the embodiments of the present disclosure, the original audio is processed into tracks to obtain multiple track data, wherein the original audio refers to the audio obtained after mixing multiple sounds, the target track data among the multiple track data are adjusted, and the adjusted target track data and the track data among the multiple track data except the target track data are mixed to obtain the target audio. The playback effect of each track can be adjusted independently, so that the target audio finally played is more diversified, and the audience can also feel the timbre and details of each track more intuitively, providing the audience with a richer auditory experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 is a flow chart of an audio processing method provided by an embodiment of the present disclosure;

[0020] Figure 2 is a flow chart of a method for adjusting audio track data provided by an embodiment of the present disclosure;

[0021] Figure 3 is a flow chart of a method for obtaining a target adjustment parameter and a target parameter value provided by an embodiment of the present disclosure;

[0022] Figure 4is a flow chart of a method for mixing multiple audio track data provided by an embodiment of the present disclosure;

[0023] Figure 5 is a flow chart of a method for dividing original audio tracks provided by an embodiment of the present disclosure;

[0024] Figure 6 is a flowchart of another method for adjusting audio track data provided by an embodiment of the present disclosure;

[0025] Figure 7 is a flowchart of another method for mixing multiple audio track data provided by an embodiment of the present disclosure;

[0026] Figure 8 is a structural schematic diagram of an audio processing device provided by an embodiment of the present disclosure;

[0027] Fig. 9 It is a structural diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0031] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] Figure 1 is a flow chart of an audio processing method provided by an embodiment of the present disclosure, and the method can be executed by an audio processing device. Figure 1 As shown, the audio processing method provided in this embodiment includes the following steps:

[0034] S101, performing track processing on original audio to obtain multiple track data, wherein the original audio refers to audio obtained by mixing multiple sounds.

[0035] The original audio in the embodiments of the present disclosure may be understood as audio obtained by mixing multiple sounds. For example, the original audio may be audio obtained by mixing multiple musical instruments and / or vocals. Normally, the audio in audio playback software is the original audio obtained by mixing multiple audio tracks.

[0036] In the disclosed embodiment, the audio processing device may perform track processing on the original audio obtained by mixing multiple sounds, and separate multiple audio track data from the original audio.

[0037] In an exemplary implementation of the disclosed embodiment, the audio processing device can obtain the original audio selected by the user, and input the original audio into a pre-trained track separation model, and the track separation model can separate multiple track data from the original audio. For example, the track separation model can be obtained by supervised training of a U-Net neural network based on training data, and the training data includes multiple groups of mixed audio and multiple track data constituting the mixed audio. After the mixed audio is down-sampled, the obtained data is written into a circular buffer, and data is extracted from the circular buffer and input into the U-Net neural network, and then the output data is written into another circular buffer, and the data therein is up-sampled, and the processed data is compared with the track data included in the training data to determine the loss, and then the neural network is trained based on the loss until the loss is less than a preset threshold. Among them, the multiple track data in the training data can include various types of musical instruments and / or vocal track data. For example, when the multiple track data include drum track data, piano track data, and guitar track data, the trained track separation model can separate drum tracks, piano tracks, and guitar tracks from the mixed audio.

[0038] S102: Adjust target audio track data among the multiple audio track data.

[0039] In the disclosed embodiment, after obtaining multiple audio track data corresponding to the original audio, the audio processing device can select target audio track data from them for adjustment to obtain adjusted target audio track data. For example, the target audio track data can be a default audio track that needs to be adjusted, such as vocal track data and drum track data, or it can be audio track data customized by the user, or it can be all audio track data, which is not limited here.

[0040] In an exemplary implementation of the disclosed embodiment, the audio processing device may pre-acquire an adjustment method for adjusting the audio track data, and adjust the audio track data based on the adjustment method. For example, when multiple audio track data respectively correspond to a musical instrument type, such as drums, piano, and guitar, the target audio track data corresponding to each musical instrument type may respectively correspond to an adjustment method, and the audio processing device may adjust each target audio track data based on the adjustment method corresponding to each musical instrument type.

[0041] S103: Mix the adjusted target audio track data and the audio track data other than the target audio track data in the multiple audio track data to obtain target audio.

[0042] The target audio in the embodiment of the present disclosure can be understood as the audio obtained after mixing and to be played to the user.

[0043] In the embodiment of the present disclosure, after adjusting the target audio track data, the audio processing device may mix the adjusted target audio track data with the audio track data other than the target audio track data among the multiple audio track data to obtain the target audio.

[0044] In an exemplary implementation of the embodiment of the present disclosure, the audio processing device may adjust all the audio track data among the multiple audio track data, and then mix all the adjusted audio track data to obtain the target audio.

[0045] The disclosed embodiment obtains multiple audio track data by performing track processing on the original audio, wherein the original audio refers to the audio obtained by mixing multiple sounds, and the target audio track data among the multiple audio track data is adjusted, and the adjusted target audio track data and the audio track data among the multiple audio track data except the target audio track data are mixed to obtain the target audio. The playback effect of each audio track can be adjusted independently, so that the target audio finally played is more diversified, and the audience can also feel the timbre and details of each audio track more intuitively, providing the audience with a richer auditory experience.

[0046] Figure 2 is a flow chart of a method for adjusting audio track data provided by an embodiment of the present disclosure, such as Figure 2 As shown, based on the above embodiment, the audio track data can be adjusted by the following method.

[0047] S201. Obtain target adjustment parameters for adjusting the target audio track data and corresponding target parameter values.

[0048] The adjustment parameters in the embodiments of the present disclosure may be understood as parameters used to adjust the audio playback effect. For example, the adjustment parameters may include volume adjustment parameters, sound effect adjustment parameters, etc., which are not limited here. The target adjustment parameters may be understood as adjustment parameters for adjusting the target audio track data, and the target parameter value is the value of the specific parameter to be adjusted.

[0049] In the disclosed embodiment, the audio processing device may first obtain target adjustment parameters and corresponding target parameter values ​​used to adjust various types of target audio track data before adjusting the target audio track data.

[0050] In an exemplary implementation of the disclosed embodiment, the audio processing device may obtain a preset adjustment rule, determine the adjustment parameters and corresponding parameter values ​​for adjusting each type of audio track data based on the adjustment rule, and further determine the target adjustment parameters and target parameter values ​​corresponding to the type of target audio track data. For example, the preset rule determines that the vocal track data corresponds to the first adjustment parameter and the first parameter value, the drum track data corresponds to the second adjustment parameter and the second parameter value, and the piano track data corresponds to the third adjustment parameter and the third parameter value. The target audio track data only includes the vocal track data and the drum track data but does not include the piano track data. Then, it is determined that the vocal track data in the target audio track data corresponds to the first adjustment parameter and the first parameter value, and the drum track data corresponds to the second adjustment parameter and the second parameter value.

[0051] In another exemplary implementation of the disclosed embodiment, the audio processing device may obtain the adjustment parameters and corresponding parameter values ​​customized by the user, and determine them as the target adjustment parameters and target parameter values ​​for adjusting the target audio track data; if the user does not set the adjustment parameters and parameter values, the system default adjustment parameters and parameter values ​​are determined as the target adjustment parameters and target parameter values.

[0052] S202. For each target audio track data, adjust the target adjustment parameter of the target audio track data to the corresponding target parameter value.

[0053] In the embodiment of the present disclosure, after determining the target adjustment parameters and target parameter values, the audio processing device can determine the target adjustment parameters and target parameter values ​​corresponding to each target audio track data of the target adjustment in multiple audio track data, and then adjust the target adjustment parameters of the target audio track data to the corresponding target parameter values.

[0054] In an exemplary implementation of the embodiments of the present disclosure, when the target adjustment parameters include volume adjustment parameters, the audio processing device can increase the volume, decrease the volume or mute the target audio track data, and adjust the volume to the volume parameter value corresponding to the volume adjustment parameter. When the target adjustment parameters include sound effect adjustment parameters, an echo can be added to the target audio track data, such as the vocal track data, or bass enhancement can be performed on the drum track data, etc., and the sound effect parameter value corresponding to the sound effect adjustment parameter can be adjusted, which is not limited here.

[0055] The disclosed embodiment obtains the target adjustment parameters and the corresponding target parameter values ​​for adjusting the target audio track data, and for each target audio track data, adjusts the target adjustment parameters of the target audio track data to the corresponding target parameter values. The target audio track data can be accurately adjusted based on the target adjustment parameters and the target parameter values, thereby making the mixed target audio more diversified and better meeting user needs.

[0056] Figure 3 is a flow chart of a method for obtaining a target adjustment parameter and a target parameter value provided by an embodiment of the present disclosure, such as Figure 3 As shown, based on the above embodiment, the target adjustment parameter and the target parameter value can be obtained by the following method.

[0057] S301. Search for adjustment parameters and parameter values ​​corresponding to the music genre label information of the original audio in a preset correspondence relationship between music genre label information, adjustment parameters, and corresponding parameter values.

[0058] In the disclosed embodiment, the audio processing device may obtain a correspondence between pre-set music genre label information, adjustment parameters, and corresponding parameter values, and search for corresponding adjustment parameters and parameter values ​​in the correspondence according to the music genre label information of the original audio.

[0059] In an exemplary implementation of the embodiments of the present disclosure, the audio processing device can obtain the genre tag information of the original audio while obtaining the original audio through an application program interface, or determine the genre tag information through keywords in the streaming media, and then when determining the target adjustment parameters and target parameter values, based on the acquired genre tag information, search for the adjustment parameters and parameter values ​​corresponding to the genre tag information in the correspondence between the preset genre tag information, the adjustment parameters and the corresponding parameter values.

[0060] S302. Among the adjustment parameters and parameter values ​​corresponding to the music genre tag information, filter at least one target adjustment parameter whose track type to be adjusted by the adjustment parameter is consistent with the track type of the target track data, and determine the parameter value corresponding to the target adjustment parameter as the target parameter value.

[0061] The audio track type in the embodiment of the present disclosure can be understood as the basis for track division when the original audio is divided into tracks. For example, the audio track type can include piano tracks, violin tracks, drum tracks, etc., and can also include vocal tracks, accompaniment tracks, etc., which are not limited here.

[0062] In the disclosed embodiment, the audio processing device can obtain the track type of the target track data, and after obtaining the adjustment parameters and parameter values ​​corresponding to the genre tag information, obtain the track type that each adjustment parameter is requested to adjust, and based on the track type of the target track data and the track type to be adjusted by the adjustment parameters, screen out at least one target adjustment parameter whose track type to be adjusted by the adjustment parameters is consistent with the track type of the target track data, determine it as the target adjustment parameter, and determine the parameter value corresponding to the target adjustment parameter as the target parameter value.

[0063] In an exemplary implementation of the embodiments of the present disclosure, if the adjustment parameters corresponding to the genre tag information include A adjustment parameters, B adjustment parameters and C adjustment parameters, and the corresponding track types are A track type, B track type and C track type respectively, and the track types of the target track data include A track type, B track type and D track type, then the A adjustment parameters and the B adjustment parameters are determined as the target adjustment parameters, and the parameter values ​​corresponding to the A adjustment parameters and the B adjustment parameters are determined as the target parameter values.

[0064] The disclosed embodiment searches for the adjustment parameters and parameter values ​​corresponding to the style tag information of the original audio in the correspondence between the preset style tag information and the adjustment parameters and the corresponding parameter values, and selects at least one target adjustment parameter whose track type to be adjusted by the adjustment parameter is consistent with the track type of the target track data among the adjustment parameters and parameter values ​​corresponding to the style tag information, and determines the parameter value corresponding to the target adjustment parameter as the target parameter value, thereby realizing automatic adjustment of the original audio and making the adjusted target audio better reflect the style of the original audio.

[0065] Figure 4 is a flow chart of a method for mixing multiple audio track data provided by an embodiment of the present disclosure, such as Figure 4 As shown, based on the above embodiment, multiple audio track data can be mixed by the following method.

[0066] S401: Obtain a mixing mode selected by a user.

[0067] The mixing mode in the embodiment of the present disclosure can be understood as the mixing method adopted when mixing multiple audio track data to obtain mixed audio. For example, the mixing mode may include a Vector Base Amplitude Panning (VBAP) mixing mode, a mid-side processing (MS) mixing mode, etc., which are not limited here.

[0068] In the embodiment of the present disclosure, the audio processing device may first obtain a mixing mode selected by a user before mixing a plurality of adjusted audio track data.

[0069] In an exemplary implementation of the embodiment of the present disclosure, the audio processing device can obtain a mixing mode selected by the user from a plurality of preset mixing modes through voice input, text input, touch instructions, etc.

[0070] S402: If the mixing mode selected by the user is the vector-based amplitude shift mixing mode, the number and position information of the speakers are obtained.

[0071] In the embodiment of the present disclosure, the audio processing device may obtain the number and position information of the speakers used to play the target audio after determining that the mixing mode selected by the user is the vector basis amplitude shift mixing mode.

[0072] In an exemplary implementation of the embodiment of the present disclosure, the audio processing device may obtain the quantity and position information of the speakers from a configuration file of the audio playback system.

[0073] S403: If the number of speakers is greater than or equal to two, for each speaker, based on a vector basis amplitude translation algorithm, determine a mixing parameter corresponding to the speaker according to a preset virtual sound position and position information of the speaker.

[0074] The virtual sound position in the embodiment of the present disclosure can be understood as the position of a virtual audio playback source simulated by cooperating with multiple audio playback devices, such as speakers. Audio playback devices including speakers may or may not exist at the virtual sound position. For example, the virtual sound position may be set by the user or by the system default, which is not limited here. Different audio track data may have different virtual sound positions.

[0075] The mixing parameters in the embodiments of the present disclosure may include gain values.

[0076] In the embodiment of the present disclosure, the audio processing device can determine whether the number of speakers is greater than or equal to two after obtaining the number and position information of the speakers. If the number of speakers is greater than or equal to two, it is determined that the conditions for simulating the virtual sound position are currently met. At this time, for each speaker, a vector basis amplitude translation algorithm can be used to determine the mixing parameters required for the speaker when simulating the sound effect of the virtual sound position based on the pre-set virtual sound position and the position information of the speaker.

[0077] In an exemplary implementation of the embodiment of the present disclosure, if the number of speakers is greater than or equal to two, and the speakers are located on the same straight line, the virtual sound emission position can be set on the straight line. Taking the number of speakers as two as an example, the vector basis amplitude translation algorithm can be used to determine the two-dimensional mixing parameters corresponding to each speaker according to the following method:

[0078]

[0079]

[0080] Among them, g represents the gain value of the speaker, p represents the sound source vector of the virtual sound position, and p T represents the transpose of p, l1 and l2 represent the unit vectors of the two speakers respectively, l1=[l11 l 12 ],l2=[l 21 l 22 ],g scaled represents the normalized gain value, C represents the total volume, g1 and g2 represent the mixing parameters of the two speakers, specifically the gain values ​​of the two speakers, p, l1, and l2 can be determined according to the virtual sound position and the positions of the two speakers, respectively. scaled and C are both preset known quantities, and the values ​​of g1 and g2 can be determined by the above method.

[0081] If the two speakers are at symmetrical angles relative to the listener, a vector-based amplitude translation algorithm can be used to determine the two-dimensional mixing parameters corresponding to each speaker according to the following method:

[0082]

[0083]

[0084] in, is the symmetry angle of the two loudspeakers relative to the listener, It is the angle between the virtual sound source and the listener.

[0085] In another exemplary implementation of the embodiment of the present disclosure, if the number of speakers is greater than two and the speakers are not located on the same straight line, a vector basis amplitude translation algorithm can be used to set the virtual sound position in the space formed by the speakers. Taking the number of speakers as two as an example, the mixing parameters corresponding to the speakers can be determined according to the following method:

[0086]

[0087]

[0088] Among them, l1, l2, l3 represent the unit vectors of the three speakers respectively, l1 = [l 11 l 12 l13 ],l2=[l 21 l 22 l 23 ],l3=[l 31 l 32 l 33 ], g1, g2, g3 represent the mixing parameters of the three speakers respectively, specifically the gain values ​​of the three speakers. The values ​​of g1, g2, g3 can be determined by the above method.

[0089] S404: For each speaker, mix the adjusted target audio track data and audio track data other than the target audio track data among the multiple audio track data based on the mixing parameters corresponding to the speaker to obtain mixed audio corresponding to the speaker.

[0090] In the embodiment of the present disclosure, after determining the mixing parameters corresponding to each speaker, the audio processing device can, for each speaker, mix the adjusted target audio track data and the audio track data of multiple audio track data other than the target audio track data based on the mixing parameters corresponding to the speaker to obtain the mixed audio corresponding to the speaker.

[0091] In an exemplary implementation of the disclosed embodiment, when the virtual sound position includes different virtual sound positions of different audio tracks, the audio processing device can determine, for each speaker, which audio track data is included in the audio to be played by the speaker, and then mix these audio track data based on the mixing parameters to obtain mixed audio corresponding to each speaker.

[0092] S405: Compensate the mixed audio corresponding to the speaker based on the frequency and phase compensation parameters of the speaker to obtain the target audio corresponding to the speaker.

[0093] The frequency and phase compensation parameters in the embodiments of the present disclosure may be understood as parameters for adjusting the frequency and phase in order to compensate for the hardware differences between different speakers so that each speaker can have the same playback effect when playing the same audio.

[0094] In the disclosed embodiment, the audio processing device may adjust each mixed audio for each speaker based on pre-acquired frequency and phase compensation parameters of the speaker after obtaining the mixed audio corresponding to the speaker, so as to obtain the adjusted target audio corresponding to the speaker.

[0095] The disclosed embodiment obtains a mixing mode selected by a user. If the mixing mode selected by the user is a vector basis amplitude shift mixing mode, the number and position information of the speakers are obtained. If the number of speakers is greater than or equal to two, for each speaker, based on a vector basis amplitude shift algorithm, the mixing parameters corresponding to the speaker are determined according to a preset virtual sound position and the position information of the speaker. For each speaker, the adjusted target audio track data and the audio track data of the multiple audio track data other than the target audio track data are mixed based on the mixing parameters corresponding to the speaker to obtain a mixed audio corresponding to the speaker. The mixed audio corresponding to the speaker is compensated based on the frequency and phase compensation parameters of the speaker to obtain the target audio corresponding to the speaker. The mixed audio can simulate the effect of the sound source being the virtual sound position during mixing, thereby making the stereo effect more realistic and immersive, and further enhancing the user's auditory experience.

[0096] Figure 5 is a flow chart of a method for dividing original audio tracks provided by an embodiment of the present disclosure, such as Figure 5 As shown, based on the above embodiment, the original audio can be divided into tracks by the following method.

[0097] S501. Obtain a play mode selected by a user, where the play mode includes a singing mode and a listening mode.

[0098] The playback mode in the embodiments of the present disclosure can be understood as a mode for playing audio, the singing mode can be understood as an audio playback mode suitable for users to sing songs while playing audio, and the listening mode can be understood as an audio playback mode suitable for users to listen to music. For example, the audio played in the singing mode does not include human voice, and the audio played in the listening mode includes human voice.

[0099] In the disclosed embodiment, the audio processing device may obtain the playback mode selected by the user before performing track splitting processing on the original audio, wherein the user may select a playback mode from a plurality of playback modes through voice input, text input, or touch commands.

[0100] S502: If the playback mode selected by the user is the singing mode, the original audio is subjected to vocal separation to obtain vocal track data and accompaniment track data.

[0101] In the embodiment of the present disclosure, after determining that the playback mode selected by the user is the singing mode, the audio processing device can perform vocal separation processing on the original audio when dividing the tracks, and divide the original audio into vocal track data containing only the vocals and the accompaniment track data after removing the vocals.

[0102] In an exemplary implementation of the embodiments of the present disclosure, the audio processing device can separate the vocal track data and the accompaniment track data from the original audio based on a pre-trained vocal separation model. The training data used by the vocal separation model includes the mixed audio and the vocal track data and the accompaniment track data that constitute the mixed audio. The structure and training method of the vocal separation model are similar to those of the track separation model in S101 and will not be repeated here.

[0103] S503: If the playing mode selected by the user is the listening mode, the original audio is divided into tracks according to the timbre to obtain a plurality of track data, each of which corresponds to a timbre.

[0104] In the disclosed embodiment, after determining that the playback mode selected by the user is the listening mode, the audio processing device can perform track processing on various timbres contained in the original audio according to the timbre to obtain multiple track data corresponding to the multiple timbres.

[0105] The disclosed embodiment obtains the playback mode selected by the user, wherein the playback mode includes a singing mode and a listening mode. If the playback mode selected by the user is the singing mode, the original audio is separated from the human voice to obtain human voice track data and accompaniment track data. If the playback mode selected by the user is the listening mode, the original audio is divided into tracks according to the timbre to obtain multiple track data, each track data corresponding to a timbre. The tracks can be separated according to the playback mode selected by the user, so that the target audio played subsequently can better meet the user's needs and further enhance the user experience.

[0106] Figure 6 is a flowchart of another method for adjusting audio track data provided by an embodiment of the present disclosure. Figure 6 As shown, based on the above embodiment, the audio track data can be adjusted by the following method.

[0107] S601. If the playback mode selected by the user is the singing mode, the singing method selected by the user is obtained, and the singing method includes an accompaniment method and a lead singing method.

[0108] In the embodiment of the present disclosure, the audio processing device can further obtain the singing method selected by the user after determining that the playback mode selected by the user is the singing mode, and determine whether the singing method selected by the user is the accompaniment method or the lead singing method. The specific acquisition method is similar to S501 and will not be repeated here.

[0109] S602: If the singing mode is an accompaniment mode, the vocal track data is muted.

[0110] In the embodiment of the present disclosure, the audio processing device can mute the vocal track data obtained by separating the original audio after determining that the singing method selected by the user is the accompaniment method.

[0111] S603: If the singing mode is the lead singing mode, the volume of the vocal track data is increased.

[0112] In the embodiment of the present disclosure, the audio processing device can increase the volume of the vocal track data separated from the original audio after determining that the singing method selected by the user is the lead singing method. For example, the amplitude of the volume increase can be determined according to the user's custom settings or the system default settings.

[0113] In an exemplary implementation of the disclosed embodiment, the audio processing device may, after separating the original audio to obtain the vocal track data and the accompaniment track data, perform pitch-up or pitch-down processing on the vocal track data and / or the accompaniment track data based on the user's instructions. For example, when the singing mode selected by the user is the accompaniment mode, the accompaniment track data is pitch-up or pitch-down processed separately; when the singing mode selected by the user is the lead singing mode, the vocal track data and the accompaniment track data are pitch-up or pitch-down processed simultaneously, so that the user can adjust the target audio played according to his or her own vocal range to achieve a better singing effect.

[0114] The disclosed embodiment obtains the singing method selected by the user if the playback mode selected by the user is the singing mode, and the singing method includes an accompaniment method and a lead singing method. If the singing method is the accompaniment method, the vocal track data is muted; if the singing method is the lead singing method, the volume of the vocal track data is increased. When the user wants to sing a song, the adjustment method for adjusting the audio track data can be further determined according to the singing method expected by the user, so that the target audio obtained after mixing can better meet the needs of the user, thereby further improving the user experience.

[0115] Figure 7 is a flowchart of another method for mixing multiple audio track data provided by an embodiment of the present disclosure. Figure 7 As shown, based on the above embodiment, multiple audio track data can be mixed by the following method.

[0116] S701. When the playback mode is the singing mode, collect the singing audio of the user.

[0117] In the embodiment of the present disclosure, the audio processing device may collect the user's singing audio through an audio collection device, such as a microphone, when determining that the playback mode is the singing mode.

[0118] In an exemplary implementation of an embodiment of the present disclosure, the audio processing device can adjust the user's singing audio after collecting the user's singing audio, such as adjusting the volume of the singing audio, or adding an echo sound effect to the singing audio, and can also transpose the singing audio and add the transposed audio as harmony to the singing audio.

[0119] S702. Mix the adjusted target audio track data and the audio track data other than the target audio track data among the multiple audio track data with the singing audio to obtain the target audio.

[0120] In the disclosed embodiment, after collecting the user's singing audio, the audio processing device can mix the user's singing audio with the adjusted target audio track data and the audio track data other than the target audio track data among the multiple audio track data to obtain the target audio containing the user's singing audio.

[0121] In an exemplary implementation of the disclosed embodiment, the audio processing device may mix the target track data, the track data other than the target track data in the multiple track data, and the singing audio according to the mixing method selected by the user, such as setting different mixing modes, or setting different virtual sound positions for the singing audio and the multiple track data. The specific implementation method is similar to S403-S405 and will not be repeated here.

[0122] The disclosed embodiment collects the singing audio of the user when the playback mode is the singing mode, mixes the adjusted target track data and the track data of the multiple track data except the target track data with the singing audio to obtain the target audio, and can mix the user's singing audio with the processed original audio, so that when the target audio is played through the speaker, the singing audio and the processed original audio are played as a whole, and can also facilitate the user to export the target audio including the singing audio, thereby further improving the user experience.

[0123] Figure 8 is a structural diagram of an audio processing device provided by an embodiment of the present disclosure, such as Figure 8 As shown, the audio processing device 800 includes: a track splitting module 810, an adjustment module 820, and a mixing module 830, wherein the track splitting module 810 is used to perform track splitting processing on the original audio to obtain multiple track data, and the original audio refers to the audio obtained after mixing multiple sounds; the adjustment module 820 is used to adjust the target track data among the multiple track data; the mixing module 830 is used to mix the adjusted target track data and the track data among the multiple track data except the target track data to obtain the target audio.

[0124] Optionally, the audio processing device 800 also includes: an adjustment parameter determination module, used to obtain target adjustment parameters and corresponding target parameter values ​​for adjusting the target audio track data; the adjustment module 820 is specifically used to adjust the target adjustment parameters of the target audio track data to the corresponding target parameter values ​​for each target audio track data.

[0125] Optionally, the adjustment parameter determination module includes: a search unit, used to search for the adjustment parameters and parameter values ​​corresponding to the style tag information of the original audio in the correspondence between the preset style tag information and the adjustment parameters and the corresponding parameter values; a screening unit, used to screen at least one target adjustment parameter whose track type to be adjusted by the adjustment parameter is consistent with the track type of the target track data among the adjustment parameters and parameter values ​​corresponding to the style tag information, and determine the parameter value corresponding to the target adjustment parameter as the target parameter value.

[0126] Optionally, the audio processing device 800 also includes: a first acquisition module, used to obtain a mixing mode selected by a user; a second acquisition module, used to obtain the number and position information of the speakers if the mixing mode selected by the user is a vector basis amplitude shift mixing mode; a mixing parameter determination module, used to determine the mixing parameters corresponding to the speakers for each speaker based on a vector basis amplitude shift algorithm, according to a pre-set virtual sound position and the position information of the speakers if the number of speakers is greater than or equal to two; the mixing module 830 includes: a mixing unit, used to mix the adjusted target audio track data and the audio track data of the multiple audio track data other than the target audio track data for each speaker based on the mixing parameters corresponding to the speakers, to obtain the mixed audio corresponding to the speakers; a compensation unit, used to compensate the mixed audio corresponding to the speakers based on the frequency and phase compensation parameters of the speakers, to obtain the target audio corresponding to the speakers.

[0127] Optionally, the audio processing device 800 also includes: a third acquisition module, used to obtain a playback mode selected by the user, wherein the playback mode includes a singing mode and a listening mode; the track separation module 810 includes: a first track separation unit, used to separate the vocals of the original audio if the playback mode selected by the user is a singing mode, and obtain vocal track data and accompaniment track data; a second track separation unit, used to separate the original audio according to the timbre if the playback mode selected by the user is a listening mode, and obtain multiple track data, each track data corresponding to a timbre.

[0128] Optionally, the audio processing device 800 also includes: a fourth acquisition module, which is used to obtain the singing method selected by the user if the playback mode selected by the user is the singing mode, and the singing method includes the accompaniment method and the lead singing method; the adjustment module 820 includes: a first adjustment unit, which is used to mute the vocal track data if the singing method is the accompaniment method; and a second adjustment unit, which is used to increase the volume of the vocal track data if the singing method is the lead singing method.

[0129] Optionally, the audio processing device 800 also includes: a collection module, used to collect the user's singing audio when the playback mode is the singing mode; the mixing module 830, specifically used to mix the adjusted target audio track data and the audio track data of the multiple audio track data except the target audio track data with the singing audio to obtain the target audio.

[0130] The audio processing device provided in this embodiment can execute the method described in any of the above embodiments. Its execution method and beneficial effects are similar and will not be described in detail here.

[0131] Fig. 9 It is a structural diagram of a computer device provided in an embodiment of the present disclosure.

[0132] like Fig. 9 As shown, the computer device may include a processor 910 and a memory 920 storing computer program instructions.

[0133] Specifically, the processor 910 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0134] The memory 920 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 920 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 920 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 920 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 920 is a non-volatile solid-state memory. In a particular embodiment, the memory 920 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0135] The processor 910 reads and executes the computer program instructions stored in the memory 920 to perform the steps of the audio processing method provided in the embodiment of the present disclosure.

[0136] In one example, the computer device may further include a transceiver 930 and a bus 940. Fig. 9 As shown, the processor 910, the memory 920 and the transceiver 930 are connected via a bus 940 and communicate with each other.

[0137] The bus 940 includes hardware, software or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 940 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0138] The embodiments of the present disclosure further provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the audio processing method provided by the embodiments of the present disclosure.

[0139] The above-mentioned storage medium may, for example, include a memory 920 of computer program instructions, and the above-mentioned instructions may be executed by the processor 910 of the audio processing device to complete the audio processing method provided by the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc. The above-mentioned computer program may be written in any combination of one or more programming languages ​​to perform the program code of the operation of the embodiment of the present disclosure, and the programming language includes an object-oriented programming language, such as Java, C++, etc., and also includes a conventional procedural programming language, such as "C" language or a similar programming language. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device, partially on the remote computing device, or entirely on the remote computing device or server.

[0140] An embodiment of the present disclosure also provides a vehicle, which includes a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the various processes and effects in the above-mentioned embodiments of the present disclosure can be implemented, which will not be elaborated here.

[0141] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An audio processing method, characterized in that: include: Performing track processing on the original audio to obtain multiple track data, wherein the original audio refers to the audio obtained by mixing multiple sounds; Adjusting target audio track data among the plurality of audio track data; The adjusted target audio track data and the audio track data other than the target audio track data among the multiple audio track data are mixed to obtain the target audio.

2. The method according to claim 1, characterized in that Before adjusting the target audio track data among the plurality of audio track data, the method further includes: Obtaining a target adjustment parameter for adjusting the target audio track data and a corresponding target parameter value; The adjusting the target audio track data among the plurality of audio track data comprises: For each of the target audio track data, the target adjustment parameter of the target audio track data is adjusted to the corresponding target parameter value.

3. The method according to claim 2, characterized in that Obtaining a target adjustment parameter for adjusting the target audio track data and a corresponding target parameter value, including: In the preset correspondence relationship between the music genre label information, the adjustment parameter and the corresponding parameter value, searching for the adjustment parameter and the parameter value corresponding to the music genre label information of the original audio; Among the adjustment parameters and parameter values ​​corresponding to the genre tag information, at least one target adjustment parameter whose track type to be adjusted by the adjustment parameter is consistent with the track type of the target track data is screened, and the parameter value corresponding to the target adjustment parameter is determined as the target parameter value.

4. The method according to claim 1, characterized in that Before mixing the adjusted target audio track data and the audio track data other than the target audio track data among the plurality of audio track data to obtain the target audio, the method further includes: Get the mixing mode selected by the user; If the mixing mode selected by the user is the vector-based amplitude panning mixing mode, the number and position information of the speakers are obtained; If the number of speakers is greater than or equal to two, for each speaker, based on a vector basis amplitude translation algorithm, a mixing parameter corresponding to the speaker is determined according to a preset virtual sound position and position information of the speaker; The step of mixing the adjusted target audio track data and the audio track data other than the target audio track data among the plurality of audio track data to obtain the target audio includes: For each speaker, mixing the adjusted target audio track data and audio track data other than the target audio track data among the plurality of audio track data based on the mixing parameters corresponding to the speaker to obtain mixed audio corresponding to the speaker; Based on the frequency and phase compensation parameters of the speaker, the mixed audio corresponding to the speaker is compensated to obtain the target audio corresponding to the speaker.

5. The method according to claim 1, characterized in that Before the original audio is processed into tracks to obtain a plurality of audio track data, the method further includes: Obtaining a playback mode selected by a user, wherein the playback mode includes a singing mode and a listening mode; The original audio is processed into tracks to obtain multiple track data, including: If the playback mode selected by the user is the singing mode, the original audio is subjected to vocal separation to obtain vocal track data and accompaniment track data; If the playback mode selected by the user is the listening mode, the original audio is divided into tracks according to the timbre to obtain a plurality of track data, each of which corresponds to a timbre.

6. The method according to claim 5, characterized in that Before adjusting the target audio track data among the plurality of audio track data, the method further includes: If the playback mode selected by the user is the singing mode, then the singing method selected by the user is obtained, and the singing method includes an accompaniment singing method and a lead singing method; The adjusting the target audio track data among the plurality of audio track data comprises: If the singing mode is an accompaniment mode, the vocal track data is muted; If the singing mode is the lead singing mode, the volume of the vocal track data is increased.

7. The method according to claim 5, characterized in that Before mixing the adjusted target audio track data and the audio track data other than the target audio track data among the plurality of audio track data to obtain the target audio, the method further includes: When the playing mode is the singing mode, collecting the singing audio of the user; The step of mixing the adjusted target audio track data and the audio track data other than the target audio track data among the plurality of audio track data to obtain the target audio includes: The adjusted target track data and track data other than the target track data among the multiple track data are mixed with the singing audio to obtain the target audio.

8. An audio processing device, characterized in that: include: A track splitting module is used to process the original audio into tracks to obtain multiple track data, wherein the original audio refers to the audio obtained by mixing multiple sounds; An adjustment module, used for adjusting target audio track data among the plurality of audio track data; A mixing module is used to mix the adjusted target audio track data and the audio track data other than the target audio track data in the multiple audio track data to obtain target audio.

9. A computer device, characterized in that: include: Memory; processor; and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A vehicle, characterized in that: include: Memory; processor; and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

Images