Audio adjustment method and device, electronic equipment and storage medium
By obtaining user historical singing audio and identifying and adjusting audio clips that do not conform to the tone characteristics, the problem of being able to finely adjust the tone in the prior art is solved, personalized adjustments of the user's selected songs are realized, and singing effect and user experience are improved.
Patent Information
- Application Number
- CN202411136811.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-29
AI Technical Summary
Existing cloud music karaoke applications cannot make detailed adjustments to the overall tone of the songs selected by the user, resulting in the user's singing effect not meeting expectations.
By obtaining the user's historical singing audio, determining the user's range information, identifying audio clips in the original singing audio that do not meet the user's tone characteristics, and performing tone change processing and synthesis processing to generate personalized and adjusted audio.
It realizes personalized adjustments to the selected tracks based on the user's tone characteristics and singing ability, so that users can be more handy when singing, improve user experience and achieve the expected results.
Smart Images

Figure CN119049433A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio processing technology, and in particular to an audio adjustment method, device, electronic device and storage medium. Background Art
[0002] In the Cloud Music Karaoke application, users can choose songs to sing from a rich resource of songs.
[0003] In the prior art, since each user has different timbre and range, the unique timbre and range of the original singer of the selected song may not be applicable to the user. In addition, although the existing karaoke applications provide two modes, original singer and accompaniment, they still lack the function of personalized adjustment according to the user's voice characteristics. Even if some advanced karaoke applications provide a pitch change function, they often simply adjust the overall pitch of the selected song and cannot make fine adjustments, which leads to the user's singing result not achieving the expected effect. Summary of the invention
[0004] The purpose of the present application is to provide an audio adjustment method, device, electronic device and storage medium to address the deficiencies in the above-mentioned prior art, so as to solve the problem that the prior art cannot finely adjust the overall pitch of the song selected by the user, resulting in the user's singing effect not achieving the expected effect.
[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:
[0006] In a first aspect, an embodiment of the present application provides an audio adjustment method, the method comprising:
[0007] Obtain the user's historical singing audio and the original singing audio of the song to be sung;
[0008] Determine user vocal range information based on the historical singing audio, where the user vocal range information is used to characterize the user's timbre characteristics;
[0009] Determining an audio segment to be adjusted according to the user's vocal range information, wherein the audio segment to be adjusted is an audio segment in the original singing audio that does not meet the timbre characteristics of the user;
[0010] The audio segment to be adjusted is subjected to pitch shifting processing, and the audio segment subjected to pitch shifting processing and the audio segment in the original audio that is not subjected to pitch shifting processing are synthesized to obtain the adjusted audio.
[0011] In a second aspect, an embodiment of the present application provides an audio adjustment device, the device comprising:
[0012] An acquisition module is used to acquire the user's historical singing audio and the original singing audio of the song to be sung;
[0013] A first determination module, used to determine user vocal range information based on the historical singing audio, wherein the user vocal range information is used to characterize the user's timbre characteristics;
[0014] A second determining module is used to determine an audio segment to be adjusted according to the user's vocal range information, wherein the audio segment to be adjusted is an audio segment in the original singing audio that does not meet the timbre characteristics of the user;
[0015] The processing module is used to perform pitch shifting processing on the audio segment to be adjusted, and to synthesize the pitch shifted audio segment and the audio segment in the original audio that has not been pitch shifted, so as to obtain the adjusted audio.
[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the audio adjustment method as described in any one of the first aspects above.
[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the audio adjustment method as described in any one of the first aspects above are executed.
[0018] According to the audio adjustment method, device, electronic device and storage medium of the embodiment of the present application, the user's historical singing audio and the original singing audio of the song to be sung are obtained, and based on the historical singing audio, the user's vocal range information used to characterize the user's timbre characteristics is determined, and then according to the user's vocal range information, the audio segment to be adjusted that does not meet the user's timbre characteristics in the original singing audio is determined, and the audio segment to be adjusted is subjected to pitch change processing, and the audio segment after pitch change processing and the audio segment not subjected to pitch change processing in the original singing audio are synthesized to obtain the adjusted audio. According to the embodiment of the present application, the user's vocal range information determined based on the user's historical singing audio is used as the basis for audio analysis and adjustment, and the audio segment to be adjusted in the original singing audio that exceeds the user's singing ability is identified, and the pitch of the audio segment to be adjusted is adjusted to make it easier for the user to sing, and the audio that has finally been subjected to pitch change and synthesis processing has been personalized and adjusted to adapt to the user's timbre characteristics and singing ability. In this way, the user's selected song is personalized and adjusted according to the user's specific timbre characteristics, so that it is more suitable for the user's singing ability, which can not only improve the user experience, but also enable the user to show his or her unique voice characteristics when singing, and the user's singing effect can also achieve the expected effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0020] Figure 1 A schematic diagram of a flow chart of an audio adjustment method provided in an embodiment of the present application is shown;
[0021] Figure 2 A schematic diagram of a process for determining user vocal range information provided by an embodiment of the present application is shown;
[0022] Figure 3 A schematic diagram showing a flow chart of a method for determining a sound range duration provided in an embodiment of the present application;
[0023] Figure 4 A schematic flow chart of a method for determining loudness range information provided by an embodiment of the present application is shown;
[0024] Figure 5 A schematic diagram showing a flow chart of a method for determining pitch range information provided in an embodiment of the present application;
[0025] Figure 6 A schematic diagram of a flow chart of a method for determining an audio segment to be adjusted provided in an embodiment of the present application is shown;
[0026] Figure 7 A schematic diagram showing a flow chart of a method for determining a vocal range information provided in an embodiment of the present application;
[0027] Figure 8 A schematic diagram of the structure of an audio adjustment device provided in an embodiment of the present application is shown;
[0028] Fig. 9 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0029] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.
[0030] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.
[0031] In order to enable those skilled in the art to use the content of this application, the following implementation is provided in conjunction with the specific application scenario of "adjusting the audio rhythm according to the user's vocal range characteristics". For those skilled in the art, the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the audio adjustment method, it should be understood that this is only an exemplary embodiment.
[0032] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.
[0033] Vocal range refers to the range of pitches that can be produced by an instrument, user, or piece of music. For users, the vocal range of each user varies greatly. For example, some users have a wider vocal range and can easily sing high and low notes, while some users have a narrower vocal range and cannot reach certain pitches. In the Cloud Music Karaoke application, the vocal range of the original song selected by the user may cover a wide range of high and low notes. For users with a narrower vocal range, they cannot control the vocal range of the original song. In addition, each user's timbre is unique, and it is difficult for users to imitate and master the unique timbre of the original song. For example, the original song has a unique hoarse voice, a thick bass, or a clear high note, and the user's timbre is far different from that of the original singer. In addition, the user may have problems with confidence when singing, resulting in the final singing effect not meeting the expected effect.
[0034] An embodiment of the present application provides an audio adjustment method, which generates a user portrait based on the user's historical singing data, determines the user's vocal range information, such as the vocal range duration and the vocal range duration range (including pitch range information and loudness range information), and then matches the user's selected song based on the user's vocal range information, determines the segments in the user's selected song that exceed the user's singing ability, such as high-pitched segments or low-pitched segments that exceed the user's limit, and adjusts the segments to be within the user's limit, and outputs the adjusted audio, so that the user can achieve the expected singing effect by singing based on the adjusted audio.
[0035] Figure 1 A schematic diagram of a flow chart of an audio adjustment method provided in an embodiment of the present application is shown. Figure 1 As shown, the method specifically comprises the following steps:
[0036] S101, obtaining the user's historical singing audio and the original singing audio of the song to be sung.
[0037] Optionally, the historical singing audio refers to the audio data of various songs sung by the user at different times or occasions, and the original singing audio of the song to be sung refers to the audio data corresponding to the original version of the song to be sung selected by the user.
[0038] S102. Determine the user's vocal range information based on historical singing audio, where the user's vocal range information is used to characterize the user's timbre characteristics.
[0039] Optionally, by analyzing the user's historical singing audio, data that can characterize the user's timbre characteristics is extracted to obtain a comprehensive portrait of the user's timbre characteristics, including the user's best and most comfortable vocal range, pitch, and loudness control capabilities, so as to better understand the user's singing ability and characteristics, and facilitate the use of the user's vocal range information to adjust the audio melody of the song to be sung to match the user's singing ability.
[0040] Exemplarily, the user's vocal range information includes the vocal range duration corresponding to different vocal ranges, pitch range information, and loudness range information. The vocal range duration corresponding to different vocal ranges indicates the length of time the user can sing in different vocal ranges (such as bass, middle, and high). The pitch range information indicates the span between the highest and lowest notes that the user can reach when singing. Based on the pitch range information, the user's vocal range width, that is, the high and low limits of the vocal range, can be reflected. The loudness range information indicates the volume changes when the user sings, including the dynamic range between the softest and strongest sounds that the user can produce.
[0041] S103: Determine an audio segment to be adjusted according to the user's vocal range information, where the audio segment to be adjusted is an audio segment in the original audio that does not satisfy the user's timbre characteristics.
[0042] Optionally, based on the user's vocal range information, the pitch, volume and other attributes of the original audio of the song to be sung are matched and compared with the user's vocal range information, and audio clips in the original audio that exceed the user's vocal range or do not match the user's timbre characteristics are identified, and the audio clips are marked as audio clips to be adjusted. The audio clips to be adjusted may be due to factors such as the volume being too high or too low, the loudness being too loud or too soft, which makes it impossible for the user to comfortably or accurately replicate the original singing effect.
[0043] S104, performing pitch shifting processing on the audio segment to be adjusted, and synthesizing the pitch shifted audio segment and the audio segment in the original audio that has not been pitch shifted, to obtain an adjusted audio.
[0044] Optionally, the purpose of the tone change process is to make the tone of the audio clip more suitable for the user's vocal range, so the tone change process is performed for the audio clip to be adjusted, which can be to adjust the pitch of the audio clip to be adjusted. This process does not change the audio speed, that is, the track rhythm remains unchanged. After completing the tone change process, the audio clip after the tone change process is reintegrated with the audio clip in the original singing audio that is not subjected to the tone change process to obtain the adjusted audio. The adjusted audio obtained through the tone change and synthesis process is more in line with the user's vocal range and timbre characteristics as a whole, so that the user is more handy when singing along.
[0045] Based on this, according to the audio adjustment method provided in the embodiment of the present application, the user's vocal range information determined based on the user's historical singing audio is used as the basis for audio analysis and adjustment, and the audio segment to be adjusted in the original singing audio that exceeds the user's singing ability is identified, and the pitch of the audio segment to be adjusted is adjusted to make it easier for the user to sing, and the audio that has finally been processed by pitch change and synthesis has been personalized and can adapt to the user's timbre characteristics and singing ability. In this way, the user's selected track is personalized and adjusted according to the user's specific timbre characteristics to make it more suitable for the user's singing ability, which can not only improve the user experience, allowing the user to show their unique voice characteristics when singing, but also the user's singing effect can achieve the expected effect.
[0046] Figure 2 A schematic diagram of a method for determining user vocal range information provided by an embodiment of the present application is shown. Figure 2 As shown, the above step S102 determines the user's vocal range information based on the historical singing audio, and specifically includes the following steps:
[0047] S201, dividing the historical singing audio into multiple singing audio segments.
[0048] Exemplarily, the historical singing audio can be segmented into multiple singing audio clips according to a preset segmentation strategy. The preset segmentation strategy includes but is not limited to segmentation according to time. Specifically, the historical singing audio is segmented into multiple time segments of fixed lengths. For example, the total length of the historical singing audio is 3 minutes, and the audio is segmented into segments of 5 seconds each, resulting in 36 singing audio clips.
[0049] It should be noted that the sampling frequency of audio is defined as the number of samples collected per second, so the historical singing audio can also be divided into smaller time segments through the sampling frequency, each time segment is usually 1 second or smaller time unit, and then multiple singing audio segments are obtained.
[0050] S202, performing frequency domain analysis on each singing audio clip to determine the duration of the range corresponding to each tone zone.
[0051] As a possible implementation, Figure 3 The flowchart of a method for determining a duration of a sound range provided in an embodiment of the present application is shown. Figure 3 As shown, the above step S202 specifically includes the following steps:
[0052] S301, using Fourier transform strategy to perform time-frequency conversion processing on each singing audio clip to obtain frequency domain information of each singing audio clip.
[0053] Optionally, the frequency domain information includes fundamental frequency information for representing the pitch of the singing audio clip, and each singing audio clip can be converted into the frequency domain through a fast Fourier transform strategy to identify the fundamental frequency and harmonic components in each singing audio clip. In this process, each singing audio clip in the time domain is decomposed into different frequency components. Since the singing audio clip is a real signal, the corresponding spectrum has conjugate symmetry, so only the positive frequency part of the frequency component is returned during the time-frequency conversion process.
[0054] It should be noted that only the positive frequency part is returned in the above process because the negative frequency part and the positive frequency part contain the same frequency domain information, which can reduce redundant data, simplify the analysis process, and further reduce the amount of calculation and storage requirements to improve the speed and efficiency of time-frequency conversion processing.
[0055] S302: Determine the sound range corresponding to each singing audio clip according to the frequency domain information of each singing audio clip.
[0056] Optionally, pre-set range boundary information is obtained, and the sound zone corresponding to each singing audio segment is determined according to the fundamental frequency information and range boundary information of each singing audio segment, and the range boundary information includes frequency range information corresponding to each sound zone.
[0057] Exemplarily, the vocal range includes a bass range, a middle range, and a high range. The audio signal of each singing audio clip is converted into the frequency domain by fast Fourier transform, and the fundamental frequency information of each singing audio clip is detected. The fundamental frequency is the frequency component with the largest amplitude in the frequency domain, which usually represents the main pitch of the singing audio clip. On this basis, based on the pre-set vocal range boundary information and the fundamental frequency information of each singing audio clip, the vocal range to which each singing audio clip belongs is determined by comparing the fundamental frequency information with the frequency range corresponding to each vocal range. Specifically, check whether the fundamental frequency of the singing audio clip is within the frequency range of the bass range, the middle range, or the high range. If the fundamental frequency of the singing audio clip falls within the frequency range of a certain audio, the singing audio clip is considered to belong to that vocal range.
[0058] The range boundary information includes the frequency ranges of the bass area, the middle area, and the high area. For example, the frequency range of the bass area can be defined as 20 Hz to 200 Hz, the frequency range of the middle area can be defined as 200 Hz to 2000 Hz, and the frequency range of the high area can be defined as 2000 Hz to 20000 Hz. The frequency ranges of the above different sound areas can be adjusted according to specific needs, and are not limited thereto.
[0059] S303. Determine the duration of the range corresponding to each tone zone according to the length of the target singing audio segment corresponding to each tone zone. The target singing audio segment is a singing audio segment belonging to the same tone zone among multiple singing audio segments.
[0060] Optionally, each singing audio clip is classified into a corresponding range through the detected fundamental frequency information of each singing audio clip, that is, the singing audio clip is judged to belong to the bass range, middle range, and high range according to the fundamental frequency value, and the target singing audio clips belonging to the same range are accumulated to determine the range duration corresponding to each range.
[0061] Exemplarily, the singing audio clips belonging to the same pitch range are collected and classified, for example, all the singing audio clips belonging to the bass range are classified into one group, all the singing audio clips belonging to the middle pitch range are classified into one group, and all the singing audio clips belonging to the high pitch range are classified into one group. For each pitch range, the total duration corresponding to the pitch range is calculated based on the length of all the singing audio clips in the pitch range. Specifically, the length of the singing audio clip reflects the duration of the singing audio clip, which can be determined based on the number of singing audio clips and the sampling rate. On the basis of obtaining the duration of each singing audio clip, the duration of all target singing audio clips belonging to the same pitch range can be added together to obtain the pitch range duration corresponding to each pitch range, that is, the total duration of the bass range, the middle pitch range and the high pitch range.
[0062] Based on this, the range duration corresponding to each vocal zone can indirectly reflect the proportion of time the user spends in different vocal zones. For example, the user sings in the middle vocal zone most of the time, which means that the user is most comfortable in the middle vocal zone, or the melody lines of the singable songs are concentrated in the middle vocal zone. Therefore, by determining the range duration corresponding to each vocal zone, we can better understand and grasp the user's vocal range.
[0063] S203, performing loudness analysis on each singing audio clip to determine loudness range information.
[0064] As a possible implementation, Figure 4 A flow chart of a method for determining loudness range information provided by an embodiment of the present application is shown. Figure 4 As shown, the above step S203 specifically includes the following steps:
[0065] S401. Perform loudness analysis on each singing audio segment to obtain loudness information of each singing audio segment, where the loudness information includes a root mean square value for representing the loudness of the singing audio segment.
[0066] Exemplarily, the loudness information is used to reflect the sound intensity corresponding to the singing audio clip, and the loudness of each singing audio clip can be determined by calculating the root mean square value, where the root mean square value is the square root of the average value of the square of the audio signal amplitude. Specifically, each singing audio clip is converted into a digital signal to obtain a series of sample points, and each sample point is squared, and the average of the square values of all sample points is calculated, and then the root mean square value is obtained by square rooting the average.
[0067] S402: Determine loudness range information according to the pitch range and loudness information corresponding to each singing audio segment.
[0068] Exemplarily, the loudness range information includes a bass range loudness range, a middle range loudness range, and a high range loudness range. For each range, the loudness range of the range is determined based on the root mean square value of all singing audio clips in the range. Specifically, based on the maximum and minimum root mean square values in the bass range, the middle range loudness range, and the high range loudness range, the bass range loudness range, the middle range loudness range, and the high range loudness range are determined respectively.
[0069] Based on this, by determining the loudness range of different sound zones, the distribution of sound energy in different sound zones can be determined. For example, a sound zone with a larger loudness range indicates that the dynamic range of the sound zone is wider, and a sound zone with a smaller loudness range indicates that the sound of the sound zone is more stable, thereby providing a reference for audio adjustment.
[0070] S204, performing pitch analysis on each singing audio clip to determine pitch range information.
[0071] As a possible implementation, Figure 5 FIG. 1 is a flow chart showing a method for determining pitch range information provided by an embodiment of the present application. Figure 5 As shown, the above step S204 specifically includes the following steps:
[0072] S501, using a preset pitch detection strategy to determine the pitch corresponding to each singing audio clip.
[0073] Exemplarily, preset pitch detection strategies such as the YIN algorithm and the HPS algorithm, wherein the YIN algorithm is a method for detecting pitch by calculating a differential function and a cumulative differential function, and is mainly applicable to single audio signals. The HPS algorithm is a method for estimating the fundamental frequency by performing multiple harmonic multiplications on the spectrum of an audio signal to enhance the fundamental frequency component, and is mainly applicable to multi-audio signals. The above pitch detection strategy is used to perform audio analysis on each singing audio clip, and the pitch value of each singing audio clip is calculated, and the pitch value reflects the tone characteristics of the user when singing.
[0074] S502: Determine the pitch range information according to the pitch range and pitch corresponding to each singing audio clip.
[0075] Optionally, the pitch range information includes a bass pitch range, a middle pitch range and a treble pitch range, and the bass pitch range, the middle pitch range and the treble pitch range respectively correspond to the user's pitch coverage range in the bass, middle and treble areas, that is, the frequency and distribution of the user's voice in the bass, middle and treble areas when the user is singing.
[0076] Exemplarily, based on the tone ranges corresponding to each singing audio clip, the singing audio clips belonging to the same tone range are first distinguished, and for each tone range, the highest and lowest values of the corresponding pitch of the singing audio clip in the tone range are used to determine the pitch interval corresponding to the tone range, and the tone range corresponding to each tone range can be obtained.
[0077] Based on this, while determining the loudness information corresponding to each singing audio clip, the pitch information corresponding to each singing audio clip is further determined, where the loudness information reflects the strength of the sound, and the pitch information reflects the frequency of the sound. The range and characteristics of the audio clip are jointly determined based on the correspondence between loudness and pitch. By combining the loudness information and pitch information, it is possible to more accurately determine which range each singing audio clip belongs to, and understand the performance of the singing audio clip in the corresponding range.
[0078] Figure 6 A schematic diagram of a flow chart of a method for determining an audio segment to be adjusted provided in an embodiment of the present application is shown. Figure 6 As shown, the above step S103 determines the audio segment to be adjusted according to the user's vocal range information, and specifically includes the following steps:
[0079] S601, segmenting the original singing audio to obtain a plurality of original singing audio segments.
[0080] Correspondingly, the original audio can also be divided into multiple time segments of fixed length according to time. For example, if the total length of the original audio is 3 minutes, the audio can be divided into segments of 5 seconds each, and 36 original audio segments can be obtained.
[0081] S602: Perform audio analysis on each original singing audio segment to determine the range information of each original singing audio segment.
[0082] Optionally, Figure 7 FIG. 1 is a flow chart of a method for determining a range of sound provided by an embodiment of the present application. Figure 7 As shown, the above step S602 specifically includes the following steps:
[0083] S701: Perform fundamental frequency detection on each original audio segment to obtain fundamental frequency information of each original audio segment.
[0084] Exemplarily, the audio signal of each original audio clip is converted into frequency domain information through fast Fourier transform to perform fundamental frequency detection on each original audio clip to determine the fundamental frequency information of each original audio clip, that is, the pitch. The fundamental frequency information is the key factor in determining the pitch.
[0085] S702: Determine the sound range corresponding to each original singing audio segment according to the fundamental frequency information of each original singing audio segment and the preset sound range boundary information.
[0086] Exemplarily, in the same manner as the above-mentioned processing method for the singing audio clip, the audio signal of each original singing audio clip is converted into the frequency domain through fast Fourier transform, and the fundamental frequency information of each original singing audio clip is detected. On this basis, based on the pre-set range boundary information and the fundamental frequency information of each original singing audio clip, the sound zone to which each original singing audio clip belongs is determined by comparing the fundamental frequency information with the frequency range corresponding to each sound zone. Specifically, check whether the fundamental frequency of the original singing audio clip is within the frequency range of the bass zone, the middle zone or the treble zone. If the fundamental frequency of the original singing audio clip falls within the frequency range of a certain audio, the original singing audio clip is considered to belong to that sound zone.
[0087] S703. Determine the duration of the range corresponding to each tone zone according to the length of the target original singing audio segment corresponding to each tone zone, wherein the target original singing audio segment is an original singing audio segment belonging to the same tone zone among the multiple original singing audio segments.
[0088] Optionally, in the same manner as the above-mentioned processing method for singing audio clips, each original singing audio clip is classified into a corresponding range through the detected fundamental frequency information of each original singing audio clip, that is, the fundamental frequency value is used to determine which range the original singing audio clip belongs to, namely, the bass, middle and high range. The target original singing audio clips belonging to the same range are accumulated to determine the range duration corresponding to each range.
[0089] Exemplarily, the original singing audio clips belonging to the same pitch range are collected and classified, for example, all the original singing audio clips belonging to the bass range are classified into one group, all the original singing audio clips belonging to the middle pitch range are classified into one group, and all the original singing audio clips belonging to the high pitch range are classified into one group. For each pitch range, the total duration corresponding to the pitch range is calculated based on the length of all the original singing audio clips in the pitch range. Specifically, the length of the original singing audio clip reflects the duration of the original singing audio clip, which can be determined based on the number of original singing audio clips and the sampling rate. On the basis of obtaining the duration of each original singing audio clip, the duration of all target original singing audio clips belonging to the same pitch range can be added together to obtain the pitch range duration corresponding to each pitch range, that is, the total duration of the bass range, the middle pitch range and the high pitch range.
[0090] Based on this, by determining the vocal range duration corresponding to each vocal zone, the user's vocal range can be better understood and mastered.
[0091] S603: Determine the audio segment to be adjusted according to the user's vocal range information and the vocal range information of each original singing audio segment.
[0092] Optionally, the user vocal range information is compared with the vocal range information of each original singing audio clip to determine whether each original singing audio clip meets the user vocal range condition, and the original singing audio clip that does not meet the user vocal range condition is used as the audio clip to be adjusted; wherein the user vocal range condition is determined based on the user vocal range information.
[0093] Exemplarily, the range information of the original singing audio clip refers to the pitch range actually sung in the track, including all the pitches appearing in the track. Based on the above-mentioned preset pitch detection strategy, the fundamental frequency of each original singing audio clip can be obtained, and then the range information of each original singing audio clip is determined based on the fundamental frequency. On this basis, the user's range information is compared with the range information of each original singing audio clip to check whether the pitch in each original singing audio clip falls completely or mostly within the user's comfortable range. Specifically, check whether the lowest and highest notes of the original singing audio clip are within the user's range. If the range of the original singing audio clip falls completely or mostly within the user's range, it is considered that the original singing audio clip meets the user's range condition. On the contrary, if the range of the original singing audio clip exceeds the user's range, that is, there is a pitch lower than the user's bass area or higher than the user's treble area, it is considered that the original singing audio clip does not meet the user's range condition and needs to be adjusted, and the original singing audio clip that does not meet the user's range condition is marked as an audio clip to be adjusted.
[0094] Based on this, the user's vocal range information is compared with the vocal range information of the original audio clip to determine the audio clip to be adjusted, and then the audio clip to be adjusted is adjusted to adapt to the user's singing ability. This processing not only conforms to the user's vocal range but also maintains the integrity and attractiveness of the music. It can effectively personalize the music material, ensure its adaptability and singability, and thus improve the effect of music production and performance.
[0095] As a possible implementation manner, the above-mentioned step S104 performs pitch shifting processing on the audio segment to be adjusted, including: determining the pitch shifting attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted; and performing pitch shifting processing on the audio segment to be adjusted according to the pitch shifting attribute of the audio segment to be adjusted, wherein the pitch shifting processing includes high pitch lowering processing and low pitch raising processing.
[0096] Optionally, the pitch change attribute indicates the direction of pitch adjustment of the audio segment to be adjusted, including up-pitch and down-pitch. Up-pitch means that the pitch of the audio segment to be adjusted needs to be adjusted upward, while down-pitch means that the pitch of the audio segment to be adjusted needs to be adjusted downward. In this way, by analyzing the fundamental frequency information of the audio segment to be adjusted, it is determined whether the pitch of the audio segment to be adjusted needs to be increased or decreased, ensuring that the audio after pitch change processing matches the timbre characteristics of the user.
[0097] Optionally, the above step of determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted includes:
[0098] If the fundamental frequency value of the audio segment to be adjusted exceeds the user's high pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a falling pitch; conversely, if the fundamental frequency value of the audio segment to be adjusted exceeds the user's low pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a rising pitch.
[0099] For example, if the fundamental frequency value of the audio clip to be adjusted is higher than the user's high pitch range, it indicates that the pitch of the audio clip to be adjusted exceeds the user's natural and comfortable sound range. In this case, the audio clip to be adjusted should be down-tuned to return its pitch to the user's comfort zone. On the contrary, if the fundamental frequency value of the audio clip to be adjusted is lower than the user's low pitch range, it indicates that the pitch of the audio clip to be adjusted is too low. In this case, the audio clip to be adjusted needs to be up-tuned, which is also to return the pitch to the user's natural pitch range.
[0100] Based on this, the use of pitch shifting can help compensate for the limitations of the user's vocal range. By raising or lowering the pitch, it can ensure that the user's singing remains within the most comfortable and best-performing pitch range, thereby improving the user's singing effect.
[0101] Based on the same inventive concept, an audio adjustment device corresponding to the audio adjustment method is also provided in the embodiment of the present application. Since the principle of solving the problem by the audio adjustment device in the embodiment of the present application is similar to the above-mentioned audio adjustment method in the embodiment of the present application, the implementation of the audio adjustment device can refer to the implementation of the audio adjustment method, and the repeated parts will not be repeated.
[0102] Reference Figure 8 , which is a schematic diagram of the structure of an audio adjustment device provided in an embodiment of the present application, the audio adjustment device 800 includes: an acquisition module 801, a first determination module 802, a second determination module 803, and a processing module 804, wherein:
[0103] The acquisition module 801 is used to acquire the user's historical singing audio and the original singing audio of the song to be sung;
[0104] A first determination module 802 is used to determine the user's vocal range information based on the historical singing audio, where the user's vocal range information is used to characterize the user's timbre characteristics;
[0105] The second determining module 803 is used to determine the audio segment to be adjusted according to the user's vocal range information, where the audio segment to be adjusted is the audio segment in the original singing audio that does not meet the user's timbre characteristics;
[0106] The processing module 804 is used to perform pitch shifting processing on the audio segment to be adjusted, and synthesize the audio segment after pitch shifting processing and the audio segment in the original audio that has not been pitch shifted processing to obtain the adjusted audio.
[0107] Based on this, according to the audio adjustment device of the embodiment of the present application, the user's vocal range information determined based on the user's historical singing audio is used as the basis for audio analysis and adjustment, and the audio segment to be adjusted in the original singing audio that exceeds the user's singing ability is identified, and the pitch of the audio segment to be adjusted is adjusted to make it easier for the user to sing, and the audio that has finally been processed by pitch change and synthesis has been personalized and can adapt to the user's timbre characteristics and singing ability. In this way, the user's selected track is personalized and adjusted according to the user's specific timbre characteristics to make it more suitable for the user's singing ability, which can not only improve the user experience, allowing the user to show his or her unique voice characteristics when singing, but also the user's singing effect can achieve the expected effect.
[0108] In a possible implementation manner, the first determining module 802 is specifically configured to:
[0109] Split the historical singing audio into multiple singing audio segments;
[0110] Perform frequency domain analysis on each singing audio clip to determine the duration of the range corresponding to each tone range;
[0111] Perform loudness analysis on each singing audio clip to determine loudness range information;
[0112] Perform pitch analysis on each singing audio clip to determine the pitch range information.
[0113] In a possible implementation manner, the first determining module 802 is specifically configured to:
[0114] Using Fourier transform strategy to perform time-frequency conversion on each singing audio clip, and obtain frequency domain information of each singing audio clip;
[0115] Determine the pitch range corresponding to each singing audio clip according to the frequency domain information of each singing audio clip;
[0116] According to the length of the target singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target singing audio segment is a singing audio segment belonging to the same vocal range among multiple singing audio segments.
[0117] In a possible implementation manner, the first determining module 802 is specifically configured to:
[0118] Acquire preset range boundary information, where the range boundary information includes frequency range information corresponding to each range;
[0119] According to the fundamental frequency information and range boundary information of each singing audio segment, the sound range corresponding to each singing audio segment is determined.
[0120] In a possible implementation manner, the first determining module 802 is further configured to:
[0121] Performing loudness analysis on each singing audio clip to obtain loudness information of each singing audio clip, wherein the loudness information includes a root mean square value for representing the loudness of the singing audio clip;
[0122] The loudness range information is determined based on the pitch range and loudness information corresponding to each singing audio clip.
[0123] In a possible implementation manner, the first determining module 802 is further configured to:
[0124] Using a preset pitch detection strategy, determine the pitch corresponding to each singing audio clip;
[0125] Determine the pitch range information based on the pitch range and pitch corresponding to each singing audio clip.
[0126] In a possible implementation manner, the second determining module 803 is specifically configured to:
[0127] Segment the original singing audio to obtain multiple original singing audio segments;
[0128] Performing audio analysis on each original singing audio segment to determine the range information of each original singing audio segment;
[0129] The audio segments to be adjusted are determined according to the user's vocal range information and the vocal range information of each original singing audio segment.
[0130] In a possible implementation manner, the second determining module 803 is specifically configured to:
[0131] Performing fundamental frequency detection on each original audio segment to obtain fundamental frequency information of each original audio segment;
[0132] Determine the pitch range corresponding to each original singing audio segment according to the fundamental frequency information of each original singing audio segment and the preset pitch range boundary information;
[0133] According to the length of the target original singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target original singing audio segment is an original singing audio segment belonging to the same vocal range among multiple original singing audio segments.
[0134] In a possible implementation manner, the second determining module 803 is specifically configured to:
[0135] The user vocal range information is compared with the vocal range information of each original singing audio clip to determine whether each original singing audio clip meets the user vocal range condition, and the original singing audio clip that does not meet the user vocal range condition is used as the audio clip to be adjusted; wherein the user vocal range condition is determined based on the user vocal range information.
[0136] In a possible implementation manner, the processing module 804 is specifically configured to:
[0137] Determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted;
[0138] According to the pitch change attribute of the audio segment to be adjusted, the audio segment to be adjusted is pitched, and the pitch change processing includes high pitch lowering processing and low pitch raising processing.
[0139] In a possible implementation manner, the processing module 804 is specifically configured to:
[0140] If the fundamental frequency value of the audio segment to be adjusted exceeds the user's high pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a falling pitch; conversely, if the fundamental frequency value of the audio segment to be adjusted exceeds the user's low pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a rising pitch.
[0141] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.
[0142] The present application embodiment also provides an electronic device 900, such as Fig. 9 As shown, it is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of the present application, including: a processor 901, a memory 902, and optionally, a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the electronic device 900 is running, the processor 901 communicates with the memory 902 through the bus 903. When the machine-readable instructions are executed by the processor 901, the following processing is performed:
[0143] Obtain the user's historical singing audio and the original singing audio of the song to be sung;
[0144] Based on historical singing audio, determine the user's vocal range information, which is used to characterize the user's timbre characteristics;
[0145] Determine, according to the user's vocal range information, an audio segment to be adjusted, where the audio segment to be adjusted is an audio segment in the original singing audio that does not satisfy the user's timbre characteristics;
[0146] The audio segment to be adjusted is subjected to pitch shifting processing, and the audio segment subjected to pitch shifting processing and the audio segment of the original audio that is not subjected to pitch shifting processing are synthesized to obtain the adjusted audio.
[0147] In a possible implementation manner, when the processor 901 executes the instruction to determine the user's vocal range information based on the historical singing audio, it is specifically configured to:
[0148] Split the historical singing audio into multiple singing audio segments;
[0149] Perform frequency domain analysis on each singing audio clip to determine the duration of the range corresponding to each tone range;
[0150] Perform loudness analysis on each singing audio clip to determine loudness range information;
[0151] Perform pitch analysis on each singing audio clip to determine the pitch range information.
[0152] In a possible implementation manner, when the processor 901 executes the instruction of performing frequency domain analysis on each singing audio segment to determine the range duration corresponding to each range, it is specifically used to:
[0153] Using Fourier transform strategy to perform time-frequency conversion on each singing audio clip, and obtain frequency domain information of each singing audio clip;
[0154] Determine the pitch range corresponding to each singing audio clip according to the frequency domain information of each singing audio clip;
[0155] According to the length of the target singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target singing audio segment is a singing audio segment belonging to the same vocal range among multiple singing audio segments.
[0156] In a possible implementation manner, when the processor 901 executes an instruction to determine the sound range corresponding to each singing audio segment according to the frequency domain information of each singing audio segment, it is specifically used to:
[0157] Acquire preset range boundary information, where the range boundary information includes frequency range information corresponding to each range;
[0158] According to the fundamental frequency information and range boundary information of each singing audio segment, the sound range corresponding to each singing audio segment is determined.
[0159] In a possible implementation manner, when the processor 901 executes the instruction to perform loudness analysis on each singing audio segment and determine the loudness range information, it is specifically configured to:
[0160] Performing loudness analysis on each singing audio clip to obtain loudness information of each singing audio clip, wherein the loudness information includes a root mean square value for representing the loudness of the singing audio clip;
[0161] The loudness range information is determined based on the pitch range and loudness information corresponding to each singing audio clip.
[0162] In a possible implementation manner, when the processor 901 executes the instruction to perform pitch analysis on each singing audio segment to determine pitch range information, it is specifically configured to:
[0163] Using a preset pitch detection strategy, determine the pitch corresponding to each singing audio clip;
[0164] Determine the pitch range information based on the pitch range and pitch corresponding to each singing audio clip.
[0165] In a possible implementation manner, when the processor 901 executes the instruction of determining the audio segment to be adjusted according to the user's vocal range information, it is specifically configured to:
[0166] Segment the original singing audio to obtain multiple original singing audio segments;
[0167] Performing audio analysis on each original singing audio segment to determine the range information of each original singing audio segment;
[0168] The audio segments to be adjusted are determined according to the user's vocal range information and the vocal range information of each original singing audio segment.
[0169] In a possible implementation manner, when the processor 901 executes the instruction of performing audio analysis on each original singing audio segment to determine the range information of each original singing audio segment, it is specifically configured to:
[0170] Performing fundamental frequency detection on each of the original audio clips to obtain fundamental frequency information of each of the original audio clips;
[0171] Determine the pitch range corresponding to each original singing audio segment according to the fundamental frequency information of each original singing audio segment and the preset pitch range boundary information;
[0172] According to the length of the target original singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target original singing audio segment is an original singing audio segment belonging to the same vocal range among multiple original singing audio segments.
[0173] In a possible implementation manner, when the processor 901 executes the instruction to determine the audio segment to be adjusted according to the user's vocal range information and the vocal range information of each original singing audio segment, it is specifically configured to:
[0174] The user vocal range information is compared with the vocal range information of each original singing audio clip to determine whether each original singing audio clip meets the user vocal range condition, and the original singing audio clip that does not meet the user vocal range condition is used as the audio clip to be adjusted; wherein the user vocal range condition is determined based on the user vocal range information.
[0175] In a possible implementation manner, when executing the instruction to perform pitch shifting processing on the audio segment to be adjusted, the processor 901 is specifically configured to:
[0176] Determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted;
[0177] According to the pitch change attribute of the audio segment to be adjusted, the audio segment to be adjusted is pitched, and the pitch change processing includes high pitch lowering processing and low pitch raising processing.
[0178] In a possible implementation manner, when the processor 901 executes the instruction of determining the pitch shift attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted, it is specifically configured to:
[0179] If the fundamental frequency value of the audio segment to be adjusted exceeds the user's high pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a falling pitch; conversely, if the fundamental frequency value of the audio segment to be adjusted exceeds the user's low pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a rising pitch.
[0180] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are performed:
[0181] Obtain the user's historical singing audio and the original singing audio of the song to be sung;
[0182] Based on historical singing audio, determine the user's vocal range information, which is used to characterize the user's timbre characteristics;
[0183] Determine, according to the user's vocal range information, an audio segment to be adjusted, where the audio segment to be adjusted is an audio segment in the original singing audio that does not satisfy the user's timbre characteristics;
[0184] The audio segment to be adjusted is subjected to pitch shifting processing, and the audio segment subjected to pitch shifting processing and the audio segment of the original audio that is not subjected to pitch shifting processing are synthesized to obtain the adjusted audio.
[0185] In a possible implementation manner, when the processor executes the instruction to determine the user's vocal range information based on the historical singing audio, it is specifically used to:
[0186] Split the historical singing audio into multiple singing audio segments;
[0187] Perform frequency domain analysis on each singing audio clip to determine the duration of the range corresponding to each tone range;
[0188] Perform loudness analysis on each singing audio clip to determine loudness range information;
[0189] Perform pitch analysis on each singing audio clip to determine the pitch range information.
[0190] In a possible implementation manner, when the processor executes the instruction of performing frequency domain analysis on each singing audio segment to determine the range duration corresponding to each range, it is specifically used to:
[0191] Using Fourier transform strategy to perform time-frequency conversion on each singing audio clip, and obtain frequency domain information of each singing audio clip;
[0192] Determine the pitch range corresponding to each singing audio clip according to the frequency domain information of each singing audio clip;
[0193] According to the length of the target singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target singing audio segment is a singing audio segment belonging to the same vocal range among multiple singing audio segments.
[0194] In a possible implementation manner, when the processor executes the instruction of determining the sound range corresponding to each singing audio segment according to the frequency domain information of each singing audio segment, it is specifically used to:
[0195] Acquire preset range boundary information, where the range boundary information includes frequency range information corresponding to each range;
[0196] According to the fundamental frequency information and range boundary information of each singing audio segment, the sound range corresponding to each singing audio segment is determined.
[0197] In a possible implementation manner, when the processor executes the instruction to perform loudness analysis on each singing audio segment and determine the loudness range information, it is specifically used to:
[0198] Performing loudness analysis on each singing audio clip to obtain loudness information of each singing audio clip, wherein the loudness information includes a root mean square value for representing the loudness of the singing audio clip;
[0199] The loudness range information is determined based on the pitch range and loudness information corresponding to each singing audio clip.
[0200] In a possible implementation manner, when the processor executes the instruction to perform pitch analysis on each singing audio segment to determine pitch range information, it is specifically used to:
[0201] Using a preset pitch detection strategy, determine the pitch corresponding to each singing audio clip;
[0202] Determine the pitch range information based on the pitch range and pitch corresponding to each singing audio clip.
[0203] In a possible implementation manner, when the processor executes the instruction to determine the audio segment to be adjusted according to the user's vocal range information, it is specifically configured to:
[0204] Segment the original singing audio to obtain multiple original singing audio segments;
[0205] Performing audio analysis on each original singing audio segment to determine the range information of each original singing audio segment;
[0206] The audio segments to be adjusted are determined according to the user's vocal range information and the vocal range information of each original singing audio segment.
[0207] In a possible implementation manner, when the processor executes the instruction of performing audio analysis on each original singing audio segment to determine the range information of each original singing audio segment, it is specifically used to:
[0208] Performing fundamental frequency detection on each of the original audio clips to obtain fundamental frequency information of each of the original audio clips;
[0209] Determine the pitch range corresponding to each original singing audio segment according to the fundamental frequency information of each original singing audio segment and the preset pitch range boundary information;
[0210] According to the length of the target original singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target original singing audio segment is an original singing audio segment belonging to the same vocal range among multiple original singing audio segments.
[0211] In a possible implementation manner, when the processor executes the instruction to determine the audio segment to be adjusted according to the user's vocal range information and the vocal range information of each original singing audio segment, it is specifically configured to:
[0212] The user vocal range information is compared with the vocal range information of each original singing audio clip to determine whether each original singing audio clip meets the user vocal range condition, and the original singing audio clip that does not meet the user vocal range condition is used as the audio clip to be adjusted; wherein the user vocal range condition is determined based on the user vocal range information.
[0213] In a possible implementation manner, when the processor executes the instruction to perform pitch shift processing on the audio segment to be adjusted, it is specifically configured to:
[0214] Determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted;
[0215] According to the pitch change attribute of the audio segment to be adjusted, the audio segment to be adjusted is pitched, and the pitch change processing includes high pitch lowering processing and low pitch raising processing.
[0216] In a possible implementation manner, when the processor executes the instruction of determining the pitch shift attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted, the processor is specifically configured to:
[0217] If the fundamental frequency value of the audio segment to be adjusted exceeds the user's high pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a falling pitch; conversely, if the fundamental frequency value of the audio segment to be adjusted exceeds the user's low pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a rising pitch.
[0218] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0219] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.
[0220] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.
Claims
1. An audio adjustment method, characterized in that: include: Obtain the user's historical singing audio and the original singing audio of the song to be sung; Determine user vocal range information based on the historical singing audio, where the user vocal range information is used to characterize the user's timbre characteristics; Determining an audio segment to be adjusted according to the user's vocal range information, wherein the audio segment to be adjusted is an audio segment in the original singing audio that does not meet the timbre characteristics of the user; The audio segment to be adjusted is subjected to pitch shifting processing, and the audio segment subjected to pitch shifting processing and the audio segment in the original audio that is not subjected to pitch shifting processing are synthesized to obtain the adjusted audio.
2. The audio adjustment method according to claim 1, characterized in that: The user vocal range information includes the vocal range duration, pitch range information and loudness range information corresponding to different vocal zones; The determining of the user's vocal range information based on the historical singing audio includes: Dividing the historical singing audio into a plurality of singing audio segments; Performing frequency domain analysis on each of the singing audio clips to determine the duration of the range corresponding to each range; Performing loudness analysis on each of the singing audio clips to determine the loudness range information; Perform pitch analysis on each of the singing audio clips to determine the pitch range information.
3. The audio adjustment method according to claim 2, characterized in that: The performing frequency domain analysis on each of the singing audio clips to determine the range duration corresponding to each range includes: Performing time-frequency conversion processing on each of the singing audio clips using a Fourier transform strategy to obtain frequency domain information of each of the singing audio clips; Determine the sound range corresponding to each of the singing audio clips according to the frequency domain information of each of the singing audio clips; According to the length of the target singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target singing audio segment is a singing audio segment belonging to the same vocal range among multiple singing audio segments.
4. The audio adjustment method according to claim 3, characterized in that: The step of determining the sound range corresponding to each of the singing audio segments according to the frequency domain information of each of the singing audio segments comprises: Acquire preset range boundary information, wherein the range boundary information includes frequency range information corresponding to each sound zone; The sound range corresponding to each of the singing audio segments is determined according to the fundamental frequency information and the sound range boundary information of each of the singing audio segments.
5. The audio adjustment method according to claim 2, characterized in that: The step of performing loudness analysis on each of the singing audio clips to determine the loudness range information includes: Performing loudness analysis on each of the singing audio segments to obtain loudness information of each of the singing audio segments, wherein the loudness information includes a root mean square value for representing the loudness of the singing audio segment; The loudness range information is determined according to the pitch range corresponding to each of the singing audio segments and the loudness information.
6. The audio adjustment method according to claim 2, characterized in that: The step of performing pitch analysis on each of the singing audio segments to determine the pitch range information includes: Using a preset pitch detection strategy, determine the pitch corresponding to each of the singing audio clips; The pitch range information is determined according to the pitch range and the pitch corresponding to each of the singing audio clips.
7. The audio adjustment method according to claim 1, characterized in that: The step of determining the audio segment to be adjusted according to the user's voice range information includes: Segmenting the original singing audio to obtain multiple original singing audio segments; Performing audio analysis on each of the original singing audio segments to determine the range information of each of the original singing audio segments; The audio segment to be adjusted is determined according to the user's vocal range information and the vocal range information of each of the original audio segments.
8. The audio adjustment method according to claim 7, characterized in that: The step of performing audio analysis on each of the original singing audio segments to determine the range information of each of the original singing audio segments includes: Performing fundamental frequency detection on each of the original audio clips to obtain fundamental frequency information of each of the original audio clips; Determine the sound range corresponding to each of the original singing audio segments according to the fundamental frequency information of each of the original singing audio segments and the preset sound range boundary information; According to the length of the target original singing audio segment corresponding to each vocal range, the duration of the vocal range corresponding to each vocal range is determined, and the target original singing audio segment is an original singing audio segment belonging to the same vocal range among multiple original singing audio segments.
9. The audio adjustment method according to claim 8, characterized in that: The step of determining the audio segment to be adjusted according to the user's vocal range information and the vocal range information of each of the original singing audio segments includes: The user vocal range information is compared with the vocal range information of each of the original audio clips to determine whether each of the original audio clips meets the user vocal range condition, and the original audio clips that do not meet the user vocal range condition are used as the audio clips to be adjusted; wherein the user vocal range condition is determined based on the user vocal range information.
10. The audio adjustment method according to claim 1, characterized in that: The performing pitch change processing on the audio segment to be adjusted includes: Determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted; According to the pitch change attribute of the audio segment to be adjusted, the audio segment to be adjusted is subjected to pitch change processing, wherein the pitch change processing includes high pitch lowering processing and low pitch raising processing.
11. The audio adjustment method according to claim 10, characterized in that: The determining the pitch change attribute of the audio segment to be adjusted according to the fundamental frequency information of the audio segment to be adjusted includes: If the fundamental frequency value of the audio segment to be adjusted exceeds the user's high pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a falling pitch; conversely, if the fundamental frequency value of the audio segment to be adjusted exceeds the user's low pitch range, the pitch change attribute of the audio segment to be adjusted is determined to be a rising pitch.
12. An audio adjustment device, characterized in that: include: An acquisition module is used to acquire the user's historical singing audio and the original singing audio of the song to be sung; A first determination module, used to determine user vocal range information based on the historical singing audio, wherein the user vocal range information is used to characterize the user's timbre characteristics; A second determining module is used to determine an audio segment to be adjusted according to the user's vocal range information, wherein the audio segment to be adjusted is an audio segment in the original singing audio that does not meet the timbre characteristics of the user; The processing module is used to perform pitch shifting processing on the audio segment to be adjusted, and to synthesize the pitch shifted audio segment and the audio segment in the original audio that has not been pitch shifted, so as to obtain the adjusted audio.
13. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the steps of the audio adjustment method as described in any one of claims 1 to 11.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the audio adjustment method according to any one of claims 1 to 11 are executed.
Citation Information
Cited By
Audio processing method and device, electronic equipment and storage medium
CN119943012A
Fitting system and method of original singing audio information, and computer storage medium
CN121838696A
A method and system for generating audio from sheet music
CN122929980A
