Audio splicing method, device and storage medium

Abstract

The present application discloses an audio splicing method, device and storage medium in the field of audio processing technology. After obtaining two audio clips to be spliced, the present application first determines the overlapping part of the two audio clips and the fundamental frequency of the two audio clips in the overlapping part; if the fundamental frequency pitch difference of the two audio clips exceeds the pitch threshold, the duration of the overlapping part of the two audio clips is increased, and after the two audio clips are updated based on the overlapping part after the increased duration, the splicing operation is performed on the updated two audio clips. For any two audio clips to be spliced, when it is determined that the fundamental frequency difference between the two is large, the scheme first increases the overlapping part of the two audio clips to make the two audio clips as continuous as possible after splicing, so that the sound of the splicing result is relatively natural, and there will be no jamming or strange sound. Correspondingly, the audio splicing device and storage medium provided by the present application also have the above-mentioned technical effects.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to an audio splicing method, device and storage medium. Background Art

[0002] Currently, existing audio splicing technologies, such as Ola (Overlap and add) or Crossfade (fade in and out), can splice different voice clips. However, these splicing technologies are only applicable to splicing human voices. For singing, especially high-pitched singing, splicing often causes freezes and strange sounds. Summary of the invention

[0003] In view of this, the purpose of this application is to provide an audio splicing method, device and storage medium, which can splice various audio clips, and the splicing result sounds relatively natural without any freeze or strange sound. The specific scheme is as follows:

[0004] To achieve the above objectives, on the one hand, the present application provides an audio splicing method, comprising:

[0005] Obtain a first audio clip and a second audio clip to be spliced;

[0006] Determine an overlapping portion between the tail of the first audio segment and the head of the second audio segment, extract a first fundamental frequency of the first audio segment in the overlapping portion, and extract a second fundamental frequency of the second audio segment in the overlapping portion;

[0007] If the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds a pitch threshold, increasing the duration of the overlapping portion;

[0008] updating the first audio segment and the second audio segment based on the overlapped portion after the duration is increased;

[0009] A splicing operation is performed on the updated first audio segment and the updated second audio segment.

[0010] Preferably, the pitch threshold is two semitones.

[0011] Preferably, increasing the duration of the overlapping portion comprises:

[0012] The duration of the increased overlapping portion is determined based on a preset sampling rate and the pitch difference.

[0013] Preferably, the determining the duration of the increased overlapping portion based on the preset sampling rate and the pitch difference comprises:

[0014] The product of the preset sampling rate and the pitch difference is calculated, and the duration of the increased overlapping portion is determined based on the product.

[0015] Preferably, it also includes:

[0016] If there is an invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency, or if there is no invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency but the pitch difference does not exceed the pitch threshold, a splicing operation is performed on the first audio segment and the second audio segment.

[0017] Preferably, it also includes:

[0018] Determining a difference in signal amplitude between overlapping portions of two audio clips to be spliced;

[0019] If the signal amplitude difference is not less than the amplitude threshold and the inverse phase splicing meets the preset condition, the phase of an audio segment at the subsequent splicing position is flipped.

[0020] Preferably, the step of determining the signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​comprises:

[0021] The signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​is calculated according to the first formula; the first formula is:

[0022] Where Δp represents the audio segment x on which the splicing operation is performed. A and audio clip x B The signal amplitude difference of the overlapping part, L is the audio segment x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

[0023] Preferably, performing a splicing operation on two audio clips includes:

[0024] The concatenation operation is performed on the two audio clips based on a second formula, where the second formula is:

[0025]

[0026] Where y(n) represents the audio segment x A and audio clip x B The splicing result at sampling point n, L is the audio segment x A and audio clip x B The length of the overlapping part, w(l) is the window function, N A For audio clip x A The length, N BFor audio clip x B The length of , l = [0,L].

[0027] Preferably, the preset condition is: - -p T |<|p + -p T |;

[0028] Among them, p - is the phase difference of the overlapping part, p + is the phase sum of the overlapping parts, For audio clip x A The mean of the absolute values ​​of the signal amplitudes in the overlapping parts, For audio clip x B The mean absolute value of the signal amplitude in the overlapping part, p T =(p A +p B ) / 2, L is the audio clip x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

[0029] On the other hand, the present application also provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned audio splicing method.

[0030] On the other hand, the present application further provides a storage medium, in which computer executable instructions are stored. When the computer executable instructions are loaded and executed by a processor, the aforementioned audio splicing method is implemented.

[0031] The audio splicing method provided by the present application, after obtaining the first audio segment and the second audio segment to be spliced, first determines the overlapping part of the tail of the first audio segment and the head of the second audio segment, extracts the first fundamental frequency of the first audio segment in the overlapping part, and extracts the second fundamental frequency of the second audio segment in the overlapping part; if the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds the pitch threshold, it indicates that the fundamental frequency difference of the two audio segments is large, and the direct splicing operation is prone to jamming and strange sound. To solve this problem, increase the duration of the overlapping part; update the first audio segment and the second audio segment based on the increased overlapping part; perform the splicing operation on the updated first audio segment and the updated second audio segment. This scheme increases the duration of the overlapping part of the two audio segments, thereby increasing the possibility of signal continuity of the two audio segments. It can be seen that for any two audio segments to be spliced, when it is determined that the fundamental frequency difference between the two is large, the scheme first increases the duration of the overlapping part of the two audio segments, so that the two audio segments are as continuous as possible after splicing, so that the splicing result can be relatively natural to hear, without jamming and strange sound.

[0032] Correspondingly, the audio splicing device and storage medium provided by this application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0034] Figure 1 A schematic diagram of a physical architecture applicable to the present application provided for the present application;

[0035] Figure 2 A flow chart of an audio splicing method provided by this application;

[0036] Figure 3 A schematic diagram of the comparison of an overlapping portion before and after enlargement provided by the present application;

[0037] Figure 4 Another flow chart of the audio splicing method provided by this application;

[0038] Figure 5 A comparison diagram of the splicing effects of the present invention and the prior art provided in this application;

[0039] Figure 6 An application diagram of an audio splicing solution provided in this application;

[0040] Figure 7A server structure diagram provided for this application;

[0041] Figure 8 A terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, in the embodiments of the present application, "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0043] Existing audio splicing technologies, such as Ola (Overlap and add) or Crossfade (fade in and out), can splice different voice clips. However, these splicing technologies are only suitable for splicing human voices. For singing, especially high-pitched singing, splicing often causes jamming and strange sounds. In view of the above-mentioned problems, the present application proposes an audio splicing solution, which can splice various audio clips, and the splicing result sounds relatively natural without jamming or strange sounds.

[0044] For ease of understanding, the physical framework to which this application applies is first introduced.

[0045] It should be understood that the audio splicing method provided in the present application can be applied to a system or program with an audio splicing function. Specifically, the system or program with an audio splicing function can be run in a server, a personal computer, or other device.

[0046] like Figure 1 As shown, Figure 1 This is a schematic diagram of the physical architecture applicable to this application. Figure 1 In the embodiment, a system or program with an audio splicing function can be run on a server. After the server obtains a first audio segment and a second audio segment to be spliced ​​from other terminal devices through a network, the server first determines the overlapping part between the tail of the first audio segment and the head of the second audio segment, extracts the first fundamental frequency of the first audio segment in the overlapping part, and extracts the second fundamental frequency of the second audio segment in the overlapping part; if the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds a pitch threshold, increases the duration of the overlapping part; updates the first audio segment and the second audio segment based on the overlapping part after the increased duration, and performs a splicing operation on the updated first audio segment and the updated second audio segment.

[0047] Depend on Figure 1 It can be seen that the server can establish communication connections with multiple devices, and the server obtains audio clips to be spliced ​​from these devices, and then the server splices the obtained audio clips according to the solution provided in this application.

[0048] Figure 1 A variety of terminal devices are shown in FIG. In actual scenarios, more or fewer types of terminal devices may participate in the audio splicing process. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in an actual scenario, multiple servers may be involved, and the specific number of servers depends on the actual scenario.

[0049] It should be noted that the audio splicing method provided in this embodiment can be performed offline, that is, the server locally stores the audio segments to be spliced, and it can directly use the solution provided in this application to splice the audio.

[0050] It is understandable that the above-mentioned system and program with audio splicing function can also be run on a personal mobile terminal or as a cloud service program. The specific operation mode depends on the actual scenario and is not limited here.

[0051] Specifically, after splicing multiple audio segments, you can get spliced ​​songs, character dialogues, etc. It can be seen that this solution can be used in tools such as voice editing and song editing.

[0052] Combined with the above commonalities, please see Figure 2 , Figure 2 A flow chart of an audio splicing method provided in an embodiment of the present application. Figure 2 As shown, the audio splicing method may include the following steps:

[0053] S201: Acquire a first audio segment and a second audio segment to be spliced.

[0054] In this embodiment, the first audio segment and the second audio segment may be audio of a person's speech or audio of a song.

[0055] S202: Determine an overlapping portion between the tail of the first audio segment and the head of the second audio segment, extract a first fundamental frequency of the first audio segment in the overlapping portion, and extract a second fundamental frequency of the second audio segment in the overlapping portion.

[0056] Audio processing is generally performed in frames. In order to ensure the continuity of the signal, adjacent frames will be repeated during framing. For example, when the unit frame length is 20ms (milliseconds), the first frame is 0ms~20ms, then the second frame can be 10ms~30ms, and the 10ms~20ms is the overlapping part of the two. Generally, the overlapping part is half, one third, etc. of the unit frame length, and the specific value can be flexibly determined according to the actual situation. Accordingly, the first audio segment and the second audio segment usually refer to: two frames with a length equal to the unit frame length, such as the first frame 0ms~20ms and the second frame 10ms~30ms in the above example.

[0057] It should be noted that the signals of the 10ms to 20ms part of the time overlap between the first frame and the second frame in the above example are not necessarily the same. Therefore, it is not appropriate to directly assume that the signal difference between the first frame and the second frame is only: 0ms to 10ms, 20ms to 30ms, and the signal difference of the overlapping part of the two frames also needs to be considered.

[0058] In order to distinguish the signal difference between the first audio clip and the second audio clip in the overlapping part, this embodiment extracts the first fundamental frequency of the first audio clip in the overlapping part, extracts the second fundamental frequency of the second audio clip in the overlapping part, and judges the pitch difference between the first fundamental frequency and the second fundamental frequency when the extracted first fundamental frequency and the second fundamental frequency are both valid. If the pitch difference between the first fundamental frequency and the second fundamental frequency (i.e., the tone difference) exceeds the pitch threshold, it means that the signal difference between the first audio clip and the second audio clip in the overlapping part is large, and it is difficult to eliminate the signal mutation by directly using the splicing technology for splicing, and it is easy to cause discontinuity of hearing, strange sound, etc. If there is an invalid fundamental frequency in the first fundamental frequency and the second fundamental frequency, or there is no invalid fundamental frequency in the first fundamental frequency and the second fundamental frequency but the pitch difference between the first fundamental frequency and the second fundamental frequency does not exceed the pitch threshold, it means that the signal difference between the first audio clip and the second audio clip in the overlapping part is not too large, and the direct use of the splicing technology can basically eliminate the signal mutation and ensure the connectivity of the signal splicing, so there is no need to increase the duration of the overlapping part. Therefore, in a specific implementation, if there is an invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency, or if there is no invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency but the height difference does not exceed the pitch threshold, a splicing operation is performed on the first audio segment and the second audio segment.

[0059] Among them, open source tools such as pYin can be used to extract the fundamental frequency in the audio and convert it into pitch information. In a specific implementation, the pitch threshold is two semitones. Generally, when the fundamental frequency changes by more than 4 semitones, strange sounds often appear at the splicing transition position, while Ola and Crossfade can eliminate the pitch differences within two semitones.

[0060] S203: If the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds the pitch threshold, increase the duration of the overlapping portion, update the first audio segment and the second audio segment based on the overlapping portion after the increased duration, and perform a splicing operation on the updated first audio segment and the updated second audio segment.

[0061] In order to solve the problem that the signal difference between the first audio segment and the second audio segment in the overlapping part is large, this embodiment increases the overlapping part, so that after the first audio segment and the second audio segment are updated based on the increased overlapping part, a splicing operation is performed on the updated first audio segment and the updated second audio segment.

[0062] See also Figure 3 , assuming that the first frame is 0ms~20ms and the second frame is 10ms~30ms, the overlap between the two is 10ms~20ms. If the overlap is increased by 5ms and the starting position of each audio segment remains unchanged, the new overlap is 10ms~25ms, and then the first frame is updated from 0ms~20ms to 0ms~25ms, and the second frame is updated from 10ms~30ms to 10ms~35ms.

[0063] It can be seen that after obtaining the first audio segment and the second audio segment to be spliced, the present embodiment first determines the overlapping part of the tail of the first audio segment and the head of the second audio segment, extracts the first fundamental frequency of the first audio segment in the overlapping part, and extracts the second fundamental frequency of the second audio segment in the overlapping part; if the first fundamental frequency and the second fundamental frequency are both valid, and the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds the pitch threshold, it indicates that the fundamental frequency difference of the two audio segments is large, and the direct splicing operation is prone to jamming and strange sound. To solve this problem, the scheme increases the overlapping part of the two to increase the possibility of signal continuity of the two audio segments, and after updating the first audio segment and the second audio segment based on the increased overlapping part, the splicing operation is performed on the updated first audio segment and the updated second audio segment. It can be seen that for any two audio segments to be spliced, when it is determined that the fundamental frequency difference between the two is large, the scheme first increases the overlapping part of the two to make the two audio segments as continuous as possible after splicing, so that the splicing result can be relatively natural to hear, and there will be no jamming and strange sound.

[0064] Based on the above embodiment, it should be noted that, when both the first fundamental frequency and the second fundamental frequency are valid and the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds the pitch threshold, how much the overlapping part can be increased can generally be determined based on the value range of the unit frame length. For example, assuming that the value range of the unit frame length is Xms to Yms, then the length of the overlapping part can be changed at will if the length of the updated segment (i.e., the unit frame length) does not exceed the value range of the unit frame length.

[0065] In a specific embodiment, increasing the duration of the overlapping portion includes: determining the increased duration of the overlapping portion based on a preset sampling rate and a pitch difference. Wherein, determining the increased duration of the overlapping portion based on a preset sampling rate and a pitch difference includes: calculating the product of the preset sampling rate and the pitch difference, and determining the increased duration of the overlapping portion based on the product. For example: taking half of the product of the preset sampling rate and the pitch difference as the increased duration of the overlapping portion, and rounding up if the division is not exact. Of course, the degree of increase of the overlapping portion can be flexibly set in the case where the length of the updated segment does not exceed the value range of the unit frame length.

[0066] Based on the above embodiment, it should be noted that, in a specific implementation, it further includes: determining the signal amplitude difference of the overlapping parts of the two audio clips to be spliced; if the energy difference is not less than the amplitude threshold and the inverse splicing meets the preset conditions, then performing a phase flip on the audio clip at the later splicing position. Otherwise, directly splicing the two audio clips.

[0067] In a specific implementation, determining the signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​includes: calculating the signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​according to a first formula; the first formula is: Where Δp represents the audio segment x on which the splicing operation is performed. A and audio clip x B The signal amplitude difference of the overlapping part, L is the audio segment x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

[0068] In a specific implementation, performing a splicing operation on two audio clips includes: performing a splicing operation on the two audio clips based on a second formula, where the second formula is:

[0069]

[0070] Where y(n) represents the audio segment x A and audio clip x B The splicing result at sampling point n, L is the audio segment x A and audio clip x B The length of the overlapping part, w(l) is the window function, N A For audio clip x A The length, N B For audio clip x BThe length of the final concatenated signal is N. y =N A +N B -L.

[0071] The preset conditions are: - -p T |<|p + -p T |; Among them, p - is the phase difference of the overlapping part, p + is the phase sum of the overlapping parts, For audio clip x A The mean of the absolute values ​​of the signal amplitudes in the overlapping parts, For audio clip x B The mean absolute value of the signal amplitude in the overlapping part, p T =(p A +p B ) / 2, L is the audio clip x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

[0072] See also Figure 4 In a specific implementation, the audio splicing process may include:

[0073] 1. Get audio clips A and B.

[0074] 2. Extract the fundamental frequency.

[0075] Specifically, open source tools such as pYin can be used to extract the fundamental frequency (i.e., fundamental frequency) and convert it into pitch information. The conversion formula is: c=12·log2(f / 440)+69, where c represents the note number and f represents the fundamental frequency value.

[0076] 3. Determine the transition duration (i.e. the length of the overlapping part).

[0077] If the end of audio clip A and the beginning of B (i.e. the overlapping part) are unvoiced, the transition duration remains unchanged, and the existing method (such as crossfade) is used for subsequent splicing. Unvoiced means that the signals of the overlapping part of A and B have no valid fundamental frequency, so there is no need to consider the fundamental frequency difference at this time.

[0078] If the signals of the overlapping parts of A and B are both voiced sounds, then the effective fundamental frequency can be extracted from them. When the pitch change of the two extracted effective fundamental frequencies is within 2 semitones (including 2 semitones), the transition duration remains unchanged, and the existing method is used for splicing. At this time, it is believed that the existing method can eliminate the fundamental frequency difference within 2 semitones (including 2 semitones).

[0079] If the pitch variation of the two extracted effective fundamental frequencies exceeds 2 semitones, the transition duration is increased to allow the overlapping parts to transition naturally and prevent freezes. Where Δc represents the pitch difference between two effective fundamental frequencies, L is the sampling rate, and the symbol Indicates rounding up.

[0080] If the transition duration remains unchanged, the existing method is used to splice the two original audio segments A and B. If the transition duration is increased, the audio segments A and B need to be updated based on the increased transition duration, and then the updated A and the updated B are spliced. The update steps of A and B can refer to the relevant introduction above, which will not be repeated here.

[0081] 4. Determine whether the phase of B at the subsequent splicing position needs to be flipped.

[0082] Calculate the energy difference of the two segments to be spliced ​​in the overlapping part. If the energy difference is too large, flip the phase of B when the anti-phase splicing meets the preset conditions; otherwise, keep the phase of B unchanged.

[0083] Specifically, calculate in the range of l = [0, L]: and Then calculate Δp=p + -p - , L represents the duration of the overlapping part of the two segments to be spliced. + is the phase sum of the overlapping parts, p - is the phase difference of the overlapping part, which is the sum of the phases minus the phase difference. The larger the Δp, the greater the energy difference of the overlapping part; otherwise, it indicates that the energy difference of the overlapping part is small. When the energy difference is small, flipping the phase of B can make the phase of the splicing result more continuous.

[0084] Specifically, if two audio segments satisfy x A =-x B , then x A +x B = 0, that is, the amplitude of the spliced ​​part is 0, which is unacceptable. Because there will be a lag in the sense of hearing. Therefore, this embodiment uses this step to perform splicing prejudgment in order to confirm x A +x B The amplitude value is within a reasonable range, not a very small value.

[0085] Therefore, the larger the Δp, the more positive the splicing (i.e., splicing x A and x B ) and reverse splicing (i.e. splicing x A and -x B ), so when Δp ≥ Thr, the phase of B needs to be suitable for splicing. When Δp < Thr, whether the phase of B is flipped has little effect on the splicing result, so the phase of B can be kept unchanged.

[0086] Based on the above, it can be determined that: if Δp<Thr, then x B (n) = x B (n). If Δp≥Thr and meets the preset condition |p - -p T |<|p + -p T |When x B (n) = -x B (n), where n∈[0,N B ], where the threshold Thr is an empirical value, such as: Thr = -0.002. If Δp ≥ Thr, but does not meet the preset condition |p - -p T |<|p + -p T |When x B (n) = x B (n).

[0087] in, Represents x A The mean of the absolute values ​​of the amplitudes at the end; Represents x B The mean of the absolute values ​​of the amplitudes at the beginning; p T =(p A +p B ) / 2, indicating a compromise value. - With p T The difference (i.e. |p - -p T |) is less than p + With p T The difference (i.e. |p + -p T |), it means that the inverted splicing can make the phase of the splicing result more continuous, so flip x B On the contrary, if Δp ≥ Thr, but does not meet the preset condition |p - -p T |<|p + -p T |, then whether the phase of B is flipped has little effect on the splicing result, and x can be directly spliced. A and xB , without flipping x B phase.

[0088] 5. Splice two audio clips.

[0089] The formula for concatenating two audio clips is:

[0090]

[0091] Where y(n) represents the audio segment x A and audio clip x B The splicing result at sampling point n, L is the audio segment x A and audio clip x B The length of the overlapping part, w(l) is the window function, N A For audio clip x A The length, N B For audio clip x B The length of N, l = [0, L]. y =N A +N B -L

[0092] Wherein, w(l) can be a raised cosine window function, a Hamming window function, etc. Among them, the raised cosine window function can make the spliced ​​signal still maintain the best amplitude change. When using a function such as a Hamming window function to add a window, if the length of the overlapping part is not half of the unit frame length, consider post-processing the signal after splicing to remove the amplitude jitter.

[0093] When w(l) is a raised cosine window function,

[0094] It can be seen that this embodiment can meet the naturalness and smoothness of splicing for any audio, can achieve natural fundamental frequency transition for high-pitched singing, and reduce the problems of stuttering and strange sound in the process of singing signal synthesis / splicing. Figure 5 .like Figure 5 As shown, Figure 5 At A in the middle, the existing general splicing solution (such as crossfade) is directly used to complete the splicing, while Figure 5 The splicing is completed at B using this embodiment, and the two solutions are spliced ​​for the same pair of audio clips. Specifically, the splicing effect at A corresponds to Figure 5 The waveform indicated by C in the figure has an obvious sudden increase, the waveform is not smooth enough, and it is easy to have a strange sound; while the splicing effect at B corresponds to Figure 5 The waveform indicated by D and D in the figure has reduced the obvious sudden increase, and the waveform rises more slowly, thus reducing the probability of abnormal situations such as strange sounds, and making the sound of the spliced ​​part relatively natural.

[0095] The following describes a specific application scenario example to introduce the audio splicing solution provided by the present application. The following describes a specific application scenario example to introduce the solution provided by the present application, which can splice any audio.

[0096] See also Figure 6 The specific process includes:

[0097] S601, the user selects several audios to be spliced ​​on the terminal;

[0098] S602, the terminal sends the audio selected by the user to the server;

[0099] S603: After receiving the audio, the server divides the audio into frames, and then executes the method provided in the present application for the audio frames at the splicing position, and finally obtains the spliced ​​audio;

[0100] S604, the server sends the spliced ​​audio to the terminal;

[0101] S605: After the terminal receives the spliced ​​audio, the user can play the spliced ​​audio.

[0102] The terminal may be a smart phone, a television, a tablet computer, a laptop computer, a desktop computer or other device.

[0103] In actual application, the present application can be set in an audio and video editing tool, which is installed on a user terminal, so that the user can splice audio in real time.

[0104] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below and related implementation steps of the above embodiment can be referenced to each other.

[0105] Furthermore, the present application also provides an electronic device. The electronic device can be Figure 7 The server 50 shown may also be Figure 8 Terminal 60 is shown. Figure 7 and Figure 8 All of them are structural diagrams of electronic devices according to an exemplary embodiment, and the contents in the diagrams cannot be regarded as any limitation on the scope of use of the present application.

[0106] Figure 7A schematic diagram of the structure of a server provided in an embodiment of the present application. The server 50 may specifically include: at least one processor 51, at least one memory 52, a power supply 53, a communication interface 54, an input / output interface 55, and a communication bus 56. The memory 52 is used to store a computer program, which is loaded and executed by the processor 51 to implement the relevant steps in the audio splicing disclosed in any of the aforementioned embodiments.

[0107] In this embodiment, the power supply 53 is used to provide working voltage for each hardware device on the server 50; the communication interface 54 can create a data transmission channel between the server 50 and external devices, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 55 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0108] In addition, the memory 52, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 521, a computer program 522 and data 523, etc. The storage method can be temporary storage or permanent storage.

[0109] The operating system 521 is used to manage and control the hardware devices and computer programs 522 on the server 50, so as to realize the operation and processing of the data 523 in the memory 52 by the processor 51, and it can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the audio splicing method disclosed in any of the aforementioned embodiments, the computer program 522 can further include a computer program that can be used to complete other specific tasks. In addition to including data such as application update information, the data 523 can also include data such as application developer information.

[0110] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal 60 may specifically include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0111] Generally, the terminal 60 in this embodiment includes: a processor 61 and a memory 62 .

[0112] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0113] The memory 62 may include one or more computer-readable storage media, which may be non-transitory. The memory 62 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 62 is at least used to store the following computer program 621, wherein, after the computer program is loaded and executed by the processor 61, the relevant steps in the audio splicing method performed by the terminal side disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 62 may also include an operating system 622 and data 623, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 622 may include Windows, Unix, Linux, etc. Data 623 may include, but is not limited to, update information of the application.

[0114] In some embodiments, the terminal 60 may further include a display screen 63 , an input / output interface 64 , a communication interface 65 , a sensor 66 , a power supply 67 , and a communication bus 68 .

[0115] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 60, and the terminal 60 may include more or less components than those shown in the figure.

[0116] A storage medium provided in an embodiment of the present application is introduced below. The storage medium described below and the related implementation steps of the above embodiment can be referenced to each other.

[0117] Furthermore, the embodiment of the present application also discloses a storage medium, in which a computer executable instruction is stored, and when the computer executable instruction is loaded and executed by a processor, the audio splicing method disclosed in any of the above embodiments is implemented. For the specific steps of the method, reference can be made to the corresponding contents disclosed in the above embodiments, and no further description will be given here.

[0118] It should be pointed out that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An audio splicing method, characterized in that: include: Obtain a first audio clip and a second audio clip to be spliced; Determine an overlapping portion between the tail of the first audio segment and the head of the second audio segment, extract a first fundamental frequency of the first audio segment in the overlapping portion, and extract a second fundamental frequency of the second audio segment in the overlapping portion; If the pitch difference between the first fundamental frequency and the second fundamental frequency exceeds the pitch threshold, the duration of the overlapping portion is increased; wherein the length of the overlapping portion after the increased duration does not exceed the value range of the unit frame length; wherein the increasing the duration of the overlapping portion comprises: determining the increased duration of the overlapping portion based on a preset sampling rate and the pitch difference; updating the first audio segment and the second audio segment based on the overlapped portion after the duration is increased; Performing a splicing operation on the updated first audio segment and the updated second audio segment; Among them, it also includes: Determining a difference in signal amplitude between overlapping portions of two audio clips to be spliced; If the signal amplitude difference is not less than the amplitude threshold and the inverse phase splicing meets the preset conditions, the phase of the audio segment at the later splicing position is flipped; otherwise, the two audio segments are directly spliced; Among them, the preset condition is: - -p T |<|p + -p T |; Among them, p - is the phase difference of the overlapping part, p + is the phase sum of the overlapping parts, p T =(p A +p B ) / 2, p A The audio segment x to be spliced A The mean absolute value of the signal amplitude in the overlapping part, p B The audio segment x to be spliced B The mean of the absolute values ​​of the signal amplitudes in the overlapping parts.

2. The method according to claim 1, characterized in that The pitch threshold is two semitones.

3. The method according to claim 1, characterized in that The determining the duration of the increased overlapping portion based on the preset sampling rate and the pitch difference comprises: The product of the preset sampling rate and the pitch difference is calculated, and the duration of the increased overlapping portion is determined based on the product.

4. The method according to claim 1, characterized in that: Also includes: If there is an invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency, or if there is no invalid fundamental frequency between the first fundamental frequency and the second fundamental frequency but the pitch difference does not exceed the pitch threshold, a splicing operation is performed on the first audio segment and the second audio segment.

5. The method according to claim 1, characterized in that: The step of determining the signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​includes: The signal amplitude difference of the overlapping parts of the two audio clips to be spliced ​​is calculated according to the first formula; the first formula is: Where Δp represents the audio segment x on which the splicing operation is performed. A and audio clip x B The signal amplitude difference of the overlapping part, L is the audio segment x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

6. The method according to claim 1, characterized in that L is the audio segment x A and audio clip x B The length of the overlapping part, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

7. The method according to any one of claims 1 to 4, characterized in that: Performs a concatenation operation on two audio clips, including: The concatenation operation is performed on the two audio clips based on a second formula, where the second formula is: Where y(n) represents the audio segment x A and audio clip x B The splicing result at sampling point n, L is the audio segment x A and audio clip x B The length of the overlapping part, w(l) is the window function, N A For audio clip x A The length, N B For audio clip x B The length of , l = [0,L].

8. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images