A voice mimicry comparison method, apparatus and related products

Abstract

The application discloses a sound imitation comparison method and device and related products. Based on frame features, frame-level distances between target frames in a target feature sequence and imitation frames in an imitation feature sequence are calculated to obtain a plurality of frame-level distances; corresponding positions between a first audio frame in the target feature sequence and each imitation frame are respectively taken as candidate path starting points, audio frame paths are searched from each candidate path starting point based on the plurality of frame-level distances, and an audio frame path with the minimum candidate cumulative distance is determined as a target alignment path; based on the target alignment path, frame features that are aligned with each other in the target feature sequence and the imitation feature sequence are calculated and processed to obtain a spectral distance, a frequency band profile distance, an energy distance and a fundamental frequency distance; based on the spectral distance, the frequency band profile distance, the energy distance and the fundamental frequency distance, calculation and processing are performed to obtain an imitation similarity score of an imitation audio relative to a target audio. In this way, the adaptability and accuracy of sound imitation comparison are improved.

Description

Technical Field

[0001] This application relates to the field of audio signal processing technology, and in particular to a sound imitation and comparison method, apparatus and related products. Background Technology

[0002] With the development of audio processing technology, voice imitation comparison has been widely used in scenarios such as game interaction, dubbing practice, and language repetition. In the process of voice imitation, a target audio is usually provided in advance, and the imitator speaks according to the target audio to generate the imitation audio. Then, the target audio and the imitation audio need to be compared to determine the degree of similarity between the imitation audio and the target audio.

[0003] In related technologies, audio processing is typically performed on the target audio and the imitation audio to extract the corresponding audio features of the two audio segments, and the similarity between the two audio segments is determined by the differences between the audio features. However, the actual generated imitation audio is easily affected by factors such as the recording environment, the start time of recording, and the duration of the sound, which may cause differences in time position and sound features between the target audio and the imitation audio, making it difficult to accurately reflect the actual degree of imitation of the target audio by the imitation audio.

[0004] Therefore, how to improve the accuracy of sound imitation comparison results is a key issue of concern to those skilled in the art. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a sound imitation comparison method, apparatus, and related products to improve the accuracy of sound imitation comparison results.

[0006] The embodiments of this application disclose the following technical solutions: The first aspect of this application provides a sound imitation comparison method. This sound imitation comparison method includes: Acquire the target audio and the mimicry audio generated for the target audio; Audio processing is performed on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in chronological order of audio frames, and the frame features include spectral features, energy features and fundamental frequency features; Based on the frame features, calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence to obtain multiple frame-level distances; Using the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, audio frame paths are searched from each candidate path starting point based on the multiple frame-level distances, and the audio frame path with the smallest cumulative candidate distance is determined as the target alignment path. Based on the target alignment path, the frame features aligned with each other in the target feature sequence and the imitation feature sequence are calculated to obtain spectral distance, frequency band profile distance, energy distance, and fundamental frequency distance; wherein, the spectral distance is used to characterize the energy distribution difference of each pair of aligned frames in the target alignment path across multiple frequency bands, the frequency band profile distance is used to characterize the difference between the target frequency band energy profile and the imitation frequency band energy profile formed after grouping the multiple frequency bands, the energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, and the fundamental frequency distance is used to characterize the tone difference between each pair of aligned frames in the target alignment path; The similarity score of the imitated audio relative to the target audio is obtained by calculating the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance.

[0007] A second aspect of this application provides a sound imitation comparison device. The sound imitation comparison device includes: An audio acquisition unit is used to acquire target audio and simulated audio generated for the target audio; A feature sequence acquisition unit is used to perform audio processing on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in the time order of audio frames, and the frame features include spectral features, energy features and fundamental frequency features; A frame-level distance acquisition unit is used to calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence based on the frame features, and obtain multiple frame-level distances; The target path determination unit is used to take the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, search for audio frame paths from each of the candidate path starting points based on the multiple frame-level distances, and determine the audio frame path with the smallest cumulative candidate distance as the target alignment path. Multiple distance acquisition units are configured to calculate and process the frame features that are aligned with each other in the target feature sequence and the imitation feature sequence based on the target alignment path, to obtain spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance; wherein, the spectral distance is used to characterize the energy distribution difference of each pair of aligned frames in the target alignment path across multiple frequency bands, the frequency band contour distance is used to characterize the difference between the target frequency band energy contour and the imitation frequency band energy contour formed after grouping the multiple frequency bands, the energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, and the fundamental frequency distance is used to characterize the tone difference between each pair of aligned frames in the target alignment path; The similarity scoring unit is used to perform calculations based on the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance to obtain the similarity score of the imitated audio relative to the target audio.

[0008] A third aspect of this application provides a computer device. The computer device includes: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the sound imitation comparison method provided in the first aspect.

[0009] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the steps of the sound imitation and comparison method provided in the first aspect.

[0010] Compared with the prior art, this application has the following advantages: In this application's technical solution, firstly, a target audio and a simulated audio generated based on the target audio are acquired; then, audio processing is performed on the target audio and the simulated audio to obtain a target feature sequence corresponding to the target audio and a simulated feature sequence corresponding to the simulated audio; wherein, the feature sequence includes multiple frame features arranged in the temporal order of the audio frames, and the frame features include spectral features, energy features, and fundamental frequency features. Thus, this application can convert the target audio and the simulated audio into feature sequences that possess both temporal order and multiple sound feature dimensions, providing a data foundation for the temporal alignment and sound feature comparison of the two audio segments.

[0011] Subsequently, based on frame features, the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence is calculated, obtaining multiple frame-level distances. The corresponding positions between the first audio frame in the target feature sequence and each imitation frame are used as candidate path starting points. Audio frame paths are searched from each candidate path starting point based on multiple frame-level distances, and the audio frame path with the smallest cumulative candidate distance is determined as the target alignment path. Thus, this application no longer requires the target audio and imitation audio to correspond starting from their respective first audio frames. Instead, it can determine the starting position matching the target audio from multiple positions in the imitation feature sequence, ensuring that preceding audio frames not involved in sound imitation do not affect subsequent feature comparisons. This allows for adaptation to situations where the effective sounds of the target audio and imitation audio are offset in time, improving the accuracy of time alignment between the target audio and imitation audio.

[0012] Next, after obtaining the target alignment path, the frame features aligned with each other in the target feature sequence and the imitation feature sequence are calculated based on the target alignment path to obtain the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. Finally, the imitation similarity score of the imitation audio relative to the target audio is obtained based on the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. Specifically, the spectral distance characterizes the energy distribution differences of each pair of aligned frames in the target alignment path across multiple frequency bands; the frequency band contour distance characterizes the difference between the target frequency band energy contour and the imitation frequency band energy contour formed after grouping multiple frequency bands; the energy distance characterizes the signal strength differences between each pair of aligned frames in the target alignment path; and the fundamental frequency distance characterizes the pitch differences between each pair of aligned frames in the target alignment path. Thus, this application can comprehensively evaluate the imitation audio from multiple aspects such as spectral details, overall spectral contour, sound intensity, and pitch, reducing the influence of single sound feature comparison results on the overall score, and enabling the obtained imitation similarity score to more comprehensively and accurately reflect the actual degree of imitation of the target audio relative to the target audio.

[0013] As can be seen, this application achieves temporal position matching and multi-dimensional sound feature comparison between the target audio and the imitated audio by searching for the target alignment path based on multiple candidate path starting points and performing multi-dimensional distance calculation on the frame features that are aligned with each other based on the target alignment path, thereby improving the adaptability and accuracy of sound imitation comparison. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a sound imitation comparison method provided in this application embodiment; Figure 2 A flowchart illustrating the method for obtaining a feature sequence in a sound imitation comparison provided in this application embodiment; Figure 3 A flowchart illustrating the similarity score obtained in a voice imitation comparison method provided in this application embodiment; Figure 4 A flowchart illustrating the similarity score obtained in another voice imitation comparison method provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of a sound imitation comparison device provided in an embodiment of this application. Detailed Implementation

[0016] As described earlier, with the development of audio processing technology, voice imitation comparison has been widely used in scenarios such as game interaction, dubbing practice, and language repetition. In the process of voice imitation, a target audio is usually provided in advance, and the imitator speaks according to the target audio to generate the imitation audio. Then, the target audio and the imitation audio need to be compared to determine the degree of similarity between the imitation audio and the target audio.

[0017] In related technologies, audio processing is typically performed on the target audio and the imitation audio to extract the corresponding audio features of the two audio segments, and the similarity between the two audio segments is determined by the differences between the audio features. However, the actual generated imitation audio is easily affected by factors such as the recording environment, the start time of recording, and the duration of the sound, which may cause differences in time position and sound features between the target audio and the imitation audio, making it difficult to accurately reflect the actual degree of imitation of the target audio by the imitation audio.

[0018] Therefore, how to improve the accuracy of sound imitation comparison results is a key issue of concern to those skilled in the art.

[0019] To address the above issues, this application provides a solution, proposing a sound imitation comparison method, apparatus, and related products, aiming to improve the accuracy of sound imitation comparison results. In this application's technical solution, firstly, a target audio and an imitation audio generated for the target audio are acquired; then, audio processing is performed on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in temporal order of the audio frames, and the frame features include spectral features, energy features, and fundamental frequency features. Thus, this application can convert the target audio and the imitation audio into feature sequences that possess both temporal order and multiple sound feature dimensions, providing a data foundation for time alignment and sound feature comparison of two audio segments.

[0020] Subsequently, based on frame features, the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence is calculated, obtaining multiple frame-level distances. The corresponding positions between the first audio frame in the target feature sequence and each imitation frame are used as candidate path starting points. Audio frame paths are searched from each candidate path starting point based on multiple frame-level distances, and the audio frame path with the smallest cumulative candidate distance is determined as the target alignment path. Thus, this application no longer requires the target audio and imitation audio to correspond starting from their respective first audio frames. Instead, it can determine the starting position matching the target audio from multiple positions in the imitation feature sequence, ensuring that preceding audio frames not involved in sound imitation do not affect subsequent feature comparisons. This allows for adaptation to situations where the effective sounds of the target audio and imitation audio are offset in time, improving the accuracy of time alignment between the target audio and imitation audio.

[0021] Next, after obtaining the target alignment path, the frame features aligned with each other in the target feature sequence and the imitation feature sequence are calculated based on the target alignment path to obtain the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. Finally, the imitation similarity score of the imitation audio relative to the target audio is obtained based on the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. Specifically, the spectral distance characterizes the energy distribution differences of each pair of aligned frames in the target alignment path across multiple frequency bands; the frequency band contour distance characterizes the difference between the target frequency band energy contour and the imitation frequency band energy contour formed after grouping multiple frequency bands; the energy distance characterizes the signal strength differences between each pair of aligned frames in the target alignment path; and the fundamental frequency distance characterizes the pitch differences between each pair of aligned frames in the target alignment path. Thus, this application can comprehensively evaluate the imitation audio from multiple aspects such as spectral details, overall spectral contour, sound intensity, and pitch, reducing the influence of single sound feature comparison results on the overall score, and enabling the obtained imitation similarity score to more comprehensively and accurately reflect the actual degree of imitation of the target audio relative to the target audio.

[0022] As can be seen, this application achieves temporal position matching and multi-dimensional sound feature comparison between the target audio and the imitated audio by searching for the target alignment path based on multiple candidate path starting points and performing multi-dimensional distance calculation on the frame features that are aligned with each other based on the target alignment path, thereby improving the adaptability and accuracy of sound imitation comparison.

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0024] First, it should be noted that the data collection and processing in this application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0025] For example, in a game interaction scenario, the terminal device can play a 3-second target audio clip. After the target audio finishes playing, the imitator begins to mimic the sound. Since the sound acquisition begins before the actual imitation begins, the obtained imitation audio may include a 0.5-second prelude, 2.8 seconds of the actual imitated sound, and 0.3 seconds of ambient sound at the end. After obtaining the target audio and the imitation audio, the server can determine the audio content actually participating in the comparison through the following steps and calculate the imitation similarity score between the target audio and the imitation audio. Finally, based on the imitation similarity score, encouraging or disappointing feedback is given.

[0026] See Figure 1 This figure is a flowchart of a sound imitation comparison method provided in an embodiment of this application. Figure 1 The sound imitation comparison methods shown include: S101: Obtain the target audio and the mimicry audio generated for the target audio.

[0027] In this step, the target audio can be understood as the audio used as a sound imitation object or a sound comparison benchmark. The target audio can be pre-stored in the terminal device, server, or audio storage device, or it can be obtained by the terminal device or server from audio resources according to the sound imitation task. The imitation audio can be understood as the audio generated by imitating the sound content in the target audio. The imitation audio can be acquired in real time by the terminal device, or it can be read by the electronic device from local storage space or network storage space.

[0028] For example, the terminal device can first play the target audio and, in response to an audio acquisition operation, control the microphone to acquire sound signals to obtain the imitation audio, where the audio can be PCM (Pulse Code Modulation) audio. For instance, the target audio can be a pre-stored audio clip of an animal sound; after the electronic device plays the animal sound audio, it acquires the imitator's voice to form the imitation audio. As another example, the target audio can be a line of dialogue, and the imitation audio can be a recording generated after the imitator imitates the same lines, tone, or rhythm.

[0029] This application can also configure audio identifiers for the target audio and the emulated audio separately to distinguish them during subsequent processing. It should be noted that before subsequent processing, the target audio and the emulated audio can be converted to the same sampling rate, number of channels, and data format. For example, both the target audio and the emulated audio can be converted to single-channel pulse-code modulation audio. In one example, a sampling rate of 16000 Hz (Hertz) can be used.

[0030] Thus, by obtaining the target audio and the mimicry audio generated for the target audio, this application can provide input data for subsequent effective audio detection, feature extraction, time alignment, and similarity calculation based on a unified audio object.

[0031] S102: Perform audio processing on the target audio and the imitation audio to obtain the target feature sequence corresponding to the target audio and the imitation feature sequence corresponding to the imitation audio.

[0032] In this step, the feature sequence includes multiple frame features arranged in chronological order of the audio frames. Frame features include spectral features, energy features, and fundamental frequency features. The target feature sequence can be understood as a data sequence formed by arranging the frame features corresponding to each target frame in the target audio according to their chronological order. The imitation feature sequence can be understood as a data sequence formed by arranging the frame features corresponding to each imitation frame in the imitation audio according to their chronological order.

[0033] An audio frame can be understood as an audio data unit divided from audio according to a preset time length. The audio frame corresponding to the target audio is called the target frame, and the audio frame corresponding to the imitation audio is called the imitation frame. Frame features are used to characterize an audio frame from different feature dimensions. Among them, spectral features are used to characterize the distribution of signal energy in the audio frame across multiple frequency bands; energy features are used to characterize the signal strength of the audio frame; and fundamental frequency features are used to characterize the pitch or periodic sound components in the audio frame.

[0034] For example, the target valid audio can be divided into target frames T1, T2, and T3, and frame features F1, F2, and F3 can be obtained respectively. Then the target feature sequence can be represented as [F1, F2, F3]. The imitated valid audio can be divided into imitated frames R1, R2, R3, R4, and R5, and frame features G1, G2, G3, G4, and G5 can be obtained respectively. Then the imitated feature sequence can be represented as [G1, G2, G3, G4, G5].

[0035] Thus, this application can convert continuous audio waveforms into multi-dimensional frame features arranged in time order, enabling the target audio and the simulated audio to be compared and time-aligned in a unified feature space.

[0036] The process of "performing audio processing on the target audio and the imitation audio to obtain the target feature sequence corresponding to the target audio and the imitation feature sequence corresponding to the imitation audio" in this application is specifically as follows: Figure 2 As shown. Figure 2 This is a flowchart illustrating the method for obtaining feature sequences in a sound imitation comparison embodiment provided in this application. Figure 2 This may include steps S1021-S1024, which are specifically described as follows: S1021: Perform detection processing on the target audio and the imitation audio respectively to obtain the target effective audio in the target audio and the imitation effective audio in the imitation audio.

[0037] It should be noted that the target audio and the imitated audio may include silent segments, ambient sounds, or other audio segments that are not involved in the sound imitation, located before or after the effective sound. Directly segmenting and extracting features from the complete audio will not only increase the amount of data for subsequent processing but may also allow invalid audio segments to participate in time alignment and similarity calculations, thereby affecting the comparison results.

[0038] Based on this, this application detects the effective sound regions in the target audio and the imitation audio respectively, extracts the target effective audio from the target audio, and extracts the imitation effective audio from the imitation audio. Specifically, the target effective audio is used to characterize audio segments in the target audio that satisfy the audio activity conditions, and the imitation effective audio is used to characterize audio segments in the imitation audio that satisfy the audio activity conditions.

[0039] For example, if the duration of the imitated audio is 3.6 seconds, where 0 to 0.5 seconds is the preceding blank space, 0.5 to 3.3 seconds is the actual imitated sound, and 3.3 to 3.6 seconds is the following ambient sound, then the effective sound range corresponding to 0.5 to 3.3 seconds can be determined through detection processing, and the effective imitated audio can be obtained.

[0040] The process of "detecting and processing the target audio and the imitation audio separately to obtain the target effective audio in the target audio and the imitation effective audio in the imitation audio" in this application is as follows: First, frame energy is extracted from both the target audio and the simulated audio to obtain the target frame energy sequence for the target audio and the simulated frame energy sequence for the simulated audio. Understandably, this application first divides the target audio and the simulated audio into multiple detection frames and calculates the frame energy of each detection frame. The frame energy characterizes the signal strength of the corresponding audio frame (detection frame). It should be noted that the frame energy can be linear energy, mean square energy, logarithmic energy, or other values ​​that can characterize the signal strength of the audio frame. For example, if the target audio includes 5 detection frames, the frame energy of each detection frame can be [42dB, 40dB, 25dB, 20dB, 38dB], where dB represents decibels.

[0041] In this application, the energy of multiple frames in the target audio can be arranged according to the temporal order of each detection frame in the target audio to obtain a target frame energy sequence. Similarly, the energy of multiple frames in the imitation audio can be arranged according to the temporal order of each detection frame in the imitation audio to obtain an imitation frame energy sequence. The frame energy sequence includes multiple frame energies arranged in the temporal order of the audio frames.

[0042] Subsequently, quantile calculations are performed on the target frame energy sequence and the emulated frame energy sequence to obtain the first target energy quantile and the second target energy quantile corresponding to the target audio, as well as the first emulated energy quantile and the second emulated energy quantile corresponding to the emulated audio. Understandably, this application sorts the multiple frame energies in the target frame energy sequence according to energy from high to low to obtain a sorted target frame energy sequence, and then extracts the first target energy quantile and the second target energy quantile from this sorted target frame energy sequence. Similarly, the multiple frame energies in the emulated frame energy sequence are sorted according to energy from high to low to obtain a sorted emulated frame energy sequence, and then the first emulated energy quantile and the second emulated energy quantile are extracted from this sorted emulated frame energy sequence.

[0043] The first energy quantile (first target energy quantile, first simulacrum energy quantile) characterizes the noise floor of the corresponding audio, while the second energy quantile (second target energy quantile, second simulacrum energy quantile) characterizes the medium energy level of the corresponding audio. For example, the first energy quantile can be the 5% energy quantile, and the second energy quantile can be the 50% energy quantile. The 5% energy quantile is determined by a subset of audio frames with lower energy and can approximately reflect the noise floor of the corresponding audio; the 50% energy quantile corresponds to the middle position of the frame energy distribution and can reflect the medium energy level of the corresponding audio.

[0044] Next, based on the first target energy quantile and the second target energy quantile, a target activity detection threshold corresponding to the target audio is determined; and based on the first imitation energy quantile and the second imitation energy quantile, an imitation activity detection threshold corresponding to the imitation audio is determined. It is understood that this application can determine the target activity detection threshold corresponding to the target audio based on the difference between the first target energy quantile and the second target energy quantile; and, based on the difference between the first imitation energy quantile and the second imitation energy quantile, determine the imitation activity detection threshold corresponding to the imitation audio.

[0045] It should be noted that the target activity detection threshold and the imitation activity detection threshold are determined based on the frame energy distribution of the corresponding audio. When the target audio and the imitation audio have different background sound levels or recording volumes, they can obtain different activity detection thresholds. The activity detection threshold can be understood as the VAD (Voice Activity Detection) threshold.

[0046] In one feasible implementation, the activity detection thresholds (target activity detection threshold, simulated activity detection threshold) can be obtained by formula (1), which is as follows:

[0047] in, Indicates the activity detection threshold; Indicates the first energy quantile (unit: dB); This represents the second energy quantile. It should be noted that... This is the upper limit of the dynamic range to avoid missing detections due to excessively high thresholds in extremely quiet environments. For example, in a typical home environment (noise floor of about 30-40dB), a dynamic range of 8dB can cover most of the energy difference between speech and noise. This is the lower limit of the dynamic range. To maintain a certain level of detection sensitivity even in extremely noisy environments, when... and When the difference is too small (less than 3dB), it indicates that the energy of the audio segment is generally low, which may be pure noise or a very quiet recording. In this case, the threshold is still calculated based on the minimum 3dB difference to ensure basic speech detection capability.

[0048] Finally, based on the target activity detection threshold and the target frame energy sequence, effective target audio is extracted from the target audio; and based on the imitation activity detection threshold and the imitation frame energy sequence, effective imitation audio is extracted from the imitation audio. Understandably, this application can compare the energy of each target frame in the target frame energy sequence with the target activity detection threshold to determine target audio frames that meet the target activity detection threshold (e.g., greater than or equal to the target activity detection threshold), and concatenate multiple target audio frames that meet the target activity detection threshold to determine the effective target audio.

[0049] Accordingly, the energy of each imitation frame in the imitation frame energy sequence is compared with the imitation activity detection threshold to determine the imitation audio frames that meet the imitation activity detection threshold (e.g., greater than or equal to the imitation activity detection threshold). Multiple imitation audio frames that meet the imitation activity detection threshold are then concatenated to determine the effective imitation audio.

[0050] In one feasible implementation, this application can pre-configure the number of edge detection frames and determine the start and end edges of valid audio based on the number of edge detection frames. For example, the number of edge detection frames can be set to 2 frames. When the frame energy of two consecutive audio frames both meet the corresponding target activity detection threshold, the start edge of the detected valid sound can be determined; when the frame energy of two consecutive audio frames does not meet the corresponding target activity detection threshold, the end edge of the detected valid sound can be determined. Then, the target valid audio or simulated valid audio can be extracted based on the determined start and end edges. Thus, compared to determining the valid audio edge based solely on the frame energy of a single audio frame, confirming the audio activity state through two consecutive audio frames can reduce edge misjudgments caused by instantaneous sound fluctuations or single-frame anomalies, improving the stability of the valid audio edge and the accuracy of valid audio extraction.

[0051] Thus, compared to existing VAD methods in related technical solutions, which often rely on fixed energy thresholds or assume the audio start segment is silent, resulting in high false detection rates in real-world noisy environments (such as homes, outdoors, and public places), potentially misclassifying valid audio segments as silent and truncating them, this application determines the activity detection threshold based on the frame energy distribution of both the target audio and the simulated audio. This allows the activity detection threshold to be adjusted according to the noise floor and medium energy level of the corresponding audio, reducing the possibility of false detection under different sound environments with a fixed activity detection threshold. Furthermore, by jointly confirming the start and end edges of valid sound using two consecutive audio frames, the impact of instantaneous energy fluctuations in a single audio frame on the edges of valid audio can be reduced. This improves the extraction accuracy and edge stability of both the target valid audio and the simulated valid audio, and reduces the number of invalid audio frames entering subsequent feature extraction and path search processes, thereby improving subsequent processing efficiency.

[0052] S1022: Perform frame segmentation processing on the target valid audio and the simulated valid audio respectively to obtain multiple target frames and multiple simulated frames.

[0053] In this step, multiple time windows can be set on the target effective audio and the simulated effective audio according to preset frame length and preset frame shift, respectively, to obtain multiple target frames in the target effective audio and multiple simulated frames in the simulated effective audio. The preset frame length is used to characterize the audio duration covered by each audio frame, and the preset frame shift is used to characterize the time interval between the start positions of two adjacent audio frames. Adjacent audio frames may partially overlap to improve the continuity of audio features changing over time.

[0054] For example, the preset frame length can be 25 milliseconds (corresponding to 400 sampling points), and the preset frame shift can be 10 milliseconds (corresponding to 160 sampling points). For instance, for a target valid audio of 100 milliseconds, the first target frame can be obtained from the 0 millisecond position, with a frame length of [0, 25); the second target frame can be obtained from the 10 millisecond position, with a frame length of [10, 35); the third target frame can be obtained from the 20 millisecond position, with a frame length of [20, 45); and so on.

[0055] S1023: Perform feature extraction on each of the target frames and each of the imitation frames respectively to obtain the frame features corresponding to each of the target frames and the frame features corresponding to each of the imitation frames respectively.

[0056] In this step, for each target frame, spectral feature extraction, energy feature extraction, and fundamental frequency feature extraction are performed to obtain the frame features corresponding to the target frame; and for each imitation frame, spectral feature extraction, energy feature extraction, and fundamental frequency feature extraction are performed to obtain the frame features corresponding to the imitation frame.

[0057] For example, the spectral features can be log-Mel spectral features (Log-Mel features, 24-dimensional, 512-point FFT), where 512-point FFT represents a 512-point Fast Fourier Transform; the energy features can be the logarithmic energy of the audio frame; the fundamental frequency features can include the fundamental frequency, and may also include voiced / unvoiced information used to characterize whether the audio frame contains effective periodic sound components. The fundamental frequency features are obtained based on the autocorrelation method, with the Mel filter bank starting at 64Hz (lowest frequency) and cutting off at 4000Hz (highest frequency). It should be noted that the extraction processes for the spectral features, energy features, and fundamental frequency features are existing technologies and will not be elaborated upon here.

[0058] For example, the spectral characteristics of a target frame may include the energy of 24 frequency bands: [2.1, 2.5, 3.0, ..., 1.8]; the energy characteristic of the target frame may be 5.6, and the fundamental frequency characteristic may be 220Hz.

[0059] S1024: Arrange the frame features corresponding to each target frame according to the time order of the audio frames to obtain the target feature sequence; and arrange the frame features corresponding to each imitation frame according to the time order of the audio frames to obtain the imitation feature sequence.

[0060] In this step, the frame features corresponding to each target frame can be arranged according to the chronological order of each target frame in the target valid audio to obtain a target feature sequence; and the frame features corresponding to each imitation frame can be arranged according to the chronological order of each imitation frame in the imitation valid audio to obtain an imitation feature sequence. For example, if target frames T1, T2, and T3 are located at the beginning, middle, and end of the target audio in sequence, then the corresponding frame features F1, F2, and F3 are arranged in sequence to form the target feature sequence [F1, F2, F3].

[0061] Each sequence position in the target feature sequence corresponds to a target frame, and each sequence position in the imitation feature sequence corresponds to an imitation frame. The sequence position can represent the temporal order of the corresponding audio frame in the corresponding audio, and is used as the positional basis when constructing the audio frame matrix and searching for audio frame paths.

[0062] Thus, by first extracting the valid audio, and then performing frame segmentation, feature extraction, and temporal sorting on the target valid audio and the imitated valid audio in the same way, this application can obtain the target feature sequence and the imitated feature sequence corresponding to the data structure, providing standardized input for subsequent frame-level distance calculation and path search.

[0063] S103: Based on the frame features, calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence to obtain multiple frame-level distances.

[0064] In this step, the frame-level distance is used to characterize the degree of difference in frame features between a target frame and a mimic frame. A smaller frame-level distance indicates that the frame features of the corresponding target frame and mimic frame are closer; a larger frame-level distance indicates that the differences in their frame features are more significant. This application can select each target frame in the target feature sequence and each mimic frame in the mimic feature sequence, and calculate a frame-level distance based on the frame features corresponding to the selected target frame and mimic frame respectively. Thus, multiple frame-level distances corresponding to combinations of multiple sets of target frames and mimic frames can be obtained.

[0065] For example, if the target feature sequence includes target frames T1, T2, and T3, and the imitation feature sequence includes imitation frames R1, R2, R3, and R4, then this application can calculate the frame-level distances corresponding to T1-R1, T1-R2, T1-R3, T1-R4; T2-R1, T2-R2, T2-R3, T2-R4; and T3-R1, T3-R2, T3-R3, T3-R4 respectively, obtaining a total of 12 frame-level distances.

[0066] It should be noted that the frame-level distance can be calculated based on spectral and energy characteristics, or it can be further calculated by combining fundamental frequency characteristics. In one feasible implementation, the frame-level distance can be calculated using formula (2), which is as follows:

[0067] in, represents the frame-level distance between the i-th target frame and the j-th imitation frame; D represents the number of feature dimensions involved in the calculation; This represents the distance weight corresponding to the k-th feature dimension; This represents the feature value of the i-th target frame in the k-th feature dimension; This represents the feature value of the j-th simulated frame in the k-th feature dimension. It should be noted that the distance weight for the energy dimension can be 2.0, while the distance weight for the other dimensions (spectral dimension, fundamental frequency dimension) can be 1.0.

[0068] Understandably, for any target frame and any imitation frame, the frame features corresponding to the target frame and the imitation frame are obtained respectively. Based on the correspondence between the same feature dimensions, the absolute value of the difference between the feature values ​​of the target frame and the imitation frame in each feature dimension is calculated to obtain the feature difference corresponding to each feature dimension. Then, each feature difference is multiplied by its corresponding dimension weight to obtain the weighted feature difference for each feature dimension. Finally, the weighted feature differences are summed, and the summation result is divided by the number of feature dimensions involved in the calculation to obtain the frame-level distance between the target frame and the imitation frame.

[0069] For example, the target frame has feature values ​​of [2, 4, 6, 5] in the four feature dimensions involved in the calculation, and the imitation frame has feature values ​​of [3, 4, 5, 4] in the corresponding four feature dimensions. The dimension weights of the first three feature dimensions are all 1, and the dimension weight of the fourth energy dimension is 2. First, the absolute value of the difference between the feature values ​​in the corresponding feature dimensions is calculated, resulting in [1, 0, 1, 1]. Then, it is multiplied by the corresponding dimension weights to obtain the weighted feature difference [1, 0, 1, 2]. The four weighted feature differences are then summed to obtain 4. The summation result is divided by the number of feature dimensions involved in the calculation, 4, to obtain the frame-level distance between the target frame and the imitation frame as 1.

[0070] S104: Using the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, search for audio frame paths from each candidate path starting point based on the multiple frame-level distances, and determine the audio frame path with the smallest cumulative candidate distance as the target alignment path.

[0071] In this step, the corresponding position is used to characterize the frame correspondence between a target frame and a mimic frame. The candidate path starting point is used to characterize the matching position formed between the first audio frame in the target feature sequence and a mimic frame in the mimic feature sequence. The candidate path starting point is used as the starting position for audio frame path search. The target alignment path can be understood as the audio frame path determined from multiple audio frame paths, used to establish the temporal correspondence between the target feature sequence and the mimic feature sequence. Each corresponding position in the target alignment path can form a pair of aligned frames, which includes a target frame and a mimic frame aligned with each other.

[0072] The process of "using the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as the starting points of the candidate paths" in this application is as follows: First, an audio frame matrix is ​​constructed based on the arrangement positions of each target frame in the target feature sequence and each imitation frame in the imitation feature sequence. Each position in the audio frame matrix corresponds to one target frame and one imitation frame.

[0073] Understandably, this application sets the arrangement of each target frame in the target feature sequence as one matrix dimension of the audio frame matrix, and sets the arrangement of each imitation frame in the imitation feature sequence as another matrix dimension of the audio frame matrix, thereby constructing the audio frame matrix. For example, if the target feature sequence includes n target frames and the imitation feature sequence includes m imitation frames, then the audio frame matrix can include n rows and m columns. The matrix position in the i-th row and j-th column corresponds to the i-th target frame in the target feature sequence and the j-th imitation frame in the imitation feature sequence, and can store or associate the frame-level distance between the target frame and the imitation frame.

[0074] For example, if the target feature sequence includes three target frames T1, T2, and T3, and the imitation feature sequence includes four imitation frames R1, R2, R3, and R4, then a 3x4 audio frame matrix can be constructed. Here, (1,1) corresponds to T1 and R1; (1,2) corresponds to T1 and R2; (2,3) corresponds to T2 and R3; and (3,4) corresponds to T3 and R4.

[0075] Subsequently, in the DTW (Dynamic Time Warping) dynamic programming calculation, when initializing the cumulative distance of the first row (i=0), this application sets all positions from j=0 to j=m or j=m-1 as valid starting candidate points, instead of only starting the calculation from j=0. That is, the matrix positions corresponding to the first audio frame in the target feature sequence and each imitation frame are determined as the starting points of each candidate path. Specifically, the matrix positions corresponding to the first audio frame in the imitation feature sequence and other target frames besides the first audio frame in the target feature sequence are determined as invalid positions. Invalid positions are used to indicate that it is not allowed to start searching for audio frame paths from the corresponding positions. This ensures that: when the target frame index i=0, the cumulative distance C(0,j) can be directly set as the frame-level distance d(0,j) between the corresponding target frame and the imitation frame, where j can range from 0 to m or 0 to m-1; when the imitation frame index j=0 but i>0, it is set as invalid (it is not allowed to start from the position i>0 and j=0).

[0076] For example, using the aforementioned 3x4 matrix example, T1 is the first target frame in the target feature sequence. Therefore, (1,1), (1,2), (1,3), and (1,4) can all be identified as candidate path starting points. Among them, (2,1) and (3,1) in the first column can be identified as invalid positions.

[0077] Thus, compared to the standard Dynamic Time Warping (DTW) method in related technical solutions, which requires the target audio and the recorded audio to be aligned at both ends of the time axis, and where alignment failure leads to distorted similarity scores when there is pre-noise, delay, or blank segments in the recording, this application addresses this issue by setting multiple candidate path start points and invalid positions. This allows the target audio to participate in alignment from the first target frame while opening up the path start position on the emulated audio side, ignoring audio frames in the emulated audio that precede the actual emulated content, thereby improving adaptability to pre-noise, ambient sound, or recording delays.

[0078] The process of "searching for audio frame paths from each of the candidate path starting points based on the multiple frame-level distances, and determining the audio frame path with the smallest cumulative candidate distance as the target alignment path" in this application is as follows: First, based on the frame-level distances corresponding to each matrix position in the audio frame matrix (excluding invalid positions), a path search is performed from each candidate path starting point to obtain multiple candidate audio frame paths. Understandably, this application can use each candidate path starting point as the path's starting position and search for subsequent matrix positions in the audio frame matrix according to the time order of the target frame and the emulated frame to obtain multiple candidate audio frame paths.

[0079] For example, in the aforementioned 3x4 matrix, starting from the candidate path starting point (1,2), the path can proceed sequentially as follows: (1,2) → (2,3) → (3,4). This path indicates that T1 is aligned with R2, T2 with R3, and T3 with R4. As another example, starting from the candidate path starting point (1,1), another candidate audio frame path can be obtained: (1,1) → (2,2) → (3,3).

[0080] Subsequently, the frame-level distances corresponding to the matrix positions traversed by each candidate audio frame path are accumulated to obtain the initial cumulative distance for each candidate audio frame path. The initial cumulative distance characterizes the overall frame feature differences between the multiple pairs of target frames and imitation frames established by the candidate audio frame path. A smaller initial cumulative distance indicates that the target frames and imitation frames on that candidate audio frame path are more closely related overall, and vice versa.

[0081] Understandably, for each candidate audio frame path, the frame-level distances corresponding to each matrix position traversed by the candidate audio frame path can be obtained, and the obtained frame-level distances can be accumulated to obtain the initial cumulative distance corresponding to the candidate audio frame path. For example, for the candidate audio frame path: (1,2)→(2,3)→(3,4), the frame-level distances corresponding to the three matrix positions are 0.8, 0.5, and 0.7 respectively, so its initial cumulative distance can be: 0.8+0.5+0.7=2.0.

[0082] Next, based on the number of imitation frames covered by each candidate audio frame path and the number of target frames included in the target feature sequence, the duration ratio corresponding to each candidate audio frame path is determined. Understandably, for each candidate audio frame path, the number of imitation frames covered by that candidate audio frame path can be counted, and the number of target frames included in the target feature sequence can be obtained. The duration ratio corresponding to that candidate audio frame path is then determined based on the number of imitation frames and the number of target frames.

[0083] In this application, the imitation frame interval covered by the candidate audio frame path can be regarded as the imitation audio interval actually used by the path to match the target audio. Since the target frame and the imitation frame can use the same frame shift, the ratio between the number of imitation frames covered by the candidate audio frame path and the number of target frames can be used to approximately characterize the ratio between the duration of the imitation audio and the duration of the target audio corresponding to the candidate audio frame path.

[0084] For example, if the target feature sequence includes 100 target frames, and a candidate audio frame path covers 90 imitation frames, then the duration ratio corresponding to this candidate audio frame path can be: 90 / 100 = 0.9. As another example, if another candidate audio frame path only covers 30 imitation frames, then its duration ratio can be: 30 / 100 = 0.3.

[0085] Next, based on the relationship between each duration ratio and the preset duration ratio range, the duration penalty value corresponding to each candidate audio frame path is determined. Specifically, if the duration ratio is within the preset duration ratio range, the corresponding duration penalty value can be set to zero, indicating that there is no significant difference between the simulated audio duration covered by the candidate audio frame path and the target audio duration requiring penalty. If the duration ratio is not within the preset duration ratio range, the corresponding duration penalty value can be determined based on the degree of deviation of the duration ratio from the preset duration ratio range, with the duration penalty value being positively correlated with the degree of deviation.

[0086] For example, the preset duration ratio range can be from 0.45 to 1.8. When the duration ratio is less than 0.45 or greater than 1.8, a duration penalty can be triggered. For instance, a duration ratio of 0.9 falls between 0.45 and 1.8, so the corresponding duration penalty value can be 0. As another example, if a duration ratio of 0.3 is less than 0.45, the corresponding duration penalty value can be determined based on the degree of deviation of 0.3 from 0.45.

[0087] Finally, based on the initial cumulative distance and duration penalty value corresponding to each candidate audio frame path, the candidate cumulative distance corresponding to each candidate audio frame path is determined. This application can add the duration penalty value corresponding to each candidate audio frame path to the initial cumulative distance corresponding to that candidate audio frame path to obtain the candidate cumulative distance for that candidate audio frame path. For example, if the initial cumulative distance of the first candidate audio frame path is 2.0, its duration ratio is 0.9, and its duration penalty value is 0, then: candidate cumulative distance = 2.0. If the initial cumulative distance of the second candidate audio frame path is 1.8, its duration ratio is 0.3, and its corresponding duration penalty value is 0.6, then: candidate cumulative distance = 1.8 + 0.6 = 2.4.

[0088] Furthermore, the cumulative candidate distances corresponding to each candidate audio frame path can be compared, and the candidate audio frame path with the smallest cumulative candidate distance can be determined as the target alignment path. For example, if the cumulative candidate distance corresponding to the first candidate audio frame path is 2.0 and the cumulative candidate distance corresponding to the second candidate audio frame path is 2.4, then the first candidate audio frame path can be determined as the target alignment path.

[0089] It should be noted that this application can also record the start and end matrix positions of the target alignment path to determine the imitation frame intervals that actually participate in subsequent distance calculations in the imitation feature sequence. Thus, this application not only allows the imitation audio to match the target audio from different positions, but also uses the number of imitation frames covered by the candidate audio frame path to judge the reasonableness of the path's duration, and corrects the candidate cumulative distance through a duration penalty value. Therefore, it can reduce the possibility of excessively short paths covering only a few similar audio frames or paths with obviously abnormal coverage durations being identified as target alignment paths, improving the integrity and reliability of the target alignment path while adapting to pre-recording gaps and recording delays.

[0090] In another feasible implementation, this application can further limit the search range of candidate audio frame paths based on a preset DTW bandwidth. The DTW bandwidth can be determined based on the DTW bandwidth ratio and the number of audio frames in the target feature sequence and the imitation feature sequence, and the lower limit of the actual search bandwidth can be limited by a preset minimum DTW bandwidth. For example, the DTW bandwidth ratio can be 0.20, and the minimum DTW bandwidth can be 8 frames. This application can determine the initial DTW bandwidth based on the larger of the number of target frames and the number of imitation frames, and the DTW bandwidth ratio of 0.20; when the determined initial DTW bandwidth is less than 8 frames, 8 frames can be determined as the actual DTW bandwidth. Therefore, candidate audio frame paths can be primarily searched within regions in the audio frame matrix that have a relatively close correspondence with time positions, reducing the participation of matrix positions that deviate significantly from the normal time correspondence in path searching. Simultaneously, by setting a minimum bandwidth, a certain amount of path adjustment space is reserved for cases with a small number of audio frames.

[0091] S105: Based on the target alignment path, calculate and process the frame features that are aligned with each other in the target feature sequence and the imitation feature sequence to obtain the spectral distance, frequency band contour distance, energy distance and fundamental frequency distance.

[0092] In this step, based on the corresponding positions in the target alignment path, multiple pairs of frame features aligned with each other in the target feature sequence and the imitation feature sequence are determined, and the corresponding frame distance is calculated for each pair of aligned frames. The frame distance can be calculated using the formula (2) above. Then, the frame distances of the same type corresponding to multiple pairs of aligned frames in the target alignment path are statistically analyzed to obtain the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance.

[0093] Among them, the spectral distance is used to characterize the energy distribution differences of each pair of aligned frames in the target alignment path across multiple frequency bands. A frequency band is used to characterize a frequency division region within the sound frequency range. For example, when the spectral feature uses a 24-dimensional Log-Mel spectral feature, the 24-dimensional Log-Mel feature values ​​correspond to 24 frequency bands respectively; that is, this spectral feature is used to directly compare the 24 frequency bands. The frequency band contour distance is used to characterize the difference between the target frequency band energy contour formed after grouping multiple frequency bands and the simulated frequency band energy contour. For example, the 24 Mel frequency bands can be grouped together... The frequency band is divided into four groups (low frequency, mid frequency, mid-high frequency, and high frequency) according to the frequency range. The average energy of each frequency band group is calculated, and the energy change trends formed by multiple frequency band groups are compared to capture the overall trend of the spectrum shape. The energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, that is, the frame-level distance based on the logarithmic energy feature, which reflects the volume change trend. The fundamental frequency distance is used to characterize the pitch difference between each pair of aligned frames in the target alignment path, that is, the weighted distance based on the fundamental frequency and voiced / unvoiced sounds, which reflects the pitch similarity.

[0094] Specifically, the process in this application of "calculating and processing the frame features that are aligned with each other in the target feature sequence and the imitation feature sequence based on the target alignment path to obtain the spectral distance, frequency band contour distance, energy distance and fundamental frequency distance" is as follows: First, for each pair of aligned frames in the target alignment path, distance is calculated based on the spectral features corresponding to the target frame and the imitation frame in the aligned frame, respectively, to obtain the spectral frame distance corresponding to the aligned frame. The spectral frame distance is used to directly compare the differences between the spectral feature values ​​corresponding to each frequency band in the target frame and the imitation frame, focusing on reflecting the fine-grained differences in the energy distribution corresponding to each individual frequency band. In this application, the spectral features may include feature values ​​corresponding to multiple frequency bands. For example, when the spectral features adopt 24-dimensional Log-Mel spectral features, a target frame may include 24 target frequency band feature values, and an imitation frame may include 24 imitation frequency band feature values.

[0095] In this application, the spectral features corresponding to the target frame and the imitation frame can be substituted into the frame-level distance calculation method shown in formula (2). That is, the absolute difference between the target spectral feature value and the imitation spectral feature value on the same frequency band is calculated respectively. Multiple absolute differences are weighted according to the corresponding dimension weights, and the sum of multiple weighted spectral feature differences is divided by the number of spectral feature dimensions involved in the calculation, so as to obtain the spectral frame distance corresponding to the pair of aligned frames.

[0096] For ease of explanation, assume a target frame shows only three frequency bands with spectral characteristics of [2, 5, 7], and the corresponding imitation frame has spectral characteristics of [3, 4, 6] for its three frequency bands. Assuming each of the three spectral dimensions has a weight of 1, the absolute differences between the three spectral dimensions are [1, 1, 1]. Summing the three weighted feature differences to obtain 3, and then dividing by the number of spectral dimensions involved in the calculation (3), yields a spectral frame distance of 1 for the pair of aligned frames. It should be noted that in actual processing, when using 24-dimensional Log-Mel spectral features, the feature differences across the 24 corresponding frequency bands can be calculated in the same way to obtain the corresponding spectral frame distance. Therefore, the spectral frame distance can reflect the fine-grained energy distribution differences of a pair of aligned frames across multiple frequency bands.

[0097] Subsequently, according to a preset frequency band grouping method, the multiple frequency bands corresponding to the target frame and the emulated frame are grouped to obtain multiple target frequency band groups and multiple emulated frequency band groups. A frequency band group is a set of frequency bands obtained by dividing multiple frequency bands according to their frequency range (filter support range). For example, when the spectral characteristics include 24 Mel frequency bands, these 24 Mel frequency bands can be divided into four frequency band groups—low frequency band, mid frequency band, mid-high frequency band, and high frequency band—according to frequency from low to high. For example, frequency bands 0 to 4 can be divided into the first frequency band group (5 bands in total, corresponding to 80-676Hz), frequency bands 5 to 9 into the second frequency band group (5 bands in total, corresponding to 552-1508Hz), frequency bands 10 to 15 into the third frequency band group (6 bands in total, corresponding to 1309-3194Hz), and frequency bands 16 to 23 into the fourth frequency band group (8 bands in total, corresponding to 2843-7600Hz). This specific grouping is only an example and does not constitute a limitation on the frequency band grouping method.

[0098] Next, the average energy of the multiple frequency bands included in each target frequency band group and each emulation frequency band group can be calculated separately to obtain the target average energy corresponding to each target frequency band group and the emulation average energy corresponding to each emulation frequency band group. Then, the multiple target average energies can be arranged in ascending order of frequency for each frequency band group (i.e., low frequency band, mid frequency band, mid-high frequency band, and high frequency band) to form the target frequency band energy profile corresponding to the target frame; and the multiple emulation average energies can be arranged in the same order to form the emulation frequency band energy profile corresponding to the emulation frame.

[0099] For example, if the average energy of the four target frequency bands corresponding to the target frame is [2, 5, 7, 3], then the target frequency band energy profile of the target frame can be represented as [2, 5, 7, 3]. Similarly, if the average energy of the four imitation frequency bands corresponding to the emulation frame is [3, 4, 6, 2], then the emulation frequency band energy profile of the emulation frame can be represented as [3, 4, 6, 2]. It can be understood that each profile value in the target frequency band energy profile and the emulation frequency band energy profile corresponds to the average energy of different frequency bands arranged from low to high frequency, and therefore can be used to describe the overall energy variation shape of an audio frame in different frequency regions.

[0100] Next, distance calculation is performed based on the target frequency band energy profile and the simulated frequency band energy profile to obtain the frequency band profile frame distance corresponding to the aligned frame. The frequency band profile frame distance compares the average energy of each group after grouping multiple frequency bands, focusing on reflecting the overall spectral shape difference from low to high frequencies. In other words, this frequency band profile frame distance is used to characterize the difference in frequency band energy change trends between the target frame and the simulated frame after frequency band grouping. This application can use multiple profile values ​​in the two frequency band energy profiles as corresponding feature dimensions, and calculate the distance between the target frequency band energy profile and the simulated frequency band energy profile based on the frame-level distance calculation method shown in formula (2) to obtain the frequency band profile frame distance corresponding to the paired aligned frames.

[0101] Specifically, this application can calculate the absolute difference between the target average energy and the simulated average energy corresponding to the same frequency band group, and calculate the feature differences corresponding to multiple frequency band groups according to formula (2) to obtain the frequency band profile frame distance. For example, the target frequency band energy profile is: [2, 5, 7, 3], and the simulated frequency band energy profile is: [3, 4, 6, 2]. The absolute differences of the four frequency band groups are: [1, 1, 1, 1]. With the dimension weights of the four frequency band groups all being 1, the sum of the four weighted feature differences is obtained as 4, and then divided by the number of frequency band groups involved in the calculation, 4, to obtain the frequency band profile frame distance corresponding to the pair of aligned frames as 1.

[0102] Then, distance calculation is performed based on the energy features corresponding to the target frame and the imitation frame, respectively, to obtain the energy frame distance corresponding to the aligned frame. For each pair of aligned frames, this application can obtain the energy features corresponding to the target frame and the imitation frame in the pair of aligned frames, and obtain the energy frame distance corresponding to the pair of aligned frames based on the frame-level distance calculation method shown in formula (2). Among them, the energy frame distance is used to reflect the difference in sound intensity between the target frame and the imitation frame.

[0103] For example, if the energy feature value of the target frame is 5.2 and the energy feature value of the imitation frame is 4.8, then the absolute difference between the two is: |5.2-4.8|=0.4. If the dimension weight corresponding to the energy dimension is 2, then the energy feature difference can be weighted based on formula (2) to obtain the corresponding energy frame distance.

[0104] Accordingly, the distance is calculated based on the fundamental frequency features corresponding to the target frame and the imitation frame, respectively, to obtain the fundamental frequency frame distance corresponding to the aligned frame. For each pair of aligned frames, this application can obtain the fundamental frequency features corresponding to the target frame and the imitation frame in the pair of aligned frames, and obtain the fundamental frequency frame distance corresponding to the pair of aligned frames based on the frame-level distance calculation method shown in formula (2).

[0105] For example, when both the target frame and the emulation frame are determined to be audio frames with valid fundamental frequencies, the difference between the corresponding fundamental frequencies of the two audio frames can be calculated. For instance, if the fundamental frequency of the target frame is 220Hz and the fundamental frequency of the emulation frame is 210Hz, then the fundamental frequency difference between the two is 10Hz, and the corresponding fundamental frequency frame distance can be calculated based on this fundamental frequency difference. As another example, when the target frame has a valid fundamental frequency while the emulation frame is determined not to have a valid fundamental frequency, the fundamental frequency frame distance can also be adjusted based on the difference in the voiced / unvoiced states of the two frames, so that the fundamental frequency frame distance can reflect not only the fundamental frequency difference but also the difference in the periodicity of the sound.

[0106] In one feasible implementation, this application can pre-set a fundamental frequency effective threshold and determine whether the fundamental frequency of the audio frame is a valid fundamental frequency based on the relationship between the voiced / unvoiced information (probability) corresponding to the audio frame and the fundamental frequency effective threshold. For example, the fundamental frequency effective threshold can be 0.15. When the voiced / unvoiced information corresponding to the audio frame meets the validity condition corresponding to the fundamental frequency effective threshold, the fundamental frequency corresponding to the audio frame can be determined as a valid fundamental frequency, and the fundamental frequency distance can be calculated based on the valid fundamental frequencies of the target frame and the emulation frame; when the corresponding condition is not met, the fundamental frequency frame distance can be determined by combining the voiced / unvoiced states of the target frame and the emulation frame.

[0107] Finally, statistical processing is performed on the spectral frame distance, frequency band contour frame distance, energy frame distance, and fundamental frequency frame distance corresponding to each pair of alignment frames in the target alignment path to obtain the spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. Understandably, the spectral distance can be obtained by averaging the multiple spectral frame distances corresponding to multiple pairs of alignment frames; the frequency band contour distance can be obtained by averaging the multiple frequency band contour frame distances; the energy distance can be obtained by averaging the multiple energy frame distances; and the fundamental frequency distance can be obtained by averaging the multiple fundamental frequency frame distances.

[0108] For example, if the target alignment path includes three pairs of alignment frames with corresponding spectral frame distances of 0.8, 1.0, and 1.2, then the spectral distance can be (0.8 + 1.0 + 1.2) / 3 = 1.0. As another example, if the frequency band profile frame distances corresponding to the three pairs of alignment frames are 0.4, 0.5, and 0.6, then the frequency band profile distance can be (0.4 + 0.5 + 0.6) / 3 = 0.5.

[0109] Thus, compared to existing audio similarity schemes that rely on a single feature, this application first uses the frame-level distance calculation method shown in formula (2) to calculate the feature distance of a single pair of mutually aligned audio frames, and then statistically analyzes the distance of the same type of frames based on multiple pairs of aligned frames in the target alignment path. This allows the spectral distance, frequency band contour distance, and energy distance to be based on the feature differences between the mutually aligned audio frames. At the same time, the fundamental frequency distance is obtained by combining the fundamental frequency and voiced / unvoiced information, thereby representing the differences between the target audio and the imitated audio from different aspects such as spectral details, overall spectral shape, sound intensity, and pitch, providing more comprehensive distance data for subsequent multi-dimensional similarity fusion.

[0110] S106: Based on the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance, a similarity score for the imitated audio relative to the target audio is obtained through calculation.

[0111] In this step, the imitation similarity score is used to quantify the degree of similarity between the imitated audio and the target audio. The imitation similarity score can be within a preset score range. For example, the preset score range can be 0 to 100 points, with higher scores indicating a greater degree of similarity between the imitated audio and the target audio.

[0112] Specifically, the process of "calculating and processing based on the spectral distance, the frequency band contour distance, the energy distance, and the fundamental frequency distance to obtain the imitation similarity score of the imitated audio relative to the target audio" in this application is as follows: First, similarity mapping is performed on the spectral distance, band contour distance, energy distance, and fundamental frequency distance to obtain spectral similarity scores, band contour similarity scores, energy similarity scores, and fundamental frequency similarity scores. In one feasible implementation, similarity mapping can be achieved through formula (3), which is as follows:

[0113] Where S represents the similarity score (spectral similarity score / band contour similarity score / energy similarity score / fundamental frequency similarity score). This indicates the current distance to be mapped (spectral distance / band profile distance / energy distance / fundamental frequency distance); This indicates the distance corresponding to zero points; This indicates the distance corresponding to high resolution. It should be noted that the high resolution distance corresponding to the spectral distance can be 0.95, and the zero resolution distance can be 2.35; the high resolution distance corresponding to the frequency band contour distance can be 0.55, and the zero resolution distance can be 2.05; the high resolution distance corresponding to the energy distance can be 0.45, and the zero resolution distance can be 2.20.

[0114] Next, a weighted calculation is performed based on the spectral similarity score and a preset spectral weight to obtain a spectral weighted score. For example, the preset spectral weight can be 30%. A weighted calculation is also performed based on the band contour similarity score and a preset band contour weight to obtain a band contour weighted score. For example, the preset band contour weight can be 35%. A weighted calculation is then performed based on the energy similarity score and a preset energy weight to obtain an energy weighted score. For example, the preset energy weight can be 30%. Finally, a weighted calculation is performed based on the fundamental frequency similarity score and a preset fundamental frequency weight to obtain a fundamental frequency weighted score. For example, the preset fundamental frequency weight can be 5%.

[0115] Finally, the similarity score between the imitated audio and the target audio is obtained by fusing the spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score.

[0116] In one feasible implementation, the imitation similarity score can be calculated using formula (4), which is specifically embodied as follows:

[0117] in, Indicates the similarity score of the imitation. Indicates the preset spectrum weights. Indicates the spectral similarity score. Indicates the preset frequency band contour weight. This represents the frequency band contour similarity score. This indicates the preset energy weight. Indicates the energy similarity score. Indicates the preset fundamental frequency weight. This represents the fundamental frequency similarity score.

[0118] Therefore, compared to related technologies that use a single sound feature for similarity evaluation, where accidental similarities or anomalous differences in a particular feature can significantly impact the overall evaluation result, this application calculates spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance separately, converts these different distances into corresponding similarity scores, and fuses them using different preset weights. This allows multiple sound features, such as spectral details, overall spectral contour, sound intensity, and pitch, to participate in the similarity evaluation, thereby improving the ability of the imitation similarity score to represent the degree of actual sound imitation.

[0119] Furthermore, the process of "fusing the spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score to obtain the similarity score of the imitated audio relative to the target audio" in this application is as follows: First, the spectral weighted score, frequency band contour weighted score, energy weighted score, and fundamental frequency weighted score are summed to obtain the initial imitation similarity score. It should be noted that while weighted fusion of multiple sound features can improve the comprehensiveness of the evaluation results, simply applying fixed weights to multiple similarity scores may still produce an initial score that is not entirely consistent with the actual degree of imitation under certain signal conditions.

[0120] For example, when the imitated audio contains only a small number of local sounds from the target audio, but these few sounds are highly similar to the corresponding segments in the target audio, a high initial imitation similarity score may be generated. Another example is that when the imitation audio has a low sound intensity, the calculation results of some sound features may lack sufficient signal basis. Yet another example is that when the frequency band profiles are significantly different but other local sound features are similar, the initial score may not fully reflect the differences in the overall spectral structure.

[0121] Based on this, this application further performs contrast calibration on the initial imitation similarity score based on multiple signal quality-related judgment results, which will be described below in conjunction with steps S1-S5 (it should be noted that steps S1-S5 are not shown in the accompanying drawings): S1: A joint determination is made based on the spectral similarity score and the energy similarity score to obtain the spectral energy determination result.

[0122] In this step, the spectral energy determination result can be used to characterize whether the target audio and the imitated audio have a high degree of similarity in both spectral distribution and sound intensity dimensions. For example, the target alignment path includes three pairs of alignment frames: (T1, R2), (T2, R3), and (T3, R4). This application compares the spectral features of the target frame and the imitated frame in the three pairs of alignment frames, obtaining a spectral similarity score of 85; and compares the energy features of the target frame and the imitated frame in the three pairs of alignment frames, obtaining an energy similarity score of 83.

[0123] In one feasible implementation, a preset high similarity threshold can be set to 80 points. When both the spectral similarity score and the energy similarity score are higher than 80 points, a spectral energy determination result can be obtained, characterizing that the target audio and the imitated audio have a high degree of consistency in both spectrum and sound intensity. For example, if the spectral similarity score is 85 points, but the energy similarity score is only 60 points, then although the target audio and the imitated audio have a high degree of similarity in spectral features, they do not reach a corresponding high degree of similarity in sound intensity, thus obtaining a spectral energy determination result different from the aforementioned situation.

[0124] Thus, by jointly considering the comparison results of the target frame and the imitation frame in terms of both spectral and energy features, the possibility of overestimating the overall matching degree due to the mere accidental similarity of a single sound feature can be reduced, and a basis for determining the adjustment degree of the initial imitation similarity score can be provided.

[0125] S2: Based on the changing trends of the target frequency band energy profile and the simulated frequency band energy profile along the target alignment path, and the changing trend of the fundamental frequency feature along the target alignment path, correlation determination is performed to obtain the frequency band fundamental frequency determination result.

[0126] In this step, the fundamental frequency determination result can be used to characterize the degree of consistency between the target audio and the simulated audio in terms of sound structure reflected by changes in frequency band energy profile and pitch.

[0127] Specifically, this application can obtain the target frequency band energy profile and fundamental frequency characteristics corresponding to the target frame and the imitation frequency band energy profile and fundamental frequency characteristics corresponding to the imitation frame in each pair of alignment frames according to the arrangement order of each pair of alignment frames in the target alignment path. For multiple target frames in the target alignment path, the target frequency band energy profiles corresponding to each target frame can be organized according to the arrangement order of each target frame in the target alignment path to determine the changing trend of the target frequency band energy profile along the target alignment path; correspondingly, the changing trend of the imitation frequency band energy profile along the target alignment path can be determined according to the imitation frequency band energy profiles corresponding to multiple imitation frames. At the same time, the fundamental frequency characteristics corresponding to each target frame and each imitation frame can be obtained according to the alignment frame order in the target alignment path to determine the changing trend of the corresponding fundamental frequency characteristics along the target alignment path.

[0128] For example, the target alignment paths sequentially include (T1, R2), (T2, R3), and (T3, R4). Assume that the target frequency band energy profiles corresponding to target frames T1, T2, and T3 sequentially exhibit enhancement, continued enhancement, and weakening over time, and the imitation frequency band energy profiles corresponding to imitation frames R2, R3, and R4 also exhibit similar enhancement, continued enhancement, and weakening changes; simultaneously, the fundamental frequency characteristics of the target frame and the corresponding imitation frame also show a change of first increasing and then decreasing along the above alignment order. In this case, it can be considered that the target audio and the imitation audio have strong consistency in terms of frequency band profile changes and fundamental frequency changes.

[0129] In one feasible implementation, a correlation coefficient can be further calculated based on the aforementioned change information, and the fundamental frequency determination result of the frequency band can be determined based on the relationship between the correlation coefficient and a preset correlation threshold. For example, the preset correlation threshold can be 0.5. When the calculated correlation coefficient is 0.7, since 0.7 is greater than 0.5, a fundamental frequency determination result of the frequency band that characterizes the change in frequency band profile and the change in fundamental frequency can be obtained.

[0130] Thus, based on the established temporal correspondence between the target frame and the imitation frame, this application can further consider the coordinated changes of different sound features over time, thereby providing a supplementary evaluation of the matching degree between the target audio and the imitation audio from the perspective of sound structure changes.

[0131] S3: Validity is determined based on the audio duration of the target audio and the imitated audio, the energy features in the imitated feature sequence, and the frequency band contour similarity score, and the audio validity determination result is obtained.

[0132] In this step, the audio validity determination results include duration validity determination results, energy validity determination results, and contour validity determination results.

[0133] To determine the validity of audio duration, this application can obtain the audio durations of both the target audio and the imitated audio, and make a judgment based on the ratio between the audio duration of the imitated audio and the audio duration of the target audio. It should be noted that when the imitated audio is significantly shorter than the target audio, even if a small number of sound segments in the imitated audio are similar to parts of the target audio, it may not be sufficient to indicate that the imitated audio completely imitates the target audio. Therefore, further determination of duration validity is needed to mitigate the impact of insufficient statistical results caused by excessively short audio.

[0134] For example, if the target audio is 3 seconds long and the imitation audio is 0.9 seconds long, then the duration ratio between the imitation audio and the target audio is 0.9 / 3 = 0.3. In one feasible implementation, the preset duration validity threshold can be 0.45. When the duration ratio 0.3 is lower than 0.45, a duration validity determination result can be obtained to characterize the insufficient duration of the imitation audio. It should be noted that the duration penalty in the aforementioned open starting point path search is mainly used to reduce the possibility that candidate audio frame paths with obviously abnormal durations are identified as target alignment paths; the duration validity determination in this step is used to further calibrate the initial imitation similarity score that has already been obtained.

[0135] For determining energy validity, this application can identify multiple imitation frames participating in the actual sound imitation comparison based on the target alignment path, and obtain the energy features corresponding to these imitation frames in the imitation feature sequence. Then, the energy features corresponding to the multiple imitation frames can be statistically analyzed to obtain energy statistics representing the intensity of the imitation audio sound actually participating in the comparison. For example, if the energy features corresponding to the multiple imitation frames in the target alignment path are 20, 25, 22, 23, and 20, then these energy features can be averaged to obtain an average energy of 22.

[0136] In one feasible implementation, the average energy corresponding to the imitated audio can be compared with a preset energy validity condition. For example, when the average energy is lower than the 45th percentile of the corresponding energy distribution, it indicates that the recording volume is too low, and an energy validity judgment result can be obtained to characterize the low sound intensity of the imitated audio. Thus, the energy validity judgment is not based on all audio frames in the imitated audio that did not participate in the actual alignment, but can be based on the imitated frames in the target alignment path that actually establish a correspondence with the target frame, so that the judgment result can more accurately reflect the sound intensity of the audio content participating in the sound imitation comparison.

[0137] For contour validity determination, this application can make a judgment based on the relationship between the frequency band contour similarity score and the preset contour validity threshold. The frequency band contour similarity score characterizes the degree of similarity in the overall spectral shape between the aligned target frame and the imitated frame. For example, the preset contour validity threshold can be 42 points. When the frequency band contour similarity score is 38 points, since 38 points is lower than 42 points, a contour validity determination result can be obtained that characterizes a large difference in frequency band contours between the target audio and the imitated audio.

[0138] Thus, by judging whether the data used for scoring is sufficiently valid from three aspects—audio duration, sound intensity of the imitation frames actually involved in alignment, and frequency band contour similarity between the target frame and the imitation frame—it is possible to identify situations that may cause the initial imitation similarity score to be too high or its credibility to be reduced, providing a basis for subsequent targeted calibration of the initial imitation similarity score.

[0139] S4: Determine the contrast calibration parameters based on the spectral energy determination results, the frequency band fundamental frequency determination results, and the audio validity determination results.

[0140] In this step, the contrast calibration parameter is used to characterize the degree of adjustment to the initial imitation similarity score.

[0141] Specifically, when the spectral energy determination results indicate that both the spectral similarity score and the energy similarity score are within a high range, a first positive calibration parameter can be determined. This first positive calibration parameter can be used to reduce the corresponding weights in the cumulative distance matrix (audio frame matrix) to enhance the positive impact of a high-confidence match between the target audio and the imitated audio on the final score.

[0142] For example, after comparing the spectral features of the target frame and the imitated frame aligned in the target alignment path, the obtained spectral similarity score is 85 points; after comparing the energy features, the obtained energy similarity score is 83 points. Since both the spectral similarity score and the energy similarity score are higher than 80 points, the spectral energy determination result can characterize that the current alignment segment has high consistency in both spectral and sound intensity aspects. At this time, a first positive calibration parameter can be determined to reduce the corresponding weight in the cumulative distance matrix, thereby reducing the cumulative distance corresponding to the high-confidence alignment relationship and enhancing the corresponding matching result. This enhancement corresponds to a score improvement of approximately +6 points.

[0143] When the frequency band fundamental frequency determination result indicates a preset correlation between the frequency band profile and the fundamental frequency change trend, a second positive calibration parameter can be determined. This second positive calibration parameter can be used to enhance the positive impact on the final score when the target audio and the simulated audio exhibit consistency in sound structure changes.

[0144] For example, based on multiple target frames and multiple emulation frames in the target alignment path, the target frequency band energy profile, the emulation frequency band energy profile, and the corresponding fundamental frequency change information are obtained respectively, and a correlation coefficient of 0.7 is calculated based on the above change information. If the preset correlation threshold is 0.5, since 0.7 is greater than 0.5, it can be determined that the target audio and the emulation audio have strong consistency in the sound structure reflected by the frequency band profile and pitch change, and a second positive calibration parameter is determined. This enhancement corresponds to a score effect of approximately +6 points.

[0145] When the duration validity determination result indicates that the duration of the simulated audio is insufficient, a first negative calibration parameter can be determined. This first negative calibration parameter can be used to increase the crossing penalty term in the matrix during dynamic time warping distance calculation, thereby increasing the path cost corresponding to audio frame paths with significantly insufficient duration.

[0146] For example, if the target audio has a duration of 3 seconds and the imitated audio has a duration of 0.9 seconds, the duration ratio between the two is 0.3. Since 0.3 is less than 0.45, the imitated audio can be considered too short, and insufficient imitated audio content may not be enough to form stable statistical results. In this case, a first negative calibration parameter can be determined, and the crossing penalty term in the matrix can be increased according to the first negative calibration parameter, thereby increasing the corresponding dynamic time warping distance and reducing the possibility of obtaining a high similarity score for excessively short imitated audio. This duration penalty corresponds to a score effect of approximately -10 points.

[0147] When the average energy of the imitated audio is low according to the energy validity determination result, a second negative calibration parameter can be determined. This second negative calibration parameter can be used to reduce the weight of the energy dimension during distance calculation, thereby reducing the impact of energy features on the sound similarity judgment result under low sound intensity conditions.

[0148] For example, energy features corresponding to multiple imitation frames actually involved in the sound comparison within the target alignment path can be obtained, and the average energy of the imitation audio can be determined based on these energy features. When this average energy is lower than the 45th percentile of the corresponding energy distribution, the overall sound intensity of the imitation sound actually involved in the comparison can be considered low. In cases of excessively low sound intensity, the comparison results in the energy dimension may not be sufficient to stably reflect the relationship between the sound intensity of the target audio and the imitation audio. Therefore, a second negative calibration parameter can be determined, and the weight corresponding to the energy dimension during the distance calculation process can be dynamically reduced based on this second negative calibration parameter. This processing corresponds to a scoring effect of approximately -5 points.

[0149] When the frequency band contour difference between the target audio and the simulated audio is significant, a third negative calibration parameter can be determined. This third negative calibration parameter can be used to dynamically attenuate the weights corresponding to the frequency band contour dimensions during multi-dimensional scoring fusion.

[0150] For example, based on multiple target frames and mimic frames aligned with each other in the target alignment path, the target frequency band energy contour and the mimic frequency band energy contour are compared, resulting in a frequency band contour similarity score of 38. If the preset contour validity threshold is 42, since 38 is lower than 42, it can be considered that the overall spectral shape difference between the target audio and the mimic audio is significant. In this case, a third negative calibration parameter can be determined, and the weights corresponding to the frequency band contour dimension in the multi-dimensional score fusion process can be dynamically attenuated according to the third negative calibration parameter. This processing corresponds to a score effect of approximately -9 points.

[0151] When the contour validity determination result indicates a high degree of similarity in the frequency band contours between the target audio and the imitated audio, a third positive calibration parameter can be determined. This third positive calibration parameter can be used to dynamically increase the weight corresponding to the frequency band contour dimension during the multi-dimensional scoring fusion process, thereby improving the contribution of the frequency band contour dimension to the fusion result.

[0152] For example, based on multiple target frames and mimic frames aligned with each other in the target alignment path, the target frequency band energy contour and the mimic frequency band energy contour are compared, resulting in a frequency band contour similarity score of 85. If the preset contour reward threshold is 80, since 85 is higher than 80, it can be considered that the overall spectral shape between the target audio and the mimic audio has a high degree of consistency. At this point, a third positive calibration parameter can be determined, and the weight corresponding to the frequency band contour dimension in the multi-dimensional score fusion process can be dynamically increased according to the third positive calibration parameter. This processing corresponds to a score improvement of approximately +9 points.

[0153] Thus, this application can adjust the distance calculation parameters, path calculation parameters, or score fusion parameters directly related to different sound matching situations, so as to establish a correspondence between the contrast calibration and the actual sound feature comparison results of the target audio and the imitation audio, thereby providing a parameter basis for obtaining the calibrated imitation similarity score.

[0154] S5: Based on the contrast calibration parameters, the initial imitation similarity score is calibrated to obtain the imitation similarity score of the imitation audio relative to the target audio.

[0155] In this step, based on the contrast calibration parameters determined in step S5 above, the processing parameters in the distance calculation process, path calculation process, or score fusion process corresponding to the contrast calibration parameters can be adjusted, and the initial imitation similarity score can be calibrated based on the adjusted processing results to obtain the imitation similarity score of the imitation audio relative to the target audio.

[0156] For example, the initial imitation similarity score between a target audio and the imitated audio is 78 points. If both the spectral similarity score and the energy similarity score are higher than 80 points, the cumulative distance matrix can be adjusted according to the corresponding first positive calibration parameter. This adjustment has an effect on the final score equivalent to increasing it by approximately 6 points. Simultaneously, if the average energy of the imitated audio is lower than the 45th percentile, the energy dimension weight can be reduced according to the second negative calibration parameter. This adjustment has an effect on the final score equivalent to decreasing it by approximately 5 points. After these processing steps, the calibrated imitation similarity score can be obtained based on the adjusted distance and the fusion result.

[0157] For example, a certain imitated audio might have a high initial imitation similarity score, but the duration ratio between the imitated and target audio is only 0.3, and the frequency band contour similarity score is 38. Since the duration ratio is below 0.45, the crossing penalty term in the matrix can be increased according to the first negative calibration parameter; simultaneously, since the frequency band contour similarity score is below 42, the dimension weight corresponding to the frequency band contour dimension can be decreased according to the third negative calibration parameter. In this way, this application can update the initial imitation similarity score based on the adjusted path distance and multi-dimensional fusion results, thereby reducing the possibility of an overly high score when the imitated audio is too short or the overall spectral shape differs significantly.

[0158] Therefore, this application can enhance the scoring performance when the target audio and the imitation audio have high consistency in multiple sound feature dimensions, while reducing the possibility of overscoring when the imitation audio is too short, the sound intensity is too low, or the overall spectral shape is too different, thereby improving the reliability of high-confidence matching results and imitation similarity scores.

[0159] Figure 3 This is a flowchart illustrating a voice imitation comparison method for obtaining a similarity score, provided in an embodiment of this application. Figure 3First, the target audio and the mimic audio generated for the target audio are acquired, and detection processing is performed on the target audio and the mimic audio respectively to obtain the target effective audio and the mimic effective audio. Then, downsampling processing is applied to the target effective audio and the mimic effective audio to ensure they have a uniform sampling rate. Further, feature extraction is performed on the processed target effective audio and mimic effective audio to obtain corresponding spectral features, energy features, and fundamental frequency features, forming a target feature sequence and a mimic feature sequence. Next, open start point alignment is performed based on the target feature sequence and the mimic feature sequence to search for audio frame paths from multiple candidate path start points in the mimic feature sequence and determine the target alignment path. Based on the target alignment path, the frame features aligned in the target feature sequence and the mimic feature sequence are calculated to obtain spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance. The spectral distance characterizes the energy distribution differences of the aligned frames across multiple frequency bands, the frequency band contour distance characterizes the difference between the target frequency band energy contour and the mimic frequency band energy contour, the energy distance characterizes the signal strength differences between aligned frames, and the fundamental frequency distance characterizes the pitch differences between aligned frames. Furthermore, similarity mapping and weighted fusion are performed on the above multiple distances, and contrast calibration is performed based on the spectral energy determination result, the frequency band fundamental frequency determination result, and the audio validity determination result. Finally, the similarity score of the imitated audio relative to the target audio is obtained, and corresponding feedback can be output based on the similarity score.

[0160] Figure 4 A flowchart illustrating the similarity score obtained in another voice imitation comparison method provided in this application embodiment. Figure 4 First, the target audio and the generated imitation audio are acquired. Detection processing is then performed on both to obtain the effective target audio and the effective imitation audio. Next, frame-by-frame processing and feature extraction are performed on both the effective target audio and the effective imitation audio to obtain frame features corresponding to each target frame and each imitation frame. These features are then arranged chronologically to obtain target feature sequences and imitation feature sequences. Further, the corresponding positions between the first audio frame in the target feature sequence and each imitation frame are used as candidate path starting points. Audio frame paths are searched based on multiple frame-level distances, and the audio frame path with the smallest cumulative candidate distance is determined as the target alignment path. If the target alignment path is valid, the aligned frame features are calculated based on the target alignment path; if the target alignment path is invalid, global DTW (Dynamic Time-Divergence) can be used. Subsequently, spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance are obtained based on the target alignment path, and similarity mapping and weighted calculations are performed on each. An initial imitation similarity score is obtained through fusion processing.

[0161] After obtaining the initial imitation similarity score, the feature comparison results between the target audio and the imitated audio, as well as the audio validity, can be further determined to meet the preset calibration conditions. If the reward conditions are met, corresponding positive calibration processing can be applied; if the penalty conditions are met, corresponding negative calibration processing can be applied; if the corresponding calibration conditions are not met, no corresponding score adjustment can be performed. Thus, based on the calibration processing results, an imitation similarity score is obtained between the imitated audio and the target audio, and corresponding feedback can be output according to the imitation similarity score, thereby completing the sound imitation comparison between the target audio and the imitated audio.

[0162] In summary, the sound imitation comparison method provided in this embodiment can improve the adaptability of time alignment between the target audio and the imitation audio, and more comprehensively evaluate the degree of imitation of the target audio relative to the imitation audio through multiple sound feature dimensions. Specifically, the adaptive VAD threshold estimation method in this application can automatically adapt to different noise environments, significantly reduce the false detection rate, and has stronger robustness in diverse real-world usage scenarios such as home and outdoor environments. The open start point (DTW) alignment method allows recordings to start matching at any position without requiring precise silence endpoint detection at the front end, simplifying system design while improving alignment success rate and making the scoring more accurate. The multi-metric feature fusion method comprehensively considers features from four dimensions: Mel spectrum, band profile, energy, and fundamental frequency. Through a weight allocation of (30%, 35%, 30%, 5%), it ensures the accuracy of the overall similarity judgment and improves the ability to capture the spectral shape through the innovative feature of band profile, which can effectively capture the overall direction of the spectral shape. Based on a technical calibration method that considers five signal quality indicators—spectrum-energy joint judgment, frequency band-fundamental frequency correlation, duration validity, energy validity, and contour validity—this approach effectively suppresses common misjudgment scenarios (such as excessively short recordings, abnormal energy, and contour mismatch), significantly improving the discriminative power of highly similar samples and the reliability of the overall score. Furthermore, the algorithm in this application is designed for resource-constrained embedded scenarios; the core algorithm modules do not rely on heap memory allocation and can achieve full-process processing through static buffers, making it suitable for running on Cortex-M4F-level microcontrollers.

[0163] Based on the sound imitation comparison method provided in the foregoing embodiments, this application also provides a sound imitation comparison device. Figure 5 This is a schematic diagram of the structure of a sound imitation comparison device provided in an embodiment of this application. Figure 5 As shown, the sound imitation and comparison device includes: Audio acquisition unit 501 is used to acquire target audio and simulated audio generated for the target audio; The feature sequence acquisition unit 502 is used to perform audio processing on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in the time order of the audio frames, and the frame features include spectral features, energy features and fundamental frequency features; The frame-level distance acquisition unit 503 is used to calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence based on the frame features, and obtain multiple frame-level distances; The target path determination unit 504 is used to take the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, search for audio frame paths from each of the candidate path starting points based on the multiple frame-level distances, and determine the audio frame path with the smallest candidate cumulative distance as the target alignment path. Multiple distance acquisition units 505 are used to calculate and process the frame features that are aligned with each other in the target feature sequence and the imitation feature sequence based on the target alignment path to obtain spectral distance, frequency band contour distance, energy distance and fundamental frequency distance; wherein, the spectral distance is used to characterize the energy distribution difference of each pair of aligned frames in the target alignment path in multiple frequency bands, the frequency band contour distance is used to characterize the difference between the target frequency band energy contour and the imitation frequency band energy contour formed after grouping the multiple frequency bands, the energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, and the fundamental frequency distance is used to characterize the tone difference between each pair of aligned frames in the target alignment path; The similarity scoring unit 506 is used to perform calculations based on the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance to obtain the similarity score of the imitated audio relative to the target audio.

[0164] In one feasible implementation, the feature sequence obtaining unit 502 is specifically used for: The target audio and the simulated audio are detected and processed separately to obtain the target valid audio in the target audio and the simulated valid audio in the simulated audio; The target valid audio and the simulated valid audio are processed into frames to obtain multiple target frames and multiple simulated frames. Feature extraction is performed on each target frame and each imitation frame to obtain the frame features corresponding to each target frame and each imitation frame respectively. Arrange the frame features corresponding to each target frame in the time order of the audio frames to obtain the target feature sequence, and arrange the frame features corresponding to each imitation frame in the time order of the audio frames to obtain the imitation feature sequence.

[0165] In one feasible implementation, the feature sequence obtaining unit 502 is specifically used for: Frame energy is extracted from the target audio and the simulated audio respectively to obtain the target frame energy sequence corresponding to the target audio and the simulated frame energy sequence corresponding to the simulated audio; wherein, the frame energy sequence includes multiple frame energies arranged in chronological order of audio frames, and the frame energy is used to characterize the signal strength of the corresponding audio frame; The energy sequences of the target frame and the emulated frame are calculated separately to obtain the first target energy quantile and the second target energy quantile corresponding to the target audio, as well as the first emulated energy quantile and the second emulated energy quantile corresponding to the emulated audio; wherein the first energy quantile is used to characterize the noise floor of the corresponding audio, and the second energy quantile is used to characterize the medium energy level of the corresponding audio. Based on the first target energy quantile and the second target energy quantile, a target activity detection threshold corresponding to the target audio is determined, and based on the first imitation energy quantile and the second imitation energy quantile, an imitation activity detection threshold corresponding to the imitation audio is determined; Based on the target activity detection threshold and the target frame energy sequence, the target effective audio is extracted from the target audio, and based on the imitation activity detection threshold and the imitation frame energy sequence, the imitation effective audio is extracted from the imitation audio.

[0166] In one feasible implementation, the target path determination unit 504 is specifically used for: An audio frame matrix is ​​constructed based on the arrangement positions of each target frame in the target feature sequence and the arrangement positions of each imitation frame in the imitation feature sequence; wherein each matrix position in the audio frame matrix corresponds to one target frame and one imitation frame; The matrix positions corresponding to the first audio frame in the target feature sequence and each of the imitation frames are determined as the starting points of each candidate path; wherein, the matrix positions corresponding to the first audio frame in the imitation feature sequence and other target frames other than the first audio frame in the target feature sequence are determined as invalid positions, and the invalid positions are used to indicate that it is not allowed to start searching for audio frame paths from the corresponding positions.

[0167] In one feasible implementation, the target path determination unit 504 is specifically used for: Based on the frame-level distances corresponding to each matrix position in the audio frame matrix, excluding the invalid positions, a path search is performed from each candidate path starting point to obtain multiple candidate audio frame paths. The frame-level distances corresponding to the matrix positions traversed by each candidate audio frame path are accumulated to obtain the initial cumulative distances corresponding to each candidate audio frame path. Based on the number of imitation frames covered by each candidate audio frame path and the number of target frames included in the target feature sequence, the duration ratio corresponding to each candidate audio frame path is determined. Based on the relationship between each of the duration ratios and the preset duration ratio range, a duration penalty value is determined for each of the candidate audio frame paths; wherein, when the duration ratio is within the preset duration ratio range, the duration penalty value is zero, and when the duration ratio is not within the preset duration ratio range, the duration penalty value is positively correlated with the degree of deviation of the duration ratio from the preset duration ratio range. Based on the initial cumulative distance and duration penalty value corresponding to each of the candidate audio frame paths, the candidate cumulative distance corresponding to each of the candidate audio frame paths is determined. The candidate audio frame path with the smallest cumulative distance is determined as the target alignment path.

[0168] In one feasible implementation, the plurality of distance acquisition units 505 are specifically used for: For each pair of alignment frames in the target alignment path, distance is calculated based on the spectral features corresponding to the target frame and the mimic frame in the alignment frame to obtain the spectral frame distance corresponding to the alignment frame; According to a preset frequency band grouping method, the multiple frequency bands corresponding to the target frame and the emulated frame are grouped to obtain multiple target frequency band groups and multiple emulated frequency band groups; wherein, the frequency band group is a set of frequency bands obtained by dividing multiple frequency bands according to frequency range; The average energy of each target frequency band group and each emulation frequency band group is calculated respectively, and the average energy is arranged in order of frequency from low to high according to the corresponding frequency band group to obtain the target frequency band energy profile and emulation frequency band energy profile corresponding to the alignment frame. Based on the target frequency band energy profile and the simulated frequency band energy profile, distance calculation is performed to obtain the frequency band profile frame distance corresponding to the aligned frame; Distance calculation is performed based on the energy features corresponding to the target frame and the emulated frame respectively to obtain the energy frame distance corresponding to the aligned frame; and distance calculation is performed based on the fundamental frequency features corresponding to the target frame and the emulated frame respectively to obtain the fundamental frequency frame distance corresponding to the aligned frame. Statistical processing is performed on the spectral frame distance, frequency band profile frame distance, energy frame distance, and fundamental frequency frame distance corresponding to each pair of alignment frames in the target alignment path to obtain the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance.

[0169] In one feasible implementation, the similarity scoring unit 506 is specifically used for: Similarity mapping is performed on the spectral distance, the frequency band contour distance, the energy distance, and the fundamental frequency distance respectively to obtain spectral similarity scores, frequency band contour similarity scores, energy similarity scores, and fundamental frequency similarity scores; A weighted spectral score is obtained by weighting the spectral similarity score and the preset spectral weights. A weighted score for the frequency band contour is obtained by weighting the frequency band contour similarity score and the preset frequency band contour weight. A weighted energy score is obtained by weighting the energy similarity score and the preset energy weight. A weighted score is obtained by weighting the fundamental frequency similarity score and the preset fundamental frequency weight. The similarity score of the imitated audio relative to the target audio is obtained by fusing the spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score.

[0170] In one feasible implementation, the similarity scoring unit 506 is specifically used for: The spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score are summed to obtain an initial imitation similarity score. A joint determination is made based on the spectral similarity score and the energy similarity score to obtain the spectral energy determination result; Based on the changing trends of the target frequency band energy profile and the simulated frequency band energy profile along the target alignment path, and the changing trend of the fundamental frequency feature along the target alignment path, correlation determination is performed to obtain the frequency band fundamental frequency determination result; The validity of the target audio and the imitated audio is determined based on the audio duration, the energy features in the imitated feature sequence, and the frequency band contour similarity score, to obtain an audio validity determination result; wherein, the audio validity determination result includes a duration validity determination result, an energy validity determination result, and a contour validity determination result; Based on the spectral energy determination result, the frequency band fundamental frequency determination result, and the audio validity determination result, a contrast calibration parameter is determined; wherein, the contrast calibration parameter is used to characterize the degree of adjustment to the initial imitation similarity score; The initial imitation similarity score is calibrated based on the contrast calibration parameters to obtain the imitation similarity score of the imitation audio relative to the target audio.

[0171] The sound imitation comparison device provided in this application embodiment has the same beneficial effects as the sound imitation comparison method provided in the above embodiments, and will not be described again here.

[0172] This application also provides a computer device. The computer device includes: A memory that stores computer programs.

[0173] A processor is used to execute a computer program in memory to implement some or all of the steps in the sound imitation comparison method described in the foregoing embodiments.

[0174] This application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements some or all of the steps in the sound imitation comparison method described in the foregoing embodiments.

[0175] It should be noted that the "first" and "second" in the names such as "first" and "second" (if they exist) mentioned in the embodiments of this application are only used as name identifiers and do not represent the first and second in order.

[0176] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0177] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A sound imitation comparison method, characterized in that, include: Acquire the target audio and the mimicry audio generated for the target audio; Audio processing is performed on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in chronological order of audio frames, and the frame features include spectral features, energy features and fundamental frequency features; Based on the frame features, calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence to obtain multiple frame-level distances; Using the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, audio frame paths are searched from each candidate path starting point based on the multiple frame-level distances, and the audio frame path with the smallest cumulative candidate distance is determined as the target alignment path. Based on the target alignment path, the frame features aligned with each other in the target feature sequence and the imitation feature sequence are calculated to obtain spectral distance, frequency band profile distance, energy distance, and fundamental frequency distance; wherein, the spectral distance is used to characterize the energy distribution difference of each pair of aligned frames in the target alignment path across multiple frequency bands, the frequency band profile distance is used to characterize the difference between the target frequency band energy profile and the imitation frequency band energy profile formed after grouping the multiple frequency bands, the energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, and the fundamental frequency distance is used to characterize the tone difference between each pair of aligned frames in the target alignment path; The similarity score of the imitated audio relative to the target audio is obtained by calculating the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance.

2. The method according to claim 1, characterized in that, The step of performing audio processing on the target audio and the imitated audio to obtain the target feature sequence corresponding to the target audio and the imitated feature sequence corresponding to the imitated audio includes: The target audio and the simulated audio are detected and processed separately to obtain the target valid audio in the target audio and the simulated valid audio in the simulated audio; The target valid audio and the simulated valid audio are processed into frames to obtain multiple target frames and multiple simulated frames. Feature extraction is performed on each target frame and each imitation frame to obtain the frame features corresponding to each target frame and each imitation frame respectively. Arrange the frame features corresponding to each target frame in the time order of the audio frames to obtain the target feature sequence, and arrange the frame features corresponding to each imitation frame in the time order of the audio frames to obtain the imitation feature sequence.

3. The method according to claim 2, characterized in that, The step of detecting and processing the target audio and the imitation audio respectively to obtain the target valid audio in the target audio and the imitation valid audio in the imitation audio includes: Frame energy is extracted from the target audio and the simulated audio respectively to obtain the target frame energy sequence corresponding to the target audio and the simulated frame energy sequence corresponding to the simulated audio; wherein, the frame energy sequence includes multiple frame energies arranged in chronological order of audio frames, and the frame energy is used to characterize the signal strength of the corresponding audio frame; The energy sequences of the target frame and the emulated frame are calculated separately to obtain the first target energy quantile and the second target energy quantile corresponding to the target audio, as well as the first emulated energy quantile and the second emulated energy quantile corresponding to the emulated audio; wherein the first energy quantile is used to characterize the noise floor of the corresponding audio, and the second energy quantile is used to characterize the medium energy level of the corresponding audio. Based on the first target energy quantile and the second target energy quantile, a target activity detection threshold corresponding to the target audio is determined, and based on the first imitation energy quantile and the second imitation energy quantile, an imitation activity detection threshold corresponding to the imitation audio is determined; Based on the target activity detection threshold and the target frame energy sequence, the target effective audio is extracted from the target audio, and based on the imitation activity detection threshold and the imitation frame energy sequence, the imitation effective audio is extracted from the imitation audio.

4. The method according to claim 1, characterized in that, The step of using the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as the starting points of the candidate paths includes: An audio frame matrix is ​​constructed based on the arrangement positions of each target frame in the target feature sequence and the arrangement positions of each imitation frame in the imitation feature sequence; wherein each matrix position in the audio frame matrix corresponds to one target frame and one imitation frame; The matrix positions corresponding to the first audio frame in the target feature sequence and each of the imitation frames are determined as the starting points of each candidate path; wherein, the matrix positions corresponding to the first audio frame in the imitation feature sequence and other target frames other than the first audio frame in the target feature sequence are determined as invalid positions, and the invalid positions are used to indicate that it is not allowed to start searching for audio frame paths from the corresponding positions.

5. The method according to claim 4, characterized in that, The step of searching for audio frame paths from each of the candidate path starting points based on the multiple frame-level distances, and determining the audio frame path with the smallest cumulative candidate distance as the target alignment path, includes: Based on the frame-level distances corresponding to each matrix position in the audio frame matrix, excluding the invalid positions, a path search is performed from each candidate path starting point to obtain multiple candidate audio frame paths. The frame-level distances corresponding to the matrix positions traversed by each candidate audio frame path are accumulated to obtain the initial cumulative distances corresponding to each candidate audio frame path. Based on the number of imitation frames covered by each candidate audio frame path and the number of target frames included in the target feature sequence, the duration ratio corresponding to each candidate audio frame path is determined. Based on the relationship between each of the duration ratios and the preset duration ratio range, a duration penalty value is determined for each of the candidate audio frame paths; wherein, when the duration ratio is within the preset duration ratio range, the duration penalty value is zero, and when the duration ratio is not within the preset duration ratio range, the duration penalty value is positively correlated with the degree of deviation of the duration ratio from the preset duration ratio range. Based on the initial cumulative distance and duration penalty value corresponding to each of the candidate audio frame paths, the candidate cumulative distance corresponding to each of the candidate audio frame paths is determined. The candidate audio frame path with the smallest cumulative distance is determined as the target alignment path.

6. The method according to claim 1, characterized in that, The step of calculating and processing mutually aligned frame features in the target feature sequence and the mimicry feature sequence based on the target alignment path to obtain spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance includes: For each pair of alignment frames in the target alignment path, distance is calculated based on the spectral features corresponding to the target frame and the mimic frame in the alignment frame to obtain the spectral frame distance corresponding to the alignment frame; According to a preset frequency band grouping method, the multiple frequency bands corresponding to the target frame and the emulated frame are grouped to obtain multiple target frequency band groups and multiple emulated frequency band groups; wherein, the frequency band group is a set of frequency bands obtained by dividing multiple frequency bands according to frequency range; The average energy of each target frequency band group and each emulation frequency band group is calculated respectively, and the average energy is arranged in order of frequency from low to high according to the corresponding frequency band group to obtain the target frequency band energy profile and emulation frequency band energy profile corresponding to the alignment frame. Based on the target frequency band energy profile and the simulated frequency band energy profile, distance calculation is performed to obtain the frequency band profile frame distance corresponding to the aligned frame; Distance calculation is performed based on the energy features corresponding to the target frame and the emulated frame respectively to obtain the energy frame distance corresponding to the aligned frame; and distance calculation is performed based on the fundamental frequency features corresponding to the target frame and the emulated frame respectively to obtain the fundamental frequency frame distance corresponding to the aligned frame. Statistical processing is performed on the spectral frame distance, frequency band profile frame distance, energy frame distance, and fundamental frequency frame distance corresponding to each pair of alignment frames in the target alignment path to obtain the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance.

7. The method according to claim 6, characterized in that, The process of calculating the similarity score between the imitated audio and the target audio based on the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance includes: Similarity mapping is performed on the spectral distance, the frequency band contour distance, the energy distance, and the fundamental frequency distance respectively to obtain spectral similarity scores, frequency band contour similarity scores, energy similarity scores, and fundamental frequency similarity scores; A weighted spectral score is obtained by weighting the spectral similarity score and the preset spectral weights. A weighted score for the frequency band contour is obtained by weighting the frequency band contour similarity score and the preset frequency band contour weight. A weighted energy score is obtained by weighting the energy similarity score and the preset energy weight. A weighted score is obtained by weighting the fundamental frequency similarity score and the preset fundamental frequency weight. The similarity score of the imitated audio relative to the target audio is obtained by fusing the spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score.

8. The method according to claim 7, characterized in that, The process of fusing the spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score to obtain the similarity score between the imitated audio and the target audio includes: The spectrum-weighted score, the frequency band contour-weighted score, the energy-weighted score, and the fundamental frequency-weighted score are summed to obtain an initial imitation similarity score. A joint determination is made based on the spectral similarity score and the energy similarity score to obtain the spectral energy determination result; Based on the changing trends of the target frequency band energy profile and the simulated frequency band energy profile along the target alignment path, and the changing trend of the fundamental frequency feature along the target alignment path, correlation determination is performed to obtain the frequency band fundamental frequency determination result; The validity of the target audio and the imitated audio is determined based on the audio duration, the energy features in the imitated feature sequence, and the frequency band contour similarity score, to obtain an audio validity determination result; wherein, the audio validity determination result includes a duration validity determination result, an energy validity determination result, and a contour validity determination result; Based on the spectral energy determination result, the frequency band fundamental frequency determination result, and the audio validity determination result, a contrast calibration parameter is determined; wherein, the contrast calibration parameter is used to characterize the degree of adjustment to the initial imitation similarity score; The initial imitation similarity score is calibrated based on the contrast calibration parameters to obtain the imitation similarity score of the imitation audio relative to the target audio.

9. A sound imitation and comparison device, characterized in that, include: An audio acquisition unit is used to acquire target audio and simulated audio generated for the target audio; A feature sequence acquisition unit is used to perform audio processing on the target audio and the imitation audio to obtain a target feature sequence corresponding to the target audio and an imitation feature sequence corresponding to the imitation audio; wherein, the feature sequence includes multiple frame features arranged in the time order of audio frames, and the frame features include spectral features, energy features and fundamental frequency features; A frame-level distance acquisition unit is used to calculate the frame-level distance between the target frame in the target feature sequence and the imitation frame in the imitation feature sequence based on the frame features, and obtain multiple frame-level distances; The target path determination unit is used to take the corresponding positions between the first audio frame in the target feature sequence and each imitation frame as candidate path starting points, search for audio frame paths from each of the candidate path starting points based on the multiple frame-level distances, and determine the audio frame path with the smallest cumulative candidate distance as the target alignment path. Multiple distance acquisition units are configured to calculate and process the frame features aligned with each other in the target feature sequence and the imitation feature sequence based on the target alignment path, to obtain spectral distance, frequency band contour distance, energy distance, and fundamental frequency distance; wherein, the spectral distance is used to characterize the energy distribution difference of each pair of aligned frames in the target alignment path across multiple frequency bands, the frequency band contour distance is used to characterize the difference between the target frequency band energy contour and the imitation frequency band energy contour formed after grouping the multiple frequency bands, the energy distance is used to characterize the signal strength difference between each pair of aligned frames in the target alignment path, and the fundamental frequency distance is used to characterize the tone difference between each pair of aligned frames in the target alignment path; The similarity scoring unit is used to perform calculations based on the spectral distance, the frequency band profile distance, the energy distance, and the fundamental frequency distance to obtain the similarity score of the imitated audio relative to the target audio.

10. A computer device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the sound imitation comparison method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the sound imitation comparison method according to any one of claims 1-8.

Images