Tone similarity detection method, model training method, device, equipment and medium

Abstract

The invention provides a tone similarity detection method, a model training method, devices, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning and speech synthesis. The specific implementation scheme is as follows: acquiring at least one sample audio data pair of a sample audio; the sample audio data pair comprises sample original audio data and sample synthetic audio data; performing feature extraction on the sample audio data pair to obtain a sample audio feature pair; and training a twin network by using the sample audio feature pair to obtain a timbre similarity detection model. According to the technical scheme, the timbre similarity detection accuracy can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of deep learning and speech synthesis technology. Background Art

[0002] With the rapid advancement of deep learning and artificial intelligence technologies, Text-to-Speech (TTS) technology has achieved qualitative improvements in speech synthesis similarity and naturalness. Currently, this technology has been widely used in many fields such as voice assistants, language learning, and personalized TTS generation. However, with the increasing demand for personalized and customized experience of applications (APPs), how to make speech synthesis models generate high-quality and high-similarity audio to meet user expectations has become the most important issue in evaluating the effect of personalized TTS generation.

[0003] In the process of using personalized TTS generation, the TTS model mainly realizes personalized TTS synthesized audio through speaker encoding or transfer learning in deep learning. However, due to the complex speech synthesis process, less original audio and uneven quality, it is very difficult to measure the timbre similarity between the original audio and the audio synthesized by the personalized TTS model. Therefore, it is particularly important to efficiently and accurately identify the timbre similarity between the original audio and the synthesized audio. Summary of the invention

[0004] The present invention provides a timbre similarity detection method and a model training method, device, equipment and medium.

[0005] According to one aspect of the present disclosure, a method for training a timbre similarity detection model is provided, the method comprising:

[0006] Acquire at least one sample audio data pair of sample audio; the sample audio data pair includes sample original audio data and sample synthesized audio data;

[0007] Extracting features from the sample audio data pair to obtain a sample audio feature pair;

[0008] The sample audio feature pairs are used to train the twin network to obtain a timbre similarity detection model.

[0009] According to another aspect of the present disclosure, a method for detecting timbre similarity is provided, the method comprising:

[0010] Acquire at least one target audio data pair of target audio; the target audio data pair includes target original audio data and target synthesized audio data;

[0011] Performing feature extraction on the target audio data pair to obtain a target audio feature pair;

[0012] Using a timbre similarity detection model to perform similarity detection on at least one target audio feature pair to obtain a timbre detection result;

[0013] The timbre similarity detection model is trained based on the training method of the timbre similarity detection model provided in any embodiment of the present disclosure.

[0014] According to another aspect of the present disclosure, a training device for a timbre similarity detection model is provided, the device comprising:

[0015] A sample audio pair acquisition module, used to acquire at least one sample audio data pair of the sample audio; the sample audio data pair includes sample original audio data and sample synthesized audio data;

[0016] A sample feature pair determination module, used for extracting features from the sample audio data pair to obtain a sample audio feature pair;

[0017] The model training module is used to train the twin network using the sample audio feature pairs to obtain a timbre similarity detection model.

[0018] According to another aspect of the present disclosure, a device for detecting timbre similarity is provided, the device comprising:

[0019] A target audio pair acquisition module, used to acquire at least one target audio data pair of a target audio; the target audio data pair includes target original audio data and target synthesized audio data;

[0020] A target feature pair determination module, used to extract features from the target audio data pair to obtain a target audio feature pair;

[0021] A timbre detection module, used to perform similarity detection on at least one target audio feature pair using a timbre similarity detection model to obtain a timbre detection result;

[0022] The timbre similarity detection model is trained based on the training method of the timbre similarity detection model described in any embodiment of the present disclosure.

[0023] According to another aspect of the present disclosure, there is provided an electronic device, the electronic device comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the timbre similarity detection model or the timbre similarity detection method described in any embodiment of the present disclosure.

[0027] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the training method of the timbre similarity detection model or the timbre similarity detection method described in any embodiment of the present disclosure.

[0028] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method for training a timbre similarity detection model or the method for detecting timbre similarity as described in any embodiment of the present disclosure.

[0029] According to the technology disclosed in the present invention, it is possible to improve the accuracy of detecting the timbre similarity.

[0030] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0032] Figure 1 is a flow chart of a method for training a timbre similarity detection model provided according to an embodiment of the present disclosure;

[0033] Figure 2 is a flowchart of another method for training a timbre similarity detection model provided according to an embodiment of the present disclosure;

[0034] Figure 3 is a flow chart of a timbre similarity detection method provided according to an embodiment of the present disclosure;

[0035] Figure 4 It is a structural schematic diagram of a training device for a timbre similarity detection model provided according to an embodiment of the present disclosure;

[0036] Figure 5 A structural diagram of a timbre similarity detection device provided according to an embodiment of the present disclosure;

[0037] Figure 6 The block diagram is a block diagram of an electronic device for implementing the training method of the timbre similarity detection model or the timbre similarity detection method according to the embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0039] It should be noted that the terms "first", "second", "sample", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] In addition, it should be noted that in the technical solution of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of relevant data of sample audio and target audio involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0041] At present, personalized TTS evaluation mainly uses two methods for quantification: subjective evaluation and objective evaluation. Subjective evaluation mainly uses the mean subjective score (MOS) and ABX test. Objective evaluation mainly uses the Mel Cepstral Distortion (MCD) value for analysis. Among them, MOS is calculated by inviting multiple testers to score the generated speech, usually between 1 and 5 points, 1 for very bad and 5 for very good. This method directly reflects the user's subjective feeling about the similarity of synthesized personalized audio timbre. The ABX test also invites multiple testers to randomly listen to two audios and evaluate whether they are similar. The number of similarities calculated by statistics reflects the degree of similarity of audio timbre. MCD evaluates the similarity between two audios by calculating the Euclidean distance between the Mel Cepstral features of the two audios.

[0042] However, the current evaluation method has the following problems: MOS and ABX tests are both subjective evaluations, and they mainly rely on the evaluator to listen to the audio results and give scores. However, due to the different factors such as the evaluator's personal preferences, hearing sensitivity and evaluation experience, multiple tests must be conducted to eliminate the potential impact of the tester's random selection. The whole process is time-consuming and costly, and it is difficult to ensure the consistency of multiple evaluation results. The MCD-based method requires the same length of audio at both ends, has no pre-training knowledge, and the evaluation scope is limited to the same scene of two audio contents.

[0043] Figure 1 This is a flow chart of a method for training a timbre similarity detection model provided according to an embodiment of the present disclosure. This method is applicable to the case of how to perform timbre similarity detection on original audio and synthesized audio, and is particularly applicable to the case of timbre similarity detection on two audio segments with irrelevant contents. This method can be executed by a training device for a timbre similarity detection model, which can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the training function of the timbre similarity detection model, such as a server. Figure 1 As shown, the training method of the timbre similarity detection model of this embodiment may include:

[0044] S101, obtaining at least one sample audio data pair of sample audio.

[0045] In this embodiment, the sample audio refers to the audio used to train the twin network. The so-called sample audio data pair includes sample original audio data and sample synthesized audio data, wherein the sample original audio data refers to the real audio that has not been processed; the sample synthesized audio data refers to the synthesized audio data corresponding to the sample original audio data; it should be noted that the audio lengths of the sample original audio data and the sample original synthesized audio data are the same.

[0046] An optional method is to obtain at least one sample audio data pair of the sample audio by collecting audio data on the product line and public data sets on the Internet.

[0047] S102, extracting features from the sample audio data pairs to obtain sample audio feature pairs.

[0048] In this embodiment, the sample audio feature pair refers to the features corresponding to the sample audio data; including the features of the sample original audio data and the features of the sample merged audio data.

[0049] In an optional manner, feature extraction may be performed on the sample original audio data and the sample synthesized audio data in the sample audio data pair to obtain a sample audio feature pair. For example, the spectrogram, linear predictive coding, or Mel-frequency cepstrum features of the sample original audio data and the sample synthesized audio data may be extracted to obtain a sample audio feature pair.

[0050] S103, using sample audio feature pairs to train the twin network to obtain a timbre similarity detection model.

[0051] In this embodiment, the twin network refers to a network composed of two sub-convolutional neural networks with the same structure and shared weights. The timbre similarity detection model is used to detect and evaluate the timbre similarity between two audio segments.

[0052] Specifically, the sample audio features can be input into the twin network for iterative training until the training stop condition is met, and the training is stopped, and the twin network at the time of stopping the training is used as the timbre similarity detection model. It should be noted that the training stop condition can be that the number of iterations meets the set number of iterations, or the training loss is stable within the set range; wherein the set number of iterations and the set range can be set by those skilled in the art according to actual conditions.

[0053] The technical solution provided by the embodiment of the present disclosure obtains at least one sample audio data pair of the sample audio; the sample audio data pair includes the sample original audio data and the sample synthesized audio data, and then extracts features from the sample audio data pair to obtain the sample audio feature pair, and then uses the sample audio feature pair to train the twin network to obtain a timbre similarity detection model. The above technical solution can realize automatic timbre similarity detection through the twin network, significantly shortening the evaluation time and saving manpower.

[0054] On the basis of the above embodiments, as an optional method of the present disclosure, obtaining at least one sample audio data pair of sample audio includes: according to the original audio length of the sample original audio data of the sample audio, cropping the sample original synthesized audio data of the sample audio to obtain at least one sample synthesized audio data; constructing a sample audio data pair according to the sample original audio data and the sample synthesized audio data.

[0055] The sample audio includes sample original audio data and sample original synthesized audio data; the sample original audio data refers to the real audio data that has not been processed; the so-called sample original synthesized audio data refers to the synthesized original audio data; it should be noted that the audio lengths of the sample original audio data and the sample original synthesized audio can be the same or different; generally, the original synthesized audio length of the sample original synthesized audio is greater than the original audio length of the sample original audio. The so-called original audio length refers to the audio length of the sample original audio data. The so-called original synthesized audio length refers to the audio length of the sample original synthesized audio data.

[0056] Specifically, the sample original synthesized audio data can be cropped according to the original audio length of the sample original audio data, and the sample original synthesized audio data can be cropped into at least one sample synthesized audio data, and the audio length of the sample synthesized audio data is the same as the original audio length. Then, the sample original audio data and the sample synthesized audio data are combined to form a sample audio data pair.

[0057] It can be understood that by cutting the original synthesized audio data of the sample to make it consistent with the audio length of the original audio data of the sample, subsequent timbre similarity evaluations are comparable.

[0058] On the basis of the above embodiments, as an optional method of the present disclosure, feature extraction is performed on the sample audio data pair to obtain the sample audio feature pair, including: framing the sample audio data pair to obtain a sample framed audio signal pair; windowing the sample framed audio signal pair to obtain a sample windowed audio signal pair; Mel filtering the sample windowed audio signal pair to obtain a sample Mel spectrum pair; performing logarithmic operation on the sample Mel spectrum pair to obtain a sample decibel scale feature pair; normalizing the sample decibel scale feature pair to obtain a sample audio feature pair.

[0059] Among them, the sample framed audio signal pair refers to the audio signal pair after the sample audio data pair is framed, including the sample original framed audio signal and the sample synthesized framed audio signal. The so-called sample windowed audio signal pair refers to the audio signal pair after the sample framed audio signal pair is windowed, including the sample original windowed audio signal and the sample synthesized windowed audio signal. The so-called sample Mel spectrum pair refers to the Mel spectrum pair obtained after filtering the sample windowed audio signal pair, including the sample original Mel spectrum and the sample synthesized Mel spectrum. The so-called sample decibel scale feature pair refers to the feature pair after logarithmic operation of the sample Mel spectrum pair, including the sample original decibel scale feature and the sample synthesized decibel scale feature.

[0060] Specifically, since the audio signal is a short-term stable signal, that is, the audio changes relatively smoothly in a short period of time, it can be considered that the audio signal has a short-term stable characteristic. It is generally believed that the 10ms-50ms voice signal segment is a quasi-steady-state process. In the present disclosure, the sampling rate of the audio signal is 44.1KHz, and 1s of audio will have 44100 sampling values. A window of 2048 is used to frame the sample original audio data in the sample audio data to obtain the sample original framed audio signal. In this way, each frame of the audio signal obtained is relatively stable and approximates a periodic signal. Similarly, a sample synthesized framed audio signal can be obtained.

[0061] In the framing process, there are two frames that may be between two syllables or in the transition from initial consonant to final consonant. The windowing operation causes spectrum leakage at the boundary. In the present disclosure, a Hanning window is used to reduce the probability of spectrum leakage, and the window moving step size is selected to be 512. The sample original framed audio signal and the sample synthesized framed audio signal are respectively subjected to windowing processing to obtain the sample original windowed audio signal and the sample synthesized windowed audio signal.

[0062] After framing and windowing, the sample windowed audio signal pair is discretized. For example, the sample original windowed audio signal and the sample synthesized windowed audio signal can be discretized by short-time Fourier transform, and then the sample original windowed audio signal and the sample synthesized windowed audio signal in the discretized sample windowed audio signal pair are Mel-filtered to obtain the sample original Mel spectrum and the sample synthesized Mel spectrum.

[0063] In order to facilitate subsequent processing and analysis, logarithmic operations are performed on the sample original Mel spectrum and the sample synthesized Mel spectrum in the sample Mel spectrum pair to obtain the sample original decibel scale feature and the sample synthesized decibel scale feature.

[0064] Finally, on the one hand, in order to prevent the subsequent twin network from preventing the feature data range from changing too much, resulting in obvious bias in model learning; on the other hand, in order to speed up the model training process and reduce excessive fluctuations in training gradients, the mean variance normalization method can be used to normalize the sample original decibel scale features and the sample synthetic decibel scale features in the sample decibel scale feature pairs, respectively, to obtain sample audio feature pairs, which will help improve the model learning effect.

[0065] It can be understood that compared with spectrogram, linear prediction coding and Mel-cepstral features, the Mel-spectrogram features extracted from the sample audio data pairs disclosed in the present invention not only provide full-band energy information, but also are more in line with the perception of the human ear, and perform better in speech synthesis timbre evaluation.

[0066] On the basis of the above embodiments, as an optional method of the present disclosure, the twin network includes a first network and a second network; the first network and the second network share weights; the first network and the second network have the same network structure; the first network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network includes at least one convolutional layer; the second feature extraction network includes at least one fully connected layer; the second network includes a third feature extraction network and a fourth feature extraction network, wherein the third feature extraction network includes at least one convolutional layer; the fourth feature extraction network includes at least one fully connected layer. It should be noted that the size of the convolution kernel of each convolutional layer is different.

[0067] Preferably, in the present disclosure, the first feature extraction network includes 4 convolutional layers; the second feature extraction network includes 3 fully connected layers; the third feature extraction network includes 4 convolutional layers; and the fourth feature extraction network includes 3 fully connected layers.

[0068] It is understandable that this network structure design can effectively learn the timbre similarity between two audio segments.

[0069] Figure 2 1 is a flowchart of another method for training a timbre similarity detection model according to an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes "using sample audio features to train the twin network to obtain a timbre similarity detection model" and provides an optional implementation scheme. Figure 2 As shown, the training method of the timbre similarity detection model of this embodiment may include:

[0070] S201, obtaining at least one sample audio data pair of sample audio.

[0071] The sample audio data pair includes sample original audio data and sample synthesized audio data.

[0072] S202, extracting features from the sample audio data pairs to obtain sample audio feature pairs.

[0073] S203, using the first feature extraction network and the third feature extraction network to extract scale features from the sample audio feature pairs to obtain sample timbre multi-scale feature pairs.

[0074] In this embodiment, the sample timbre multi-scale feature pair refers to a feature pair obtained after multi-scale feature pair extraction of the sample audio feature pair, including the sample original timbre multi-scale feature and the sample synthesized timbre multi-scale feature.

[0075] Specifically, a first feature extraction network is used to perform multi-scale feature extraction on the sample original audio features in the sample audio feature pair to obtain multi-scale features of the sample original timbre; a third feature extraction network is used to perform multi-scale feature extraction on the sample synthetic audio features in the sample audio feature pair to obtain multi-scale features of the sample synthetic timbre.

[0076] S204, using the second feature extraction network and the fourth feature extraction network to extract distance features from the sample timbre multi-scale feature pairs to obtain the sample timbre Euclidean distance pairs.

[0077] In this embodiment, the sample timbre Euclidean distance pair refers to the one obtained by performing distance feature extraction on the sample timbre multi-scale feature pair, including the sample original timbre Euclidean distance pair and the sample synthesized timbre Euclidean distance pair.

[0078] Specifically, the second feature extraction network is used to extract distance features from the multi-scale features of the sample original timbre to obtain the Euclidean distance of the sample original timbre; the fourth feature extraction network is used to extract distance features from the multi-scale features of the sample synthetic timbre to obtain the Euclidean distance of the sample synthetic timbre.

[0079] S205: Determine a training loss based on at least one sample timbre Euclidean distance pair of the sample audio.

[0080] In this embodiment, the training loss refers to the loss used to train the twin network.

[0081] An optional method is to determine the similarity between the sample original timbre Euclidean distance and the sample synthesized timbre Euclidean distance in each sample timbre Euclidean distance pair, and then average the similarities corresponding to at least one sample timbre Euclidean distance pair to obtain the training loss.

[0082] S206, using training loss to train the twin network to obtain a timbre similarity detection model.

[0083] Specifically, the twin network is iteratively trained using training loss until the training stop condition is met, and the training is stopped. The twin network corresponding to the time of stopping training is used as a timbre similarity detection model.

[0084] The technical solution provided by the embodiment of the present disclosure uses a first feature extraction network and a third feature extraction network to extract scale features from sample audio feature pairs to obtain sample timbre multi-scale feature pairs, uses a second feature extraction network and a fourth feature extraction network to extract distance features from sample timbre multi-scale feature pairs to obtain sample timbre Euclidean distance pairs, determines the training loss based on at least one sample timbre Euclidean distance pair of the sample audio, uses the training loss to train the twin network, and obtains a timbre similarity detection model. The above technical solution extracts multi-scale features and timbre Euclidean distances of sample audio features and sample synthetic audio features in parallel, which can improve the feature extraction efficiency and thus improve the similarity detection effect of the model.

[0085] On the basis of the above embodiments, as an optional method of the present disclosure, a first feature extraction network and a third feature extraction network are used to perform scale feature extraction on sample audio feature pairs to obtain sample timbre multi-scale feature pairs, including: using the first feature extraction network to perform multi-scale feature extraction on sample original audio features in the sample audio feature pairs to obtain sample original timbre multi-scale features; using the third feature extraction network to perform multi-scale feature extraction on sample synthetic audio features in the sample audio feature pairs to obtain sample synthetic timbre multi-scale features; constructing sample timbre multi-scale feature pairs based on the sample original timbre multi-scale features and the sample synthetic timbre multi-scale features.

[0086] Specifically, the original audio features of the sample are input into the first feature extraction network, and multi-scale feature extraction is performed through the convolution layer of the first feature extraction network to obtain the multi-scale features of the original timbre of the sample. The synthesized audio features of the sample are input into the third feature extraction network, and multi-scale feature extraction is performed through the convolution layer of the third feature extraction network to obtain the multi-scale features of the synthesized timbre of the sample. Then, the multi-scale features of the original timbre of the sample and the multi-scale features of the synthesized timbre of the sample are combined to form a multi-scale feature pair of the timbre of the sample.

[0087] It should be noted that after each convolutional layer is processed, the ReLU nonlinear activation function is used to perform nonlinear processing on the data processed by the convolutional layer, which can significantly improve the nonlinear expression ability of the network; in order to solve the problem of large amount of high-dimensional audio data, batch normalization and maximum pooling methods are used to process the data after nonlinear processing, which can accelerate the network training process while reducing the model's sensitivity to data and retaining the significant features of extracted timbre.

[0088] It is understandable that the multi-scale features of timbre can be extracted through the convolution layer, thus providing support for the subsequent effective learning of the similarity between two timbres.

[0089] On the basis of the above embodiments, as an optional method of the present disclosure, a second feature extraction network and a fourth feature extraction network are used to perform distance feature extraction on a sample timbre multi-scale feature pair to obtain a sample timbre Euclidean distance pair, including: using the second feature extraction network to perform distance feature extraction on a sample original timbre multi-scale feature pair in the sample timbre multi-scale feature pair to obtain a sample original timbre Euclidean distance; using the fourth feature extraction network to perform distance feature extraction on a sample synthesized timbre multi-scale feature pair in the sample timbre multi-scale feature pair to obtain a sample synthesized timbre Euclidean distance; constructing a sample timbre Euclidean distance pair according to the sample original timbre Euclidean distance and the sample synthesized timbre Euclidean distance.

[0090] Specifically, the multi-scale features of the original timbre of the sample are input into the second feature extraction network, and the distance feature is extracted through the fully connected layer in the second feature extraction network to obtain the Euclidean distance of the original timbre of the sample. The multi-scale features of the synthesized timbre of the sample are input into the fourth feature extraction network, and the distance feature is extracted through the fully connected layer in the fourth feature extraction network to obtain the Euclidean distance of the synthesized timbre of the sample. Then, the Euclidean distance of the original timbre of the sample and the Euclidean distance of the synthesized timbre of the sample form a sample timbre Euclidean distance pair.

[0091] It can be understood that by extracting the Euclidean distance features of multi-scale features in parallel, the high-dimensional abstract features extracted by the convolutional layer are converted into typical audio timbre features, thereby facilitating the detection and evaluation of timbre similarity.

[0092] On the basis of the above embodiments, as an optional method of the present disclosure, the training loss is determined according to at least one sample timbre Euclidean distance pair of the sample audio, including: for each sample timbre Euclidean distance pair, using a contrast loss function to calculate the loss of the sample timbre Euclidean distance pair to obtain the contrast loss; and determining the training loss according to at least one contrast loss corresponding to the sample audio.

[0093] Specifically, for each sample timbre Euclidean distance pair, a contrast loss function is used to calculate the contrast loss corresponding to the sample timbre Euclidean distance pair. For example, the threshold m in the contrast loss function is set to 2, ||γ0-γ1|| represents the two timbre Euclidean distances in the sample timbre Euclidean distance pair, namely the sample original timbre Euclidean distance and the sample synthesized timbre Euclidean distance, similar sample y=1, dissimilar sample y=0, details are as follows: Loss_F(γ0, γ1, y)=(1-y)||γ0-γ1||y·max(0, m-||γ0-γ1||). Then, the average of at least one contrast loss corresponding to the sample audio is calculated to obtain the training loss.

[0094] It can be understood that the training loss determined by taking the average of the audio similarities of multiple segments of sample audio makes the timbre similarity results output by the model more accurate.

[0095] Figure 3 This is a flow chart of a timbre similarity detection method provided according to an embodiment of the present disclosure. This method is applicable to the case of how to detect timbre similarity between original audio and synthesized audio, and is particularly applicable to the case of detecting timbre similarity between two audio segments with irrelevant contents. This method can be executed by a timbre similarity detection device, which can be implemented in software and / or hardware, and can be integrated into an electronic device carrying a timbre similarity detection function, such as a server. Figure 3 As shown, the timbre similarity detection method of this embodiment may include:

[0096] S301, obtaining at least one target audio data pair of target audio.

[0097] In this embodiment, the target audio refers to the audio that needs to be evaluated and detected for timbre similarity; optionally, the target audio data pair includes target original audio data and target synthesized audio data; wherein the target original audio data refers to the real audio data of the target audio that has not been processed; and the target synthesized audio data refers to the audio data after the target audio has been synthesized.

[0098] S302, extracting features from the target audio data pair to obtain a target audio feature pair.

[0099] In this embodiment, the target audio feature pair refers to a feature pair obtained after extracting spectrum features from a target audio data pair; it includes target original audio features and target synthesized audio features.

[0100] An optional method is to extract the spectrogram, linear prediction coding, or Mel-frequency cepstrum features of the target audio data pair to obtain the target audio feature pair.

[0101] Another optional method is to extract the Mel spectrum features of the target audio data pair to obtain the target audio feature pair.

[0102] S303: Perform similarity detection on at least one target audio feature pair using a timbre similarity detection model to obtain a timbre detection result.

[0103] In this embodiment, the timbre detection result refers to the timbre similarity between the two audios in the target audio data pair, which can be expressed in percentage. The so-called timbre similarity detection model is used to perform similarity detection of two audios, and can be trained based on the training method of the timbre similarity detection model provided in any embodiment of the present disclosure.

[0104] Specifically, the target audio feature pair is input into a timbre similarity detection model, and after being processed by the model, a timbre similarity detection result of the target audio is obtained.

[0105] The technical solution provided by the embodiment of the present disclosure obtains at least one target audio data pair of the target audio; the target audio data pair includes the target original audio data and the target synthesized audio data, and then extracts features from the target audio data pair to obtain a target audio feature pair, and then uses a timbre similarity detection model to perform similarity detection on at least one target audio feature pair to obtain a timbre detection result. The above technical solution, by using the timbre similarity detection model to perform similarity evaluation and detection of two audio segments, can significantly shorten the evaluation time, save manpower, realize automatic detection of timbre similarity, and obtain reliable and repeatable evaluation and detection results.

[0106] It should be noted that the features in the present disclosure can all be represented in the form of a matrix or a vector.

[0107] The timbre similarity detection model disclosed in the present invention can perform fully automated online and / or offline detection and evaluation on any two audio clips with different contents.

[0108] Figure 41 is a schematic diagram of the structure of a training device for a timbre similarity detection model provided according to an embodiment of the present disclosure. This embodiment is applicable to the case of how to perform timbre similarity detection on original audio and synthesized audio, and is particularly applicable to the case of timbre similarity detection on two audio segments with irrelevant contents. The device can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the training function of the timbre similarity detection model, such as a server. Figure 4 As shown, the training device 400 of the timbre similarity detection model of this embodiment includes:

[0109] The sample audio pair acquisition module 401 is used to acquire at least one sample audio data pair of the sample audio; the sample audio data pair includes sample original audio data and sample synthesized audio data;

[0110] A sample feature pair determination module 402 is used to extract features from the sample audio data pair to obtain a sample audio feature pair;

[0111] The model training module 403 is used to train the twin network using sample audio feature pairs to obtain a timbre similarity detection model.

[0112] The technical solution provided by the embodiment of the present disclosure obtains at least one sample audio data pair of the sample audio; the sample audio data pair includes the sample original audio data and the sample synthesized audio data, and then extracts features from the sample audio data pair to obtain the sample audio feature pair, and then uses the sample audio feature pair to train the twin network to obtain a timbre similarity detection model. The above technical solution can realize automatic timbre similarity detection through the twin network, significantly shortening the evaluation time and saving manpower.

[0113] Furthermore, the twin network includes a first network and a second network; weights are shared between the first network and the second network; the network structures of the first network and the second network are the same; the first network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network includes at least one convolutional layer; the second feature extraction network includes at least one fully connected layer; the second network includes a third feature extraction network and a fourth feature extraction network, wherein the third feature extraction network includes at least one convolutional layer; the fourth feature extraction network includes at least one fully connected layer.

[0114] Furthermore, the model training module 403 includes:

[0115] A sample multi-scale feature pair determination unit is used to extract scale features from the sample audio feature pair using the first feature extraction network and the third feature extraction network to obtain a sample timbre multi-scale feature pair;

[0116] A sample Euclidean distance pair determination unit is used to extract distance features from the sample timbre multi-scale feature pairs using the second feature extraction network and the fourth feature extraction network to obtain the sample timbre Euclidean distance pairs;

[0117] A training loss determination unit, configured to determine a training loss based on at least one sample timbre Euclidean distance pair of the sample audio;

[0118] The model training unit is used to train the twin network using training loss to obtain a timbre similarity detection model.

[0119] Furthermore, the sample multi-scale feature pair determination unit is specifically used for:

[0120] Using the first feature extraction network to perform multi-scale feature extraction on the sample original audio features in the sample audio feature pair, to obtain the sample original timbre multi-scale features;

[0121] Using a third feature extraction network to perform multi-scale feature extraction on the sample synthetic audio features in the sample audio feature pair, to obtain the sample synthetic timbre multi-scale features;

[0122] According to the multi-scale features of the original timbre of the sample and the multi-scale features of the synthesized timbre of the sample, a multi-scale feature pair of the sample timbre is constructed.

[0123] Furthermore, the sample Euclidean distance pair determination unit is specifically used for:

[0124] A second feature extraction network is used to extract distance features from the multi-scale features of the original timbre of the sample in the multi-scale feature pair of the sample timbre, and the Euclidean distance of the original timbre of the sample is obtained;

[0125] The fourth feature extraction network is used to extract distance features from the multi-scale features of the sample synthesized timbre in the multi-scale features of the sample timbre, and the Euclidean distance of the sample synthesized timbre is obtained;

[0126] According to the Euclidean distance of the original timbre of the sample and the Euclidean distance of the synthesized timbre of the sample, a sample timbre Euclidean distance pair is constructed.

[0127] Furthermore, the training loss determination unit is specifically used to:

[0128] For each sample timbre Euclidean distance pair, a contrast loss function is used to calculate the loss of the sample timbre Euclidean distance pair to obtain a contrast loss;

[0129] A training loss is determined based on at least one contrast loss corresponding to the sample audio.

[0130] Furthermore, the sample feature pair determination module 402 is specifically configured to:

[0131] Performing frame processing on the sample audio data pair to obtain a sample frame audio signal pair;

[0132] Performing windowing processing on the sample framed audio signal pair to obtain a sample windowed audio signal pair;

[0133] Performing Mel filtering on the sample windowed audio signal pair to obtain a sample Mel spectrum pair;

[0134] Perform logarithmic operation on the sample Mel spectrum pair to obtain the sample decibel scale feature pair;

[0135] The sample decibel scale feature pair is normalized to obtain a sample audio feature pair.

[0136] Furthermore, the sample audio pair acquisition module 401 is specifically used for:

[0137] According to the original audio length of the sample original audio data of the sample audio, the sample original synthesized audio data of the sample audio is cut to obtain at least one sample synthesized audio data;

[0138] A sample audio data pair is constructed based on the sample original audio data and the sample synthesized audio data.

[0139] Figure 5 A schematic diagram of the structure of a timbre similarity detection device provided according to an embodiment of the present disclosure. The method is applicable to the case of how to detect the timbre similarity of original audio and synthesized audio, and is particularly applicable to the case of detecting the timbre similarity of two audio segments with irrelevant contents. The device can be implemented in software and / or hardware, and can be integrated into an electronic device carrying the timbre similarity detection function, such as a server. Figure 5 As shown, the timbre similarity detection device 500 of this embodiment includes:

[0140] The target audio pair acquisition module 501 is used to acquire at least one target audio data pair of the target audio; the target audio data pair includes target original audio data and target synthesized audio data;

[0141] A target feature pair determination module 502 is used to extract features from a target audio data pair to obtain a target audio feature pair;

[0142] The timbre detection module 503 is used to perform similarity detection on at least one target audio feature pair using a timbre similarity detection model to obtain a timbre detection result;

[0143] The timbre similarity detection model is trained based on the training method of the timbre similarity detection model provided in any embodiment of the present disclosure.

[0144] The technical solution provided by the embodiment of the present disclosure obtains at least one target audio data pair of the target audio; the target audio data pair includes the target original audio data and the target synthesized audio data, and then extracts features from the target audio data pair to obtain a target audio feature pair, and then uses a timbre similarity detection model to perform similarity detection on at least one target audio feature pair to obtain a timbre detection result. The above technical solution, by using the timbre similarity detection model to perform similarity evaluation and detection of two audio segments, can significantly shorten the evaluation time, save manpower, realize automatic detection of timbre similarity, and obtain reliable and repeatable evaluation and detection results.

[0145] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0146] Figure 6 The block diagram is a block diagram of an electronic device for implementing the training method of the timbre similarity detection model or the timbre similarity detection method according to the embodiment of the present disclosure. Figure 6 The block diagram is a block diagram of an electronic device for implementing the training method of the timbre similarity detection model or the timbre similarity detection method according to the embodiment of the present disclosure. Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0147] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0148] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0149] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the training method of the timbre similarity detection model or the timbre similarity detection method. For example, in some embodiments, the training method of the timbre similarity detection model or the timbre similarity detection method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the training method of the timbre similarity detection model or the timbre similarity detection method described above may be executed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the timbre similarity detection model training method or the timbre similarity detection method in any other appropriate manner (for example, by means of firmware).

[0150] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0152] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0155] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0156] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0157] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0158] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0159] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a timbre similarity detection model, comprising: Obtaining at least one sample audio data pair of sample audio; The sample audio data pair includes sample original audio data and sample synthesized audio data; Extracting features from the sample audio data pair to obtain a sample audio feature pair; The sample audio feature pairs are used to train the twin network to obtain a timbre similarity detection model.

2. The method according to claim 1, wherein: The twin network includes a first network and a second network; weights are shared between the first network and the second network; the network structures of the first network and the second network are the same; the first network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network includes at least one convolutional layer; the second feature extraction network includes at least one fully connected layer; the second network includes a third feature extraction network and a fourth feature extraction network, wherein the third feature extraction network includes at least one convolutional layer; the fourth feature extraction network includes at least one fully connected layer.

3. The method according to claim 2, wherein: The method of using the sample audio feature pair to train the twin network to obtain a timbre similarity detection model includes: Using the first feature extraction network and the third feature extraction network, extracting scale features from the sample audio feature pairs to obtain sample timbre multi-scale feature pairs; Using the second feature extraction network and the fourth feature extraction network, extracting distance features from the sample timbre multi-scale feature pairs to obtain sample timbre Euclidean distance pairs; Determining a training loss based on at least one sample timbre Euclidean distance pair of the sample audio; The twin network is trained using the training loss to obtain a timbre similarity detection model.

4. The method according to claim 3, wherein: The first feature extraction network and the third feature extraction network are used to extract scale features from the sample audio feature pair to obtain a sample timbre multi-scale feature pair, including: Using the first feature extraction network to perform multi-scale feature extraction on the sample original audio features in the sample audio feature pair to obtain multi-scale features of the sample original timbre; Using the third feature extraction network to perform multi-scale feature extraction on the sample synthetic audio features in the sample audio feature pair to obtain the sample synthetic timbre multi-scale features; A sample timbre multi-scale feature pair is constructed according to the sample original timbre multi-scale feature and the sample synthesized timbre multi-scale feature.

5. The method according to claim 3, wherein: The method of using the second feature extraction network and the fourth feature extraction network to extract distance features from the sample timbre multi-scale feature pairs to obtain sample timbre Euclidean distance pairs includes: Using the second feature extraction network to extract distance features from the multi-scale features of the sample original timbre in the multi-scale feature pair of the sample timbre, to obtain the Euclidean distance of the sample original timbre; Using the fourth feature extraction network to extract distance features from the sample synthetic timbre multi-scale features in the sample timbre multi-scale feature pair, to obtain the sample synthetic timbre Euclidean distance; A sample timbre Euclidean distance pair is constructed according to the sample original timbre Euclidean distance and the sample synthesized timbre Euclidean distance.

6. The method according to claim 3, wherein: The determining of the training loss according to at least one sample timbre Euclidean distance pair of the sample audio comprises: For each sample timbre Euclidean distance pair, a contrast loss function is used to calculate the loss of the sample timbre Euclidean distance pair to obtain a contrast loss; A training loss is determined according to at least one contrast loss corresponding to the sample audio.

7. The method according to claim 1, wherein: The step of extracting features from the sample audio data pair to obtain a sample audio feature pair includes: Performing frame processing on the sample audio data pair to obtain a sample frame audio signal pair; Performing windowing processing on the sample framed audio signal pair to obtain a sample windowed audio signal pair; Performing Mel filtering on the sample windowed audio signal pair to obtain a sample Mel spectrum pair; Performing a logarithmic operation on the sample Mel spectrum pair to obtain a sample decibel scale feature pair; The sample decibel scale feature pair is normalized to obtain a sample audio feature pair.

8. The method according to claim 1, wherein: The obtaining of at least one sample audio data pair of the sample audio comprises: According to the original audio length of the sample original audio data of the sample audio, the sample original synthesized audio data of the sample audio is cut to obtain at least one sample synthesized audio data; A sample audio data pair is constructed according to the sample original audio data and the sample synthesized audio data.

9. A method for detecting timbre similarity, comprising: Acquire at least one target audio data pair of the target audio; The target audio data pair includes target original audio data and target synthesized audio data; Performing feature extraction on the target audio data pair to obtain a target audio feature pair; Using a timbre similarity detection model to perform similarity detection on at least one target audio feature pair to obtain a timbre detection result; The timbre similarity detection model is trained based on the training method of the timbre similarity detection model described in any one of claims 1-8.

10. A training device for a timbre similarity detection model, comprising: A sample audio pair acquisition module, used to acquire at least one sample audio data pair of the sample audio; The sample audio data pair includes sample original audio data and sample synthesized audio data; A sample feature pair determination module, used for extracting features from the sample audio data pair to obtain a sample audio feature pair; The model training module is used to train the twin network using the sample audio feature pairs to obtain a timbre similarity detection model.

11. The device according to claim 10, wherein: The twin network includes a first network and a second network; weights are shared between the first network and the second network; the network structures of the first network and the second network are the same; the first network includes a first feature extraction network and a second feature extraction network, wherein the first feature extraction network includes at least one convolutional layer; the second feature extraction network includes at least one fully connected layer; the second network includes a third feature extraction network and a fourth feature extraction network, wherein the third feature extraction network includes at least one convolutional layer; the fourth feature extraction network includes at least one fully connected layer.

12. The device according to claim 11, wherein The model training module includes: A sample multi-scale feature pair determination unit, configured to extract scale features from the sample audio feature pair using the first feature extraction network and the third feature extraction network to obtain a sample timbre multi-scale feature pair; A sample Euclidean distance pair determination unit, configured to extract distance features from the sample timbre multi-scale feature pairs using the second feature extraction network and the fourth feature extraction network to obtain sample timbre Euclidean distance pairs; A training loss determination unit, configured to determine a training loss based on at least one sample timbre Euclidean distance pair of the sample audio; The model training unit is used to train the twin network using the training loss to obtain a timbre similarity detection model.

13. The device according to claim 12, wherein: The sample multi-scale feature pair determination unit is specifically used to: Using the first feature extraction network to perform multi-scale feature extraction on the sample original audio features in the sample audio feature pair to obtain multi-scale features of the sample original timbre; Using the third feature extraction network to perform multi-scale feature extraction on the sample synthetic audio features in the sample audio feature pair to obtain the sample synthetic timbre multi-scale features; A sample timbre multi-scale feature pair is constructed according to the sample original timbre multi-scale feature and the sample synthesized timbre multi-scale feature.

14. The device according to claim 12, wherein: The sample Euclidean distance pair determination unit is specifically used for: Using the second feature extraction network to extract distance features from the multi-scale features of the sample original timbre in the multi-scale feature pair of the sample timbre, to obtain the Euclidean distance of the sample original timbre; Using the fourth feature extraction network to extract distance features from the sample synthetic timbre multi-scale features in the sample timbre multi-scale feature pair, to obtain the sample synthetic timbre Euclidean distance; A sample timbre Euclidean distance pair is constructed according to the sample original timbre Euclidean distance and the sample synthesized timbre Euclidean distance.

15. The device according to claim 12, wherein: The training loss determination unit is specifically used to: For each sample timbre Euclidean distance pair, a contrast loss function is used to calculate the loss of the sample timbre Euclidean distance pair to obtain a contrast loss; A training loss is determined according to at least one contrast loss corresponding to the sample audio.

16. The device according to claim 10, wherein: The sample feature pair determination module is specifically used to: Performing frame processing on the sample audio data pair to obtain a sample frame audio signal pair; Performing windowing processing on the sample framed audio signal pair to obtain a sample windowed audio signal pair; Performing Mel filtering on the sample windowed audio signal pair to obtain a sample Mel spectrum pair; Performing a logarithmic operation on the sample Mel spectrum pair to obtain a sample decibel scale feature pair; The sample decibel scale feature pair is normalized to obtain a sample audio feature pair.

17. The device according to claim 10, wherein: The sample audio pair acquisition module is specifically used for: According to the original audio length of the sample original audio data of the sample audio, the sample original synthesized audio data of the sample audio is cut to obtain at least one sample synthesized audio data; A sample audio data pair is constructed according to the sample original audio data and the sample synthesized audio data.

18. A timbre similarity detection device, comprising: A target audio pair acquisition module, used to acquire at least one target audio data pair of a target audio; The target audio data pair includes target original audio data and target synthesized audio data; A target feature pair determination module, used to extract features from the target audio data pair to obtain a target audio feature pair; A timbre detection module, used to perform similarity detection on at least one target audio feature pair using a timbre similarity detection model to obtain a timbre detection result; Wherein, the timbre similarity detection model is trained based on the training method of the timbre similarity detection model described in any one of claims 1-8.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the timbre similarity detection model described in any one of claims 1-8, or the timbre similarity detection method described in claim 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the training method for a timbre similarity detection model according to any one of claims 1 to 8, or the timbre similarity detection method according to claim 9.

21. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the training method for the timbre similarity detection model according to any one of claims 1 to 8, or the timbre similarity detection method according to claim 9.

Images