Method for speaker verification using the statistic of similarities between speech segments

Abstract

The present invention relates to a method for identifying a speaker using similarity statistics between voice segments, wherein two different voice data are divided into segments of a predetermined length to extract speaker embeddings, and a single speaker similarity is calculated based on the statistics of similarities calculated for each combination of speaker embeddings for the different voice data to determine whether they are the same person.

Description

Technology Field

[0001] The present invention relates to a technology for identifying a speaker based on the similarity of speaker embeddings extracted from voice data. Background Technology

[0002] Speaker recognition is a technology that estimates speaker information from a given speech, and it is classified into detailed technologies such as speaker verification, speaker identification, and speaker diarization.

[0003] Among these, speaker identification is a technology that determines whether the speakers of two given voices are the same person; generally, it extracts speaker embeddings from each voice and determines whether the speakers are the same based on the similarity calculated by comparing the two speaker embeddings.

[0004] Speaker embeddings refer to vectors of a fixed size created by extracting features from speech of arbitrary length. Recently, speaker embeddings are mostly extracted by implementing speaker embedding extraction models using artificial neural networks. Speaker embedding extraction models are designed to extract speaker embeddings of a fixed size regardless of the length of the input speech through pooling layers within the neural network.

[0005] These existing technologies have a problem where speaker identification performance deteriorates because it affects speaker embeddings when there is a large difference between the length of the speech used for embedding training and the length of the speech used for comparison in speaker identification.

[0006] Specifically, the ECAPA-TDNN method, which is currently a representative method for speaker embedding based on human neural networks, uses fixed-length speech segments of about 3 seconds for speaker embedding training. However, the two speech utterances used for speaker identification generally have arbitrary lengths. If the lengths of the two speech utterances used for speaker identification differ significantly from 3 seconds or if there is a large difference in length between the two speech utterances, it is possible to extract speaker embeddings of the same size, but the ability of the extracted speaker embeddings to express speaker characteristics is significantly affected by the length, which can degrade speaker identification performance.

[0007] To compensate for the impact of such differences in utterance length, approaches have been proposed, such as introducing deep layer aggregation during the speaker embedding extraction process or normalizing similarity by considering utterance length during the comparison of similarities between speaker embeddings; however, these methods have limitations in that they fail to fundamentally resolve the problem of differences in utterance length. Prior art literature

[0008] Republic of Korea Patent Application No. 10-2020-0169364 Republic of Korea Patent Application No. 10-2016-0124770 Republic of Korea Patent Application No. 10-2015-0151042 The problem to be solved

[0009] The objective of the present invention is to solve the above problem by providing a method and apparatus for speaker identification using similarity statistics between voice segments, wherein the speaker is identified by similarity statistics between speaker embeddings extracted from the segments by dividing the voice into fixed-length segments.

[0010] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood from the description below. means of solving the problem

[0011] A method for identifying a speaker using similarity statistics between voice segments according to one aspect of the present invention for achieving the aforementioned purpose is a method performed by an electronic device, comprising the steps of: dividing two different voice data into segments of a predetermined length; extracting speaker embeddings from each divided segment; calculating similarity for each combination of speaker embeddings for two different voice data; calculating speaker similarity based on similarity statistics for each combination of speaker embeddings; and determining whether the speaker is the same person based on the speaker similarity. Effects of the invention

[0012] According to the present invention, by dividing speech into fixed-length segments and extracting speaker embeddings for each segment, the problem in which differences in utterance length affect the expression of speaker characteristics in speaker embeddings is effectively resolved.

[0013] In addition, by identifying the speaker based on speaker similarity calculated from the statistics of similarities between speaker embeddings for each segment divided from different voices, it is possible to expect the effect of preventing the problem of degraded speaker identification performance caused by length differences between comparison voices.

[0014] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims. Brief explanation of the drawing

[0015] FIG. 1 is a block diagram of an electronic device in which a speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention is performed. FIG. 2 is a flowchart of a speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention. FIG. 3 is an example diagram illustrating the results produced in each processing step of a speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention. Specific details for implementing the invention

[0016] The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text.

[0017] The present invention aims to solve the problem of performance degradation caused by differences in speech length when identifying a speaker based on the similarity of speaker embeddings extracted from speech data.

[0018] In particular, the present invention is characterized by the technical feature of fundamentally solving the problem in which the ability of speaker embeddings to express speaker characteristics deteriorates due to differences in utterance length by dividing voice data into segments of a predetermined length and extracting speaker embeddings for each segment.

[0019] The speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention may be implemented as computer instructions that perform designated functions and are loaded into the processor or memory of an electronic device capable of data processing (e.g., a general-purpose computer, a special-purpose computer, a portable notebook computer, a network computer), and performed by the electronic device.

[0020] Referring to FIG. 1, an electronic device (100) in which a speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention is performed may include a communication interface (110), an input / output interface (120), a memory (130), and a processor (140).

[0021] The communication interface (110) may communicate with an external electronic device through a network.

[0022] For example, the communication interface (110) can communicate with an external electronic device through a short-range communication network such as Bluetooth, WiFi direct or IrDA (infrared data association) or a long-range communication network such as the Internet or a computer network (e.g., LAN or WAN).

[0023] The input / output interface (120) may be connected to an input / output device to receive user input or output data processing results through the input / output device.

[0024] Input / output devices are used to receive user input and output results based on user input, and may include, but are not limited to, a mouse, keyboard, display, etc.

[0025] The memory (130) includes a computer-readable recording medium and may store commands for performing a speaker identification method using similarity statistics between an operating system and a voice segment according to an embodiment of the present invention.

[0026] The processor (140) may execute commands stored in memory (130) by performing basic arithmetic, logic, and input / output operations, and may execute functions of a speaker identification method using similarity statistics between voice segments according to an embodiment of the present invention.

[0027] The processor (140) can receive two different voice data for speaker verification from the user through the communication interface (110) or the input / output interface (120) (S10).

[0028] In this case, the two different voice data may each have an arbitrary length. For the convenience of explanation, the two different voice data will be referred to as the first voice data and the second voice data, respectively.

[0029] The processor (140) may divide the first voice data and the second voice data input from the outside into segments of a preset length, respectively (S20).

[0030] The processor (140) may divide the first voice data and the second voice data into segments of the same length as the length of the voice used to train a learning model for extracting speaker embeddings (S20).

[0031] For example, when extracting speaker embeddings using an ECAPA-TDNN method learning model that primarily uses 3-second speech segments for speaker embedding learning, the processor (140) may divide the first speech data and the second speech data into 3-second segments.

[0032] That is, the length of the segment divided by the processor (140) may be pre-set to correspond to the length of the speech used in training the learning model for extracting speaker embeddings.

[0033] The processor (140) may use the voice data itself as a segment for the voice data if the length of the voice data is shorter than a preset length (S20).

[0034] The processor (140) may divide the voice data into multiple segments having a preset length if the length of the voice data is greater than the preset length (S20).

[0035] The processor (140) may divide each voice data into multiple segments by dividing each voice data into equal parts by the value calculated by dividing each voice data length by the preset length when the length of the voice data is an integer multiple of the preset length (S20).

[0036] However, since the length of voice data is generally not a multiple of the preset length, the processor (140) may not allow overlap between segments and may divide the voice data into segments of a preset length and discard the remaining segments, or may allow overlap between segments so that no segments are discarded from the voice data and divide the voice data into segments of a preset length that overlap each other (S20).

[0037] For example, the first voice data ( The length of ) and the pre-set length is, In this case, the processor (140) receives the first voice data ( ) itself is the first voice data ( It can be calculated as a segment for ).

[0038] When overlapping is not allowed, the processor (140) [allows] the first voice data ( The length from the point of ) Make the segments non-overlapping (= , here Is It may involve dividing into (referring to the largest number among the integers below) and discarding the remaining interval.

[0039] When overlapping is allowed, the processor (140) [allows] the first voice data ( The length from the point of ) The segment immediately preceding the segment and the overlapping interval according to the mathematical formula below ( Overlapping by ) (= , here Is It can be divided into (meaning the smallest number among the integers above).

[0040]

[0041] First voice data ( Segment partitioning was explained based only on ), but the second voice data ( Segment partitioning is also possible in the same way.

[0042] Referring to Fig. 3, an arbitrary length ( The first voice data having ) ) and arbitrary length( 2nd voice data having ) ) is entered, , In this case, the processor (130) receives the first voice data ( ) each pre-set length ( having ) segments Divided into, and the second voice data ( ) each pre-set length ( having ) segments It can be divided into.

[0043] When the voice data to be compared consists of multiple separately collected utterances, the processor (130) can combine the utterances into a single utterance and then perform segment division, or perform segment division for each utterance and output segments for the utterances constituting the voice data.

[0044] Afterwards, the processor (130) may extract speaker embeddings from each segment that has been divided (S30).

[0045] The processor (130) extracts segments divided from the first voice data into a pre-trained learning model to extract embeddings of a preset size from the voice data. and segments partitioned from the second voice data Speaker embeddings for the first speech data as speaker embeddings are extracted from each segment by inputting each of them Speaker embeddings for the second voice data It may be possible to extract (S30).

[0046] Here, the learning model may utilize existing technologies developed and used to extract embeddings from speech data; for example, x-vector, ECAPA-TDNN, WavLM, etc., may be used, but is not limited thereto.

[0047] In Fig. 3 , I explained it by assuming a situation, or In this case, the processor (130) receives the first voice data ( ) or second voice data ( A single speaker embedding from itself or It may be extracting.

[0048] According to the above configuration, the length of the voice data to be compared is divided into segments to match the length of the voice data used to train the learning model for extracting speaker embeddings, and then speaker embeddings are extracted, thereby preventing the problem where the characteristics of speaker embeddings extracted from the voice vary depending on the length.

[0050] Afterwards, the processor (130) may calculate similarity for each combination of speaker embeddings for two different voice data (S40).

[0051] The processor (130) has a plurality of first speaker embeddings extracted from segments divided from the first voice data. and multiple second speaker embeddings extracted from segments partitioned from the second voice data Between segments depending on the combination Individual similarities of dogs It may be possible to produce (S40).

[0052] Here is a speaker embedding extracted from the i-th segment partitioned from the first voice data ( ) and speaker embeddings extracted from the j-th segment partitioned from the second voice data ( It means the similarity between ).

[0053] The similarity between speaker embeddings can be calculated using existing similarity calculation methods. For example, the similarity between speaker embeddings may be calculated using cosine similarity; however, since this is not a core aspect of the present invention, a detailed explanation will be omitted.

[0054] The processor (130) may calculate speaker similarity based on the statistics of similarities calculated for each speaker embedding combination (S50).

[0055] The processor (130) determines the single speaker similarity between the first voice data and the second voice data as a statistical amount of individual similarities between segments ( It may be possible to produce ) (S50).

[0056] Here, the individual similarity between segments is the similarity between speaker embeddings extracted from segments for different speech data ( It means ).

[0057] In one example, the processor (130) may calculate the average of the individual similarities between segments as the speaker similarity between the first voice data and the second voice data (S50).

[0058] In another example, the processor (130) may calculate the maximum value among the individual similarities between segments as the speaker similarity between the first voice data and the second voice data (S50).

[0059] In another example, the processor (130) may calculate the average of the similarities belonging to a preset percentage in descending order among the individual similarities between segments as the speaker similarity between the first voice data and the second voice data (S50).

[0060] According to the above configuration, reliable speaker similarity can be provided even with differences in length between different voice data to be compared.

[0061] Afterwards, the processor (130) may determine whether the person is the same based on the calculated speaker similarity (S60).

[0062] The processor (130) may determine whether the calculated speaker similarity exceeds a predetermined threshold value (S60).

[0063] The processor (130) may determine whether the speaker is the same person based on whether the normalized similarity exceeds a predetermined threshold value, after normalizing the calculated speaker similarity to overcome the variability in speaker similarity caused by environmental factors other than the difference in length (S60).

[0064] For speaker similarity normalization, methods such as ZT-Norm, S_Norm, and AS-Norm may be used, or similarity normalization methods based on more complex PLDA schemes may be used.

[0065] In addition, normalized similarity values ​​can be converted into a likelihood ratio (LR) and presented as supplementary data for determining whether they are the same person.

[0066] Normalizing speaker similarity and determining whether they are the same person by comparing it with a threshold value can utilize widely used existing methods and is not a characteristic feature of the present invention, so a detailed explanation will be omitted.

[0067] According to the present invention, voice data to be compared is first divided into segments having a fixed length corresponding to the voice length used for training a learning model used for extracting speaker embeddings, and speaker embeddings are extracted from each segment, thereby preventing the problem of voice feature reflection varying depending on the length when extracting speaker embeddings.

[0068] In addition, by individually calculating the similarity between speaker embeddings extracted from segments according to the combination of segments for different voice data being compared, and calculating a single speaker similarity using the statistics of the calculated similarities, the variability of speaker similarity caused by differences in voice length can be reduced.

[0069] Accordingly, the problem where differences in speech utterance lengths affect the similarity between speaker embeddings and comparison speech can be fundamentally resolved.

[0070] Therefore, the advantage of being able to reliably identify the speaker even for speech with significant differences in length can be expected.

[0071] A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the claims and their equivalents should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0072] 100 : Electronic device 110: Communication interface 120: Input / Output Interface 130 : Memory 140 : Processor

Claims

Claim 1 A method for identifying a speaker using similarity statistics between voice segments, comprising: a step of dividing a first voice data and a second voice data into segments of a predetermined length, each performed by an electronic device; a step of extracting speaker embeddings from the segments divided in the first voice data and the segments divided in the second voice data, respectively; a step of calculating similarity for each combination of a first speaker embedding and a second speaker embedding according to a combination of a plurality of first speaker embeddings extracted from the segments divided in the first voice data and a plurality of second speaker embeddings extracted from the segments divided in the second voice data, thereby calculating individual similarities between segments; a step of calculating a single speaker similarity between the first voice data and the second voice data using the statistical amount of individual similarities between segments; and a step of determining whether they are the same person according to the speaker similarity. Claim 2 delete Claim 3 delete Claim 4 A method for identifying a speaker using similarity statistics between voice segments, wherein the step of dividing into segments in claim 1 is to divide the first voice data and the second voice data into segments of the same length as the voice used to train a learning model for extracting speaker embeddings, respectively.

Images