Speaker recognition method, device, electronic device and storage medium

Abstract

The present invention relates to the field of speech recognition technology, and in particular to a speaker recognition method, device, electronic device and storage medium, the method comprising: obtaining a target speech signal of a speaker to be recognized; extracting speech features based on the complex amplitude of the target speech signal on each audio component; converting the feature value of each feature point in the speech feature into a color value to obtain a target speech feature graph; determining the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be recognized and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library. The present application improves the interpretability of the recognition basis and intuitively displays the feature differences between different speakers.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a speaker recognition method, device, electronic equipment and storage medium. Background Art

[0002] Speaker recognition technology is a biometric technology that identifies the speaker by analyzing voice features. This technology can be used in a variety of application scenarios, such as voiceprint identification workstations. Traditional speaker recognition technology uses a black box deep learning model. First, the time-frequency features or acoustic features are extracted through speech preprocessing, and then the time-frequency features or acoustic features are input into the deep learning model for training to complete the classification of the speaker's identity.

[0003] However, the black-box deep learning model has weak interpretability for recognition basis and it is difficult to intuitively display the feature differences between different speakers. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a speaker recognition method, device, electronic device and storage medium, which can improve the interpretability of the recognition basis and intuitively display the characteristic differences between different speakers.

[0005] In a first aspect, an embodiment of the present application provides a method for speaker recognition, the method comprising:

[0006] Obtaining a target speech signal of a speaker to be identified;

[0007] Extracting speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified;

[0008] Converting the characteristic value of each characteristic point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified;

[0009] Based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library, the target speaker corresponding to the target speech signal is determined; the similarity includes the average nonlinear phase coupling strength between the target speech feature graph and the preset speech feature graph, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio.

[0010] In a possible implementation, the target speech feature graph includes a first target speech feature graph and / or a second target speech feature graph; the first target speech feature graph is obtained according to the following steps:

[0011] Converting the feature value of each feature point in the speech feature into a three-channel color value to obtain a first target speech feature map;

[0012] The second target speech feature graph is obtained according to the following steps:

[0013] The feature points in the speech features are grouped; the feature values ​​of the feature points in the same group in the speech features are converted into the same three-channel color values, and the feature points with the same color values ​​are connected to obtain a second target speech feature graph.

[0014] In a possible implementation, the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker is calculated by the following steps, including:

[0015] Calculating local similarity according to the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling mode in the target speech feature graph and the preset speech feature graph;

[0016] Calculating global similarity based on the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph;

[0017] The global similarity and the local similarity are weightedly summed to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker.

[0018] In a possible implementation, calculating the local similarity according to the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling mode in the target speech feature graph and the preset speech feature graph includes:

[0019] The local similarity is calculated by the following formula:

[0020] ;

[0021] in, is the local similarity, is the target speech feature map The average nonlinear phase coupling strength of Preset voice feature map The average nonlinear phase coupling strength of is the target speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, The preset speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, is the target speech feature map The nonlinear phase coupling mode and the preset speech characteristic diagram The similarity between the nonlinear phase coupling modes is and is a constant.

[0022] In a possible implementation, calculating the global similarity according to the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph includes:

[0023] The global similarity is calculated by the following formula:

[0024] ;

[0025] in, is the local similarity between the target speech feature graph and the preset speech feature graph, is the nonlinear relationship of the audio of the target speech feature map, The nonlinear relationship of the audio for the preset speech feature graph.

[0026] In a possible implementation, the determining the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library includes:

[0027] Calculating the similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library;

[0028] Performing weighted summation of the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker in the pre-established recognition library and the corresponding first weight, and the similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library and the corresponding second weight, to obtain a comprehensive score for each preset speaker; determining the target speaker corresponding to the target speech signal based on the comprehensive score of each preset speaker;

[0029] Among them, the recognition library correspondingly stores preset speakers, preset text information and preset voice feature maps; the preset voice feature map is determined based on the preset voice signal of the preset speaker for the preset text information; the numerical value corresponding to the first weight is much larger than the numerical value corresponding to the second weight.

[0030] In a second aspect, an embodiment of the present application further provides a speaker recognition device, the device comprising:

[0031] An acquisition module, used to acquire a target speech signal of a speaker to be recognized;

[0032] An extraction module, configured to extract speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified;

[0033] A conversion module, used to convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified;

[0034] A determination module is used to determine the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library; the similarity includes the average nonlinear phase coupling strength between the target speech feature graph and the preset speech feature graph, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio.

[0035] In a possible implementation, the target speech feature map includes a first target speech feature map and / or a second target speech feature map; a conversion module is specifically used to: obtain the first target speech feature map according to the following steps: convert the feature value of each feature point in the speech feature into a three-channel color value to obtain the first target speech feature map; obtain the second target speech feature map according to the following steps: group the feature points in the speech feature; convert the feature values ​​of the feature points in the same group in the speech feature into the same three-channel color value, and connect the feature points with the same color value to obtain the second target speech feature map.

[0036] In a possible implementation, the determination module is specifically used to calculate the local similarity based on the average nonlinear phase coupling strength, local fluctuation amplitude of the nonlinear phase coupling strength and nonlinear phase coupling mode in the target speech feature graph and the preset speech feature graph; calculate the global similarity based on the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph; and perform weighted summation of the global similarity and the local similarity to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker.

[0037] In a possible implementation, the determination module is specifically configured to calculate the local similarity using the following formula:

[0038] ;

[0039] in, is the local similarity, is the target speech feature map The average nonlinear phase coupling strength of Preset voice feature map The average nonlinear phase coupling strength of is the target speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, The preset speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, is the target speech feature map The nonlinear phase coupling mode and the preset speech characteristic diagram The similarity between the nonlinear phase coupling modes is and is a constant.

[0040] In a possible implementation, the determination module is specifically configured to calculate the global similarity using the following formula:

[0041] ;

[0042] in, is the local similarity between the target speech feature graph and the preset speech feature graph, is the nonlinear relationship of the audio of the target speech feature map, The nonlinear relationship of the audio for the preset speech feature graph.

[0043] In a possible implementation, the determination module is specifically used to calculate the similarity between the text information corresponding to the target voice signal and the preset text information corresponding to each preset speaker in the recognition library; weighted sum the similarity between the target voice feature graph of the speaker to be identified and the preset voice feature graph corresponding to each preset speaker in the pre-established recognition library and the corresponding first weight, and the similarity between the text information corresponding to the target voice signal and the preset text information corresponding to each preset speaker in the recognition library and the corresponding second weight to obtain a comprehensive score for each preset speaker; based on the comprehensive score of each preset speaker, determine the target speaker corresponding to the target voice signal; wherein the recognition library stores preset speakers, preset text information and preset voice feature graphs; the preset voice feature graph is determined based on the preset voice signal of the preset speaker for the preset text information; the numerical value corresponding to the first weight is much larger than the numerical value corresponding to the second weight.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the speaker recognition method as described in any one of the first aspects.

[0045] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the speaker recognition method as described in any one of the first aspects are executed.

[0046] The present application provides a speaker recognition method, device, electronic device and storage medium, the method comprising: obtaining a target speech signal of a speaker to be recognized; extracting speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between audio components in each audio component group of the speaker to be recognized; converting the feature value of each feature point in the speech features into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be recognized; determining the target speaker corresponding to the target speech signal based on the similarity between the target speech feature map of the speaker to be recognized and the preset speech feature map corresponding to each preset speaker in a pre-established recognition library; the similarity includes the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode and the similarity of the nonlinear relationship of the audio between the target speech feature map and the preset speech feature map. The present application determines the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library, thereby improving the interpretability of the recognition basis and intuitively displaying the feature differences between different speakers. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A flow chart of a speaker recognition method provided by an embodiment of the present application is shown;

[0049] Figure 2 A schematic diagram of a first speech feature graph provided in an embodiment of the present application is shown;

[0050] Figure 3 A schematic diagram of a second speech feature graph provided in an embodiment of the present application is shown;

[0051] Figure 4 A flowchart of another speaker recognition method provided by an embodiment of the present application is shown;

[0052] Figure 5 A schematic diagram of the structure of a speaker recognition device provided in an embodiment of the present application is shown;

[0053] Figure 6A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0055] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0056] In order to enable those skilled in the art to use the contents of this application, the following implementation is provided in conjunction with a specific application scenario, "the field of speech recognition technology". For those skilled in the art, the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "field of speech recognition technology", it should be understood that this is only an exemplary embodiment.

[0057] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0058] A speaker recognition method provided in an embodiment of the present application is described in detail below.

[0059] Reference Figure 1 As shown, it is a flowchart of a speaker recognition method provided in an embodiment of the present application. The exemplary steps of the embodiment of the present application are described below:

[0060] S101: Acquire a target speech signal of a speaker to be identified.

[0061] In an embodiment of the present application, an initial speech signal of a speaker to be identified is obtained from a speech acquisition device (such as a microphone); background noise of the initial speech signal is removed to obtain an intermediate speech signal; the intermediate speech signal is normalized to adjust the amplitude of the intermediate speech signal to [-1, 1] to obtain a target speech signal.

[0062] Here, a low-order Butterworth filter is used to remove the background noise of the initial speech signal, which can retain the key frequency components of the initial speech signal. Adjusting the amplitude of the intermediate speech signal to [-1, 1] can eliminate the impact of amplitude differences on speaker recognition and improve the stability of the subsequent feature extraction of the target speech signal.

[0063] S102: Extract speech features based on the complex amplitude of the target speech signal on each audio component.

[0064] In the implementation manner of the present application, each audio component group includes two audio components, and the speech feature is a two-dimensional matrix including the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified; the eigenvalue of each feature point in the speech feature is used to characterize the nonlinear phase coupling strength between the audio components in an audio component group. The nonlinear phase coupling strength between the audio components in each audio component group can be directly represented by any high-order nonlinear eigenvalue such as the bispectral amplitude between the audio components. The larger the bispectral amplitude, the higher the nonlinear phase coupling strength.

[0065] For example, the target speech signal includes audio component 1, audio component 2, and audio component 3, and an audio component group includes two audio components, then audio component 1 and audio component 2 are one audio component group, audio component 1 and audio component 3 form one audio component group, and audio component 2 and audio component 3 form one audio component group. Therefore, the speech features include the nonlinear phase coupling strength between audio component 1 and audio component 2, the nonlinear phase coupling strength between audio component 1 and audio component 3, and the nonlinear phase coupling strength between audio component 2 and audio component 3 of the speaker to be identified.

[0066] Here, the complex amplitude on each audio component represents the signal strength of each audio component (i.e., the amplitude of each frequency component); the larger the complex amplitude, the more energy the audio component occupies in the target speech signal, and the more important the audio component is. The complex amplitude of the target speech signal on all audio components characterizes the timbre, pitch and other characteristics of the speaker to be identified, and improves the accuracy of speech feature extraction. The nonlinear phase coupling strength between audio components reflects the closeness of the nonlinear phase relationship between audio components; the nonlinear phase coupling strength between the audio components of the speaker to be identified in all audio component groups characterizes the vocal tract shape, pronunciation habits and other characteristics of the speaker to be identified; the nonlinear phase coupling strength between the audio components of the speaker to be identified in all audio component groups can provide more representative feature information of the speaker to be identified, thereby improving the accuracy of speaker recognition.

[0067] Specifically, the complex amplitude of the target speech signal on each audio component is substituted into the following formula to obtain the speech feature:

[0068] ;

[0069] in, is the i-th audio component in the speech feature With the jth audio component The nonlinear phase coupling strength between is the target speech signal in the i-th audio component The complex magnitude (i.e., Fourier coefficient) of is the jth audio component of the target speech signal The complex magnitude (i.e., Fourier coefficient) of The target speech signal is in the audio component + The complex conjugate of the complex magnitude on , For all time windows The expected value of (used to eliminate the influence of random noise on the calculation of nonlinear phase coupling strength).

[0070] S103: Convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map.

[0071] In an embodiment of the present application, the target speech feature graph is used to characterize the distribution area of ​​the audio component groups corresponding to each nonlinear phase coupling strength of the speaker to be identified; the target speech feature graph includes a first target speech feature graph and / or a second target speech feature graph. The target speech feature graph is used to characterize the distribution area of ​​the audio component groups corresponding to each nonlinear phase coupling strength of the speaker to be identified. The target speech feature graph distinguishes each nonlinear phase coupling strength based on the depth and type of color.

[0072] Specifically, the feature value of each feature point in the speech feature is converted into a three-channel color value to obtain a first target speech feature map, including:

[0073] Step 1: Normalize the feature value of each feature point in the speech feature to a preset color value range (i.e., 0 to 255) using the following formula:

[0074] ;

[0075] in, is the normalized feature value of the i-th feature point in the speech feature, For voice features The eigenvalue of the i-th feature point before normalization, For voice features The largest eigenvalue in For voice features The smallest eigenvalue in .

[0076] Here, the larger the normalized eigenvalue of the feature point is, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the feature point is. Therefore, through this step, the color type and color depth of the feature point in the first target speech feature map can be more intuitively represented by the corresponding eigenvalue, which facilitates the subsequent mapping of the eigenvalue of the feature point to the color value and reduces the complexity of subsequent mapping.

[0077] Step 2: Map the normalized feature values ​​of each feature point in the speech feature to color values ​​of three channels to obtain a first speech feature map.

[0078] In an embodiment of the present application, for each feature point in the speech feature, if the normalized feature value of the feature point in the speech feature is the maximum value in the color value range (i.e., 255), it means that the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the feature point is the highest, then the normalized feature value of the feature point is determined as the color value of the first color channel (such as the red channel) corresponding to the feature point; the value obtained by subtracting the normalized feature value of the feature point from the maximum value in the color value range is determined as the color value of the second color channel (such as the blue channel) corresponding to the feature point; the color value of the third color channel (such as the green channel) corresponding to the feature point is determined to be 0.

[0079] Take the red channel as the first color channel, the blue channel as the second color channel, and the green channel as the third color channel as an example. Figure 2As shown, it is a schematic diagram of the first speech feature map provided in an embodiment of the present application; the redder the color of the pixel, the higher the nonlinear phase coupling strength between the audio components in the audio component group corresponding to the pixel; the bluer the color of the pixel, the lower the nonlinear phase coupling strength between the audio components in the audio component group corresponding to the pixel; each depth of each color represents a nonlinear phase coupling strength, and the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength can be intuitively reflected from the figure; and the figure can also intuitively reflect that the nonlinear phase coupling strength of the audio component group distributed in the lower left area is higher than the nonlinear phase coupling strength of the audio component groups in other areas.

[0080] Here, through the above steps, the speech features can be visualized through a heat map (first speech feature map). In the first speech feature map, the larger the color value of the first color channel, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point; the larger the color value of the second color channel, the lower the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point.

[0081] Specifically, the second target speech feature graph is obtained according to the following steps:

[0082] Step 1: Group the feature points in the speech features.

[0083] In an implementation manner of the present application, the feature value of each feature point in the speech feature is normalized to a preset color value range (i.e., 0 to 255) by the aforementioned formula; the feature points in the speech feature are divided into corresponding groups according to the normalized feature values; the feature value range corresponding to each group is obtained by evenly dividing the preset color value range according to the preset number of groups.

[0084] For example, if the number of preset groups is 2, the eigenvalue range of the first group is 0 to 127.5 (including 127.5). The eigenvalue range of the second group is 127.5 (excluding 127.5) to 255 (including 255). If the eigenvalue of feature point A is 25, feature point A is divided into the first group.

[0085] Step 2: convert the feature values ​​of the feature points in the same group of speech features into the same three-channel color values, and connect the feature points with the same color values ​​to obtain a second target speech feature map.

[0086] In the embodiment of the present application, the color value of the first color channel corresponding to the feature point in the i+1th group is greater than the color value of the first color channel (such as the red channel) corresponding to the feature point in the i-th group, and the color value of the second color channel (such as the blue channel) corresponding to the feature point in the i+1th group is less than the color value of the second color channel corresponding to the feature point in the i-th group; the color values ​​of the third color channels corresponding to all feature points are 0. The numerical value in the characteristic value range corresponding to the i+1th group is greater than the numerical value in the characteristic value range corresponding to the i-th group, and i is greater than 0, and less than or equal to the preset number of groups - 1.

[0087] Take the red channel as the first color channel, the blue channel as the second color channel, and the green channel as the third color channel as an example. Figure 3 As shown, it is a schematic diagram of the second speech feature map provided in an embodiment of the present application; the redder the color of the feature point, the higher the nonlinear phase coupling strength between the audio components in the audio component group corresponding to the pixel point; the bluer the color of the pixel point, the lower the nonlinear phase coupling strength between the audio components in the audio component group corresponding to the pixel point; the nonlinear phase coupling strength of the feature points in the same group is represented by a color, and the distribution area of ​​the audio component groups corresponding to each nonlinear phase coupling strength can be intuitively reflected from the figure; and the figure can also intuitively reflect that the nonlinear phase coupling strength of the audio component groups distributed in the lower left area is higher than the nonlinear phase coupling strength of the audio component groups in other areas, and the distribution area of ​​the audio component groups with similar nonlinear phase coupling strengths.

[0088] Here, through the above steps, the speech features can be visualized through a contour map (second speech feature map). In the second speech feature map, the larger the color value of the first color channel, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point; the larger the color value of the second color channel, the lower the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point.

[0089] S104: Determine a target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in the pre-established recognition library.

[0090] In the implementation mode of the present application, the similarity includes the average nonlinear phase coupling strength between the target speech feature map and the preset speech feature map, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio. The average nonlinear phase coupling strength is obtained by the average value of all pixel values ​​in the feature map (target speech feature map or preset speech feature map) (i.e., the brightness of the feature map); the local fluctuation amplitude of the nonlinear phase coupling strength is obtained by the variance of all pixel values ​​(i.e., the contrast information of the feature map), which is used to characterize the degree of change of the nonlinear phase coupling strength of the feature map in the local area; the similarity of the nonlinear phase coupling mode refers to the complex interaction between the speech signal of the speaker to be identified and the speech signal of the preset speaker, such as nonlinear phenomena such as frequency doubling and combination frequency, which is obtained by the covariance of all pixel values ​​between the two feature maps (i.e., the structural information of the feature map). The nonlinear relationship of the audio is represented by a one-dimensional vector of the feature map.

[0091] Among them, the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength, and the nonlinear phase coupling mode are used to calculate the local similarity between the target speech feature map and the preset speech feature map, and the nonlinear relationship of the audio is used to calculate the global similarity between the target speech feature map and the preset speech feature map.

[0092] Specifically, the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker is calculated by the following steps, including:

[0093] Step 1: Calculate the local similarity based on the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling mode in the target speech feature map and the preset speech feature map.

[0094] Here, the local similarity is calculated by the following formula;

[0095] ;

[0096] in, is the local similarity, is the target speech feature map The average nonlinear phase coupling strength (target speech feature map The average value of all pixel values Preset voice feature map The average nonlinear phase coupling strength (preset speech feature map The average value of all pixel values is the target speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength (i.e., the target speech feature map The variance of all pixel values Preset voice feature map The local fluctuation amplitude of the nonlinear phase coupling strength (i.e., the preset speech feature map The variance of all pixel values is the target speech feature map Nonlinear phase coupling mode and preset speech feature map The similarity between the nonlinear phase coupling patterns of the target speech and preset voice feature graphs The covariance of all pixel values ​​between and is a constant (used to avoid denominator being 0).

[0097] Here, in addition to the above-mentioned local similarity calculation formula, the local similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker can also be calculated by using MSE (mean square error) or NCC (normalized cross correlation coefficient).

[0098] Step 2: Calculate the global similarity based on the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph.

[0099] In the implementation manner of the present application, the global similarity is calculated by the following formula:

[0100] ;

[0101] in, is the local similarity between the target speech feature map and the preset speech feature map, is the nonlinear relationship of the audio of the target speech feature map (i.e., the one-dimensional vector after the target speech feature map is flattened), is the nonlinear relationship of the audio of the preset speech feature graph (i.e., the one-dimensional vector after the preset speech feature graph is flattened), for The model, for Model.

[0102] Here, in addition to the above-mentioned local similarity calculation formula, the global similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker can also be calculated by using methods such as KL divergence.

[0103] Step 3: Perform weighted summation of the global similarity and the local similarity to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker.

[0104] In addition, in addition to the above-mentioned calculation of the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker, calculations can also be performed based on deep learning methods, such as using convolutional neural networks to extract high-dimensional semantic information of bispectral features, and completing similarity evaluation through feature space distance (such as Euclidean distance).

[0105] Here, if the target speech feature graph includes a first target speech feature graph and a second target speech feature graph, the similarity between the first target speech feature graph of the speaker to be identified and the first preset speech feature graph corresponding to each preset speaker in the pre-established recognition library, and the similarity between the second target speech feature graph of the speaker to be identified and the second preset speech feature graph corresponding to each preset speaker in the pre-established recognition library are weightedly summed to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker in the pre-established recognition library.

[0106] Among them, the calculation process of the similarity between the first target speech feature graph of the speaker to be identified and the first preset speech feature graph corresponding to each preset speaker in the pre-established recognition library, and the similarity between the second target speech feature graph of the speaker to be identified and the second preset speech feature graph corresponding to each preset speaker in the pre-established recognition library, is the same as the aforementioned calculation of the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker in the pre-established recognition library.

[0107] Furthermore, the method can be tested using real speaker data to evaluate its accuracy, recall, F1 score and other indicators.

[0108] The embodiment of the present application provides a speaker recognition method, which includes: extracting speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified; converting the feature value of each feature point in the speech features into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified; based on the similarity between the target speech feature map of the speaker to be identified and the preset speech feature map corresponding to each preset speaker in the pre-established recognition library, determining the target speaker corresponding to the target speech signal; the similarity includes the average nonlinear phase coupling strength between the target speech feature map and the preset speech feature map, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio. The present application improves the interpretability of the recognition basis and intuitively displays the feature differences between different speakers.

[0109] Reference Figure 4 As shown, it is a flowchart of another speaker recognition method provided by an embodiment of the present application. The exemplary steps of the embodiment of the present application are described below:

[0110] S401: Acquire a target speech signal of a speaker to be identified.

[0111] S402: Extract speech features based on the complex amplitude of the target speech signal on each audio component.

[0112] S403: Convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map.

[0113] S404: Determine the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in the pre-established recognition library.

[0114] S405: Calculate the similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library.

[0115] In the implementation mode of the present application, the target speech signal is input into the ASR module, the speech is converted into text, and the text information corresponding to the target speech signal is output. The similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library can be cosine similarity, Jaccard similarity, etc.

[0116] S406. Perform a weighted summation of the similarities between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in the pre-established recognition library and the corresponding first weights, and the similarities between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library and the corresponding second weights, to obtain a comprehensive score for each preset speaker.

[0117] In the implementation manner of the present application, the numerical value corresponding to the first weight is much greater than the numerical value corresponding to the second weight.

[0118] S407: Determine a target speaker corresponding to the target speech signal based on the comprehensive score of each preset speaker.

[0119] In the implementation manner of the present application, the higher the comprehensive score, the greater the possibility that the preset speaker is the target speaker.

[0120] The recognition library stores preset speakers, preset text information and preset voice feature graphs correspondingly; the preset voice feature graph is determined based on a preset voice signal of a preset speaker for preset text information.

[0121] Here, the present application combines the similarity of text information and feature graphs to improve the accuracy of speaker recognition.

[0122] Based on the same inventive concept, a speaker recognition device corresponding to the speaker recognition method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned speaker recognition method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0123] Reference Figure 5 FIG. 1 is a schematic diagram of a speaker recognition device provided in an embodiment of the present application, wherein the device comprises:

[0124] An acquisition module 501 is used to acquire a target speech signal of a speaker to be identified;

[0125] An extraction module 502 is used to extract speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified;

[0126] The conversion module 503 is used to convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified;

[0127] The determination module 504 is used to determine the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in the pre-established recognition library; the similarity includes the average nonlinear phase coupling strength between the target speech feature graph and the preset speech feature graph, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio.

[0128] In a possible implementation, the target speech feature graph includes a first target speech feature graph and / or a second target speech feature graph; the conversion module 503 is specifically configured to:

[0129] The first target speech feature graph is obtained according to the following steps:

[0130] Converting the feature value of each feature point in the speech feature into a three-channel color value to obtain a first target speech feature map;

[0131] The second target speech feature graph is obtained according to the following steps:

[0132] The feature points in the speech features are grouped; the feature values ​​of the feature points in the same group in the speech features are converted into the same three-channel color values, and the feature points with the same color values ​​are connected to obtain a second target speech feature graph.

[0133] In a possible implementation, the determination module 504 is specifically used to calculate the local similarity based on the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling pattern in the target speech feature graph and the preset speech feature graph; calculate the global similarity based on the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph; and perform weighted summation of the global similarity and the local similarity to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker.

[0134] In a possible implementation, the determination module 504 is specifically configured to calculate the local similarity using the following formula:

[0135] ;

[0136] in, is the local similarity, is the target speech feature map The average nonlinear phase coupling strength of Preset voice feature map The average nonlinear phase coupling strength of is the target speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, The preset speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, is the target speech feature map The nonlinear phase coupling mode and the preset speech characteristic diagram The similarity between the nonlinear phase coupling modes is and is a constant.

[0137] In a possible implementation, the determination module 504 is specifically configured to calculate the global similarity using the following formula:

[0138] ;

[0139] in, is the local similarity between the target speech feature graph and the preset speech feature graph, is the nonlinear relationship of the audio of the target speech feature map, The nonlinear relationship of the audio for the preset speech feature graph.

[0140] In a possible implementation, the determination module 504 is specifically used to calculate the similarity between the text information corresponding to the target voice signal and the preset text information corresponding to each preset speaker in the recognition library; perform weighted summation on the similarity between the target voice feature graph of the speaker to be identified and the preset voice feature graph corresponding to each preset speaker in the pre-established recognition library and the corresponding first weight, and the similarity between the text information corresponding to the target voice signal and the preset text information corresponding to each preset speaker in the recognition library and the corresponding second weight to obtain a comprehensive score for each preset speaker; determine the target speaker corresponding to the target voice signal based on the comprehensive score of each preset speaker; wherein the recognition library stores preset speakers, preset text information and preset voice feature graphs; the preset voice feature graph is determined based on the preset voice signal of the preset speaker for the preset text information; and the numerical value corresponding to the first weight is much larger than the numerical value corresponding to the second weight.

[0141] The embodiment of the present application provides a speaker recognition device, which includes: an acquisition module 501, which is used to acquire a target speech signal of a speaker to be recognized; an extraction module 502, which is used to extract speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be recognized; a conversion module 503, which is used to convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be recognized; a determination module 504, which is used to determine the target speaker corresponding to the target speech signal based on the similarity between the target speech feature map of the speaker to be recognized and the preset speech feature map corresponding to each preset speaker in the pre-established recognition library; the similarity includes the average nonlinear phase coupling strength between the target speech feature map and the preset speech feature map, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode and the similarity of the nonlinear relationship of the audio. The present application improves the interpretability of the recognition basis and intuitively displays the feature differences between different speakers.

[0142] like Figure 6 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-mentioned speaker recognition method.

[0143] Specifically, the memory 602 and the processor 601 can be general-purpose memories and processors, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the speaker recognition method can be executed.

[0144] Corresponding to the above-mentioned speaker recognition method, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned speaker recognition method are executed.

[0145] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0146] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0148] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the information processing method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0149] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A speaker recognition method, characterized in that: The method comprises: Obtaining a target speech signal of a speaker to be identified; Extracting speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified; Converting the characteristic value of each characteristic point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified; Based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library, the target speaker corresponding to the target speech signal is determined; the similarity includes the average nonlinear phase coupling strength between the target speech feature graph and the preset speech feature graph, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio.

2. The speaker recognition method according to claim 1, characterized in that: The target speech feature graph includes a first target speech feature graph and / or a second target speech feature graph; The first target speech feature graph is obtained according to the following steps: Converting the feature value of each feature point in the speech feature into a three-channel color value to obtain a first target speech feature map; The second target speech feature graph is obtained according to the following steps: The feature points in the speech features are grouped; the feature values ​​of the feature points in the same group in the speech features are converted into the same three-channel color values, and the feature points with the same color values ​​are connected to obtain a second target speech feature graph.

3. The speaker recognition method according to claim 1, characterized in that: The similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker is calculated by the following steps, including: Calculating local similarity according to the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling mode in the target speech feature graph and the preset speech feature graph; Calculating global similarity based on the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph; The global similarity and the local similarity are weightedly summed to obtain the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker.

4. The speaker recognition method according to claim 3, characterized in that: The calculating of the local similarity according to the average nonlinear phase coupling strength, the local fluctuation amplitude of the nonlinear phase coupling strength and the nonlinear phase coupling mode in the target speech feature graph and the preset speech feature graph comprises: The local similarity is calculated by the following formula: ； in, is the local similarity, is the target speech feature map The average nonlinear phase coupling strength of Preset voice feature map The average nonlinear phase coupling strength of is the target speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, The preset speech feature map The local fluctuation amplitude of the nonlinear phase coupling strength, is the target speech feature map The nonlinear phase coupling mode and the preset speech characteristic diagram The similarity between the nonlinear phase coupling modes is and is a constant.

5. The speaker recognition method according to claim 3, characterized in that: The calculating of the global similarity according to the nonlinear relationship of the audio of the target speech feature graph and the nonlinear relationship of the audio of the preset speech feature graph comprises: The global similarity is calculated by the following formula: ； in, is the local similarity between the target speech feature graph and the preset speech feature graph, is the nonlinear relationship of the audio of the target speech feature map, The nonlinear relationship of the audio for the preset speech feature graph.

6. The speaker recognition method according to any one of claims 1 to 5, characterized in that: The determining the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in the pre-established recognition library comprises: Calculating the similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library; Performing weighted summation of the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graph corresponding to each preset speaker in the pre-established recognition library and the corresponding first weight, and the similarity between the text information corresponding to the target speech signal and the preset text information corresponding to each preset speaker in the recognition library and the corresponding second weight, to obtain a comprehensive score for each preset speaker; determining the target speaker corresponding to the target speech signal based on the comprehensive score of each preset speaker; Among them, the recognition library correspondingly stores preset speakers, preset text information and preset voice feature maps; the preset voice feature map is determined based on the preset voice signal of the preset speaker for the preset text information; the numerical value corresponding to the first weight is much larger than the numerical value corresponding to the second weight.

7. A speaker recognition device, characterized in that: The device comprises: An acquisition module, used to acquire a target speech signal of a speaker to be recognized; An extraction module, configured to extract speech features based on the complex amplitude of the target speech signal on each audio component; the speech features include the nonlinear phase coupling strength between the audio components in each audio component group of the speaker to be identified; A conversion module, used to convert the feature value of each feature point in the speech feature into a color value to obtain a target speech feature map; the target speech feature map is used to characterize the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling strength of the speaker to be identified; A determination module is used to determine the target speaker corresponding to the target speech signal based on the similarity between the target speech feature graph of the speaker to be identified and the preset speech feature graphs corresponding to each preset speaker in a pre-established recognition library; the similarity includes the average nonlinear phase coupling strength between the target speech feature graph and the preset speech feature graph, the local fluctuation amplitude of the nonlinear phase coupling strength, the nonlinear phase coupling mode, and the similarity of the nonlinear relationship of the audio.

8. The speaker recognition device according to claim 7, characterized in that: The target speech feature graph includes a first target speech feature graph and / or a second target speech feature graph; a conversion module is specifically used to: The first target speech feature graph is obtained according to the following steps: Converting the feature value of each feature point in the speech feature into a three-channel color value to obtain a first target speech feature map; The second target speech feature graph is obtained according to the following steps: The feature points in the speech features are grouped; the feature values ​​of the feature points in the same group in the speech features are converted into the same three-channel color values, and the feature points with the same color values ​​are connected to obtain a second target speech feature graph.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the speaker recognition method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the speaker recognition method according to any one of claims 1 to 6 are executed.

Images