Voiceprint network recognition method and system of CLAHE under small sample, and storage medium

Abstract

The invention relates to a voiceprint network recognition method and system of CLAHE under a small sample, and a storage medium. The method comprises the following steps: U1, converting a voice signal in a TIMIT data set into a spectrogram; u2, performing sample expansion on the spectrogram by adopting a CLAHE algorithm; u3, performing picture clipping on the spectrogram after sample expansion, and performing data dimension raising on the clipped picture to obtain data information of the spectrogram after data dimension raising; and U4, inputting the data information of the spectrogram after the dimension raising of the data into an efficient net neural network model for training and learning, and obtaining a trained efficient net neural network. According to the method, not only can the defect that the number of speaker samples is too small be made up, but also the quality of the generated samples can be improved, and accurate recognition of the speaker is achieved by combining the powerful generalization ability of the efficientNet network.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a voiceprint network recognition method, system and storage medium of CLAHE under small sample conditions. Background Art

[0002] As the key technologies of voiceprint recognition, one is feature extraction and the other is template matching. Since different speakers have different oral cavity and pronunciation habits, the voiceprint features of each person are different. Therefore, different speakers can be distinguished by extracting their voiceprint features. Domestic and foreign scholars have extracted features based on the auditory characteristics of human ears to speech. So far, the parameters extracted to characterize speech features are mainly: linear prediction coding coefficient (LPC), linear prediction cepstral coefficient (LPCC), Mel spectrum cepstral coefficient (MFCC), etc. Its representatives include Bogert, Tukey and Cooly, Hossan, Memon and Gregory, etc. The later feature extraction mainly focuses on the improvement of MFCC.

[0003] After feature extraction, technical means are needed to establish a voiceprint model for the speaker. Among them, the more outstanding models include the GMM model that uses multiple Gaussian probability density functions to characterize the model; the GMM-UBM model that is improved on the basis of GMM to be a universal background model with better generalization; and the identity vector i-vector model. Later, researchers gradually turned their attention to template matching. The representatives include Reynold and Rose, Dehak and Kenny, etc. Their research mainly includes the Gaussian mixture model (GMM) that uses multiple Gaussian probability density functions, the universal background model (UBM) with better generalization ability, the joint factor analysis (JFA) that can be used to extract voice information, and the identity vector (i-vector). In addition, with the rapid development and application of deep learning in the fields of speech signal processing and image processing, in recent years, related algorithms based on neural networks have been gradually applied to the field of voiceprint recognition and have achieved good results. When the number of target person's voices is small and the sample data is insufficient, the traditional voiceprint recognition model has low prediction accuracy and weak generalization ability of the model, which reduces the performance of the algorithm and makes the formed system unable to recognize normally. Therefore, it is necessary to propose a new method to deal with the problem of low prediction accuracy of the model under small samples. To improve the prediction accuracy of the model, we can start from the number of samples and combine it with the sample generation algorithm to quickly expand the number of samples, thereby improving the amount of data required by the model and enhancing the generalization ability of the model. The second angle is to improve the matching template of the model. Using the EfficientNet neural network with the best results in recent years, we can comprehensively adjust the image size, network width and depth, thereby improving the feature matching of the model, increasing the prediction accuracy of the model and the generalization ability of the model, and ultimately achieving rapid recognition of speakers with small samples. Summary of the invention

[0004] In view of the above problems, the present invention provides a voiceprint network recognition method, system and storage medium of CLAHE under small samples, which can not only make up for the small number of speaker samples, but also improve the quality of generated samples, and realize accurate recognition of speakers by combining the powerful generalization ability of the efficientNet network.

[0005] In order to achieve the above-mentioned purpose and other related purposes, the technical solution provided by the present invention is as follows: a voiceprint network recognition method of CLAHE under small sample conditions, the method comprising:

[0006] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0007] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0008] U3. Crop the spectrogram after sample expansion, and perform data dimension upgrade on the cropped image to obtain data information of the spectrogram after data dimension upgrade;

[0009] U4. Input the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0010] U5. Based on the trained efficientnet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint, and the data information of the recognized voiceprint is obtained.

[0011] Further, in step U1, converting the speech signal in the TIMIT data set into a spectrogram includes:

[0012] U11. The speech signal x(t) is framed and converted into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length;

[0013] U12. Perform FFT transformation on x(m,n) to obtain the frequency domain signal X(m,n);

[0014] U13. Establish the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n);

[0015] U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

[0016] Furthermore, in step U2, the sample expansion using the CLAHE algorithm includes:

[0017] U21. Calculate the probability density function p(r) according to the gray value of the spectrogram k ), p(r k )=N k / N, k = 0, 1, 2, 3, ..., L-1,

[0018] Among them, r k represents the kth gray level, N k Represents r k The total number of occurrences in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels;

[0019] U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S kNormalized to (0,255],

[0020]

[0021] Among them, k = 0, 1, 2, 3, ..., L-1.

[0022] Furthermore, in step U3, cropping the spectrogram after sample expansion includes:

[0023] U31. Assuming that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h, when the original image pixel position (X, Y) is known, according to the scaling ratio, where the horizontal axis width X / x=W / w, the vertical axis height Y / y=H / h, the pixel value f(x, y) at the pixel point (x, y) is obtained.

[0024] F(x,y)=f(W / w*x,H / h*y);

[0025] U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used to round it off. The improved mathematical model is expressed as:

[0026] F(x,y)=f([W / w*x],[H / h*y]).

[0027] Furthermore, in step U3, the cropped image is upgraded to a dimension. Since the EfficientNet network needs to input three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image, a dimension upgrade operation is required at this time to make the data of the three channels the same as the first channel data, which is convenient for the subsequent network training and the data preprocessing is completed.

[0028] Furthermore, in step U4, the step of inputting the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning includes:

[0029] U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed;

[0030] U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set;

[0031] U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to correct the parameters of the network to obtain the trained EfficientNet neural network.

[0032] Furthermore, the neuron activation function G of the trained efficientnet neural network is,

[0033]

[0034] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0035] Furthermore, the neuron activation function G of the trained efficientnet neural network is,

[0036]

[0037] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0038] Furthermore, the constraints of the neuron factors α, β and δ of the EfficientNet neural network are:

[0039]

[0040] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a voiceprint network recognition system for CLAHE under small sample conditions, including a computer device, which is programmed or configured to execute the steps of any one of the voiceprint network recognition methods for CLAHE under small sample conditions.

[0041] In order to achieve the above-mentioned purpose and other related purposes, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the voiceprint network recognition methods of CLAHE under small samples.

[0042] The present invention has the following positive effects:

[0043] 1. The present invention shows through the modeling analysis of the neural network that when the input image data is too little, there are problems such as the model cannot converge and the loss value is too large. Specifically, the loss value of the model cannot be reduced and the distortion is too large, resulting in extremely low prediction accuracy. Therefore, when the input spectrogram data is too little, the neural network cannot converge normally, and the input spectrogram data needs to be improved.

[0044] 2. The present invention directly expands the sample of the spectrogram converted from the speech signal by using the CLAHE method, thereby improving the image quality while expanding the number of samples. The CLAHE method does not require complex speech signal processing like the traditional method, and can ensure that the number of samples is rapidly expanded while greatly reducing computing resources. At the same time, the CLAHE method uniformly uses interpolation means to quickly expand small samples into a large number of samples. Therefore, the CLAHE method solves the problem of too few input samples of the neural network.

[0045] 3. The present invention can automatically extract sample information in the hidden layer of the network by adopting the EfficientNet network. Compared with the disadvantage that the traditional template matching method can only extract single data, this method can extract multiple sample information to optimize the convergence of the neural network to the model, which can not only increase the test accuracy of the model, but also reduce the computing resources required by the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0047] Figure 2 A schematic diagram of the EfficientNet neural network training of the present invention;

[0048] Figure 3 Schematic diagram of the spectrogram of the present invention (I);

[0049] Figure 4 Schematic diagram of the spectrogram of the present invention (II);

[0050] Figure 5 A schematic diagram of the accuracy of model training of the present invention;

[0051] Figure 6 A schematic diagram of the loss value of the model training of the present invention;

[0052] Figure 7 It is a schematic diagram of the prediction accuracy of the model of the present invention. DETAILED DESCRIPTION

[0053] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0054] Example 1: Figure 1 or Figure 3 or Figure 4As shown, a voiceprint network recognition method of CLAHE under a small sample size is provided, and the method comprises:

[0055] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0056] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0057] U3. Crop the spectrogram after sample expansion, and perform data dimension upgrade on the cropped image to obtain data information of the spectrogram after data dimension upgrade;

[0058] U4. Input the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0059] U5. Based on the trained efficientnet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint, and the data information of the recognized voiceprint is obtained.

[0060] In this embodiment, in step U1, converting the speech signal in the TIMIT data set into a spectrogram includes:

[0061] U11. The speech signal x(t) is framed and converted into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length;

[0062] U12. Perform FFT transformation on x(m,n) to obtain the frequency domain signal X(m,n);

[0063] U13. Establish the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n);

[0064] U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

[0065] In this embodiment, in step U2, the sample expansion using the CLAHE algorithm includes:

[0066] U21. Calculate the probability density function p(r) according to the gray value of the spectrogram k ), p(r k )=N k / N, k = 0, 1, 2, 3, ..., L-1,

[0067] Among them, r krepresents the kth gray level, N k Represents r k The total number of occurrences in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels;

[0068] U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S k Normalized to (0,255],

[0069]

[0070] Among them, k = 0, 1, 2, 3, ..., L-1.

[0071] In this embodiment, in step U3, cropping the spectrogram after sample expansion includes:

[0072] U31. Assuming that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h, when the original image pixel position (X, Y) is known, according to the scaling ratio, where the horizontal axis width X / x=W / w, the vertical axis height Y / y=H / h, the pixel value f(x, y) at the pixel point (x, y) is obtained.

[0073] F(x,y)=f(W / w*x,H / h*y);

[0074] U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used to round it off. The improved mathematical model is expressed as:

[0075] F(x,y)=f([W / w*x],[H / h*y]).

[0076] In this embodiment, in step U3, the cropped image is upgraded to a dimension. Since the EfficientNet network needs to input three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image, a dimension upgrade operation is required at this time to make the data of the three channels the same as the first channel data, which is convenient for the subsequent network training and the data preprocessing is completed.

[0077] In this embodiment, if Figure 2 As shown, in step U4, the data information of the spectrogram after the data dimension is upgraded is input into the EfficientNet neural network model for training and learning, including:

[0078] U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed;

[0079] U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set;

[0080] U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to correct the parameters of the network to obtain the trained EfficientNet neural network.

[0081] In this embodiment, the neuron activation function G of the trained efficientnet neural network is,

[0082]

[0083] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0084] In this embodiment, the neuron activation function G of the trained efficientnet neural network is,

[0085]

[0086] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0087] In this embodiment, the constraints of the neuron factors α, β and δ of the EfficientNet neural network are:

[0088]

[0089] In this embodiment, the present invention provides a voiceprint network recognition system for CLAHE with a small sample, including a computer device, which is programmed or configured to execute the steps of any one of the voiceprint network recognition methods for CLAHE with a small sample.

[0090] In this embodiment, the present invention provides a computer-readable storage medium, on which is stored a computer program programmed or configured to execute any one of the voiceprint network recognition methods of CLAHE under small samples.

[0091] Any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0092] Example 2: Based on the CLAHE voiceprint network recognition method under a small sample in Example 1, the present invention is further illustrated and described below.

[0093] like Figure 1 or Figure 3 or Figure 4 As shown, a voiceprint network recognition method of CLAHE under a small sample size is provided, and the method comprises:

[0094] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0095] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0096] U3. Crop the spectrogram after sample expansion, and perform data dimension upgrade on the cropped image to obtain data information of the spectrogram after data dimension upgrade;

[0097] U4. Input the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0098] U5. Based on the trained efficientnet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint, and the data information of the recognized voiceprint is obtained.

[0099] First, the speech signals corresponding to 10 speakers are extracted from the TIMIT dataset, where each person's speech signal has only 10 segments. The short-time Fourier transform is used to convert it into a spectrogram. After obtaining the spectrogram, all speech signals are deleted. The obtained spectrogram is sampled by the CLAHE method with contrast control, where the ClipLimit limit is set from 0.01 to 0.1 with a step size of 0.01. Each image is expanded 10 times, and finally 1000 spectrograms are obtained. After the spectrogram is expanded, the original color spectrogram is deleted to avoid interference from the original color image. Since the image pixels of the image input in the EfficientNet network are 224*224, it is necessary to crop the image, batch process the 1000 spectrograms, and use the nearest neighbor algorithm to crop the image according to the nearest pixel value. The training of the neural network requires three-channel data, and the spectrogram after the cropping operation is single-channel data, so it needs to be processed by dimensionality increase. Copy the channel data of 1000 spectrograms with a pixel value of 224*224 so that the latter two channel data are the same as the first channel data, and finally complete the data preprocessing part.

[0100] Combined with Figure 2 , randomly extract the processed data in a ratio of 8:2 and divide it into training set and test set. Then divide the training set images into training set and validation set in a ratio of 9:1 in EfficientNet network training and put them into the neural network for model training. The training effect of the model is shown in the figure below. Figure 6 and Figure 7 After the network training is completed, the previously divided test set images are imported into the trained network for model prediction, thereby completing all the basic operations of the present invention.

[0101] like Figure 5 As shown in the figure, it can be seen that when the model training reaches 6 epochs, the effect is close to 1. Both the model training accuracy and the model verification set accuracy quickly reach stability, which can achieve good prediction. Figure 5 It can be seen that as the network training progresses, the model loss converges quickly and decreases rapidly. When the number of iterations is set to 30, the accuracy is improved and the loss of the model is reduced at a relatively fast rate. As the number of iterations increases, the rate of loss reduction tends to be stable and the fluctuation is small.

[0102] In summary, the present invention can not only make up for the small number of speaker samples, but also improve the quality of generated samples, and realize accurate speaker recognition by combining the powerful generalization ability of the efficientNet network.

[0103] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A voiceprint network recognition method of CLAHE under small sample conditions, characterized in that: The method comprises: U1. Convert the speech signal in the TIMIT dataset into a spectrogram; U2. Use CLAHE algorithm to expand the sample of the spectrogram; U3. Crop the spectrogram after sample expansion, and perform data dimension upgrade on the cropped image to obtain data information of the spectrogram after data dimension upgrade; U4. Input the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network; U5. Based on the trained efficientnet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint, and the data information of the recognized voiceprint is obtained.

2. The voiceprint network recognition method of CLAHE under small sample according to claim 1 is characterized in that: In step U1, converting the speech signal in the TIMIT data set into a spectrogram includes: U11. The speech signal x(t) is framed and converted into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length; U12. Perform FFT transformation on x(m,n) to obtain the frequency domain signal X(m,n); U13. Establish the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n); U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

3. The voiceprint network recognition method of CLAHE under small sample according to claim 1 is characterized in that: In step U2, the sample expansion using the CLAHE algorithm includes: U21. Calculate the probability density function p(r) according to the gray value of the spectrogram k ), p(r k )=N k / N, k = 0, 1, 2, 3, ..., L-1, Among them, r k represents the kth gray level, N k Represents r k The total number of occurrences in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels; U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S k Normalized to (0,255], Among them, k = 0, 1, 2, 3, ..., L-1.

4. The voiceprint network recognition method of CLAHE under small sample according to claim 1 is characterized in that: In step U3, cropping the spectrogram after sample expansion includes: U31. Assuming that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h, when the original image pixel position (X, Y) is known, according to the scaling ratio, where the horizontal axis width X / x=W / w, the vertical axis height Y / y=H / h, the pixel value f(x, y) at the pixel point (x, y) is obtained. F(x,y)=f(W / w*x,H / h*y); U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used to round it off. The improved mathematical model is expressed as: F(x,y)=f([W / w*x],[H / h*y]).

5. The voiceprint network recognition method of CLAHE under small sample according to claim 1 is characterized in that: In step U3, the cropped image is upgraded to a dimension. Since the EfficientNet network needs to input three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image, a dimension upgrade operation is required at this time to make the data of the three channels the same as the first channel data, which is convenient for the subsequent network training and the data preprocessing is completed.

6. The voiceprint network recognition method of CLAHE under small sample according to claim 1 is characterized in that: In step U4, the step of inputting the data information of the spectrogram after the data dimension upgrade into the EfficientNet neural network model for training and learning includes: U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed; U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set; U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to correct the parameters of the network to obtain the trained EfficientNet neural network.

7. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 6 is characterized by: The neuron activation function G of the trained efficientnet neural network is, Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

8. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 7 is characterized by: The constraints of the neuron factors α, β and δ of the EfficientNet neural network are:

9. A voiceprint network recognition system of CLAHE under small sample conditions, including a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the voiceprint network recognition method of CLAHE under small sample conditions as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program programmed or configured to execute the voiceprint network recognition method of CLAHE under small sample conditions as described in any one of claims 1 to 8.

Images