A voiceprint network recognition method, system and storage medium for CLAHE under small sample conditions

Abstract

The present invention relates to a voiceprint network recognition method, system, and storage medium using CLAHE (Clearance Heuristics) (CLAHE) for small sample sizes. The method comprises: U1. converting speech signals from the TIMIT dataset into spectrograms; U2. performing sample expansion on the spectrograms using the CLAHE algorithm; U3. cropping the expanded spectrograms and performing data dimensionality upgrade on the cropped images to obtain data information about the upgraded spectrograms; U4. inputting the upgraded spectrogram data information into an EfficientNet neural network model for training and learning, thereby obtaining a trained EfficientNet neural network. This invention not only compensates for the limited number of speaker samples and improves the quality of generated samples, but also achieves accurate speaker recognition by leveraging the powerful generalization capabilities of the EfficientNet network.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a voiceprint network recognition method, system and storage medium of CLAHE under small sample conditions. Background Art

[0002] Key technologies in voiceprint recognition are feature extraction and template matching. Because different speakers have different oral structures and pronunciation habits, their voiceprint features are unique. Therefore, different speakers can be distinguished by extracting their voiceprint features. Domestic and international scholars have conducted feature extraction based on the human ear's auditory perception of speech. To date, the main parameters used to characterize speech features include linear predictive coding coefficients (LPC), linear predictive cepstral coefficients (LPCC), and mel-spectrometric cepstral coefficients (MFCC). Representative figures in this field include Bogert, Tukey and Cooley, Hossan, Memon, and Gregory. Later developments in feature extraction have focused on improving MFCCs.

[0003] After feature extraction, technical means are needed to build a voiceprint model for the speaker. Remarkable models include the GMM model, which uses multiple Gaussian probability density functions to represent the model; the improved GMM-UBM model, which generalizes better; and the identity vector (i-vector) model. Researchers subsequently turned their attention to template matching, with representatives such as Reynold and Rose, Dehak and Kenny, and others. Their research focuses on the Gaussian mixture model (GMM) using multiple Gaussian probability density functions; the universal background model (UBM) with better generalization; joint factor analysis (JFA) for extracting speech information; and identity vectors (i-vectors). Furthermore, with the rapid development and application of deep learning in speech signal processing and image processing, neural network-based algorithms have been increasingly applied to voiceprint recognition in recent years, achieving promising results. When the target person's speech is small and sample data is insufficient, traditional voiceprint recognition models suffer from low prediction accuracy and weak generalization, which degrades algorithm performance and makes the resulting system incapable of recognition. Therefore, a new approach is needed to address the problem of low prediction accuracy in small sample sizes. To improve the model's prediction accuracy, we can start with the number of samples and combine it with the sample generation algorithm to quickly expand the number of samples, thereby increasing the amount of data required by the model and enhancing the model's generalization ability. The second angle is to improve the model's matching template. Using the EfficientNet neural network, which has achieved the best results in recent years, we can comprehensively adjust the image size, network width, and depth to improve the model's feature matching. This will increase the model's prediction accuracy and generalization ability, ultimately achieving rapid speaker recognition with small samples. Summary of the Invention

[0004] In view of the above problems, the present invention provides a voiceprint network recognition method, system and storage medium of CLAHE under small sample conditions, which can not only make up for the small number of speaker samples, but also improve the quality of generated samples, and realize accurate speaker recognition by combining the powerful generalization ability of the EfficientNet network.

[0005] In order to achieve the above-mentioned and other related purposes, the present invention provides the following technical solutions: a voiceprint network recognition method of CLAHE under small sample conditions, the method comprising:

[0006] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0007] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0008] U3. Crop the spectrogram after sample expansion and perform data dimensionality upgrade on the cropped image to obtain data information of the spectrogram after data dimensionality upgrade;

[0009] U4. Input the data information of the spectrogram after the data dimension is upgraded into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0010] U5. Based on the trained EfficientNet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint and obtain the data information of the recognized voiceprint.

[0011] Furthermore, in step U1, converting the speech signal in the TIMIT dataset into a spectrogram includes:

[0012] U11. Frame the speech signal x(t) to convert it into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length.

[0013] U12. Perform FFT on x(m,n) to obtain the frequency domain signal X(m,n);

[0014] U13. Create the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n);

[0015] U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

[0016] Furthermore, in step U2, the sample expansion using the CLAHE algorithm includes:

[0017] U21. Calculate the probability density function p(r k ), p(r k )=N k / N, k=0,1,2,3,...,L-1,

[0018] Among them, r k represents the kth gray level, N k Represents r k The total number of times it appears in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels;

[0019] U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S kNormalized to (0,255],

[0020]

[0021] Where, k = 0, 1, 2, 3, ..., L-1.

[0022] Furthermore, in step U3, cropping the spectrogram after sample expansion includes:

[0023] U31. Assume that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h. Given the original image pixel position (X, Y), according to the scaling ratio, where the horizontal axis width X / x = W / w and the vertical axis height Y / y = H / h, we can get the pixel value f(x, y) at the pixel point (x, y).

[0024] F(x,y)=f(W / w*x,H / h*y);

[0025] U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used for rounding. The improved mathematical model is expressed as:

[0026] F(x,y)=f([W / w*x],[H / h*y]).

[0027] Furthermore, in step U3, the cropped image is upgraded to

[0055] : Since the EfficientNet network requires input of three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image, a dimensionality upgrade operation is required at this time so that the three-channel data is the same as the first channel data, which is convenient for subsequent network training and data preprocessing is completed.

[0028] Furthermore, in step U4, inputting the data information of the upgraded spectrogram into the EfficientNet neural network model for training and learning includes:

[0029] U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed;

[0030] U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set;

[0031] U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to correct the parameters of the network to obtain the trained EfficientNet neural network.

[0032] Furthermore, the neuron activation function G of the trained efficientnet neural network is,

[0033]

[0034] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0035] Furthermore, the neuron activation function G of the trained efficientnet neural network is,

[0036]

[0037] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0038] Furthermore, the constraints of the neuron factors α, β and δ of the EfficientNet neural network are as follows:

[0039]

[0040] In order to achieve the above-mentioned objectives and other related objectives, the present invention also provides a voiceprint network recognition system for CLAHE under small sample conditions, comprising a computer device programmed or configured to execute any one of the steps of the voiceprint network recognition method for CLAHE under small sample conditions.

[0041] In order to achieve the above-mentioned objectives and other related objectives, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the voiceprint network recognition methods of CLAHE under small samples.

[0042] The present invention has the following positive effects:

[0043] 1. This paper, through modeling and analysis of neural networks, shows that when too little image data is input, there are problems such as model failure to converge and excessive loss. Specifically, the model loss value cannot be reduced and the distortion is too large, resulting in extremely low prediction accuracy. Therefore, when too little spectrogram data is input, the neural network cannot converge properly, and the input spectrogram data needs to be improved.

[0044] 2. The present invention directly expands the sample size of the spectrogram converted from speech signals using the CLAHE method, thereby increasing the sample size and improving image quality. Unlike traditional methods, the CLAHE method does not require complex speech signal processing, ensuring rapid sample expansion while significantly reducing computational resources. Furthermore, the CLAHE method uniformly utilizes interpolation to rapidly expand small samples into a larger number. Therefore, the CLAHE method effectively addresses the problem of insufficient input samples for neural networks.

[0045] 3. The present invention can automatically extract sample information from the hidden layer of the network by using the EfficientNet network. Compared with the disadvantage of the traditional template matching method that can only extract single data, this method can extract multiple sample information to optimize the convergence of the neural network to the model, which can not only increase the test accuracy of the model, but also reduce the computing resources required by the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of the method flow of the present invention;

[0047] Figure 2 Schematic diagram of the EfficientNet neural network training of the present invention;

[0048] Figure 3 Schematic diagram of the spectrogram of the present invention (1);

[0049] Figure 4 Schematic diagram of the spectrogram of the present invention (II);

[0050] Figure 5 A schematic diagram of the accuracy of the model training of the present invention;

[0051] Figure 6 A schematic diagram of the loss value of the model training of the present invention;

[0052] Figure 7 Schematic diagram of the prediction accuracy of the model of the present invention. DETAILED DESCRIPTION

[0053] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0054] Example 1: Figure 1 or Figure 3 or Figure 4As shown, a voiceprint network recognition method of CLAHE under small sample conditions, the method includes:

[0055] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0056] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0057] U3. Crop the spectrogram after sample expansion and perform data dimensionality upgrade on the cropped image to obtain data information of the spectrogram after data dimensionality upgrade;

[0058] U4. Input the data information of the spectrogram after the data dimension is upgraded into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0059] U5. Based on the trained EfficientNet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint and obtain the data information of the recognized voiceprint.

[0060] In this embodiment, in step U1, converting the speech signal in the TIMIT dataset into a spectrogram includes:

[0061] U11. Frame the speech signal x(t) to convert it into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length.

[0062] U12. Perform FFT on x(m,n) to obtain the frequency domain signal X(m,n);

[0063] U13. Create the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n);

[0064] U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

[0065] In this embodiment, in step U2, the sample expansion using the CLAHE algorithm includes:

[0066] U21. Calculate the probability density function p(r k ), p(r k )=N k / N, k=0,1,2,3,...,L-1,

[0067] Among them, r krepresents the kth gray level, N k Represents r k The total number of times it appears in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels;

[0068] U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S k Normalized to (0,255],

[0069]

[0070] Where, k = 0, 1, 2, 3, ..., L-1.

[0071] In this embodiment, in step U3, cropping the spectrogram after sample expansion includes:

[0072] U31. Assume that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h. Given the original image pixel position (X, Y), according to the scaling ratio, where the horizontal axis width X / x = W / w and the vertical axis height Y / y = H / h, we can get the pixel value f(x, y) at the pixel point (x, y).

[0073] F(x,y)=f(W / w*x,H / h*y);

[0074] U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used for rounding. The improved mathematical model is expressed as:

[0075] F(x,y)=f([W / w*x],[H / h*y]).

[0076] In this embodiment, in step U3, the cropped image is upgraded to

[0055] because the EfficientNet network requires input of three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image. At this time, a dimensionality upgrade operation is required to make the data of the three channels the same as the first channel data, which is convenient for the subsequent network training and the data preprocessing is completed.

[0077] In this embodiment, if Figure 2 As shown, in step U4, the data information of the spectrogram after the data dimension is upgraded is input into the EfficientNet neural network model for training and learning, including:

[0078] U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed;

[0079] U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set;

[0080] U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to correct the parameters of the network to obtain the trained EfficientNet neural network.

[0081] In this embodiment, the neuron activation function G of the trained efficientnet neural network is,

[0082]

[0083] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0084] In this embodiment, the neuron activation function G of the trained efficientnet neural network is,

[0085]

[0086] Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

[0087] In this embodiment, the constraints of the neuron factors α, β and δ of the EfficientNet neural network are:

[0088]

[0089] In this embodiment, the present invention provides a voiceprint network recognition system for CLAHE with a small sample size, comprising a computer device programmed or configured to execute any one of the steps of the voiceprint network recognition method for CLAHE with a small sample size.

[0090] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to execute any one of the voiceprint network recognition methods of CLAHE under small sample conditions.

[0091] Any reference to memory, storage, database or other media used in the embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0092] Example 2: Based on the CLAHE voiceprint network recognition method under a small sample in Example 1, the present invention is further illustrated and described below.

[0093] like Figure 1 or Figure 3 or Figure 4 As shown, a voiceprint network recognition method of CLAHE under small sample conditions, the method includes:

[0094] U1. Convert the speech signal in the TIMIT dataset into a spectrogram;

[0095] U2. Use CLAHE algorithm to expand the sample of the spectrogram;

[0096] U3. Crop the spectrogram after sample expansion and perform data dimensionality upgrade on the cropped image to obtain data information of the spectrogram after data dimensionality upgrade;

[0097] U4. Input the data information of the spectrogram after the data dimension is upgraded into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network;

[0098] U5. Based on the trained EfficientNet neural network, the data information of the spectrogram after data dimensionality upgrade is input to recognize the voiceprint and obtain the data information of the recognized voiceprint.

[0099] First, speech signals corresponding to 10 speakers were extracted from the TIMIT dataset, with each speaker having only 10 segments. These were converted into spectrograms using a short-time Fourier transform (SFT). After obtaining the spectrograms, all speech signals were deleted. The resulting spectrograms were augmented using the contrast-controlled CLAHE method, with the ClipLimit limit set from 0.01 to 0.1 in steps of 0.01. Each image was augmented 10-fold, resulting in 1,000 spectrograms. After augmentation, the original color spectrograms were deleted to prevent interference from the original color image. Since the image input to the EfficientNet network is 224x224 pixels, it was necessary to crop the images. All 1,000 spectrograms were batch processed and cropped using the nearest neighbor algorithm based on the nearest pixel value. Since neural network training requires three-channel data, the cropped spectrograms are single-channel data, requiring dimensionality increase. Copy the channel data of 1000 spectrograms with a pixel value of 224*224 so that the latter two channel data are the same as the first channel data, and finally complete the entire data preprocessing part.

[0100] Combined with attachment Figure 2 , the processed data is randomly extracted and divided into training set and test set according to the ratio of 8:2. Then the training set images are divided into training set and validation set according to the ratio of 9:1 in EfficientNet network training, and put into the neural network for model training. The training effect of the model is as follows Figure 6 and Figure 7 After the network training is completed, the previously divided test set images are imported into the trained network for model prediction, thereby completing all the basic operations of the present invention.

[0101] like Figure 5 As shown in the figure, it can be seen that when the model training reaches 6 epochs, the effect is close to 1. Both the model training accuracy and the accuracy of the model validation set quickly reach stability, which can achieve good prediction. Figure 5 It can be seen that as network training progresses, the model loss converges quickly and decreases rapidly. When the number of iterations is set to 30, the accuracy improves and the model loss decreases at a relatively fast rate. As the number of iterations increases, the rate of loss decreases tends to be stable, with less fluctuation.

[0102] In summary, the present invention can not only make up for the small number of speaker samples, but also improve the quality of generated samples, and achieve accurate speaker recognition by combining the powerful generalization ability of the EfficientNet network.

[0103] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A CLAHE voiceprint network recognition method under small sample conditions, characterized by: The method comprises: U1. Convert the speech signal in the TIMIT dataset into a spectrogram; U2. Use CLAHE algorithm to expand the sample of the spectrogram; U3. Crop the spectrogram after sample expansion and perform data dimensionality upgrade on the cropped image to obtain data information of the spectrogram after data dimensionality upgrade; U4. Input the data information of the spectrogram after the data dimension is upgraded into the EfficientNet neural network model for training and learning to obtain a trained EfficientNet neural network; U5. Based on the trained EfficientNet neural network, input the data information of the spectrogram after the data dimension is upgraded, recognize the voiceprint, and obtain the data information of the recognized voiceprint; In step U4, the step of inputting the data information of the upgraded spectrogram into the EfficientNet neural network model for training and learning includes: U41. Based on the data information of the spectrogram after the data dimension is upgraded, a data set of the spectrogram after the data dimension is upgraded is constructed; U42. Divide the data set of the spectrogram after the data dimension upgrade into a training set and a test set in a ratio of 8:2, and divide the training set into a partial training set and a validation set; U43. Input part of the training set into the EfficientNet neural network for training and learning, and use the validation set to modify the network parameters to obtain the trained EfficientNet neural network; The neuron activation function G of the trained efficientnet neural network is, ， Wherein, r is the partial training set, α, β and δ are neuron factors of the EfficientNet neural network.

2. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 1 is characterized in that: In step U1, converting the speech signal in the TIMIT dataset into a spectrogram includes: U11. Frame the speech signal x(t) to convert it into x(m,n), where t represents the time parameter, m represents the number of frames, and n represents the frame length. U12. Perform FFT on x(m,n) to obtain the frequency domain signal X(m,n); U13. Create the function Y(m,n)=X(m,n)*X(m,n) T Draw the periodogram Y(m,n), where X(m,n) T represents the transpose of X(m,n); U14. Take 10*log 10 [Y(m,n)] draws a spectrogram.

3. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 1 is characterized in that: In step U2, the sample expansion using the CLAHE algorithm includes: U21. Calculate the probability density function p(r k ), p(r k )=N k / N,k=0,1,2,3,...,L-1, Among them, r k represents the kth gray level, N k Represents r k The total number of times it appears in the grayscale image, N represents the total number of pixels in the grayscale image, and L represents the total number of gray levels; U22. From the probability density function p(r k ) Calculate the cumulative distribution function S k , and S k Normalized to (0,255], ， Where k=0,1,2,3,...,L-1.

4. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 1 is characterized in that: In step U3, cropping the spectrogram after sample expansion includes: U31. Assume that the pixel size of the original spectrogram is W*H, and the pixel size of the scaled image is w*h. Given the original pixel position (X, Y), according to the scaling ratio, where the horizontal axis width X / x=W / w and the vertical axis height Y / y=H / h, the pixel value f(x, y) at the pixel point (x, y) is obtained. F(x,y)=f(W / w*x,H / h*y); U32. When the pixel value at the pixel point (x, y) is not an integer, the nearest neighbor interpolation algorithm is used for rounding. The improved mathematical model is expressed as: F(x,y)=f([W / w*x],[H / h*y]).

5. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 1 is characterized in that: In step U3, the cropped image is upgraded to a dimension. Since the EfficientNet network requires input of three-channel image data, and the spectrogram processed by CLAHE is a single-channel grayscale image, a dimension upgrade operation is required at this time to make the data of the three channels the same as the first channel data, which is convenient for the subsequent network training and the data preprocessing is completed.

6. The voiceprint network recognition method of CLAHE under small sample conditions according to claim 1 is characterized by: The constraints of the neuron factors α, β and δ of the EfficientNet neural network are: 。 7. A CLAHE voiceprint network recognition system under small sample conditions, including a computer device, characterized in that: The computer device is programmed or configured to execute the steps of the voiceprint network recognition method of CLAHE under small sample conditions as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program that is programmed or configured to execute the voiceprint network recognition method of CLAHE under small sample conditions as described in any one of claims 1 to 6.

Images