Voiceprint recognition method and system, storage medium, electronic equipment and vehicle

Abstract

The invention relates to a voiceprint recognition method and system, a storage medium, electronic equipment and a vehicle. The voiceprint recognition method comprises the following steps: in response to input of voice data, performing convolution of a plurality of convolution channels on the voice data by using a voiceprint recognition model, then fusing the voice data, and performing point-by-point convolution processing on fused voiceprint features to obtain voiceprint features of the voice data, and outputting a voiceprint recognition result based on the voiceprint features of the voice data. According to the embodiment of the invention, the voiceprint recognition is carried out on the voiceprint features obtained by carrying out convolution processing on the fused voiceprint features point by point after carrying out convolution processing on the voice data through the plurality of convolution channels, so that the calculation amount of the voiceprint recognition model is greatly reduced, and the voiceprint recognition model is conveniently deployed in a vehicle-mounted scene and other scenes.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a voiceprint recognition method, system, storage medium, electronic device and vehicle. Background Art

[0002] Voiceprint recognition is a new type of biometric technology that verifies the speaker's identity by recognizing the speaker's voice data. Currently, there are neural network models that can achieve high-precision voiceprint recognition, but the computational complexity of the neural network model is large and cannot be deployed in vehicle scenarios. Summary of the invention

[0003] The embodiments of the present application provide a voiceprint recognition method, system, storage medium, electronic device and vehicle. Voiceprint recognition is performed by convolving multiple convolution channels of voice data and fusing them. The fused voiceprint features are subjected to point-by-point convolution processing to obtain the fused voiceprint features. This greatly reduces the amount of calculation of the voiceprint recognition model, so that the voiceprint recognition model can be deployed in scenarios such as vehicle-mounted, so as to at least partially solve the above-mentioned technical problems.

[0004] To achieve the above-mentioned purpose, according to the first aspect of the present application, a voiceprint recognition method is provided, comprising: in response to the input of voice data, using a voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuse them, performing point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features of the voice data; and outputting a voiceprint recognition result based on the voiceprint features of the voice data.

[0005] Optionally, the voice data is subjected to multiple channel convolutions and then fused using the voiceprint recognition model, and the fused fused voiceprint features are subjected to point-by-point convolution processing to obtain the voiceprint features of the voice data, including: performing multiple channel convolutions on the voice data through a deep separable convolutional network in the voiceprint recognition model to obtain channel voiceprint features of multiple channels representing the voice data; concatenating and fusing the multiple channel voiceprint features to obtain a fused voiceprint feature; and performing point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features. The embodiment of the present application greatly reduces the computational complexity of the voiceprint recognition model by applying a deep separable convolutional network in the voiceprint recognition model, so that the voiceprint recognition model can be deployed in scenarios such as vehicle-mounted.

[0006] Optionally, the splicing and fusing of the multiple channel voiceprint features to obtain the fused voiceprint feature includes: fusing the multiple channel voiceprint features through the splicing and fusion network of the deep separable convolutional network to obtain the fused voiceprint feature; the performing point-by-point convolution processing on the fused voiceprint feature to obtain the voiceprint feature includes: performing point-by-point convolution processing on the fused voiceprint feature through the point-by-point convolutional network of the deep separable convolutional network to obtain the voiceprint feature. In the embodiment of the present application, since the deep separable convolutional network can extract features of different scales through channel-by-channel convolution and interact between features of different scales through point-by-point convolution, it reduces the amount of calculation while also ensuring that the voiceprint recognition model can achieve high-precision feature extraction, thereby improving the accuracy of the voiceprint recognition model.

[0007] Optionally, the voiceprint convolution processing is performed on the voice data through a deep separable convolutional network in a voiceprint recognition model to obtain voiceprint features; and the voiceprint recognition result is output based on the voiceprint features of the voice data, comprising: obtaining the voiceprint recognition features of the voice data according to the voiceprint features through the voiceprint recognition model; and voiceprint recognition is performed on the voice data according to the voiceprint recognition features to obtain a voiceprint recognition result.

[0008] Optionally, obtaining the voiceprint recognition feature of the speech data according to the voiceprint feature through the voiceprint recognition model includes: performing value restriction processing on the voiceprint feature through the target activation function in the voiceprint recognition model; and obtaining the voiceprint recognition feature according to the voiceprint feature after the value restriction processing through the voiceprint recognition model.

[0009] Optionally, the target activation function in the voiceprint recognition model is used to restrict the value of the voiceprint feature, including: nonlinearly processing the voiceprint feature through the target activation function in the voiceprint recognition model; and extreme value restriction of the voiceprint feature after the nonlinear processing through the target activation function. The activation function of the voiceprint recognition model in the embodiment of the present application can not only introduce nonlinear features, but also limit the maximum value of the output features to avoid excessive accuracy loss when the voiceprint recognition model is migrated to a mobile terminal such as a vehicle.

[0010] Optionally, performing voiceprint recognition on the voice data to be recognized based on the voiceprint recognition feature includes: comparing the voiceprint recognition feature with the historical voiceprint feature in a feature database; when there is a historical voiceprint feature matching the voiceprint recognition feature in the feature database, displaying a first prompt message and / or executing a corresponding voice command based on the input of the voice data; when there is no historical voiceprint feature matching the voiceprint recognition feature in the feature database, displaying a second prompt message; wherein the first prompt message is used to indicate that the voiceprint recognition is successful, and the second prompt message is used to indicate that the voiceprint recognition fails.

[0011] Optionally, the method further comprises: performing data preprocessing on the input original voice data to obtain the voice data to be recognized; wherein the data preprocessing comprises at least one of the following processes: voice interception processing, voice noise reduction processing, and voice enhancement processing.

[0012] Optionally, the method also includes: processing each historical voice data in the training data set through a target voiceprint recognition model to obtain sample voiceprint recognition features of each of the historical voice data; determining the recognition loss value of the target voiceprint recognition model using a target loss function in the target voiceprint recognition model according to the sample voiceprint recognition features and voiceprint annotation features of each of the historical voice data; adjusting the parameters of the target voiceprint recognition model so that the recognition loss value converges to obtain the voiceprint recognition model that has completed training.

[0013] Optionally, the target voiceprint recognition model is obtained by the following steps: adjusting the original convolutional network in the original voiceprint recognition model to a depthwise separable convolutional network; and / or, adjusting the original activation function in the original voiceprint recognition model to a target activation function, the target activation function being used to perform value restriction processing on the data; and / or, introducing a target loss function in the last layer of the network in the original voiceprint recognition model.

[0014] Optionally, the recognition loss value of the target voiceprint recognition model is determined based on the sample voiceprint recognition features and voiceprint annotation features of each of the historical voice data using the target loss function in the target voiceprint recognition model, including: determining the correctly identified target feature in the sample voiceprint recognition features based on the sample voiceprint recognition features and the voiceprint annotation features; adding an angle interval to the angle corresponding to the target feature to obtain an adjusted angle corresponding to the target feature; determining the recognition loss value based on the adjusted angle corresponding to the target feature and the angles corresponding to other features in the sample voiceprint recognition features. The embodiment of the present application calculates the loss function of the voiceprint recognition model by introducing a target loss function, which effectively maximizes the inter-class differences and minimizes the intra-class differences, thereby improving the accuracy of the voiceprint recognition model.

[0015] According to the second aspect of the present application, a voiceprint recognition device is provided, including: a voiceprint feature extraction module, which is used to respond to the input of voice data, use a voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuse them, and perform point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features of the voice data; a voiceprint recognition module, which is used to output a voiceprint recognition result based on the voiceprint features of the voice data.

[0016] According to a third aspect of the present application, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned voiceprint recognition method is implemented.

[0017] According to a fourth aspect of the present application, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the above-mentioned voiceprint recognition method.

[0018] According to the fifth aspect of the present application, a voiceprint recognition system is provided, comprising: a voiceprint collection module and an electronic device as described in the fourth aspect, the voiceprint collection module being communicatively connected to the electronic device; the voiceprint collection module being used to collect voice data and sending the collected voice data to the electronic device; the electronic device being used to perform voiceprint recognition processing on the received voice data and determine a voiceprint recognition result.

[0019] Optionally, the voiceprint recognition system further includes: a display module and a storage module, both of which are in communication with the electronic device; the storage module is used to store historical voiceprint feature information and user information; the display module is used to receive the voiceprint recognition result sent by the electronic device and display the received voiceprint recognition result. According to a sixth aspect of the present application, a vehicle is provided, comprising the voiceprint recognition system as described above, or comprising the electronic device as described above.

[0020] The benefit of the present application lies in that voiceprint recognition is performed by convolving multiple convolution channels of speech data and then fusing them, and then performing point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features, thereby greatly reducing the amount of calculation of the voiceprint recognition model.

[0021] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0023] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same figure numbers represent the same parts in the following description.

[0024] Figure 1 It is a flow chart of a voiceprint recognition method provided in an embodiment of the present application.

[0025] Figure 2 It is a schematic diagram of a voiceprint recognition model provided in an embodiment of the present application.

[0026] Figure 3 It is a schematic diagram of another voiceprint recognition model provided in an embodiment of the present application.

[0027] Figure 4 It is a schematic diagram of a deep separable convolutional network provided in an embodiment of the present application.

[0028] Figure 5 It is a schematic diagram of a channel-by-channel convolutional network provided in an embodiment of the present application.

[0029] Figure 6 It is a schematic diagram of a point-by-point convolutional network provided in an embodiment of the present application.

[0030] Figure 7 It is a schematic diagram of a model performance comparison result provided in an embodiment of the present application.

[0031] Figure 8 It is a block diagram of a voiceprint recognition system provided in an embodiment of the present application.

[0032] Fig. 9 It is a block diagram of a voiceprint recognition device provided in an embodiment of the present application.

[0033] Fig.10 It is a block diagram of another voiceprint recognition device provided in an embodiment of the present application.

[0034] Fig.11 It is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0036] According to the first aspect of the present application, an embodiment of the present application provides a voiceprint recognition method.

[0037] See also Figure 1 , Figure 1 This is a flow chart of a voiceprint recognition method provided by an embodiment of the present application. The voiceprint recognition method may include the following steps:

[0038] Step S110: In response to the input of voice data, the voice data is convolved by a plurality of convolution channels using a voiceprint recognition model and then fused, and the fused voiceprint features are convolved point by point to obtain the voiceprint features of the voice data;

[0039] Step S120: outputting a voiceprint recognition result based on the voiceprint features of the speech data;

[0040] The voiceprint feature is obtained by performing point-by-point convolution processing on the fused voiceprint feature after convolution of multiple convolution channels by the voiceprint recognition model.

[0041] The voiceprint recognition model in the embodiment of the present application can be constructed based on ECAPA-TDNN (Efficient CAPacityArchitecture for Speaker Verification with Time-Delay Neural Networks). Figure 2 As shown in Figure 1, it is the network structure diagram of ECAPA-TDNN. Figure 3 As shown, based on ECAPA-TDNN, the embodiment of the present application replaces the original conventional convolution, such as 3×3 convolution or 5×5 convolution, with a depthwise separable convolution network (DSC). In the depthwise convolution of the depthwise separable convolution network, each input channel independently applies a D k ×D kIn point-by-point convolution, a 1×1 convolution kernel is used to merge the output channels of depth-wise convolution. This decomposition reduces the complexity of the convolution operation because each convolution kernel only focuses on one channel instead of the combination of all channels, making the computation of depth-wise separable convolution less than that of conventional convolution, thereby reducing the amount of computation and improving the efficiency of voiceprint recognition.

[0042] It is worth mentioning that Figure 2 The network structure of each feature extraction module in can be improved as Figure 3 The network structure of the feature extraction module in the voiceprint recognition is further improved.

[0043] like Figure 4 As shown, the feature extraction of speech data by the deep separable convolutional network is divided into channel-by-channel convolution (Depthwise Convolution) and point-by-point convolution (Pointwise Convolution). In this embodiment, after performing multiple convolution channel convolution operations on the speech data through the voiceprint recognition model, the convolution operation results of each channel are spliced ​​and fused, and the fused fused voiceprint features are subjected to point-by-point convolution processing to obtain the voiceprint features of the speech data.

[0044] After the voiceprint features of the speech data are obtained, voiceprint recognition is performed according to the voiceprint features to determine the voiceprint recognition result.

[0045] In some embodiments, the voiceprint recognition method provided in the embodiments of the present application uses a voiceprint recognition model to perform convolution on multiple channels of speech data and then fuse them, and then perform point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features of the speech data, including:

[0046] Step S130: performing multiple channel convolutions on the voice data through a deep separable convolutional network in the voiceprint recognition model to obtain channel voiceprint features representing multiple channels of the voice data;

[0047] Step S140: concatenating and fusing the voiceprint features of multiple channels to obtain a fused voiceprint feature;

[0048] Step S150: performing point-by-point convolution processing on the fused voiceprint features to obtain voiceprint features.

[0049] In some embodiments, multiple channel voiceprint features are spliced ​​and fused to obtain a fused voiceprint feature, including: fusing multiple channel voiceprint features through a splicing and fusion network of a deep separable convolutional network to obtain a fused voiceprint feature; performing point-by-point convolution processing on the fused voiceprint feature to obtain a voiceprint feature includes: performing point-by-point convolution processing on the fused voiceprint feature through a point-by-point convolution network of a deep separable convolutional network to obtain the voiceprint feature.

[0050] Step S130 may perform multiple channel convolutions on the voice data through a channel-by-channel convolution network of a depthwise separable convolutional network to obtain channel voiceprint features representing multiple channels of the voice data;

[0051] Step S140 may fuse the voiceprint features of multiple channels through a concatenated fusion network of a deep separable convolutional network to obtain a fused voiceprint feature;

[0052] In step S150, the fused voiceprint features may be processed point-by-point by a point-by-point convolution network of a depthwise separable convolutional network to obtain the voiceprint features.

[0053] In the embodiment of the present application, the voice data can be convolved with multiple channels through the deep separable convolutional network in the voiceprint recognition model, so as to obtain channel voiceprint features of multiple channels representing the voice data. Then, the multiple channel voiceprint features are concatenated and fused through the splicing fusion network to obtain the fused voiceprint features, wherein the splicing fusion network can be ResNet (residual network), Inception network,

[0054] At least one of the networks of Spatial Pyramid Pooling (SPP) and Atrous Convolution. Then, the fused voiceprint features are subjected to point-by-point convolution processing to obtain the voiceprint features of the speech data.

[0055] like Figure 5 As shown, in the channel-by-channel convolution network, each channel uses a convolution kernel, and then the outputs of all convolution kernels are concatenated. In the embodiment of the present application, the output of the channel-by-channel convolution network is referred to as the channel voiceprint features of multiple channels. Optionally, before the channel-by-channel convolution network is processed, the voice data can be processed by other modules in the voiceprint recognition model to obtain the input feature map. For example, Figure 2 As shown in the figure, a module in the voiceprint recognition model is composed of a one-dimensional convolutional neural network, in which the one-dimensional convolutional neural network can be a combination of one-dimensional convolution (expanded convolution), activation function, and batch normalization. The voice data is input into the one-dimensional convolutional neural network to first obtain an input feature map. Then the input feature map is input into the depthwise separable convolutional network, and the channel voiceprint features of multiple channels are first processed by the channel-by-channel convolutional network.

[0056] like Figure 6As shown in the figure, in the point-by-point convolution network, a 1x1 convolution kernel can be used to mix the channel voiceprint features of multiple channels to generate the voiceprint features of the speech data. It can also be said that point-by-point convolution is a linear transformation of the convolution result of channel-by-channel convolution. In the depthwise separable convolution network, the point-by-point convolution network can play two roles: one is to allow the depthwise separable convolution network to freely change the number of output channels; the other is to perform channel fusion on the features output by the channel-by-channel convolution network.

[0057] In a specific embodiment, Figure 6 Maps*3 in the figure is the input layer of the point-by-point convolutional network, that is, the input feature map, which contains 3 feature maps. These 3 feature maps are fused voiceprint features. The point-by-point convolutional layer (Filters*4) of the point-by-point convolutional network contains 4 1x1 convolution kernels. Each convolution kernel is point-by-point convolved with the 3 input feature maps to generate 4 new feature maps. The main function of point-by-point convolution is to change the depth of the feature map (that is, the number of channels) while keeping the spatial resolution unchanged. Maps*4 is the output layer of the point-by-point convolutional network, that is, the output feature map, which contains 4 feature maps. The output feature map is the result of point-by-point convolution and can be used as the input of subsequent network layers, or for the final classification or regression tasks.

[0058] In some embodiments, the voiceprint convolution processing is performed on the voice data through a deep separable convolutional network in a voiceprint recognition model to obtain a voiceprint feature; the above step S120 includes:

[0059] Step S121: obtaining voiceprint recognition features of speech data according to voiceprint features through a voiceprint recognition model;

[0060] Step S122: Perform voiceprint recognition on the speech data according to the voiceprint recognition feature to obtain a voiceprint recognition result.

[0061] The feature extraction of speech data can be realized through the deep separable convolutional network. The embodiment of the present application further processes the voiceprint features output by the deep separable convolutional network through other modules in the voiceprint recognition model to obtain the voiceprint recognition features of the speech data. Figure 2 As shown in the figure, the deep separable convolutional network is located in the feature extraction module of the voiceprint recognition model, which also includes a feature aggregation module, a statistical pooling module and a fully connected layer. The feature aggregation module is used to aggregate the intermediate layer features of different levels output by the feature extraction module to generate a more comprehensive acoustic feature representation; the statistical pooling module combines the attention mechanism to calculate weighted statistical information; the fully connected layer performs further nonlinear transformation on the statistically pooled features and then outputs low-dimensional voiceprint recognition features. Figure 2 As shown, Figure 2Conv1D refers to one-dimensional convolution; ReLU refers to the activation function; FC refers to Fully-Connected; BN refers to BatchNormalization, which is used to accelerate the training process and improve the stability of the voiceprint recognition model.

[0062] In summary, the voiceprint recognition method provided in the embodiment of the present application responds to the input of voice data, uses a voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuses them, performs point-by-point convolution processing on the fused voiceprint features, obtains the voiceprint features of the voice data, and then further performs voiceprint recognition based on the voiceprint features output by the voiceprint recognition model. The embodiment of the present application uses a voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuses them, performs point-by-point convolution processing on the fused voiceprint features, obtains the voiceprint features of the voice data, and performs voiceprint recognition, which greatly reduces the amount of calculation of the voiceprint recognition model, so that the voiceprint recognition model can be deployed in scenarios such as vehicle-mounted. And because features of different scales are extracted through channel-by-channel convolution and features of different scales interact through point-by-point convolution, while reducing the amount of calculation, it also ensures that the voiceprint recognition model can achieve high-precision feature extraction and improves the accuracy of the voiceprint recognition model.

[0063] Below, the performance of the depthwise separable convolutional network is compared with that of conventional convolution.

[0064] Assume that the size of the input feature map is D k ×D k ×M, the size of the convolution kernel is D F ×D F ×M, the number of convolution kernels is N. Assuming that a convolution operation is performed on each point in the spatial position of the corresponding input feature map, then a single convolution of a conventional convolution requires a total of D k ×D k ×D F ×D F ×M times of calculation. This is because the spatial dimensions of the input feature map contain a total of D k ×D k points, and the amount of computation required to perform a convolution operation on each point is consistent with the size of the convolution kernel, i.e., D F ×D F ×M, so for a single convolution, the total calculation formula for the number of parameters is n_1 = D k ×D k ×D F ×D F ×M. Then, for N convolutions, the parameter calculation formula for conventional convolution is n_N=D k ×D k ×D F ×DF ×M×N.

[0065] A similar analysis of the depthwise separable convolutional network shows that the parameter calculation formula for channel-by-channel convolution is n_D = D k ×D k ×D F ×D F ×M, the parameter calculation formula for point-by-point convolution is n_P = M × N × D k ×D k Therefore, the parameter calculation formula of the depth-separable convolutional network is n_DSC=n_D+n_P.

[0066] Based on this, the ratio of the number of parameters of the depthwise separable convolutional network to the number of parameters of the conventional convolution is:

[0067] It can be seen that the application of deep separable convolutional networks can effectively reduce the computational complexity of the voiceprint recognition model and improve the computational efficiency of the voiceprint recognition model.

[0068] In some embodiments, the above step S121 includes:

[0069] Step S1211: performing value restriction processing on the voiceprint features through the target activation function in the voiceprint recognition model;

[0070] Step S1212: obtaining voiceprint recognition features through the voiceprint recognition model according to the voiceprint features after value restriction processing.

[0071] In this embodiment, the target activation function is an activation function used in the voiceprint model to suppress the model output value, which can be ReLU (Rectified Linear Unit) or ReLU6, where ReLU sets all negative values ​​to 0 and keeps positive values ​​unchanged. ReLU6 sets all negative values ​​to 0 and sets all positive values ​​greater than 6 to 6. As a preferred embodiment of this embodiment, ReLU6 is selected as the target activation function to limit the output between 0 and 6, which can prevent the gradient explosion problem, which is helpful for the stability and convergence of the model in some cases.

[0072] In some embodiments, the above step S1211 includes:

[0073] Step S12111: performing nonlinear processing on the voiceprint features through the target activation function in the voiceprint recognition model;

[0074] Step S12112: performing extreme value limiting processing on the voiceprint features after the nonlinear processing through the target activation function.

[0075] The extreme value restriction may be a maximum value restriction or a minimum value restriction, and the maximum value restriction may be a ReLU6 function.

[0076] In an embodiment of the present application, the maximum value restriction processing can be implemented by the ReLU6 activation function, and the minimum value restriction processing can be implemented by the Hard Sigmoid activation function.

[0077] The target activation function of the voiceprint recognition model in the embodiment of the present application can not only introduce nonlinear features, so that the voiceprint recognition model can learn complex data patterns. It can also limit the maximum value of the output feature to avoid excessive loss of accuracy when migrating the voiceprint recognition model to a mobile terminal such as a vehicle. If the output of the activation function is not restricted, the output range is zero to positive infinity, and the low-precision float16 cannot accurately describe its value, which will cause a loss of accuracy.

[0078] like Figure 3 As shown, based on ECAPA-TDNN, the embodiment of the present application replaces the original activation function ReLU with the activation function ReLU6. ReLU6 is a variant of ReLU, which is used to suppress the maximum value. The calculation formula of ReLU6 can be ReLU6=min(6,max(0,x)). Of course, other activation functions that can achieve the purpose of the embodiment of the present application can also be used in practical applications, and the embodiment of the present application is not limited to this.

[0079] In some embodiments, the training method of the voiceprint recognition model provided in the embodiments of the present application may include the following steps:

[0080] Step S001: Process each historical voice data in the training data set through the target voiceprint recognition model to obtain the sample voiceprint recognition features of each historical voice data;

[0081] Step S002: according to the sample voiceprint recognition features and voiceprint annotation features of each historical voice data, using the target loss function in the target voiceprint recognition model, determine the recognition loss value of the target voiceprint recognition model;

[0082] Step S003: Adjust the parameters of the target voiceprint recognition model to converge the recognition loss value, and obtain a trained voiceprint recognition model.

[0083] The target voiceprint recognition model is a voiceprint recognition model obtained by improving the original voiceprint recognition model.

[0084] Since it takes a long time to collect a large amount of speech data and annotate features, it may not meet the needs of algorithm development. You can collect open source data sets to further build a training data set. For example, you can collect an open source Chinese speech corpus data set on GitHub as a training data set. Of course, you can also further screen or enhance the data in the open source data set to build a more accurate and rich training data set, which is not limited in the embodiments of the present application.

[0085] The training data set includes at least one historical voice data and the voiceprint annotation features of each historical voice data. Each historical voice data is input into the target voiceprint recognition model, and the target voiceprint recognition model outputs the sample voiceprint recognition features of each historical voice data. Optionally, before the target voiceprint recognition model is formally trained, the target voiceprint recognition model can be pre-trained through the open source data set; then when the target voiceprint recognition model is trained for the first forward propagation, if there are modules with the same name in the pre-trained model and the target voiceprint recognition model, the parameters of the pre-trained model can be directly called, otherwise the parameters of the voiceprint recognition model can be initialized using a random Gaussian distribution. Optionally, the convergence speed of the voiceprint recognition model can be accelerated by transfer learning to prevent overfitting. Optionally, the maximum number of iterations can also be set for the target voiceprint recognition model, such as 100 times, and then one or more target voiceprint recognition models with the latest or optimal parameters are retained for use, or a target voiceprint recognition model is further selected from the latest or optimal multiple target voiceprint recognition models for use, such as performing multiple-dimensional indicator evaluations on these target voiceprint recognition models to select the target voiceprint recognition model with the best performance.

[0086] According to the sample voiceprint recognition features of each historical voice data output by the target voiceprint recognition model, and the voiceprint annotation features of each historical voice data in the training data set, the recognition loss value of the target voiceprint recognition model can be determined by using the target loss function in the target voiceprint recognition model, where the target loss function can be the loss function added after the last layer of the network in the original voiceprint recognition model. Then, according to the recognition loss value, the parameters of the target voiceprint recognition model are adjusted so that the recognition loss value converges, for example, the recognition loss value tends to a stable value, and the trained voiceprint recognition model can be obtained.

[0087] In some embodiments, the training method of the target voiceprint recognition model provided in the embodiments of the present application may include the following steps:

[0088] Step S004: adjusting the original convolutional network in the original voiceprint recognition model to a depth-separable convolutional network; and / or,

[0089] Step S004: adjusting the original activation function in the original voiceprint recognition model to a target activation function, where the target activation function is used to perform value restriction processing on the data; and / or,

[0090] Step S006: Introduce the target loss function into the last layer of the network in the original voiceprint recognition model.

[0091] The original voiceprint recognition model can be constructed based on ECAPA-TDNN. By adjusting at least one of the original convolutional network, the original activation function and the original loss function in the original voiceprint model, an improved original voiceprint recognition model, that is, a target voiceprint recognition model, is obtained.

[0092] Among them, at least one of the original convolutional network, the original activation function and the original loss function in the original voiceprint model is adjusted, and the original convolutional network in the original voiceprint recognition model can be adjusted to a depth-separable convolutional network; and / or, the original activation function in the original voiceprint recognition model is adjusted to a target activation function, and the target activation function is used to restrict the value of the data, and / or, the target loss function is introduced in the last layer of the network in the original voiceprint recognition model to obtain the target voiceprint recognition model.

[0093] Optionally, the embodiment of the present application introduces Arcface loss (Additive Angular Margin Loss) into the loss function of the target voiceprint recognition model. Figure 2 As shown, the target loss function Arcface loss is introduced in the AAM-SoftMax module of the target voiceprint recognition model. The embodiment of the present application introduces Arcface loss, a method of normalizing and enhancing the decision boundary, to transform the decision surface into a decision region, effectively maximizing the inter-class difference and minimizing the intra-class difference, thereby improving the accuracy of the target voiceprint recognition model.

[0094] It can be understood that the embodiment of the present application greatly reduces the computational complexity of the target voiceprint recognition model by adjusting the original convolutional network in the original voiceprint recognition model to a deep separable convolutional network, so that the voiceprint recognition model can be deployed in vehicle-mounted and other scenarios. And because the deep separable convolutional network can extract features of different scales through channel-by-channel convolution and interact between features of different scales through point-by-point convolution, it reduces the number of parameters and the amount of computation while ensuring that the target voiceprint recognition model can achieve high-precision feature extraction and improve the accuracy of the target voiceprint recognition model.

[0095] In addition, in the embodiment of the present application, the original activation function in the original voiceprint recognition model is adjusted to a target activation function, and the target activation function is used to restrict the value of the data. Since the activation function of the voiceprint recognition model can not only introduce nonlinear features, but also limit the maximum value of the output features, it is possible to avoid excessive loss of accuracy when migrating the target voiceprint recognition model to a mobile terminal such as a vehicle.

[0096] In addition, the embodiment of the present application adds a target loss function after the last layer of the network in the original voiceprint recognition model, wherein the target loss function is Arcface loss. By introducing Arcface loss to calculate the loss function of the voiceprint recognition model, the inter-class differences and the intra-class differences are effectively maximized, thereby improving the accuracy of the target voiceprint recognition model.

[0097] Based on this, in some embodiments, the above step S006 may include the following steps:

[0098] Step S0061: determining the correct target feature among the sample voiceprint recognition features according to the sample voiceprint recognition features and the voiceprint annotation features;

[0099] Step S0062: adding an angle interval to the angle corresponding to the target feature to obtain an adjusted angle corresponding to the target feature;

[0100] Step S0063: Determine the recognition loss value according to the adjusted angle corresponding to the target feature and the angles corresponding to other features in the sample voiceprint recognition feature.

[0101] Exemplarily, the calculation formula for the recognition loss value L3 is as follows:

[0102]

[0103] Where N is the total number of historical speech data in the training data set, i is the i-th historical speech data, s is the feature scale, m is the angle interval, yi is the correct target feature identified in the voiceprint recognition feature of the i-th historical speech data, j is the other features in the sample voiceprint recognition feature, θ j are the angles corresponding to other features, is the angle corresponding to the target feature, is the adjusted angle corresponding to the target feature.

[0104] Taking the original voiceprint recognition model implemented as the original ECAPA-TDNN as an example, the embodiment of the present application introduces a deep separable convolutional network, ReLU6 and Arcface loss on the basis of ECAPA-TDNN to obtain an improved ECAPA-TDNN. The performance comparison results of the original ECAPA-TDNN and the improved ECAPA-TDNN provided by the embodiment of the present application in at least one indicator are as follows: Figure 7 It can be seen that the embodiment of the present application improves the recognition accuracy of the voiceprint recognition model and reduces the number of parameters and reasoning time of the voiceprint recognition model.

[0105] In some embodiments, the above step S122 may include the following steps:

[0106] Step S1221: Compare the voiceprint recognition features with the historical voiceprint features in the feature database;

[0107] Step S1222: when there is a historical voiceprint feature in the feature database that matches the voiceprint recognition feature, display a first prompt message and / or execute a corresponding voice command based on the input of the voice data; the first prompt message is used to indicate that the voiceprint recognition is successful;

[0108] Step S1223: When there is no historical voiceprint feature matching the voiceprint recognition feature in the feature database, a second prompt message is displayed; the second prompt message is used to indicate that the voiceprint recognition has failed.

[0109] The feature database is used to store at least one historical voiceprint feature, each historical voiceprint feature can be bound to a user account, and a user account can be bound to multiple historical voiceprint features. Optionally, the voiceprint recognition feature obtained when registering a user account can be stored in the feature database and a binding relationship can be established between the voiceprint recognition feature and the registered user account.

[0110] The voiceprint recognition feature output for the voice data is compared with the historical voiceprint feature in the feature database, and whether the voiceprint recognition for the voice data is successful is determined based on the comparison result. Optionally, the comparison of the voiceprint recognition feature and the historical voiceprint feature includes calculating the similarity between the voiceprint recognition feature and the historical voiceprint feature. Thus, the voiceprint recognition feature matches the historical voiceprint feature, including that the similarity between the voiceprint recognition feature and the historical voiceprint feature is greater than a similarity threshold.

[0111] When the voiceprint recognition feature matches a historical voiceprint feature in the feature database, it is determined that the voiceprint recognition of the voice data is successful, that is, the speaker identity authentication corresponding to the voice data is passed, and a first prompt message indicating that the voiceprint recognition is successful can be displayed and / or the voice data can be audio-parsed to execute the instruction corresponding to the voice data, that is, the instruction requested by the voice data, such as unlocking, playing / stopping music, adjusting seats, turning on / off navigation services, turning on / off air conditioning, etc.

[0112] When the voiceprint recognition feature does not match any historical voiceprint features in the feature database, it is determined that the voiceprint recognition of the speech data has failed, that is, the speaker identity authentication corresponding to the speech data has not passed, and a second prompt message indicating the voiceprint recognition failure may be displayed.

[0113] The embodiments of the present application do not limit the specific content of the voiceprint recognition features. Optionally, the voiceprint recognition features include but are not limited to at least one of the following: a low-dimensional embedding vector (Embedding Vector), which is used to characterize the characteristics of the speaker; MFCCs (Mel-Frequency Cepstral Coefficients), which are intermediate features in the voiceprint recognition model processing process; normalized features, which are the results of normalizing MFCCs or other intermediate features; dynamic features, which describe the dynamic features of voice data changes; high-level features; attention-weighted features; multi-scale features; frequency domain features, which are used to describe the distribution characteristics of voice data in the frequency domain, such as spectral energy distribution; time domain features, which are used to describe the changing characteristics of voice data in the time domain, such as pitch, sound intensity, etc.; statistical features of the speaker, which are features obtained based on statistical analysis of voice data, such as voice entropy and energy distribution.

[0114] In summary, the voiceprint recognition method provided in the embodiment of the present application displays corresponding prompt information according to different voiceprint recognition results to prompt the user whether the voiceprint recognition is successful, thereby achieving a better interaction effect with the user.

[0115] In some embodiments, the voiceprint recognition method provided in the embodiments of the present application further includes:

[0116] Step S010: pre-process the input original voice data to obtain voice data.

[0117] In the embodiment of the present application, if the historical voice data in the training data set has been preprocessed, the voice data input when the voiceprint recognition model is used also needs to be preprocessed. In the embodiment of the present application, when the voiceprint recognition model is used, the data input to the voiceprint recognition model is referred to as voice data, and the original collected voice data is referred to as input voice data. Optionally, the input voice data can be collected by a microphone, and the sound pickup function of the microphone can be always on to respond to the voice input in a timely manner.

[0118] Among them, the data preprocessing of the input voice data includes at least one of the following processes: voice interception processing, voice noise reduction processing, and voice enhancement processing. Voice interception processing refers to intercepting effective voice from voice data, which can be achieved by VAD (Voice Activity Detection) algorithm. For example, when voice data input is detected, at least 3 seconds of voice data is saved from the current moment based on the voice interception processing, and then voice noise reduction processing and voice enhancement processing are performed to obtain voice data. Optionally, data preprocessing also includes quality detection processing to detect whether the quality of the voice data is qualified. If the quality of the voice data is qualified, it is input into the voiceprint recognition model to perform voiceprint recognition, otherwise it is discarded.

[0119] According to the second aspect of the present application, the embodiment of the present application further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above-mentioned voiceprint recognition method are implemented. The non-transitory computer-readable storage medium has all the beneficial effects of the above-mentioned voiceprint recognition method, and the present application will not repeat them here.

[0120] According to the third aspect of the present application, an embodiment of the present application further provides an electronic device, comprising: a memory and a processor, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the steps of the above-mentioned voiceprint recognition method. The electronic device has all the beneficial effects of the above-mentioned voiceprint recognition method, which will not be described in detail in this application. According to the fourth aspect of the present application, an embodiment of the present application further provides a voiceprint recognition system.

[0121] See also Figure 8 , Figure 8 The block diagram of a voiceprint recognition system provided by an embodiment of the present application includes: a voiceprint collection module 10 and an electronic device 20 as in the third aspect, the voiceprint collection module 10 is connected in communication with the electronic device 20; the voiceprint collection module 10 is used to collect voice data and send the collected voice data to the electronic device 20; the electronic device 20 is used to perform voiceprint recognition processing on the received voice data and determine the voiceprint recognition result.

[0122] Taking the deployment of the voiceprint recognition system in a vehicle-mounted scenario as an example, the voiceprint collection module 10 may be a microphone, and the electronic device 20 may be a vehicle-mounted chip, such as a vehicle-mounted SoC.

[0123] The microphone is used for audio data collection. The microphone is connected to the vehicle SoC. Optionally, the microphone can be installed at the B-pillar, door, handle, etc. of the vehicle to facilitate the collection of user voice data.

[0124] The vehicle-mounted SoC is deployed with a voiceprint recognition model, which is mainly used for voiceprint recognition. For example, the voice data to be recognized is processed by the voiceprint recognition model to obtain the voiceprint features of the voice data to be recognized. For another example, the voice data is further subjected to voiceprint recognition based on the voiceprint features of the voice data to be recognized to determine the identity information of the speaker corresponding to the voice data to be recognized.

[0125] Please continue reading Figure 8 The voiceprint recognition system also includes a display module 30 and a storage module 40, which are communicatively connected to the electronic device 20; the display module 30 is used to receive the voiceprint recognition result sent by the electronic device 20 and display the received voiceprint recognition result; the storage module 40 is used to store historical voiceprint feature information and user information.

[0126] The display module may be an LCD vehicle-mounted terminal display unit. Optionally, the display module is connected to the vehicle-mounted SoC in a wired manner.

[0127] The storage module includes a feature database, which is mainly used to store historical voiceprint features and user information, etc. In addition, it can also include third-party library files required to execute the voiceprint recognition method, such as computer-readable storage media.

[0128] related Figure 8 The specific implementation methods and beneficial effects of the embodiments can be found in the above method embodiments, which will not be described in detail here.

[0129] According to the fifth aspect of the present application, an embodiment of the present application also provides a voiceprint recognition device.

[0130] See also Fig. 9 , Fig. 9 is a block diagram of a voiceprint recognition device provided in an embodiment of the present application. The voiceprint recognition device may be an electronic device, or may be arranged in an electronic device. Fig. 9 As shown, the voiceprint recognition device includes: a voiceprint feature extraction module 710 and a voiceprint recognition module 720.

[0131] The voiceprint feature extraction module 710 is used to respond to the input of voice data, use the voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuse them, and perform point-by-point convolution processing on the fused fused voiceprint features to obtain the voiceprint features of the voice data; the voiceprint recognition module 720 is used to output the voiceprint recognition result based on the voiceprint features of the voice data. In some embodiments, the voiceprint feature extraction module 710 is specifically used to:

[0132] Through the deep separable convolutional network in the voiceprint recognition model, multiple convolution channels are convolved on the voice data to obtain channel voiceprint features of multiple channels representing the voice data; the multiple channel voiceprint features are spliced ​​and fused to obtain a fused voiceprint feature; the fused voiceprint feature is convolved point by point to obtain the voiceprint feature.

[0133] In some embodiments, the voiceprint feature extraction module 710 is further configured to fuse the voiceprint features of multiple channels through a concatenated fusion network of the depthwise separable convolutional network to obtain a fused voiceprint feature;

[0134] The voiceprint feature extraction module 710 is further specifically used to perform point-by-point convolution processing on the fused voiceprint feature through the point-by-point convolution network of the depthwise separable convolutional network to obtain the voiceprint feature.

[0135] In some embodiments, the voiceprint convolution processing is performed on the voice data through the deep separable convolutional network in the voiceprint recognition model to obtain the voiceprint features; the voiceprint recognition module 720 is specifically used to obtain the voiceprint recognition features of the voice data according to the voiceprint features through the voiceprint recognition model; and the voiceprint recognition is performed on the voice data according to the voiceprint recognition features to obtain the voiceprint recognition results.

[0136] In some embodiments, the voiceprint feature extraction module 710 is further used to perform value restriction processing on the voiceprint feature through the target activation function in the voiceprint recognition model; and obtain the voiceprint recognition feature according to the voiceprint feature after the value restriction processing through the voiceprint recognition model. In some embodiments, the voiceprint recognition module 720 is further used to perform nonlinear processing on the voiceprint feature through the target activation function in the voiceprint recognition model; and perform extreme value restriction processing on the voiceprint feature after the nonlinear processing through the target activation function.

[0137] In some embodiments, the voiceprint recognition module 720 is further specifically used to compare the voiceprint recognition feature with the historical voiceprint feature in the feature database; when there is a historical voiceprint feature matching the voiceprint recognition feature in the feature database, a first prompt message is displayed and / or a corresponding voice instruction is executed based on the input of the voice data; when there is no historical voiceprint feature matching the voiceprint recognition feature in the feature database, a second prompt message is displayed; wherein, the first prompt message is used to indicate that the voiceprint recognition is successful, and the second prompt message is used to indicate that the voiceprint recognition fails.

[0138] In some embodiments, the voiceprint recognition device also includes a preprocessing module (not shown in the figure), which is used to: perform data preprocessing on the input original voice data to obtain the voice data; wherein the data preprocessing includes at least one of the following processes: voice interception processing, voice noise reduction processing, and voice enhancement processing.

[0139] In some embodiments, Fig.10 As shown, the voiceprint recognition device 700 also includes a model training module 730, which is specifically used to: process each historical voice data in the training data set through a target voiceprint recognition model to obtain a sample voiceprint recognition feature of each of the historical voice data; determine the recognition loss value of the target voiceprint recognition model based on the sample voiceprint recognition feature and voiceprint annotation feature of each of the historical voice data and using the target loss function in the target voiceprint recognition model; adjust the parameters of the target voiceprint recognition model so that the recognition loss value converges to obtain the voiceprint recognition model that has completed training.

[0140] In some embodiments, Fig.10 As shown, the voiceprint recognition device 700 also includes a model training module 730, which is specifically used to: adjust the original convolutional network in the original voiceprint recognition model to a depth-separable convolutional network; and / or, adjust the original activation function in the original voiceprint recognition model to a target activation function, and the target activation function is used to perform value restriction processing on the data; and / or, introduce a target loss function in the last layer of the network in the original voiceprint recognition model.

[0141] In some embodiments, the voiceprint recognition device 700 also includes a model training module 730, which is specifically used to: determine the correct target feature identified in the sample voiceprint recognition feature based on the sample voiceprint recognition feature and the voiceprint annotation feature; add an angle interval to the angle corresponding to the target feature to obtain an adjusted angle corresponding to the target feature; determine the recognition loss value based on the adjusted angle corresponding to the target feature and the angles corresponding to other features in the sample voiceprint recognition feature.

[0142] It should be understood that, in specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. In addition, the specific implementation methods and corresponding beneficial effects of the above modules can be referred to the above method embodiments, which will not be described in detail here.

[0143] See also Fig.11 , the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0144] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.11 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Fig.11 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0145] In particular, according to some embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of some embodiments of the present application are executed.

[0146] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above, which is not specifically limited in the present application. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0147] In some embodiments of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device, or device. In some embodiments of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0148] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an adhoc peer-to-peer network), as well as any currently known or future developed network.

[0149] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0150] For example, the computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: responds to the input of voice data, uses the voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuses them, performs point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features of the voice data; and outputs the voiceprint recognition results based on the voiceprint features of the voice data.

[0151] Computer program code for performing the operations of some embodiments of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function.

[0153] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.

[0154] For example, two boxes shown in succession may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] The units described in some embodiments of the present application may be implemented by software or hardware. The units described may also be set in a processor. For example, they may be described as: a processor includes a voiceprint feature extraction module and a voiceprint recognition module. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, a voiceprint recognition module may also be described as "a unit for performing voiceprint recognition on the voice data to be recognized according to the voiceprint recognition feature".

[0156] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0157] According to the sixth aspect of the present application, an embodiment of the present application further provides a vehicle, including a voiceprint recognition device, or an electronic device, or a voiceprint recognition system, wherein the electronic device and the voiceprint recognition system can be used to perform the above-mentioned voiceprint recognition method. The vehicle has all the beneficial effects of the above-mentioned voiceprint recognition device, or electronic device, or voiceprint recognition system, which will not be described in detail in this application.

[0158] The vehicle may be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and this application does not make any specific limitation on this.

[0159] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0160] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0161] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0162] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application has used examples to describe various embodiments with different emphases, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A voiceprint recognition method, characterized in that: include: In response to the input of voice data, the voice data is convolved by a plurality of convolution channels using a voiceprint recognition model and then fused, and the fused voiceprint features are convolved point by point to obtain the voiceprint features of the voice data; Output a voiceprint recognition result based on the voiceprint features of the speech data.

2. The method according to claim 1, characterized in that The method of using the voiceprint recognition model to perform multiple convolution channel convolutions on the voice data and then fuse them, and performing point-by-point convolution processing on the fused voiceprint features to obtain the voiceprint features of the voice data includes: Performing multiple channel convolutions on the voice data through a deep separable convolutional network in a voiceprint recognition model to obtain channel voiceprint features representing multiple channels of the voice data; splicing and fusing the multiple channel voiceprint features to obtain a fused voiceprint feature; The fused voiceprint feature is subjected to point-by-point convolution processing to obtain the voiceprint feature.

3. The method according to claim 2, characterized in that The step of concatenating and fusing the plurality of channel voiceprint features to obtain a fused voiceprint feature comprises: fusing the plurality of channel voiceprint features through a concatenated fusion network of the depthwise separable convolutional network to obtain a fused voiceprint feature; The performing point-by-point convolution processing on the fused voiceprint feature to obtain the voiceprint feature includes: performing point-by-point convolution processing on the fused voiceprint feature through a point-by-point convolution network of the depthwise separable convolutional network to obtain the voiceprint feature.

4. The method according to claim 1, characterized in that The voiceprint convolution processing is performed on the voice data through a deep separable convolutional network in a voiceprint recognition model to obtain a voiceprint feature; and the voiceprint recognition result is output based on the voiceprint feature of the voice data, including: Obtaining the voiceprint recognition feature of the speech data according to the voiceprint feature through the voiceprint recognition model; The voiceprint recognition is performed on the speech data according to the voiceprint recognition feature to obtain a voiceprint recognition result.

5. The method according to claim 4, characterized in that The obtaining the voiceprint recognition feature of the speech data according to the voiceprint feature by using the voiceprint recognition model includes: Through the target activation function in the voiceprint recognition model, the voiceprint feature is subjected to value restriction processing; The voiceprint recognition feature is obtained through the voiceprint recognition model according to the voiceprint feature after the value restriction processing.

6. The method according to claim 5, characterized in that The method of performing value restriction processing on the voiceprint feature through the target activation function in the voiceprint recognition model includes: Performing nonlinear processing on the voiceprint features through a target activation function in the voiceprint recognition model; The target activation function is used to perform extreme value limiting processing on the voiceprint features after the nonlinear processing.

7. The method according to claim 4, characterized in that The performing voiceprint recognition on the voice data according to the voiceprint recognition feature includes: Comparing the voiceprint recognition feature with the historical voiceprint features in the feature database; When there is a historical voiceprint feature matching the voiceprint recognition feature in the feature database, displaying a first prompt message and / or executing a corresponding voice command based on the input of the voice data; When there is no historical voiceprint feature matching the voiceprint recognition feature in the feature database, displaying a second prompt message; The first prompt information is used to indicate that the voiceprint recognition is successful, and the second prompt information is used to indicate that the voiceprint recognition is failed.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Performing data preprocessing on the input original voice data to obtain the voice data; The data preprocessing includes at least one of the following processes: speech interception processing, speech noise reduction processing, and speech enhancement processing.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Processing each historical voice data in the training data set through the target voiceprint recognition model to obtain a sample voiceprint recognition feature of each historical voice data; According to the sample voiceprint recognition features and voiceprint annotation features of each of the historical voice data, using the target loss function in the target voiceprint recognition model, determining the recognition loss value of the target voiceprint recognition model; The parameters of the target voiceprint recognition model are adjusted to converge the recognition loss value, thereby obtaining the trained voiceprint recognition model.

10. The method according to claim 9, characterized in that The target voiceprint recognition model is obtained by the following steps: Adjust the original convolutional network in the original voiceprint recognition model to a depthwise separable convolutional network; and / or, Adjusting the original activation function in the original voiceprint recognition model to a target activation function, where the target activation function is used to perform value restriction processing on the data; and / or, The target loss function is introduced into the last layer of the network in the original voiceprint recognition model.

11. The method according to claim 9, characterized in that The step of determining the recognition loss value of the target voiceprint recognition model by using the target loss function in the target voiceprint recognition model according to the sample voiceprint recognition features and the voiceprint annotation features of each of the historical voice data comprises: Determining a correct target feature among the sample voiceprint recognition features according to the sample voiceprint recognition features and the voiceprint annotation features; Adding an angle interval to the angle corresponding to the target feature to obtain an adjusted angle corresponding to the target feature; The recognition loss value is determined according to the adjusted angle corresponding to the target feature and the angles corresponding to other features in the sample voiceprint recognition feature.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the voiceprint recognition method according to any one of claims 1 to 11 is implemented.

13. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the voiceprint recognition method according to any one of claims 1 to 11.

14. A voiceprint recognition system, characterized in that: include: A voiceprint collection module and the electronic device as claimed in claim 13, wherein the voiceprint collection module is communicatively connected to the electronic device; The voiceprint collection module is used to collect voice data and send the collected voice data to the electronic device; The electronic device is used to perform voiceprint recognition processing on the received voice data and determine the voiceprint recognition result.

15. The system according to claim 14, characterized in that Also includes: A display module and a storage module, wherein the display module and the storage module are both communicatively connected with the electronic device; The storage module is used to store historical voiceprint feature information and user information; The display module is used to receive the voiceprint recognition result sent by the electronic device and display the received voiceprint recognition result.

16. A vehicle, characterized in that: The vehicle includes the electronic device as claimed in claim 13, or the voiceprint recognition system as claimed in claim 14 or 15.

Images