Speech recognition method, medium, device and computing equipment

Abstract

The embodiments of the present disclosure provide a speech recognition method, medium, apparatus, and computing device. The speech recognition method includes: obtaining the acoustic features of the speech signal to be recognized; determining the confidence level of each speech segment contained in the speech signal to be recognized as the target speech segment based on the speech recognition model and the acoustic features; and identifying the target speech segment in the speech signal to be recognized according to the confidence level of each speech segment. On the one hand, by recognizing audio data through a speech recognition model, the efficiency and accuracy of speech recognition can be greatly improved. On the other hand, by identifying the speech segments of specific persons in the audio data, the influence of speech segments of non-specific persons can be eliminated, further improving the efficiency of speech recognition.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of speech recognition technology, and more particularly, to speech recognition methods, media, apparatuses, and computing devices. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] With the continuous development of science and technology, speech recognition technology has gradually been applied to various scenarios, such as audio review and excerpting. Taking the audio data review scenario as an example, before the audio data is released, it is necessary to identify whether the audio data contains the voice of a specific person, so as to determine the compliance of the audio data.

[0004] Currently, identifying specific individuals in audio data primarily relies on human reviewers manually identifying and judging the voice information. However, manual recognition is inefficient, and with the advent of the information age, the volume and duration of audio data are growing exponentially, making manual recognition inadequate for current review needs. Summary of the Invention

[0005] The embodiments of the present disclosure provide a speech recognition method, medium, apparatus, and computing device for solving the current problem of low efficiency in manual review of audio data.

[0006] In a first aspect of the embodiments of the present disclosure, a speech recognition method is provided, comprising: obtaining acoustic features of a speech signal to be recognized; determining, based on a speech recognition model and the acoustic features, the confidence that each speech segment contained in the speech signal to be recognized is a target speech segment, wherein the target speech segment is a speech signal emitted by a target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signal of the target person; and identifying the target speech segment in the speech signal to be recognized according to the confidence of each speech segment.

[0007] In one possible embodiment, the speech recognition model is obtained by training based on the acoustic features of the target person as follows: obtaining the acoustic features of multiple sample audio signals corresponding to the target person; inputting the acoustic features into the target voiceprint recognition model to obtain the voiceprint features corresponding to the acoustic features output by the target voiceprint recognition model; obtaining label data of the acoustic features based on the voiceprint features, and the label data of the acoustic features is used to indicate the target person to which the acoustic features belong; and training the initial speech recognition model with the label data of the acoustic features to obtain a speech recognition model.

[0008] In one possible embodiment, the method of identifying a target speech segment in a speech signal to be recognized based on a confidence level includes: identifying a target speech segment in a speech signal to be recognized based on a first speech segment having a confidence level greater than or equal to a confidence threshold; or determining that a speech segment having a confidence level less than a confidence threshold is not a target speech segment.

[0009] In a possible embodiment, the method of identifying a target speech segment in a speech signal to be identified based on a first speech segment having a confidence greater than or equal to a confidence threshold includes: obtaining a first speech segment having a confidence greater than a confidence threshold; obtaining a voiceprint feature of the first speech segment based on the acoustic features of the first speech segment; and identifying whether the first speech segment is a target speech segment based on the voiceprint feature.

[0010] In a possible implementation, the method of obtaining the voiceprint features of the first voice segment based on the acoustic features of the first voice segment includes: inputting the acoustic features of the first voice segment into a target voiceprint recognition model to obtain the voiceprint features of the first voice segment, wherein the target voiceprint recognition model is trained based on the voiceprint features of the target person.

[0011] In one possible embodiment, identifying whether a first voice segment is a target voice segment based on a voiceprint feature includes: comparing the voiceprint feature with a voiceprint feature in a database, where the database contains the voiceprint features of the target person; if it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, determining that the first voice segment is the target voice segment; if it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, determining that the first voice segment is not the target voice segment.

[0012] In a possible implementation, the speech recognition method further includes determining first time information of each target speech segment in the speech signal to be recognized, where the first time information includes a start and end time of each target speech segment.

[0013] In a possible embodiment, the speech recognition method also includes: merging the temporally continuous target speech segments according to the first time information of each target speech segment to obtain the speech segment to be processed and the second time information corresponding to each speech segment to be processed, the second time information including: the start and end time of each speech segment to be processed, and / or the duration of each speech segment to be processed.

[0014] In one possible embodiment, the speech recognition model includes: a convolutional neural network, a recurrent neural network, a self-attention mechanism and a fully connected layer; the convolutional neural network is used to extract local features from acoustic features; the recurrent neural network is used to extract global features from acoustic features; the self-attention mechanism is used to extract target features from acoustic features; the fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is the target speech segment based on local features, global features and target features.

[0015] In a second aspect of the embodiments of the present disclosure, a model training method is provided, comprising: obtaining acoustic features of a sample audio signal; and training an initial voiceprint recognition model using the acoustic features of the sample audio signal based on a cyclic loss function to obtain a target voiceprint recognition model.

[0016] In one possible implementation, the voiceprint recognition model includes: a first convolution module, a residual module based on coordinate attention, a self-attention mechanism, a second convolution module, an attention statistics pool, a fully connected layer, and a cyclic loss function module;

[0017] Among them, the first convolution module and the second convolution module are used to obtain the local features of the acoustic features of the sample audio signal; the residual module based on coordinate attention is used to obtain the weight values ​​of the convolution channels in the first convolution module and the second convolution module; the self-attention mechanism is used to obtain the target features in the extracted acoustic features; the attention statistical pool is used to aggregate the acoustic features of variable length into the acoustic features of fixed length; the fully connected layer is used to output the recognition results of the voiceprint features corresponding to the acoustic features based on the local features, target features and fixed-length acoustic features; the cyclic loss function module is used to obtain the cyclic loss function based on the recognition results output by the fully connected layer, and the cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

[0018] In one possible implementation, based on a cyclic loss function, an initial voiceprint recognition model is trained using acoustic features of a sample audio signal to obtain a target voiceprint recognition model, including:

[0019] Determine S first target persons among the target persons; for each first target person, determine N second target persons from multiple target persons whose voiceprint feature similarity with the first target person is greater than a similarity threshold according to a similarity matrix, where the similarity matrix is ​​used to indicate the voiceprint feature similarity between different target persons, where S and N are both integers greater than or equal to 1; based on a cyclic loss function, train the initial voiceprint recognition model through the acoustic features of the sample audio signals corresponding to each second target person to obtain a target voiceprint recognition model.

[0020] In a possible embodiment, the model training method further includes: inputting the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model to obtain the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model; determining the similarity between the voiceprint features of each target person based on the voiceprint feature center corresponding to each target person; and determining the similarity matrix between each target person based on the similarity between the voiceprint features of each target person.

[0021] In one possible implementation, obtaining multiple sample audio signals corresponding to a target person includes: obtaining multiple initial sample audio signals corresponding to the target person; and adding noise data to the initial sample audio signals to obtain multiple sample audio signals corresponding to the target person.

[0022] In one possible implementation, noise data is added to an initial sample audio signal to obtain multiple sample audio signals corresponding to a target person, including: obtaining identity information of the target person; determining target noise data corresponding to the identity information based on the identity information of the target person to which the initial sample audio signal belongs; and adding the target noise data to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person.

[0023] In a possible implementation, the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0024] Based on the cyclic loss function, the initial voiceprint recognition model is trained through the acoustic features of the sample audio signal to obtain the target voiceprint recognition model, including:

[0025] For the i-th training, the i-1-th voiceprint recognition model is trained using the acoustic features of the M1 first sample audio signals in the first sample set and the acoustic features of the M2 second sample audio signals in the second sample set to obtain the i-th cyclic loss function and the i-th voiceprint recognition model. The i-1-th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times; the voiceprint recognition model corresponding to the convergence of the cyclic loss function is determined as the target voiceprint recognition model;

[0026] Wherein, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

[0027] In a possible implementation, the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0028] Based on the cyclic loss function, the initial voiceprint recognition model is trained by the acoustic features of the sample audio signals to obtain the target voiceprint recognition model, including: based on the cyclic loss function, the initial voiceprint recognition model is trained by the acoustic features of the sample audio signals in the first sample set to obtain the first voiceprint recognition model; based on the cyclic loss function, the first voiceprint recognition model is trained by the acoustic features of the sample audio signals in the second sample set to obtain the target voiceprint recognition model.

[0029] In the third aspect of the embodiments of the present disclosure, a model training method is provided, including: obtaining multiple sample audio signals corresponding to a target person; obtaining acoustic features of the sample audio signals; inputting the acoustic features of the sample audio signals into a target voiceprint recognition model to obtain voiceprint features corresponding to the sample audio signals output by the target voiceprint recognition model, wherein the target voiceprint recognition model is trained using the model training method of the second aspect; obtaining label data of the acoustic features based on the voiceprint features, wherein the label data of the acoustic features is used to indicate the target person to which the acoustic features belong; and training an initial speech recognition model using the label data of the acoustic features to obtain a speech recognition model.

[0030] In one possible implementation, obtaining label data of acoustic features based on voiceprint features includes: clustering the voiceprint features to obtain the category to which each voiceprint feature belongs and the cluster center of each category; determining the label data of the voiceprint features in each category based on the label data of the cluster center; and obtaining the label data of the acoustic features based on the label data of the voiceprint features.

[0031] In the fourth aspect of the embodiment of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the speech recognition method of the first aspect and / or the model training method of the second aspect or the third aspect are implemented.

[0032] In the fifth aspect of the embodiment of the present disclosure, a speech recognition device is provided, including: an acquisition module for acquiring acoustic features of a speech signal to be recognized; a determination module for determining, based on a speech recognition model and acoustic features, the confidence that each speech segment contained in the speech signal to be recognized is a target speech segment, wherein the target speech segment is a speech signal emitted by a target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signal of the target person; and an identification module for identifying the target speech segment in the speech signal to be recognized based on the confidence of each speech segment.

[0033] In one possible embodiment, the speech recognition model is obtained by training based on the acoustic features of the target person as follows: obtaining the acoustic features of multiple sample audio signals corresponding to the target person; inputting the acoustic features into the target voiceprint recognition model to obtain the voiceprint features corresponding to the acoustic features output by the target voiceprint recognition model; obtaining label data of the acoustic features based on the voiceprint features, and the label data of the acoustic features is used to indicate the target person to which the acoustic features belong; and training the initial speech recognition model with the label data of the acoustic features to obtain the speech recognition model.

[0034] In one possible implementation, the recognition module is specifically used to: identify a target speech segment in the speech signal to be recognized based on a first speech segment whose confidence is greater than or equal to a confidence threshold; or determine that a speech segment whose confidence is less than the confidence threshold is not the target speech segment.

[0035] In one possible embodiment, the recognition module is specifically used to: obtain a first voice segment with a confidence level greater than a confidence threshold; obtain a voiceprint feature of the first voice segment based on the acoustic features of the first voice segment; and identify whether the first voice segment is a target voice segment based on the voiceprint feature.

[0036] In a possible implementation, the recognition module is specifically used to: input the acoustic features of the first voice segment into a target voiceprint recognition model to obtain the voiceprint features of the first voice segment, wherein the target voiceprint recognition model is trained based on the voiceprint features of the target person.

[0037] In one possible embodiment, the recognition module is specifically used to: compare the voiceprint feature with the voiceprint feature in the database, where the database contains the voiceprint feature of the target person; if it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, then determine that the first voice segment is the target voice segment; if it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, then determine that the first voice segment is not the target voice segment.

[0038] In a possible implementation, the determination module is further configured to determine first time information of each target speech segment in the speech signal to be recognized, where the first time information includes a start and end time of each target speech segment.

[0039] In a possible embodiment, the speech recognition device also includes: a processing module, which is used to merge the temporally continuous target speech segments according to the first time information of each target speech segment, obtain the speech segments to be processed and the second time information corresponding to each speech segment to be processed, and the second time information includes: the start and end time of each speech segment to be processed, and / or the duration of each speech segment to be processed.

[0040] In one possible implementation, the speech recognition model includes: a convolutional neural network, a recurrent neural network, a self-attention mechanism, and a fully connected layer;

[0041] The convolutional neural network is used to extract local features from acoustic features; the recurrent neural network is used to extract global features from acoustic features; the self-attention mechanism is used to extract target features from acoustic features; and the fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is the target speech segment based on local features, global features, and target features.

[0042] In the sixth aspect of the embodiment of the present disclosure, a model training device is provided, including: an acquisition module for acquiring multiple sample audio signals corresponding to a target person and acquiring the acoustic features of the sample audio signals; a training module for training an initial voiceprint recognition model based on the acoustic features of the sample audio signals based on a cyclic loss function to obtain a target voiceprint recognition model.

[0043] In one possible implementation, the voiceprint recognition model includes: a first convolution module, a residual module based on coordinate attention, a self-attention mechanism, a second convolution module, an attention statistics pool, a fully connected layer, and a cyclic loss function module;

[0044] The first convolution module and the second convolution module are used to obtain the local features of the acoustic features of the sample audio signal; the residual module based on coordinate attention is used to obtain the weight values ​​of the convolution channels in the first convolution module and the second convolution module; the self-attention mechanism is used to obtain the target features in the extracted acoustic features; the attention statistical pool is used to aggregate the acoustic features of variable length into the acoustic features of fixed length; the fully connected layer is used to output the recognition results of the voiceprint features corresponding to the acoustic features based on the local features, target features and fixed-length acoustic features; the cyclic loss function module is used to obtain the cyclic loss function based on the recognition results output by the fully connected layer. The cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

[0045] In one possible implementation, the training module is specifically used to: determine S first target persons among the target persons; for each first target person, determine N second target persons from multiple target persons whose voiceprint feature similarity with the first target person is greater than a similarity threshold according to a similarity matrix, and the similarity matrix is ​​used to indicate the voiceprint feature similarity between different target persons, where S and N are both integers greater than or equal to 1; based on a cyclic loss function, the initial voiceprint recognition model is trained through the acoustic features of the sample audio signals corresponding to each second target person to obtain a target voiceprint recognition model.

[0046] In a possible embodiment, the model training device also includes: a determination module; the acquisition module is also used to: input the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model, and obtain the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model; the determination module is used to determine the similarity between the voiceprint features of each target person based on the voiceprint feature center corresponding to each target person; and determine the similarity matrix between each target person based on the similarity between the voiceprint features of each target person.

[0047] In a possible implementation, the acquisition module is specifically configured to: acquire multiple initial sample audio signals corresponding to the target person, and add noise data to the initial sample audio signals to obtain multiple sample audio signals corresponding to the target person.

[0048] In one possible implementation, the acquisition module is specifically used to: obtain the identity information of the target person; determine the target noise data corresponding to the identity information based on the identity information of the target person to which the initial sample audio signal belongs; and add the target noise data to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person.

[0049] In a possible implementation, the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0050] The training module is specifically configured to: for the i-th training, train the i-1th voiceprint recognition model using the acoustic features of M1 first sample audio signals in the first sample set and the acoustic features of M2 second sample audio signals in the second sample set, obtain the i-th cyclic loss function and the i-th voiceprint recognition model, where the i-1th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times; determine the voiceprint recognition model corresponding to the convergence of the cyclic loss function as the target voiceprint recognition model;

[0051] Wherein, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

[0052] In a possible embodiment, the sample audio signal includes a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set; the training module is specifically used to: based on a cyclic loss function, train the initial voiceprint recognition model through the acoustic features of the sample audio signals in the first sample set to obtain a first voiceprint recognition model; based on a cyclic loss function, train the first voiceprint recognition model through the acoustic features of the sample audio signals in the second sample set to obtain a target voiceprint recognition model.

[0053] In a seventh aspect of the embodiments of the present disclosure, a model training device is provided, comprising: an acquisition module, configured to acquire a plurality of sample audio signals corresponding to a target person, and to acquire acoustic features of the sample audio signals;

[0054] A voiceprint recognition module, configured to input the acoustic features of the sample audio signal into a target voiceprint recognition model to obtain the voiceprint features corresponding to the sample audio signal output by the target voiceprint recognition model, where the target voiceprint recognition model is trained using the model training method of any one of the second or third aspects;

[0055] A label data acquisition module is used to obtain label data of acoustic features based on voiceprint features, and the label data of acoustic features is used to indicate the target person to whom the acoustic features belong;

[0056] The training module is used to train the initial speech recognition model using the labeled data of the acoustic features to obtain a speech recognition model.

[0057] In one possible implementation, the label data acquisition module is specifically used to: cluster the voiceprint features to obtain the category to which each voiceprint feature belongs and the cluster center of each category; determine the label data of the voiceprint features in each category based on the label data of the cluster center; and obtain the label data of the acoustic features based on the label data of the voiceprint features.

[0058] In an eighth aspect of an embodiment of the present disclosure, a computing device is provided, comprising: at least one processor and a memory; the memory stores computer-executable instructions; at least one processor stores computer-executable instructions in at least one memory, so that at least one processor executes the speech recognition method of the first aspect, and / or the model training method of the second aspect or the third aspect.

[0059] In the ninth aspect of the embodiment of the present disclosure, a computer program product is provided, which includes a computer program; when the computer program is executed, it implements the speech recognition method of the first aspect, and / or the model training method of the second aspect or the third aspect.

[0060] The speech recognition method, medium, apparatus, and computing device in the embodiments of the present disclosure first obtain the acoustic features of the speech signal to be recognized; then, based on the speech recognition model and the acoustic features, determine the confidence that each speech segment contained in the speech signal to be recognized is the target speech segment, wherein the target speech segment is the speech signal emitted by the target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signal of the target person; finally, based on the confidence of each speech segment, identify the target speech segment in the speech signal to be recognized. The solution in the embodiments of the present disclosure only needs to identify the acoustic features through the speech recognition model to determine the speech segment of the target person in the speech signal to be recognized. It is highly efficient, requires less manpower investment, and has more accurate recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0062] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0063] Figure 2 Schematic diagram of the process of the speech recognition method provided in the embodiment of the present disclosure Figure 1 ;

[0064] Figure 3 Schematic diagram of the principle of the speech recognition method provided in the embodiment of the present disclosure Figure 1 ;

[0065] Figure 4 Schematic diagram of the process of the speech recognition method provided in the embodiment of the present disclosure Figure 2 ;

[0066] Figure 5 Schematic diagram of the principle of the speech recognition method provided in the embodiment of the present disclosure Figure 2 ;

[0067] Figure 6 A schematic diagram of the structure of the voiceprint recognition model provided in an embodiment of the present disclosure;

[0068] Figure 7 A flowchart of a voiceprint recognition model training method provided in an embodiment of the present disclosure;

[0069] Figure 8 A schematic diagram of the structure of the speech recognition model provided in the embodiment of the present disclosure;

[0070] Figure 9 A flowchart of a method for training a speech recognition model according to an embodiment of the present disclosure;

[0071] Figure 10 A schematic diagram of the clustering results of voiceprint features provided in an embodiment of the present disclosure;

[0072] Figure 11 A schematic diagram of a program product provided by an embodiment of the present disclosure;

[0073] Figure 12 A schematic diagram of the structure of a speech recognition device provided in an embodiment of the present disclosure;

[0074] Figure 13 A schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure Figure 1 ;

[0075] Figure 14 A schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure Figure 2 ;

[0076] Figure 15 A schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0077] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0078] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0079] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0080] According to an embodiment of the present disclosure, a speech recognition method, medium, apparatus, and computing device are proposed.

[0081] In addition, the number of any elements in the drawings is for illustration and not for limitation, and any names are for distinction only and do not have any limiting meaning. The principle and spirit of the present disclosure will be explained in detail with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0083] With the continuous development of science and technology, speech recognition technology has gradually been applied to various scenarios, such as audio and video review scenarios, excerpt scenarios, etc.

[0084] In audio and video review scenarios, the continued promotion of audio and video platforms has led to an increasing number of people publishing audio data. This has the potential for the spread of illegal audio data, such as audio data containing the voice of a target individual. The release of this audio information could have a negative impact. Therefore, before publishing this audio data, it is necessary to identify the voice clips of specific individuals in the audio data so that the compliance of the audio data can be determined based on these voice clips.

[0085] In the excerpting scenario, for example, for audio recordings of some important occasions, it is necessary to extract speech fragments of specific people in the audio recordings, such as the speech fragments of the target person in the meeting, the speech fragments of the interviewee in the interview recording, etc. This process also requires identifying the voice fragments of specific people in the audio recordings, so as to extract these voice fragments.

[0086] It should be understood that the solution of this embodiment can also be applied to other similar scenarios, which will not be described in detail here.

[0087] To meet these requirements, existing technologies typically employ manual identification to identify audio data. For example, in audio data review scenarios, before the data is released, manual listening is used to identify the target person's voice clips contained in the audio data, and then manually determine whether these voice clips are legal. In excerpting scenarios, manual listening is also required to identify the voice clips of specific individuals contained in the audio data, and then extract these voice clips.

[0088] However, manual recognition is inefficient and has a high misrecognition rate. With the advent of the information age, the amount and duration of audio have grown exponentially, and manual recognition can no longer meet current speech recognition needs.

[0089] In addition, the inventors also found that since a piece of audio data may contain voice clips of specific people as well as voice clips of non-specific people, it is usually necessary to listen to the entire piece of audio data before completing the review or excerpting. Therefore, the voice clips of non-specific people in the audio data will seriously affect the review efficiency or excerpting efficiency.

[0090] In view of this, the embodiments of the present disclosure provide a speech recognition method, medium, apparatus and computing device. In the speech recognition method, a speech recognition model trained with sample audio signals of a specific person is used to recognize audio data, thereby obtaining a speech segment of the specific person in the audio data. On the one hand, compared with manual recognition, the efficiency and accuracy of speech recognition can be greatly improved by recognizing audio data with a trained speech recognition model. On the other hand, the speech recognition model can be used to identify the speech segments of a specific person in the audio data, thereby excluding the influence of speech segments of non-specific persons during review or excerpting, further improving the efficiency of speech review or excerpting.

[0091] Among them, the method and the device are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0092] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0093] Application Scenario Overview

[0094] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure, such as Figure 1 As shown, the application scenario can be an audio data review scenario or a voice excerpt scenario of a specific person.

[0095] In the application scenario provided by the embodiments of the present disclosure, the devices involved include a server 101 and / or a terminal 102 , and speech recognition is implemented through the server 101 or the terminal 102 .

[0096] The server 101 may be a single server, a server cluster, a distributed server, a centralized server, or a cloud server.

[0097] The terminal 102 may be a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities (such as a smartphone or tablet), a computing device (such as a personal computer (PC)), a vehicle-mounted device, a wearable device (such as a smart watch or smart bracelet), a smart home device (such as a smart display device), etc.

[0098] Exemplary Methods

[0099] The following combination Figure 1 The application scenario shown, refer to Figures 2 to 9To describe the method of speech recognition according to an exemplary embodiment of the present disclosure. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0100] The execution device applicable to the embodiments of the present disclosure is a computing device, such as the terminal or server mentioned above.

[0101] Figure 2 Schematic diagram of the process of the speech recognition method provided in the embodiment of the present disclosure Figure 1 .like Figure 2 As shown, the speech recognition method may include the following steps:

[0102] S201: Acquire acoustic features of a speech signal to be recognized.

[0103] It should be understood that the types of voice signals to be recognized are different for different application scenarios, and the embodiments of the present disclosure do not limit the specific types of voice signals to be recognized.

[0104] In this step, after the speech signal to be recognized is obtained, acoustic features in the speech signal to be recognized are extracted, wherein the acoustic features may be Mel-Frequency Cepstral Coefficients (MFCC) features.

[0105] S202: Determine the confidence level of each speech segment included in the speech signal to be recognized as a target speech segment based on the speech recognition model and acoustic features.

[0106] S203: Identify a target speech segment in the speech signal to be recognized according to the confidence level of each speech segment.

[0107] Among them, each speech signal to be recognized contains multiple speech segments, the acoustic features are the acoustic features of multiple speech segments, the target speech segment is the speech signal emitted by the target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signals of the target person. As for the training method of the speech recognition model, it is shown in subsequent embodiments.

[0108] In some embodiments, the speech recognition model may be a TS-VAD model. For ease of understanding, the speech recognition model in the following embodiments is illustrated using the TS-VAD model as an example, but this is not a limitation in actual applications.

[0109] In this step, the acoustic features are input into the TS-VAD model, and the TS-VAD model recognizes the acoustic features to determine the confidence level of each speech segment in the speech signal to be recognized as the target speech segment. Figure 3 To explain:

[0110] Figure 3 Schematic diagram of the principle of the speech recognition method provided in the embodiment of the present disclosure Figure 1 .like Figure 3 As shown, a time anchor frame (anchor) of a preset duration is set on the speech signal to be recognized, and each anchor corresponds to a speech segment, so as to extract the MFCC features of the speech segment corresponding to each anchor (segment 1, segment 2...segment n are multiple speech segments in the speech signal to be recognized).

[0111] Furthermore, the acoustic features corresponding to segment 1, segment 2...segment n are respectively input into the speech recognition model, and each acoustic feature is recognized by the speech recognition model to obtain the confidence that the speech segment corresponding to each acoustic feature is the target speech segment.

[0112] Furthermore, the segment with a confidence greater than a confidence threshold is determined as the target speech segment. For example, Figure 3 The target speech segments are segment 1, segment 4, segment 5, segment 7, segment 8, and segment 9; segment 2, segment 3, segment 6, and segment n are not target speech segments.

[0113] It should be noted that the preset duration and confidence threshold corresponding to each anchor can be set according to needs, and the embodiments of the present disclosure do not make specific limitations.

[0114] In the embodiment of the present disclosure, since the speech recognition model is trained based on the acoustic characteristics of the target person, it has the ability to recognize the acoustic characteristics of the target person. Therefore, when the speech recognition model is used to obtain the target speech segment in the speech signal to be recognized, compared with manual recognition, it will not miss or misrecognize, and the accuracy rate is higher.

[0115] In addition, in actual applications, by identifying the voice clips of specific people in the audio data through the speech recognition model, the influence of the voice clips of non-specific people can be eliminated. Then, in the subsequent review process or excerpting process, only these target voice clips need to be reviewed or excerpted, without the need for manual listening to the entire audio data, which can further improve the processing efficiency of audio data.

[0116] Figure 4 Schematic diagram of the process of the speech recognition method provided in the embodiment of the present disclosure Figure 2 This embodiment will be Figure 2The embodiment shown is described in more detail. Specifically, the speech recognition method of this embodiment includes the following steps:

[0117] S401: Acquire acoustic features of a speech signal to be recognized.

[0118] S402: Determine the confidence level of each speech segment included in the speech signal to be recognized as a target speech segment based on the speech recognition model and acoustic features.

[0119] S403: Determine that the speech segment with a confidence level less than a confidence level threshold is not a target speech segment.

[0120] It should be noted that steps S401 to S403 are Figure 2 The principles and effects of steps S201 to S202 in the illustrated embodiment are similar, and reference may be made to the above embodiment for details, which will not be described in detail here.

[0121] S404: Identify a target speech segment in the speech signal to be recognized according to the first speech segment whose confidence is greater than or equal to the confidence threshold.

[0122] In some embodiments, step S404 may be performed as follows: Figure 2 The solution shown in step S203 of the illustrated embodiment determines the target speech segment, that is, determines the segment with a confidence level greater than a confidence level threshold as the target speech segment.

[0123] However, in order to improve the recognition efficiency of the speech recognition model and reduce the omission of the target speech segment in the speech signal to be recognized, the speech recognition model will try to identify as many speech segments similar to the target person as possible as the target speech segment during the recognition process (that is, segments with a confidence level greater than the confidence threshold may not be the target speech segment). Therefore, in the above speech recognition process, recognition errors may occur.

[0124] In view of this, in another embodiment provided by the present disclosure, voiceprint recognition can also be performed on speech segments with a confidence level greater than a confidence threshold using a voiceprint recognition model, thereby obtaining a more accurate recognition result. Next, this solution is described in conjunction with steps S4041-S4043:

[0125] S4041. Obtain a first speech segment whose confidence level is greater than a confidence level threshold.

[0126] S4042. Based on the acoustic features of the first voice segment, obtain the voiceprint features of the first voice segment.

[0127] Specifically, the acoustic features of the first voice segment are input into the target voiceprint recognition model to obtain the voiceprint features of the first voice segment, wherein the target voiceprint recognition model is trained based on the voiceprint features of the target person. As for the training process of the target voiceprint recognition model, it is shown in subsequent embodiments.

[0128] S4043. Identify whether the first voice segment is a target voice segment based on the voiceprint feature.

[0129] In some embodiments, after obtaining the voiceprint feature, the confidence level of the voiceprint feature being the target person's voiceprint feature can be further determined through the voiceprint recognition model, thereby determining whether the first voice segment corresponding to the voiceprint feature is the target voice segment of the target person based on the confidence level.

[0130] In other embodiments, the first voice segment corresponding to the voiceprint feature may be compared with the voiceprint feature stored in the database to determine whether it is the voiceprint feature of the target person, which specifically includes the following steps:

[0131] (1) Compare the voiceprint features with those in the database;

[0132] Among them, the database contains the voiceprint features of the target person.

[0133] (2) if it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, then determining the first voice segment as the target voice segment;

[0134] (3) If it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, it is determined that the first voice segment is not the target voice segment.

[0135] At this point, the acquisition of the target speech segment has been completed.

[0136] As described in the above embodiment, since a segment of audio data may contain a target person's voice segment (e.g. Figure 3 Segments 1, 4, 5, 7, 8, and 9 shown in the figure may also contain voice segments of non-target persons (e.g. Figure 3 As shown in the fragments 2, 3, 6 and n), the voice fragments of non-target persons will seriously affect the review efficiency or the extraction efficiency.

[0137] In view of this, in the embodiment of the present disclosure, the time information of the target person's voice segment (i.e., the target voice segment) can also be obtained to accurately perform relevant processing on the target person's voice segment. Next, this solution is explained in conjunction with steps S405-S406:

[0138] S405: Determine first time information of each target speech segment in the speech signal to be recognized.

[0139] The first time information includes the start and end time of each target voice segment.

[0140] In some embodiments, after obtaining the start and end time of each target voice segment, the start and end time of each target voice segment can be output to relevant personnel (for example, audio and video reviewers or excerpters, etc.), so that the relevant personnel can review or excerpt the corresponding target voice segment based on the start and end time of each target voice segment.

[0141] In other embodiments, before outputting the start and end time of each target speech segment, the target speech segment may be further processed, such as merging, filtering, etc. Next, this solution is described in detail in conjunction with step S406:

[0142] S406 : Merge temporally consecutive target speech segments according to the first time information of each target speech segment to obtain speech segments to be processed and second time information corresponding to each speech segment to be processed.

[0143] Figure 5 Schematic diagram of the principle of the speech recognition method provided in the embodiment of the present disclosure Figure 2 .like Figure 5 As shown, adjacent target speech segments are merged, for example, segment 4 and segment 5 are merged to obtain speech segment 1 to be processed, and segments 7, 8 and 9 are merged to obtain speech segment 2 to be processed, wherein the segments that are not merged are treated as separate speech segments to be processed, for example, segment 1 is speech segment 3 to be processed.

[0144] It should be understood that after obtaining the start and end time of the target voice segment, the start and end time of the non-target person's voice segment is also known accordingly. Optionally, the non-target person's voice segments can also be merged according to the start and end time of the non-target person's voice segment in the same way, for example, Figure 5 The fragments 2 and 3 shown in FIG are merged.

[0145] In actual applications, since the speeches of the same specific person may be relatively concentrated, adjacent target voice segments may all be voice segments of the specific person, but the audio data corresponding to the same information may be located in different target voice segments. Compared with reviewing or excerpting the target voice segments one by one, reviewing or excerpting the merged voice segments to be processed can prevent important information from being missed. In addition, by merging processing, compared with reviewing or excerpting one by one, processing efficiency can also be improved.

[0146] In some embodiments, since the speaking duration of a specific person is random, the lengths of the voice segments to be processed after merging are also different. For example, voice segment 1 to be processed (segments 4 and 5) has two anchors, voice segment 2 to be processed (segments 7, 8, and 9) has three anchors, and voice segment 3 to be processed (segment 1) has one anchor. Since there is relatively less available information in the shorter voice segments to be processed, it may cause greater interference to the subsequent review or excerpting process, thereby reducing the efficiency of the review or excerpting.

[0147] In this embodiment, after the merging process, the second time information of each voice segment to be processed can also be obtained based on the first time information, wherein the second time information includes: the start and end time of each voice segment to be processed, and / or the duration of each voice segment to be processed.

[0148] Furthermore, the target filtering segment is determined according to the relationship between the duration of the speech segment to be processed and the preset duration. For example, the segment to be processed whose duration is less than the preset duration can be determined as the target filtering segment.

[0149] Among them, the preset duration can be set based on experience. For example, the preset duration can be the shortest duration of audio that can be recognized by the human ear, or it can be the pronunciation duration of a character, such as 0.1 seconds, 0.3 seconds, 0.5 seconds, etc.

[0150] For example, taking the preset duration as 0.2 seconds and the length of each anchor as 0.1 seconds, the duration of the voice segment 1 to be processed is 0.2 seconds, the duration of the voice segment 2 to be processed is 0.3 seconds, and the duration of the voice segment 3 to be processed is 0.1 seconds. Among them, the duration of the voice segment 3 to be processed is less than the preset duration, that is, the voice segment 3 to be processed is determined as the target filtering segment.

[0151] Furthermore, the target filtering segment is filtered, and the filtered voice segment to be processed is output to relevant personnel, so that the relevant personnel can review or extract the filtered voice segment to be processed.

[0152] Still taking the above example, in this step, the voice segment 3 to be processed (ie, segment 1) is filtered, and then the second time information corresponding to the voice segment 2 to be processed and the voice segment 3 to be processed is output to the relevant personnel.

[0153] In this embodiment, by filtering the shorter voice segments to be processed, the interference of voice segments with less information can be further reduced, thereby improving the review or excerpting efficiency.

[0154] Next, combine Figure 6 、 Figure 7 The structure of the voiceprint recognition model and its training process in the above embodiment are described in detail:

[0155] Figure 6 This is a schematic diagram of the structure of the voiceprint recognition model provided by the embodiment of the present disclosure. Figure 6 As shown in the figure, the voiceprint recognition model includes: a first convolution module (Conv1D), a coordinate attention-based residual module (CA-Resnetblock), a self-attention mechanism (self-attention), a second convolution module (Conv2D), an attention statistics pool (Attentive Statistics Pooling, ASP), a fully connected layer (Fully Connected) and a cyclic loss function module (Circle loss).

[0156] Among them, the first convolution module is a one-dimensional convolution module, and the second convolution module is a two-dimensional convolution module, both of which are used to obtain local features of the acoustic features of the sample audio signal. Through the above two convolution modules, the voiceprint recognition model can accurately extract local features in the acoustic features.

[0157] A coordinate attention-based residual module is used to obtain the weight values ​​of the convolution channels in the first convolution module and the second convolution module.

[0158] In this embodiment, the coordinate attention-based residual module can better capture the importance relationship between each convolution channel and adjust the weight value corresponding to each convolution channel compared to the existing residual module. Therefore, during the training process, the voiceprint recognition model can better learn the convolution channels that are more helpful for learning, thereby improving the accuracy of the voiceprint recognition model.

[0159] The self-attention mechanism is used to obtain the target features from the extracted acoustic features; the attention statistical pool is used to aggregate the acoustic features of variable length into the acoustic features of fixed length; the fully connected layer is used to output the recognition results of the voiceprint features corresponding to the acoustic features based on the local features, target features and fixed-length acoustic features.

[0160] The cyclic loss function module is used to obtain a cyclic loss function based on the recognition result output by the fully connected layer, wherein the cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

[0161] In this embodiment, by using a cyclic loss function to train the voiceprint recognition model, the distance between voiceprint features in the same category can be reduced, and the distance between voiceprint features in different categories can be increased, thereby improving the accuracy of the voiceprint recognition model.

[0162] Next, the training process of the above voiceprint recognition model is described in detail with reference to specific embodiments:

[0163] Figure 7 This is a flow chart of the voiceprint recognition model training method provided by the embodiment of the present disclosure. Figure 7 As shown, the training method of the voiceprint recognition model includes the following steps:

[0164] S701: Acquire multiple sample audio signals corresponding to a target person.

[0165] It should be noted that the embodiment of the present disclosure does not specifically limit the type of the sample audio signal. For example, it can be a custom sample data set or a public data set, such as VoxCeleb1, VoxCeleb2, and cn-Celeb, etc., which are not shown one by one here.

[0166] S702: Acquire acoustic features of a sample audio signal.

[0167] The acoustic features may be MFCC features. As for the method of extracting the MFCC features, reference may be made to the solutions in the prior art, which will not be described in detail here.

[0168] S703: Based on the cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signal to obtain a target voiceprint recognition model.

[0169] Specifically, the acoustic features of the sample audio signal are input into Figure 6 Iterative training is performed in the voiceprint recognition model shown. Based on the fully connected layer and the cyclic loss function module, a cyclic loss function is obtained for each training. The parameters of the model are adjusted by the cyclic loss function. When the cyclic loss function converges, the model obtained by the current training is determined to be the target voiceprint recognition model.

[0170] In one possible implementation, since the scenarios in the sample data may be relatively simple, the accuracy of the trained target voiceprint recognition model may be insufficient. However, during the voiceprint recognition process, a large amount of noise signals may exist in the speech signal to be recognized, making it impossible for the target voiceprint recognition model to accurately recognize the voice.

[0171] In view of this, in the embodiment of the present disclosure, noise data can also be added to the initial sample audio signal, so that the voiceprint recognition model can be trained by the sample audio signal with added noise data to improve the recognition ability of the target voiceprint recognition model and further improve the accuracy of the recognition results.

[0172] Specifically, the above step S701 specifically includes the following steps:

[0173] (1) obtaining a plurality of initial sample audio signals corresponding to a target person;

[0174] (2) Noise data is added to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person.

[0175] In some embodiments, the noise data may be any type of noise data.

[0176] In other embodiments, since different target persons are in different scenes, the real noise contained in the scene is also different. For example, if the target person is a target person and the audio acquisition scene is a speech scene, the real noise data of the scene may be: reverberation sound, camera shutter sound, etc.

[0177] Therefore, in one embodiment of the present disclosure, noise data may also be determined based on the identity information of the target person or scene information.

[0178] Taking identity information as an example, the specific solution for step (2) above is as follows:

[0179] First, obtain the target person's identity information;

[0180] Then, according to the identity information of the target person to which the initial sample audio signal belongs, target noise data corresponding to the identity information is determined;

[0181] Specifically, according to the correspondence between the identity information and the noise data, the target noise data corresponding to the target person to whom the initial sample audio signal belongs is determined.

[0182] Finally, target noise data is added to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person.

[0183] For example, before training the voiceprint recognition model, the identity information of the target person is first determined, and then the type of noise data corresponding to the target person is determined based on the correspondence between the target person and the noise data. For example, the noise data corresponding to the target person can be noise data of types such as reverberation sound and camera shutter sound.

[0184] Furthermore, according to the type of noise data, a search can be performed in a noise database or the Internet to obtain reverberation sound and / or camera shutter sound, and the obtained reverberation sound and / or camera shutter sound can be superimposed on the initial sample audio signal to obtain a sample audio signal.

[0185] In this solution, by adding noise data to the initial sample audio signal and then training the voiceprint recognition model with the added noise data, we can simulate the ambient noise at the pronunciation site. This enables the target voiceprint recognition model to accurately extract voiceprint features from noisy environments, further improving the accuracy of voiceprint recognition results. Furthermore, by determining the type of target noise data based on the target person's identity information, the added noise data is more consistent with the actual pronunciation environment, further improving the recognition capabilities of the voiceprint recognition model.

[0186] In actual applications, since the voiceprint features of different target persons are different, the higher the similarity of the voiceprint features, the more difficult it is for the voiceprint recognition model to identify them. This makes it easy for the voiceprint recognition model to misidentify audio signals with high similarity of voiceprint features.

[0187] The inventors discovered that during the training process, the higher the similarity between the voiceprint features of the sample audio signals, the better the recognition effect of the voiceprint recognition model. However, during the training process, the sample audio signals used are random. If the number of difficult samples (i.e., samples with high similarity) in the sample audio signals is small, the recognition effect of the trained voiceprint recognition model will be poor.

[0188] In view of this, the present disclosure creatively proposes a training process for a voiceprint recognition model to solve the above problems. The following describes the process in detail with reference to specific embodiments:

[0189] (1) Determine S first target persons among the target persons;

[0190] Among them, the S first target persons can be any persons among all the target persons.

[0191] The present embodiment does not limit the source of the sample audio data. For example, the sample audio data can be all the data in the public datasets VoxCeleb1, VoxCeleb2, and cn-Celeb (containing voice data of 10,363 target persons in total, of which VoxCeleb1 contains voice data of 1,251 target persons, VoxCeleb2 contains voice data of 6,112 target persons, and cn-Celeb contains voice data of 3,000 target persons). The following uses this as an example to illustrate the above steps:

[0192] First, determine any S first target persons from the public datasets VoxCeleb1, VoxCeleb2, and cn-Celeb, for example: first target person R1, first target person R2…first target person R S .

[0193] (2) For each first target person, according to the similarity matrix, determine N second target persons from multiple target persons whose voiceprint feature similarity with the first target person is greater than a similarity threshold.

[0194] The similarity matrix is ​​used to indicate the similarity of voiceprint features between different target persons. As for the method of obtaining the similarity matrix, it will be shown in the subsequent embodiments.

[0195] In this step, for each first target person, according to the similarity matrix, N second target persons are obtained from 10362 target persons (all target persons except the first target person), whose voiceprint feature similarity with the first target person is greater than the similarity threshold, for example: second target person R1', second target person R2'... second target person R N '.

[0196] (3) Based on the cyclic loss function, the initial voiceprint recognition model is trained through the acoustic features of the sample audio signals corresponding to each second target person to obtain the target voiceprint recognition model.

[0197] In this step, first, the sample audio signals corresponding to the second target persons are obtained. The embodiment of the present disclosure does not specifically limit the number of the sample audio signals corresponding to the second target persons.

[0198] In some embodiments, the sample audio signal corresponding to each second target person may be all sample audio signals of the second target person in the sample audio data. N ', for example, the corresponding sample audio signal sets are: K1, K2...K N , where K1 is the set of all sample audio signals of the second target person R1' in all sample audio signals, K2 is the set of all sample audio signals of the second target person R2' in all sample audio signals, and K N For all sample audio signals, the second target person R N 'The set of all sample audio signals, where K1, K2...K N The number of sample audio signals in the .

[0199] In other embodiments, when obtaining sample audio signals corresponding to the second target persons, the number of sample audio signals obtained for different second target persons is the same. For example, the second target persons R1', R2'...R N ', for example, the corresponding sets of sample audio signals are: K1, K2...K N , where K1, K2…K N The number of sample audio signals is the same.

[0200] Furthermore, based on the cyclic loss function, through K1, K2…K N The acoustic features corresponding to all the sample audio signals in the training set are used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

[0201] In this embodiment, since the similarity between the voiceprint features of the N second target persons and the voiceprint features of the first target person is greater than the similarity threshold, the similarity between the voiceprint features of the N second target persons is relatively high, so the voiceprint features of the sample audio signals from them are also relatively similar, that is, K1, K2...K N The voiceprint features corresponding to the sample audio signals in are more similar.

[0202] Therefore, through K1, K2…K N When the initial voiceprint recognition model is trained with the sample audio signals in the dataset, the model can not only fit the simple sample space, but also take into account the difficult sample space, thereby improving the voiceprint recognition model's ability to recognize audio signals with high similarity.

[0203] In addition, through the similarity matrix of the target person's voiceprint features, we can quickly obtain a second target person with a higher similarity to the voiceprint features of the first target person, and thus obtain sample audio data with a higher similarity based on the second target person. In this process, there is no need to calculate the similarity between each sample audio data in the data set one by one, and sample audio data with a higher similarity can be obtained. The process is relatively simple and can further improve training efficiency.

[0204] In some embodiments, the similarity matrix between different target persons can be obtained by the following steps:

[0205] (1) Inputting the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model, and obtaining the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model;

[0206] Specifically, the acoustic features of the sample audio signal corresponding to the target person are input into the initial voiceprint recognition model, the normalization parameter W of the fully connected layer in the initial voiceprint recognition model is obtained, and each row of the normalization parameter W is determined to be the voiceprint feature center of a target person.

[0207] (2) Determine the similarity matrix between target persons based on the voiceprint feature centers corresponding to each target person.

[0208] Specifically, the similarity matrix between the voiceprint features of each target person is obtained according to the following formula:

[0209] Similarity=WW T

[0210] Wherein, Similarity(i, j) represents the similarity between the voiceprint features of the i-th target person and the j-th target person, and W is the center of the voiceprint feature of each target person.

[0211] In this embodiment, since the initial voiceprint recognition model has a certain voiceprint recognition capability, the voiceprint feature center extracted by it can accurately express the voiceprint characteristics of the target person, and then an accurate similarity matrix is ​​obtained through the voiceprint feature center to obtain sample audio data with higher similarity, which can improve the voiceprint recognition model's recognition capability for high-similarity speech signals.

[0212] In addition, compared to obtaining the similarity between each sample audio signal one by one, in this solution, since the voiceprint recognition model is more efficient in extracting the voiceprint feature center, it can quickly obtain sample audio signals with higher similarity, ultimately improving training efficiency.

[0213] In actual applications, the sample audio signal may include multiple types of samples. For example, based on language type, it may include: Chinese samples, English samples, or samples of other languages, etc. Based on target persons, it may include: audio samples of female target persons, audio samples of male target persons. As for other classification methods, they will not be described here one by one.

[0214] Since the number of different samples in a sample type is different, for example, in the public datasets VoxCeleb1, VoxCeleb2, and cn-Celeb, VoxCeleb1 and VoxCeleb2 are English datasets, and cn-Celeb is a Chinese dataset, the number of English samples is much larger than that of Chinese samples. If the public dataset is directly used for training, the feature space expression ability of the voiceprint features is limited due to the small number of Chinese samples. As a result, the trained voiceprint recognition model will have a higher ability to recognize English audio signals, but a lower ability to recognize Chinese audio signals, resulting in inaccurate recognition results for Chinese audio signals.

[0215] In view of this, during the training process of the voiceprint recognition model in step S703, the present disclosure creatively proposes the following two implementation methods to solve the above problems:

[0216] For example, the sample audio signals include a first sample set (e.g., an English sample set) and a second sample set (e.g., a Chinese sample set), wherein the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0217] Method 1: First, for the i-th training, the i-1th voiceprint recognition model is trained using the acoustic features of the M1 first sample audio signals in the first sample set and the acoustic features of the M2 second sample audio signals in the second sample set to obtain the i-th cyclic loss function and the i-th voiceprint recognition model, where the i-1th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times;

[0218] Furthermore, the voiceprint recognition model corresponding to the convergence of the cyclic loss function is determined to be the target voiceprint recognition model.

[0219] Wherein, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

[0220] It should be understood that the number of M2 should be less than or equal to the total number of samples in the second sample set, and the difference between the numbers of M1 and M2 should be less than or equal to the preset value, so as to keep the gap between different samples (for example, English samples and Chinese samples) within a reasonable range during each model training process, thereby avoiding the situation where the voiceprint recognition model has different recognition capabilities for different samples.

[0221] The preset value can be set based on experience, for example, 0, 500, 1000, etc. Preferably, the preset value can be set to 0, that is, the same number of first sample audio signals and second sample audio signals are used for model training, so that the voiceprint recognition model has the same recognition ability for different samples. As for the specific values ​​of M1 and M2, the embodiment of the present disclosure does not limit them.

[0222] For example, taking the public data sets of VoxCeleb1, VoxCeleb2, and cn-Celeb, and M1 and M2 both being 3000 as an example (i.e., the first sample set is VoxCeleb1+VoxCeleb2, and the second sample set is cn-Celeb), in the first training, any 3000 first sample audio signals (1 to 3000) are obtained from the first sample set, and 3000 second sample audio signals (1 to 3000) are obtained from the second sample set. The initial voiceprint recognition model is trained for the first time using the first sample audio signals (1 to 3000) and the second sample audio signals (1 to 3000), to obtain a second voiceprint recognition model and a second cyclic loss function;

[0223] During the second training process, 3000 first sample audio signals (3001 to -6000) are obtained from the first sample set, and 3000 second sample audio signals (1 to 3000) are obtained from the second sample set. The second voiceprint recognition model is trained by the first sample audio signals (3001 to 6000) and the second sample audio signals (1 to 3000) to obtain the third voiceprint recognition model and the third cyclic loss function.

[0224] Iterative training is performed according to the above method until the output cyclic loss function converges, and the voiceprint recognition model corresponding to the cyclic loss function is determined as the target voiceprint recognition model. As for other rounds of training, they will not be described here one by one.

[0225] It should be noted that the above training process is illustrated by taking M2 as the total number of samples in the second sample set as an example. When M2 is less than the total number of samples in the second sample set, the second sample audio signal can also be cyclically obtained in the same manner as the first sample audio signal is obtained from the first sample set. For example, when M2 is 1000, in the first training, the second sample audio signal is (1~1000); in the second training, the second sample audio signal is (1001~2000); in the third training process, the second sample audio signal is (2001~3000), and in the fourth training, the second sample audio signal is (1~1000)...

[0226] In this embodiment, through the above training method, since the difference in the number of the first sample audio signal and the second sample audio signal used in each training process is less than or equal to a preset value, the difference between different samples (for example, English samples and Chinese samples) can be maintained within a reasonable range. This solves the technical problem of the limited feature space expression ability of the voiceprint features due to the small number of samples, avoids the situation where the voiceprint recognition model has different recognition capabilities for different samples, and thus improves the performance of the target voiceprint recognition model. In addition, it can also avoid the phenomenon of overfitting of the voiceprint recognition model due to the small number of sample data, thereby further improving the model performance.

[0227] Method 2: Based on the cyclic loss function, the initial voiceprint recognition model is trained by the acoustic features of the sample audio signals in the first sample set to obtain the first voiceprint recognition model.

[0228] Based on the cyclic loss function, the first voiceprint recognition model is trained using the acoustic features of the sample audio signals in the second sample set to obtain a target voiceprint recognition model.

[0229] For example, still taking VoxCeleb1+VoxCeleb2 as the first sample set and cn-Celeb as the second sample set, firstly, based on the cyclic loss function, iterative training is performed through all the sample speech data in VoxCeleb1 and VoxCeleb2 to obtain the model and cyclic loss function value obtained in each training. When the cyclic loss function value converges, the model corresponding to the cyclic loss function value is determined to be the first voiceprint recognition model.

[0230] Furthermore, the first voiceprint recognition model is trained using all sample speech data in cn-Celeb to obtain the model and cyclic loss function value obtained in each training. When the cyclic loss function value converges, the model corresponding to the cyclic loss function value is determined to be the target voiceprint recognition model.

[0231] In this embodiment, the first voiceprint recognition model obtained by training with the sample audio signals in the first sample set has the ability to recognize the first sample audio signal. By training the first voiceprint recognition model with the sample audio signals in the second sample set, the first voiceprint recognition model can be further enabled to have the ability to recognize the second sample audio signal, thereby obtaining a voiceprint recognition model with the ability to recognize two types of sample audio signals. This can avoid the situation where the voiceprint recognition model has different recognition capabilities for different samples, thereby ensuring the performance of the target voiceprint recognition model.

[0232] Next, combine Figure 8 、 Figure 9 The structure of the speech recognition model and its training process in the above embodiment are described in detail:

[0233] Figure 8 This is a schematic diagram of the structure of the speech recognition model provided in the embodiment of the present disclosure. Figure 8 As shown in the figure, the speech recognition model includes: convolutional neural network, recurrent neural network, self-attention mechanism and fully connected layer;

[0234] Among them, the convolutional neural network is used to extract local features from acoustic features, the recurrent neural network is used to extract global features from acoustic features, the self-attention mechanism is used to extract target features from acoustic features, and the fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is the target speech segment based on local features, global features and target features.

[0235] In this embodiment, a convolutional neural network can quickly extract local features from acoustic features, thereby improving model training efficiency. However, due to the small receptive field of convolutional neural networks, it is difficult to capture the global signal in the speech signal. Therefore, a recurrent neural network is added to this solution to better capture global features and improve the performance of the speech recognition model.

[0236] In addition, by introducing the self-attention mechanism layer, the model can learn the importance of which time segments and which frequency ranges, and obtain the final feature vector that is beneficial to classification through weighted averaging, further improving the performance of the speech recognition model.

[0237] Figure 9 Schematic diagram of the flow of the method for training the speech recognition model provided in the embodiment of the present disclosure. Figure 9 As shown, the training method includes the following steps:

[0238] S901: Acquire multiple sample audio signals corresponding to a target person.

[0239] S902: Acquire acoustic features of the sample audio signal.

[0240] It should be noted that steps S901 to S902 are Figure 7 Steps S701-S702 in the illustrated embodiment are similar, and specific solutions can be referred to the above embodiment, which will not be described in detail here.

[0241] In some embodiments, in training the voiceprint recognition model and the speech recognition model, since the acoustic features of the sample audio signal are both used, in actual applications, if it is necessary to train the above two models, the acoustic features of the obtained sample audio signal can be reused, that is, the training process of the two models uses the acoustic features of the same sample audio signal for training, which can simplify the training process and improve training efficiency.

[0242] S903: Input the acoustic features of the sample audio signal into the target voiceprint recognition model to obtain the voiceprint features corresponding to the sample audio signal output by the target voiceprint recognition model.

[0243] Among them, the target voiceprint recognition model is based on Figure 7 The training process can refer to the above embodiment and will not be described here in detail.

[0244] S904: Obtain label data of acoustic features according to the voiceprint features.

[0245] The label data of the acoustic feature is used to indicate the target person to which the acoustic feature belongs.

[0246] It should be understood that there are many ways to obtain label data of acoustic features, and the embodiments of the present disclosure do not make specific limitations. On the one hand, the acoustic features can be labeled by manual labeling to label the target person described by each acoustic feature, thereby obtaining label data of the acoustic features.

[0247] However, manual labeling is inefficient. In view of this, the present disclosure provides an automated labeling method to solve the above problem. The following describes step S904 in detail with reference to specific steps:

[0248] Specifically, the above step S904 includes the following steps:

[0249] (1) Clustering the voiceprint features to obtain the category to which each voiceprint feature belongs and the cluster center of each category;

[0250] (2) Determine the label data of the voiceprint features in each category based on the label data of the cluster center;

[0251] (3) Obtain the label data of the acoustic feature based on the label data of the voiceprint feature.

[0252] In this embodiment, a clustering algorithm can be used to cluster the voiceprint features. Clustering algorithms include, for example, the k-means algorithm, the DBSCAN algorithm, the OPTICS algorithm, and the like. As for the specific method of using the above clustering algorithms to cluster and obtain the category to which each voiceprint feature belongs and the cluster center of each category, reference can be made to the existing technology and will not be repeated here.

[0253] For easier understanding, please refer to Figure 10 , Figure 10 This is a schematic diagram of the clustering results of the voiceprint features provided by the embodiment of the present disclosure. Figure 10 As shown in the figure, after clustering the voiceprint features, three categories are obtained: category A, category B and category C. The cluster center of category A is cent-A, the cluster center of category B is cent-B, and the cluster center of category C is cent-C. At the same time, the label data of each cluster center is obtained.

[0254] The embodiment of the present application does not specifically limit the method of obtaining the label data of each cluster center. For example, the label data of the cluster center can be manually determined by auditioning. That is, after obtaining the cluster center of each category, the audio signal corresponding to the cluster center can be auditioned to determine the target person corresponding to the cluster center. As for other methods, they will not be listed here one by one.

[0255] Next, the distance between the voiceprint feature in each category and the cluster center is calculated (eg, Euclidean distance or cosine distance, etc.). For example, the distance between each voiceprint feature in category A and cent-A is determined.

[0256] Furthermore, target voiceprint features whose distance is less than or equal to a preset distance are determined, and the label data of the target voiceprint features is determined to be the label data of the cluster center. For example, taking cent-A as an example, if the label data of cent-A is: target person A, the label data of target voiceprint features whose distance from cent-A is less than the preset distance is determined to be target person A, and the label data of voiceprint features whose distance is greater than the preset distance is set to non-target persons.

[0257] Furthermore, based on the correspondence between the voiceprint feature and the acoustic feature, the acoustic feature corresponding to the target voiceprint feature is determined, and the label data of the target acoustic feature is determined to be the label data of the acoustic feature.

[0258] S905 . Train the initial speech recognition model using the label data of the acoustic features to obtain a target speech recognition model.

[0259] In this step, the label data of the acoustic features are input into the initial speech recognition model for training, and the loss function of each training output is obtained. It is determined that the model corresponding to the convergence of the loss function is the speech recognition model.

[0260] In this embodiment, during the training of the speech recognition model, voiceprint features are clustered to obtain cluster centers for multiple categories. The label data for each acoustic feature within that category is then determined based on the label data of the cluster centers. In this process, only the label data for the cluster centers needs to be determined. Compared to manual labeling, this method eliminates the need to audition all acoustic features, reducing manpower investment and improving model training efficiency. Furthermore, this method produces more accurate labeled data. Furthermore, by obtaining voiceprint features through a trained voiceprint recognition model, the extracted voiceprint features can accurately represent the target person's voiceprint characteristics, resulting in a highly accurate clustering result.

[0261] In some embodiments, if there are many audio signals of non-target persons in the sample audio signal, and the audio signals of non-target persons are similar, the voiceprint features corresponding to the audio signals of such non-target persons may be clustered into one category during clustering, and the speech recognition model obtained by training the speech recognition model using such audio signals is not accurate enough.

[0262] In view of this, some sample audio signals in each category can be verified by auditioning to determine whether the voiceprint features in this category are the voiceprint features corresponding to the audio signals of non-target persons. If so, such audio signals are filtered out to improve the performance of the trained speech recognition model.

[0263] Exemplary media

[0264] After introducing the method of the exemplary embodiment of the present disclosure, next, reference is made to Figure 11 A storage medium according to an exemplary embodiment of the present disclosure is described.

[0265] Figure 11 This is a schematic diagram of a program product provided by an embodiment of the present disclosure, refer to Figure 11 As shown, a program product 1100 for implementing the above method according to an embodiment of the present disclosure is described, which can be a portable compact disk read-only memory (CD-ROM) and includes program code and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0266] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0267] A readable signal medium may include a data signal transmitted in baseband or as part of a carrier wave, which carries readable program code. Such a transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium.

[0268] Program code for performing the operations disclosed herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0269] Exemplary devices

[0270] After introducing the method and medium of the exemplary embodiment of the present disclosure, next, reference is made to Figure 12 、 Figure 13 and Figure 14A device according to an exemplary embodiment of the present disclosure will be described.

[0271] Figure 12 This is a schematic diagram of the structure of the speech recognition device provided by the embodiment of the present disclosure. Figure 12 As shown, the speech recognition device 1200 includes:

[0272] An acquisition module 1201 is configured to acquire acoustic features of a speech signal to be recognized;

[0273] Determination module 1202, configured to determine, based on a speech recognition model and acoustic features, a confidence level that each speech segment contained in the speech signal to be recognized is a target speech segment, wherein the target speech segment is a speech signal uttered by a target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signal of the target person;

[0274] The recognition module 1203 is configured to recognize a target speech segment in the speech signal to be recognized based on the confidence level of each speech segment.

[0275] In one possible implementation, the speech recognition model is trained based on the acoustic characteristics of the target person as follows:

[0276] Acquiring acoustic features of multiple sample audio signals corresponding to a target person; inputting the acoustic features into a target voiceprint recognition model to obtain voiceprint features corresponding to the acoustic features output by the target voiceprint recognition model; obtaining label data of the acoustic features based on the voiceprint features, wherein the label data of the acoustic features is used to indicate the target person to which the acoustic features belong; training an initial speech recognition model using the label data of the acoustic features to obtain a speech recognition model.

[0277] In one possible implementation, the recognition module 1203 is specifically used to: identify a target speech segment in the speech signal to be recognized based on a first speech segment whose confidence is greater than or equal to a confidence threshold; or determine that a speech segment whose confidence is less than the confidence threshold is not a target speech segment.

[0278] In a possible implementation, the recognition module 1203 is specifically used to: obtain a first voice segment whose confidence is greater than a confidence threshold; obtain a voiceprint feature of the first voice segment based on the acoustic features of the first voice segment; and identify whether the first voice segment is a target voice segment based on the voiceprint feature.

[0279] In a possible implementation, the recognition module 1203 is specifically configured to input the acoustic features of the first voice segment into a target voiceprint recognition model to obtain the voiceprint features of the first voice segment, wherein the target voiceprint recognition model is trained based on the voiceprint features of the target person.

[0280] In one possible implementation, the recognition module 1203 is specifically used to: compare the voiceprint feature with the voiceprint feature in the database, where the database contains the voiceprint feature of the target person; if it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, then determine that the first voice segment is the target voice segment; if it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, then determine that the first voice segment is not the target voice segment.

[0281] In a possible implementation, the determination module 1202 is further configured to determine first time information of each target speech segment in the speech signal to be recognized, where the first time information includes the start and end time of each target speech segment.

[0282] In a possible embodiment, the speech recognition device 1200 also includes: a processing module 1204, which is used to merge the temporally continuous target speech segments according to the first time information of each target speech segment, and obtain the speech segments to be processed and the second time information corresponding to each speech segment to be processed, the second time information including: the start and end time of each speech segment to be processed, and / or the duration of each speech segment to be processed.

[0283] In one possible implementation, the speech recognition model includes: a convolutional neural network, a recurrent neural network, a self-attention mechanism, and a fully connected layer;

[0284] Among them, the convolutional neural network is used to extract local features from acoustic features; the recurrent neural network is used to extract global features from acoustic features; the self-attention mechanism is used to extract target features from acoustic features; and the fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is the target speech segment based on local features, global features and target features.

[0285] The above-mentioned speech recognition device provided by the embodiment of the present disclosure can implement each process in the above-mentioned speech recognition method embodiment and achieve the same functions and effects, which will not be described in detail here.

[0286] Figure 13 A schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure Figure 1 .like Figure 13 As shown, the model training device 1300 includes:

[0287] An acquisition module 1301 is configured to acquire a plurality of sample audio signals corresponding to a target person and acquire acoustic features of the sample audio signals;

[0288] The training module 1302 is used to train the initial voiceprint recognition model based on the acoustic features of the sample audio signal based on the cyclic loss function to obtain the target voiceprint recognition model.

[0289] In one possible implementation, the voiceprint recognition model includes: a first convolution module, a residual module based on coordinate attention, a self-attention mechanism, a second convolution module, an attention statistics pool, a fully connected layer, and a cyclic loss function module;

[0290] Among them, the first convolution module and the second convolution module are used to obtain the local features of the acoustic features of the sample audio signal; the residual module based on coordinate attention is used to obtain the weight values ​​of the convolution channels in the first convolution module and the second convolution module; the self-attention mechanism is used to obtain the target features in the extracted acoustic features; the attention statistical pool is used to aggregate the acoustic features of variable length into the acoustic features of fixed length; the fully connected layer is used to output the recognition results of the voiceprint features corresponding to the acoustic features based on the local features, target features and fixed-length acoustic features; the cyclic loss function module is used to obtain the cyclic loss function based on the recognition results output by the fully connected layer, and the cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

[0291] In one possible implementation, the training module 1302 is specifically used to: determine S first target persons among the target persons; for each first target person, determine N second target persons from multiple target persons whose voiceprint feature similarity with the first target person is greater than a similarity threshold according to a similarity matrix, and the similarity matrix is ​​used to indicate the voiceprint feature similarity between different target persons, where S and N are both integers greater than or equal to 1; based on a cyclic loss function, the initial voiceprint recognition model is trained through the acoustic features of the sample audio signals corresponding to each second target person to obtain a target voiceprint recognition model.

[0292] In a possible implementation, the model training device 1300 further includes: a determination module 1303;

[0293] The acquisition module 1301 is further configured to: input the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model, and obtain the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model;

[0294] The determination module 1303 is used to determine the similarity between the voiceprint features of each target person according to the voiceprint feature center corresponding to each target person; and determine the similarity matrix between the target persons according to the similarity between the voiceprint features of each target person.

[0295] In a possible implementation, the acquisition module 1301 is specifically configured to: acquire multiple initial sample audio signals corresponding to the target person, and add noise data to the initial sample audio signals to obtain multiple sample audio signals corresponding to the target person.

[0296] In one possible implementation, the acquisition module 1301 is specifically used to: obtain the identity information of the target person; determine the target noise data corresponding to the identity information based on the identity information of the target person to which the initial sample audio signal belongs; and add the target noise data to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person.

[0297] In a possible implementation, the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0298] The training module 1302 is specifically configured to: for the i-th training, train the i-1-th voiceprint recognition model using the acoustic features of M1 first sample audio signals in the first sample set and the acoustic features of M2 second sample audio signals in the second sample set, obtain the i-th cyclic loss function and the i-th voiceprint recognition model, where the i-1-th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times; and determine that the voiceprint recognition model corresponding to the convergence of the cyclic loss function is the target voiceprint recognition model;

[0299] Wherein, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

[0300] In a possible implementation, the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set;

[0301] The training module 1302 is specifically configured to: train the initial voiceprint recognition model based on the acoustic features of the sample audio signals in the first sample set based on the cyclic loss function to obtain a first voiceprint recognition model;

[0302] Based on the cyclic loss function, the first voiceprint recognition model is trained using the acoustic features of the sample audio signals in the second sample set to obtain a target voiceprint recognition model.

[0303] The above-mentioned model training device provided in the embodiment of the present disclosure can implement each process in the aforementioned embodiment of the training method of the voiceprint recognition model and achieve the same functions and effects, which will not be repeated here.

[0304] Figure 14 A schematic diagram of the structure of the model training device provided in the embodiment of the present disclosure Figure 2 .like Figure 14 As shown, the model training device 1400 includes:

[0305] An acquisition module 1401 is configured to acquire a plurality of sample audio signals corresponding to a target person and acquire acoustic features of the sample audio signals;

[0306] The voiceprint recognition module 1402 is configured to input the acoustic features of the sample audio signal into a target voiceprint recognition model to obtain the voiceprint features corresponding to the sample audio signal output by the target voiceprint recognition model. The target voiceprint recognition model is trained using the voiceprint recognition model training method described in the above embodiment.

[0307] The label data acquisition module 1403 is used to obtain label data of acoustic features according to the voiceprint features, and the label data of acoustic features is used to indicate the target person to whom the acoustic features belong;

[0308] The training module 1404 is used to train the initial speech recognition model using the label data of the acoustic features to obtain a speech recognition model.

[0309] In one possible implementation, the label data acquisition module 1403 is specifically used to: cluster the voiceprint features to obtain the category to which each voiceprint feature belongs and the cluster center of each category; determine the label data of the voiceprint features in each category based on the label data of the cluster center; and obtain the label data of the acoustic features based on the label data of the voiceprint features.

[0310] The above-mentioned model training device provided in the embodiment of the present disclosure can implement each process in the aforementioned speech recognition model training method embodiment and achieve the same functions and effects, which will not be repeated here.

[0311] Exemplary Computing Devices

[0312] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, Figure 15 The computing device of the exemplary embodiment of the present disclosure is described. It should be understood that Figure 15 The computing device 1500 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0313] Figure 15 This is a schematic diagram of the structure of the computing device provided in the embodiment of the present disclosure. Figure 15 As shown, computing device 1500 is implemented as a general-purpose computing device. Components of computing device 1500 may include, but are not limited to, the aforementioned at least one processing unit 1501, the aforementioned at least one storage unit 1502, and a bus 1503 connecting various system components (including processing unit 1501 and storage unit 1502).

[0314] The bus 1503 includes a data bus, a control bus, and an address bus.

[0315] The storage unit 1502 may include readable media in the form of volatile memory, such as random access memory (RAM) 1512 and / or cache memory 1522 , and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 1532 .

[0316] The storage unit 1502 may also include a program / utility 1552 having a set (at least one) of program modules 1542, such program modules 1542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0317] The computing device 1500 may also communicate with one or more external devices 1504 (e.g., a keyboard, a pointing device, etc.). Such communication may be performed via an input / output (I / O) interface 1505. Furthermore, the computing device 1500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 1506. Figure 15 As shown, network adapter 1506 communicates with other modules of computing device 1500 via bus 1503. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computing device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0318] It should be noted that although several units / modules or sub-units / modules of the timing update device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in a single unit / module. Conversely, the features and functions of a single unit / module described above can be further divided and embodied by multiple units / modules.

[0319] Furthermore, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0320] Although the spirit and principles of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the specific embodiments disclosed, and the division into various aspects does not mean that the features in these aspects cannot be combined to benefit. Such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A speech recognition method, comprising: Acquiring acoustic features of a speech signal to be recognized; Determining, based on a speech recognition model and the acoustic features, a confidence level that each speech segment contained in the speech signal to be recognized is a target speech segment, wherein the target speech segment is a speech signal uttered by a target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signals of the target person; Identifying a target speech segment in the speech signal to be recognized based on the confidence level of each speech segment; The speech recognition model is trained based on the acoustic characteristics of the target person as follows: Acquiring acoustic features of a plurality of sample audio signals corresponding to a target person; Inputting the acoustic feature into a target voiceprint recognition model to obtain a voiceprint feature corresponding to the acoustic feature output by the target voiceprint recognition model; Obtaining label data of the acoustic feature according to the voiceprint feature, wherein the label data of the acoustic feature is used to indicate the target person to whom the acoustic feature belongs; Training an initial speech recognition model using the label data of the acoustic features to obtain the speech recognition model; The target voiceprint recognition model is obtained through the following steps: Acquire multiple sample audio signals corresponding to the target person; Acquiring acoustic features of the sample audio signal; Determining S first target persons among the target persons; For each of the first target persons, determine, from the plurality of target persons, N second target persons whose voiceprint feature similarity to the first target person is greater than a similarity threshold according to a similarity matrix, where S and N are both integers greater than or equal to 1; Based on the cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals corresponding to each of the second target persons to obtain the target voiceprint recognition model.

2. The speech recognition method according to claim 1, wherein identifying the target speech segment in the speech signal to be recognized based on the confidence level comprises: identifying a target speech segment in the speech signal to be recognized according to the first speech segment whose confidence is greater than or equal to a confidence threshold; Alternatively, it is determined that the speech segment with a confidence level less than the confidence level threshold is not the target speech segment.

3. The speech recognition method according to claim 2, wherein identifying the target speech segment in the speech signal to be recognized based on the first speech segment having a confidence level greater than or equal to a confidence threshold comprises: Acquire a first speech segment whose confidence level is greater than the confidence level threshold; Acquiring a voiceprint feature of the first voice segment based on the acoustic feature of the first voice segment; According to the voiceprint feature, it is identified whether the first voice segment is a target voice segment.

4. The speech recognition method according to claim 3, wherein obtaining the voiceprint feature of the first speech segment based on the acoustic feature of the first speech segment comprises: The acoustic features of the first voice segment are input into a target voiceprint recognition model to obtain the voiceprint features of the first voice segment, wherein the target voiceprint recognition model is trained based on the voiceprint features of the target person.

5. The speech recognition method according to claim 3, wherein identifying whether the first speech segment is a target speech segment based on the voiceprint feature comprises: comparing the voiceprint feature with voiceprint features in a database, the database containing the voiceprint features of the target person; If it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, determining the first voice segment as a target voice segment; If it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, it is determined that the first voice segment is not a target voice segment.

6. The speech recognition method according to any one of claims 1 to 5, further comprising: Determine first time information of each target speech segment in the speech signal to be recognized, where the first time information includes: a start and end time of each target speech segment.

7. The speech recognition method according to claim 6, further comprising: According to the first time information of each target voice segment, the target voice segments that are continuous in time are merged and processed to obtain the voice segments to be processed and the second time information corresponding to each voice segment to be processed. The second time information includes: the start and end time of each voice segment to be processed, and / or the duration of each voice segment to be processed.

8. The speech recognition method according to any one of claims 1 to 5, wherein the speech recognition model comprises: Convolutional neural networks, recurrent neural networks, self-attention mechanisms, and fully connected layers; The convolutional neural network is used to extract local features from the acoustic features; The recurrent neural network is used to extract global features from the acoustic features; The self-attention mechanism is used to extract target features from the acoustic features; The fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is a target speech segment based on the local features, the global features and the target features.

9. A model training method comprising: Acquire multiple sample audio signals corresponding to the target person; Acquiring acoustic features of the sample audio signal; Based on the cyclic loss function, the initial voiceprint recognition model is trained by the acoustic features of the sample audio signal to obtain a target voiceprint recognition model; the target voiceprint recognition model is the target voiceprint recognition model in any one of the speech recognition methods of claims 1-8.

10. The model training method according to claim 9, wherein the initial voiceprint recognition model comprises: The first convolution module, the residual module based on coordinate attention, the self-attention mechanism, the second convolution module, the attention statistics pool, the fully connected layer, and the cyclic loss function module; The first convolution module and the second convolution module are used to obtain local features of the acoustic features of the sample audio signal; The coordinate attention-based residual module is used to obtain weight values ​​of convolution channels in the first convolution module and the second convolution module; The self-attention mechanism is used to obtain target features from the acoustic features; The attention statistics pool is used to aggregate acoustic features of variable length into acoustic features of fixed length; The fully connected layer is configured to output a recognition result of the voiceprint feature corresponding to the acoustic feature based on the local feature, the target feature, and the fixed-length acoustic feature; The cyclic loss function module is used to obtain a cyclic loss function based on the recognition result output by the fully connected layer. The cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

11. The model training method according to claim 9, wherein the training of the initial voiceprint recognition model using the acoustic features of the sample audio signal based on the cyclic loss function to obtain the target voiceprint recognition model comprises: Determining S first target persons among the target persons; For each of the first target persons, determining, from the multiple target persons, N second target persons whose voiceprint feature similarity to the first target person is greater than a similarity threshold according to a similarity matrix, where S and N are both integers greater than or equal to 1; Based on the cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals corresponding to each of the second target persons to obtain the target voiceprint recognition model.

12. The model training method according to claim 9, further comprising: Inputting the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model, and obtaining the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model; Determining the similarity between the voiceprint features of each target person based on the voiceprint feature centers corresponding to each target person; According to the similarity between the voiceprint features of each target person, a similarity matrix between the target persons is determined.

13. The model training method according to claim 9, wherein obtaining a plurality of sample audio signals corresponding to the target person comprises: Acquire a plurality of initial sample audio signals corresponding to a target person; Noise data is added to the initial sample audio signal to obtain a plurality of sample audio signals corresponding to the target person.

14. The model training method according to claim 13, wherein adding noise data to the initial sample audio signal to obtain multiple sample audio signals corresponding to the target person comprises: Obtain the target person's identity information; Determining target noise data corresponding to the identity information based on the identity information of the target person to whom the initial sample audio signal belongs; The target noise data is added to the initial sample audio signal to obtain a plurality of sample audio signals corresponding to the target person.

15. The model training method according to any one of claims 9 to 14, wherein the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set; The method of training the initial voiceprint recognition model based on the cyclic loss function and the acoustic features of the sample audio signal to obtain the target voiceprint recognition model includes: For the i-th training, the i-1-th voiceprint recognition model is trained using the acoustic features of the M1 first sample audio signals in the first sample set and the acoustic features of the M2 second sample audio signals in the second sample set to obtain the i-th cyclic loss function and the i-th voiceprint recognition model, where the i-1-th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times; Determining that the voiceprint recognition model corresponding to the convergence of the cyclic loss function is the target voiceprint recognition model; Wherein, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

16. The model training method according to any one of claims 9 to 14, wherein the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set; The method of training the initial voiceprint recognition model based on the cyclic loss function and the acoustic features of the sample audio signal to obtain the target voiceprint recognition model includes: Based on a cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals in the first sample set to obtain a first voiceprint recognition model; Based on the cyclic loss function, the first voiceprint recognition model is trained using the acoustic features of the sample audio signals in the second sample set to obtain the target voiceprint recognition model.

17. A model training method comprising: Acquire multiple sample audio signals corresponding to the target person; Acquiring acoustic features of the sample audio signal; inputting the acoustic features of the sample audio signal into a target voiceprint recognition model to obtain a voiceprint feature corresponding to the sample audio signal output by the target voiceprint recognition model, wherein the target voiceprint recognition model is trained using the model training method according to any one of claims 9 to 16; Obtaining label data of the acoustic feature according to the voiceprint feature, wherein the label data of the acoustic feature is used to indicate the target person to whom the acoustic feature belongs; The initial speech recognition model is trained using the label data of the acoustic features to obtain a target speech recognition model.

18. The model training method according to claim 17, wherein obtaining the label data of the acoustic feature according to the voiceprint feature comprises: Clustering the voiceprint features to obtain the category to which each voiceprint feature belongs and the cluster center of each category; Determining the label data of the voiceprint features in each category based on the label data of the cluster center; The label data of the acoustic feature is obtained according to the label data of the voiceprint feature.

19. A computer-readable storage medium, wherein computer-executable instructions are stored in the computer-readable storage medium. When a processor executes the computer-executable instructions, the speech recognition method described in any one of claims 1 to 8 and / or the model training method described in any one of claims 9 to 18 are implemented.

20. A speech recognition device, comprising: An acquisition module, used to acquire acoustic features of a speech signal to be recognized; a determination module for determining, based on a speech recognition model and the acoustic features, a confidence level that each speech segment contained in the speech signal to be recognized is a target speech segment, wherein the target speech segment is a speech signal uttered by a target person, and the speech recognition model is trained based on the acoustic features corresponding to the sample audio signals of the target person; A recognition module, configured to identify a target speech segment in the speech signal to be recognized based on the confidence level of each speech segment; The speech recognition model is trained based on the acoustic characteristics of the target person as follows: Acquiring acoustic features of a plurality of sample audio signals corresponding to a target person; Inputting the acoustic feature into a target voiceprint recognition model to obtain a voiceprint feature corresponding to the acoustic feature output by the target voiceprint recognition model; Obtaining label data of the acoustic feature according to the voiceprint feature, wherein the label data of the acoustic feature is used to indicate the target person to whom the acoustic feature belongs; Training an initial speech recognition model using the label data of the acoustic features to obtain the speech recognition model; The target voiceprint recognition model is obtained through the following steps: Acquire multiple sample audio signals corresponding to the target person; Acquiring acoustic features of the sample audio signal; Determining S first target persons among the target persons; For each of the first target persons, determine, from the plurality of target persons, N second target persons whose voiceprint feature similarity to the first target person is greater than a similarity threshold according to a similarity matrix, where S and N are both integers greater than or equal to 1; Based on the cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals corresponding to each of the second target persons to obtain the target voiceprint recognition model.

21. The speech recognition device according to claim 20, wherein the recognition module is specifically used to: identify the target speech segment in the speech signal to be recognized based on the first speech segment whose confidence is greater than or equal to the confidence threshold; or determine that the speech segment whose confidence is less than the confidence threshold is not the target speech segment.

22. According to the speech recognition device according to claim 21, the recognition module is specifically used to: obtain the first speech segment whose confidence is greater than the confidence threshold; obtain the voiceprint feature of the first speech segment based on the acoustic features of the first speech segment; and identify whether the first speech segment is the target speech segment based on the voiceprint feature.

23. The speech recognition device according to claim 22, wherein the recognition module is specifically configured to: input the acoustic features of the first speech segment into a target voiceprint recognition model to obtain the voiceprint features of the first speech segment, wherein: The target voiceprint recognition model is trained based on the voiceprint features of the target person.

24. The speech recognition device according to claim 22, wherein the recognition module is specifically configured to: compare the voiceprint feature with voiceprint features in a database, wherein the database contains the voiceprint features of the target person; If it is determined that the similarity between the voiceprint feature and at least one voiceprint feature in the database is greater than or equal to a similarity threshold, determining the first voice segment as a target voice segment; If it is determined that the similarity between the voiceprint feature and the voiceprint feature in the database is less than the similarity threshold, it is determined that the first voice segment is not a target voice segment.

25. The speech recognition device according to any one of claims 20 to 24, wherein the determining module is further configured to: determine first time information of each target speech segment in the speech signal to be recognized, wherein the first time information includes: The start and end time of each target speech segment.

26. The speech recognition device according to claim 25, further comprising: A processing module is used to merge the target voice segments that are continuous in time according to the first time information of each target voice segment, obtain the voice segments to be processed and the second time information corresponding to each voice segment to be processed, wherein the second time information includes: the start and end time of each voice segment to be processed, and / or the duration of each voice segment to be processed.

27. The speech recognition device according to any one of claims 20 to 24, wherein the speech recognition model comprises: Convolutional neural networks, recurrent neural networks, self-attention mechanisms, and fully connected layers; The convolutional neural network is used to extract local features from the acoustic features; The recurrent neural network is used to extract global features from the acoustic features; The self-attention mechanism is used to extract target features from the acoustic features; The fully connected layer is used to output the confidence that each speech segment contained in the speech signal to be recognized is a target speech segment based on the local features, the global features and the target features.

28. A model training device comprising: An acquisition module, configured to acquire a plurality of sample audio signals corresponding to a target person and acquire acoustic features of the sample audio signals; A training module is used to train the initial voiceprint recognition model based on the acoustic features of the sample audio signal based on a cyclic loss function to obtain a target voiceprint recognition model; the target voiceprint recognition model is the target voiceprint recognition model in any one of the speech recognition methods of claims 1-8.

29. The model training device according to claim 28, wherein the initial voiceprint recognition model comprises: The first convolution module, the residual module based on coordinate attention, the self-attention mechanism, the second convolution module, the attention statistics pool, the fully connected layer, and the cyclic loss function module; The first convolution module and the second convolution module are used to obtain local features of the acoustic features of the sample audio signal; The coordinate attention-based residual module is used to obtain weight values ​​of convolution channels in the first convolution module and the second convolution module; The self-attention mechanism is used to obtain target features from the acoustic features; The attention statistics pool is used to aggregate acoustic features of variable length into acoustic features of fixed length; The fully connected layer is configured to output a recognition result of the voiceprint feature corresponding to the acoustic feature based on the local feature, the target feature, and the fixed-length acoustic feature; The cyclic loss function module is used to obtain a cyclic loss function based on the recognition result output by the fully connected layer. The cyclic loss function is used to train the initial voiceprint recognition model to obtain the target voiceprint recognition model.

30. The model training device according to claim 28, wherein the training module is specifically configured to: determine S first target persons among the target persons; For each of the first target persons, determine N second target persons from the multiple target persons whose voiceprint feature similarity with the first target person is greater than a similarity threshold according to a similarity matrix, wherein the similarity matrix is ​​used to indicate the voiceprint feature similarity between different target persons, wherein: Both S and N are integers greater than or equal to 1; Based on the cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals corresponding to each of the second target persons to obtain the target voiceprint recognition model.

31. The model training device according to claim 28, further comprising: Identify the module; The acquisition module is further configured to: input the acoustic features of the sample audio signal corresponding to the target person into the initial voiceprint recognition model, and obtain the voiceprint feature center corresponding to the target person output by the initial voiceprint recognition model; The determination module is used to determine the similarity between the voiceprint features of each target person according to the voiceprint feature center corresponding to each target person; and determine the similarity matrix between the target persons according to the similarity between the voiceprint features of each target person.

32. According to the model training device according to claim 28, the acquisition module is specifically used to: acquire multiple initial sample audio signals corresponding to the target person, add noise data to the initial sample audio signals, and obtain multiple sample audio signals corresponding to the target person.

33. The model training device according to claim 32, wherein the acquisition module is specifically used to: acquire identity information of the target person; Determining target noise data corresponding to the identity information based on the identity information of the target person to whom the initial sample audio signal belongs; The target noise data is added to the initial sample audio signal to obtain a plurality of sample audio signals corresponding to the target person.

34. The model training device according to any one of claims 28 to 33, wherein the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set; The training module is specifically configured to: for the i-th training, train the i-1th voiceprint recognition model using the acoustic features of M1 first sample audio signals in the first sample set and the acoustic features of M2 second sample audio signals in the second sample set, obtain the i-th cyclic loss function and the i-th voiceprint recognition model, where the i-1th voiceprint recognition model is obtained by training the initial voiceprint recognition model i-1 times; and determine that the voiceprint recognition model corresponding to when the cyclic loss function converges is the target voiceprint recognition model; in, M1 and M2 are both integers greater than or equal to 1, i is an integer greater than or equal to 2, the first sample audio signal used in the i-th training is different from that used in the i-1-th training, and / or the second sample audio signal used in the i-th training is different from that used in the i-1-th training.

35. The model training device according to any one of claims 28 to 33, wherein the sample audio signals include a first sample set and a second sample set, the number of sample audio signals in the first sample set is greater than the number of sample audio signals in the second sample set, and the target person corresponding to the first sample set is different from the target person corresponding to the second sample set; The method of training the initial voiceprint recognition model based on the cyclic loss function and the acoustic features of the sample audio signal to obtain the target voiceprint recognition model includes: Based on a cyclic loss function, the initial voiceprint recognition model is trained using the acoustic features of the sample audio signals in the first sample set to obtain a first voiceprint recognition model; Based on the cyclic loss function, the first voiceprint recognition model is trained using the acoustic features of the sample audio signals in the second sample set to obtain the target voiceprint recognition model.

36. A model training device comprising: An acquisition module, configured to acquire a plurality of sample audio signals corresponding to a target person and acquire acoustic features of the sample audio signals; a voiceprint recognition module, configured to input the acoustic features of the sample audio signal into a target voiceprint recognition model, and obtain the voiceprint features corresponding to the sample audio signal output by the target voiceprint recognition model, wherein the target voiceprint recognition model is trained using the model training method according to any one of claims 9 to 16; a label data acquisition module, configured to obtain label data of the acoustic feature according to the voiceprint feature, wherein the label data of the acoustic feature is used to indicate the target person to whom the acoustic feature belongs; The training module is used to train the initial speech recognition model using the label data of the acoustic features to obtain a target speech recognition model.

37. The model training device according to claim 36, wherein the label data acquisition module is specifically configured to: The voiceprint features are clustered to obtain the category to which each voiceprint feature belongs and the cluster center of each category; based on the label data of the cluster center, the label data of the voiceprint features in each category is determined; based on the label data of the voiceprint features, the label data of the acoustic features is obtained.

38. A computing device comprising: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor and at least the computer execution instructions stored in the memory enable the at least one processor to execute the speech recognition method described in any one of claims 1 to 8, and / or the model training method described in any one of claims 9 to 18.

39. A computer program product, comprising a computer program; when the computer program is executed, the speech recognition method according to any one of claims 1 to 8 and / or the model training method according to any one of claims 9 to 18 is implemented.

Images