Speech recognition method, device, equipment, storage medium and program product

Abstract

The present application discloses a speech recognition method, device, equipment, storage medium and program product, which relate to the field of artificial intelligence. The method includes: obtaining a speech signal stream; extracting speech sequence embedding features; fusing the speech sequence embedding features to obtain fused features; recursively estimating the fused features to obtain speaker embedding features; performing speaker recognition on the speech sequence embedding features based on the speaker embedding features to obtain speaker recognition results. In the process of performing speaker recognition on the speech to generate a speaker log, the speaker embedding features corresponding to each speaker are obtained by recursively estimating the fused features, and the speaker recognition results are obtained through the attention mechanism, so that all parameters can be used in the decoding process, thereby improving the accuracy of speaker recognition on the speech. The embodiments of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a speech recognition method, apparatus, device, storage medium and program product. Background Art

[0002] Speaker log refers to the process of labeling the collected voice signal stream by distinguishing the speaking stages of different speakers, thereby detecting the identity of the speaker corresponding to each speech segment. Speaker log is used in many scenarios such as meeting recording and customer service work supervision.

[0003] In the related art, when generating a speaker log for a speech signal stream, the speech frames in the speech signal stream are encoded to obtain an embedding layer vector of each speech frame, and then the encoding vector of the speech signal stream is obtained through a long short-term memory module (LSTM), and the number of speakers is decoded from the encoding vector of the speech signal stream. Subsequently, the decoding vector and encoding vector of each speech frame are combined to perform a probability prediction corresponding to the speaker, thereby determining the speaker information corresponding to each frame.

[0004] However, in the above speaker log generation method, the information of the encoding vector is not fully utilized, resulting in a low accuracy rate of speaker recognition, that is, a low accuracy rate of speaker log generation. Summary of the invention

[0005] The embodiments of the present application provide a speech recognition method, apparatus, device, storage medium and program product, which can improve the accuracy of generating speaker logs. The technical solution is as follows:

[0006] In one aspect, a speech recognition method is provided, the method comprising:

[0007] Acquire a speech signal stream, wherein the speech signal stream is speech to be speaker recognized, and the speech signal stream includes speech frames;

[0008] Extracting a speech sequence embedding feature of the speech frame, wherein the speech sequence embedding feature is a feature extracted by a pre-trained encoder;

[0009] fusing the speech sequence embedding features of the speech frame to obtain fused features;

[0010] Recursively estimating the fused features to obtain speaker embedding features corresponding to the speakers respectively;

[0011] Based on the speaker embedding feature, speaker recognition is performed on the speech sequence embedding feature through an attention mechanism to obtain speaker recognition results corresponding to the speech frames respectively, and the speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

[0012] In another aspect, a speech recognition device is provided, the device comprising:

[0013] An acquisition module, used for acquiring a speech signal stream, wherein the speech signal stream is speech to be speaker recognized, and the speech signal stream includes speech frames;

[0014] An extraction module, used for extracting speech sequence embedding features of the speech frame, wherein the speech sequence embedding features are features extracted by a pre-trained encoder;

[0015] A fusion module, used for fusing the speech sequence embedding features of the speech frame to obtain a fusion feature;

[0016] An estimation module, used for recursively estimating the fused features to obtain speaker embedding features corresponding to the speakers respectively;

[0017] A recognition module is used to perform speaker recognition on the speech sequence embedding features through an attention mechanism based on the speaker embedding features to obtain speaker recognition results corresponding to the speech frames respectively, and the speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

[0018] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement a speech recognition method as described in any of the above-mentioned embodiments of the present application.

[0019] On the other hand, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement a speech recognition method as described in any of the above-mentioned embodiments of the present application.

[0020] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the speech recognition method described in any of the above embodiments.

[0021] The beneficial effects brought by the technical solution provided in the embodiments of the present application include at least:

[0022] In the process of speaker recognition for speech to generate speaker logs, the speaker embedding features corresponding to each speaker are obtained by recursively estimating the fused features, and then the speaker recognition results are obtained through the attention mechanism based on the speaker embedding features. The LSTM is improved to avoid decoding with input zero vectors, so that all parameters can be used in the decoding process, thereby improving the utilization efficiency of the LSTM and thus improving the accuracy of speaker recognition for speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a schematic diagram of a speaker log provided by an exemplary embodiment of the present application;

[0025] Figure 2 is a schematic diagram of the architecture of a speech recognition model provided by an exemplary embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0027] Figure 4 is a flow chart of a speech recognition method provided by an exemplary embodiment of the present application;

[0028] Figure 5 is a flow chart of a speech recognition method provided by another exemplary embodiment of the present application;

[0029] Figure 6 is a flow chart of a speech recognition method provided by another exemplary embodiment of the present application;

[0030] Figure 7 is a structural block diagram of a speech recognition device provided by an exemplary embodiment of the present application;

[0031] Figure 8 is a structural block diagram of a speech recognition device provided by another exemplary embodiment of the present application;

[0032] Fig. 9 It is a structural block diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0034] Speaker log refers to the process of labeling the collected voice signal stream by distinguishing the speaking stages of different speakers, thereby detecting the identity of the speaker corresponding to each speech segment. Speaker log is used in many scenarios such as meeting recording and customer service work supervision.

[0035] For illustration, please refer to Figure 1 , which shows a schematic diagram of a speaker log provided by an exemplary embodiment of the present application. Figure 1 As shown, the speech segments of speaker 110 and speaker 120 are identified for the speech signal stream 100 .

[0036] In the related art, in the process of generating speaker logs, the speech frames in the speech signal stream are encoded to obtain the encoding vector of each speech frame, and then the encoding vector of the speech signal stream is obtained through the long short-term memory module (Long Short-Term Memory, LSTM), and the number of speakers is decoded from the encoding vector of the speech signal stream. Subsequently, the decoding vector and encoding vector of each speech frame are combined to perform probability prediction corresponding to the speaker, thereby determining the speaker information corresponding to each frame. However, when generating speaker logs, this method does not fully utilize the information of the encoding vector, resulting in a low accuracy rate of speaker recognition, that is, a low accuracy rate of speaker log generation.

[0037] Based on the above problems, Figure 2 is a schematic diagram of the architecture of a speech recognition model provided by an exemplary embodiment of the present application, such as Figure 2 As shown, in the speech recognition model 200, it includes an end-to-end speaker verification (End-to-End Neural Diarization, EEND) module 210 based on self-attention (Self-Attention, SA), an improved LSTM module 220, and an attention layer 230;

[0038] Among them, after the speech frame in the speech signal stream is feature extracted, the encoding vector x of the speech frame is obtained 1 、x 2 , …, x t, and input into the SA-EEND module 210 to obtain an embedding layer vector, perform pooling on the embedding layer vector, and perform recursive estimation through the improved LSTM module 220 to obtain the embedding layer features corresponding to each speaker in the speech signal stream; predict the embedding layer vector output by the SA-EEND module 210 according to the embedding layer features corresponding to each speaker, so as to obtain the probability that each speech frame corresponds to the speaker, that is, whether the speech frame includes the speaker's speech utterance content; finally, obtain the speaker log of the speech signal stream according to the recognition results of all speech frames.

[0039] Figure 3 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application, such as Figure 3 As shown, the implementation environment includes a terminal 310 and a server 320, and the terminal 310 and the server 320 are connected via a communication network 330;

[0040] The terminal 310 is used to send a voice signal stream to the server 320. Optionally, the voice signal stream is recorded by the terminal 310 through a microphone; or, the voice signal stream is recorded by the terminal 310 through call signal collection; or, the voice signal stream is received by the terminal 310 from other terminals / servers.

[0041] The server 320 includes the above-mentioned speech recognition model 321. After the speech signal stream sent by the terminal 310 is recognized by the speech recognition model 321, a speaker log is obtained, and the speaker log is fed back to the terminal 310 for display.

[0042] The above-mentioned terminals can be terminal devices in various forms such as mobile phones, tablet computers, desktop computers, portable laptops, intelligent voice interaction devices, intelligent voice interaction devices, smart home appliances, car terminals, etc.

[0043] It is worth noting that the above-mentioned servers can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0044] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0045] In some embodiments, the above-mentioned server can also be implemented as a node in a blockchain system. Blockchain is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0046] The method provided in this embodiment can be applied to various scenarios such as meeting records and customer service work supervision. Illustratively, the application scenarios of this application include at least one of the following scenarios:

[0047] First, the meeting record scenario.

[0048] That is, the audio during the meeting is recorded to obtain a voice signal stream, and the voice frames in the voice signal stream are subjected to speaker recognition by the voice recognition method provided in the embodiment of the present application to obtain the speaker recognition result corresponding to each voice frame, thereby generating a speaker log corresponding to the meeting. Subsequently, when re-listening to the meeting content, the meeting speech of the target speaker can be selectively listened to according to the speaker log.

[0049] Second, the customer service work supervision scenario.

[0050] The customer service telephone / voice communication content is recorded to obtain a voice signal stream, and the voice frames in the voice signal stream are subjected to speaker recognition through the voice recognition method provided in the embodiment of the present application to obtain the speaker recognition result corresponding to each voice frame, thereby generating a speaker log corresponding to the customer service telephone communication process. Subsequently, the management personnel can extract the customer service telephone / voice content, and select the part expressed by the customer service according to the speaker log for selective listening, thereby realizing the supervision of the customer service work.

[0051] It is worth noting that the above application scenarios are only illustrative examples. The embodiments of the present application can also be applied to various scenarios such as cloud technology, artificial intelligence, Internet of Vehicles, map navigation, smart transportation, assisted driving, etc. The embodiments of the present application do not limit the specific application scenarios of the speech recognition method.

[0052] Combined with the above content, Figure 4 FIG. 1 is a flow chart of a speech recognition method provided by an exemplary embodiment of the present application, and takes the method applied to a server as an example for explanation. Figure 4 As shown, the method includes:

[0053] Step 401: Acquire a voice signal stream.

[0054] The speech signal stream is speech to be subjected to speaker recognition, and the speech signal stream includes speech frames.

[0055] The method of obtaining the voice signal stream includes at least one of the following situations:

[0056] 1. Receive the voice signal stream sent by the terminal. For example, if the terminal is a terminal that initiates customer service, the call content is recorded during the customer service stage, and after the recording is completed, the voice signal stream of the call content is sent to the server for recognition;

[0057] 2. Obtain the voice signal stream to be identified from the stored database. For example, in conference applications, the conference process is recorded, the speeches of each speaker in the conference are recorded, and stored in the server. When speaker recognition is required, the voice signal stream of the conference recording to be identified is obtained from the stored conference recording for speaker recognition.

[0058] It is worth noting that the above-mentioned method of acquiring the voice signal stream is only an illustrative example and is not limited to this in the embodiments of the present application.

[0059] Step 402: extracting speech sequence embedding features of speech frames.

[0060] The speech sequence embedding features are extracted from the trained encoder.

[0061] In some embodiments, after the speech features of the speech frames of a given speech information stream are modeled, the relationship between the speech features and the frame-by-frame speaker logs is modeled, wherein after the speech features are input into the encoder, a high-dimensional representation of the speech features, i.e., speech sequence embedding features, is obtained through layer-by-layer analysis.

[0062] Optionally, the speech sequence embedding features of the speech frame are extracted through the SA-EEND module.

[0063] Among them, the speech frame is obtained after the speech information stream is segmented according to the frame length and frame shift. For example, the speech information stream is segmented according to the frame length of 10ms (also called window length) and the frame shift of 25ms to obtain the speech frames in the speech information stream, and the speech frames are input into the speaker recognition model. First, the speech feature x of the speech frame is extracted, and the SA-EEND module in the speaker recognition model is used to perform high-dimensional feature extraction on the speech feature x to obtain the speech sequence embedding feature.

[0064] Schematically, the speech sequence embedding feature obtained by encoding the speech frame layer by layer through the encoder is expressed as the following formula 1:

[0065] Formula 1:

[0066] Among them, e represents the intermediate hidden layer representation (embedding) of the encoder, n represents the number of hidden layers (Identity, ID), the 0th layer is the initial feature, g is used to represent a single-layer Transformer encoder, x represents the speech feature of the speech frame, m represents the mth speech frame, and M represents the number of speech frames.

[0067] Step 403: fuse the speech sequence embedding features to obtain fused features.

[0068] In the related art, when performing speaker recognition, a high-dimensional representation of a whole sentence of the above encoding result is first obtained, that is, a high-dimensional representation of a whole sentence of the above speech sequence embedded features. In addition, the number of speakers is decoded from the whole sentence information.

[0069] The high-dimensional representation of a whole sentence of the speech sequence embedding feature is obtained as shown in the following formula 2:

[0070] Formula 2:

[0071] Among them, enc represents the encoder, and h and c are two parameters in the LSTM model. However, in the related art, when obtaining a high-dimensional representation of a whole sentence of the above encoding result, only the intermediate hidden layer output of the last frame is used as the modeling information of the whole sentence. However, 1. Generally speaking, the training data of the speaker's voice is often relatively long, so LSTM may lose some speaker information, and 2. LSTM will model the speech order, but in fact, there is not necessarily an actual relationship between the previous and the next part of each conversation.

[0072] Therefore, in the embodiment of the present application, the second-order statistics in the hidden layer representation are counted as fusion features through the statistics pooling layer (Statistics Pooling, SP).

[0073] Optionally, a stacked sequence corresponding to the speech sequence embedding feature is generated, and a statistical pooling process is performed on the stacked sequence to obtain a fused feature. When the statistical pooling process is performed on the stacked sequence, a mean and a standard deviation corresponding to the stacked sequence are obtained, and the mean and the standard deviation are concatenated to obtain a fused feature.

[0074] Schematically, the process of statistically fusing features through the SP layer is shown in the following formula 3:

[0075] Formula 3: Pooling(E) = Concat(avg(E), std(E))

[0076] Where E represents the stacked sequence corresponding to the speech sequence embedding feature, that is, E = {e 1 , e 2 ,…,e T}. avg indicates mean calculation, std indicates standard deviation calculation, and Concat indicates concatenation processing.

[0077] Step 404 , recursively estimate the fused features to obtain speaker embedding features corresponding to the speakers.

[0078] In the related art, the loop structure of the LSTM model is used for decoding, and the decoding process is shown in the following formula 4:

[0079] Formula 4:

[0080] Where s represents the sth speaker, and dec represents the decoder in the LSTM model. To decode each possible speaker, a zero vector is input, and the calculation formula of LSTM is as shown in Formula 5:

[0081] Formula 5:

[0082] Among them, i corresponds to the information mechanism in LSTM, f corresponds to the forget mechanism in LSTM, gate corresponds to the gate mechanism in LSTM, and t corresponds to the output mechanism in LSTM. ii , W if , W ig , W io The W matrix is ​​the matrix corresponding to different mechanisms in the calculation process, c and h are parameters, and b is the parameter corresponding to different mechanisms.

[0083] However, in the calculation process of the above formulas 4 and 5, each input of the zero vector will result in ii , W if , W ig , W io These four matrices have no gradient update, so the decoding accuracy is low. Therefore, in the embodiment of the present application, the above-mentioned fusion features are recursively estimated to obtain the speaker embedding features corresponding to each speaker.

[0084] Optionally, the fused features are recursively estimated through the LSTM network to obtain n layers of mask features, where n is a positive integer, and the speaker embedding features corresponding to the i-th speaker are determined based on the i-th layer of mask features and the i-1-th layer of fused features, where 0<i≤n. Schematically, the speaker embedding feature is determined as shown in the following formula 6:

[0085] Formula 6:

[0086] Among them, E 0 This is the initial fusion feature Pooling (E) obtained above, func is an LSTM-core (Cell), and the initial input h 0 , c 0 All are zero vectors, and the subsequent ones use the h decoded last time. n-1 and c n-1 , which will decode the speaker embedding features of several single speakers. 1 represents the first layer of mask features, e 1 That is, the speaker embedding feature corresponding to a speaker. Similarly, maskn represents the nth layer mask feature, E n-1 represents the fusion feature of the n-1th layer, e n Represents the speaker embedding feature corresponding to the n-th speaker.

[0087] That is, the product of the mask feature of the i-th layer and the fusion feature of the i-1-th layer is determined as the speaker embedding feature corresponding to the i-th speaker.

[0088] Step 405 , based on the speaker embedding features, speaker recognition is performed on the speech sequence embedding features through an attention mechanism to obtain speaker recognition results corresponding to the speech frames.

[0089] The speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

[0090] In summary, the method provided in this embodiment, in the process of performing speaker recognition on speech to generate a speaker log, obtains speaker embedding features corresponding to each speaker by recursively estimating the fused features, and then obtains the speaker recognition result through the attention mechanism based on the speaker embedding features, improves LSTM, avoids decoding with input zero vectors, so that all parameters can be used in the decoding process, improves the utilization efficiency of LSTM, and thus improves the accuracy of speaker recognition on speech.

[0091] In an optional embodiment, when speaker recognition is performed based on speaker embedding features, speaker embedding features of the entire sentence are used to decode speaker embedding features of a single person and decode attractors.

[0092] Figure 5 is a flow chart of a speech recognition method provided by another exemplary embodiment of the present application, and takes the method applied in a server as an example for explanation. Figure 5 As shown, the method includes:

[0093] Step 501: Acquire a voice signal stream.

[0094] The speech signal stream is speech to be subjected to speaker recognition, and the speech signal stream includes speech frames.

[0095] Step 502: extract the speech sequence embedding features of the speech frame.

[0096] The speech sequence embedding features are extracted from the trained encoder.

[0097] Optionally, the speech sequence embedding features of the speech frame are extracted through the SA-EEND module.

[0098] Step 503: Generate a stacked sequence corresponding to the speech sequence embedding feature.

[0099] Optionally, the speech sequence embedding features of each speech frame are stacked to obtain a stacked sequence.

[0100] Step 504: Perform statistical pooling processing on the stacked sequence to obtain fusion features.

[0101] Optionally, the statistical pooling processing method has been described in detail by the above formula three, and will not be repeated here.

[0102] Step 505, recursively estimate the fusion features through the long short-term memory network to obtain n layers of mask features, where n is a positive integer.

[0103] Optionally, a LSTM-Cell is used to process the initial fusion features to obtain the first layer of mask features, where the initial fusion features are based on the initial h 0 , c 0 The zero vector is used as a parameter for estimation, and the h decoded last time is used later. n-1 and c n-1 .

[0104] Step 506: Determine the speaker embedding feature corresponding to the i-th speaker based on the i-th layer mask features and the i-1-th layer fusion features, 0<i≤n.

[0105] Optionally, the product of the mask feature of the i-th layer and the fusion feature of the i-1-th layer is determined as the speaker embedding feature corresponding to the i-th speaker.

[0106] Step 507, query the speaker embedding features through the attention mechanism and the speech sequence embedding features to obtain attractors corresponding to the speaker embedding features respectively.

[0107] Optionally, the stacked sequence is projected onto the key matrix and the value matrix, a query matrix is ​​constructed according to the speaker embedding features, the speaker embedding features are queried according to the key matrix, the value matrix and the query matrix, and attractors corresponding to the speaker embedding features are obtained.

[0108] Schematically, an attention mechanism is introduced to query the corresponding attractor through the decoded embedding. First, the stacked sequence E of the speech sequence embedding feature stack is projected into the key matrix K and the value matrix V, and the query matrix Q is the above speaker embedding feature e 1 To e n Then, the proportional dot product form attention mechanism can be expressed as the following formula 7:

[0109] Formula 7:

[0110] Wherein, d represents a preset ratio, and the speaker embedding feature and the first feature of each speaker are processed through the attention mechanism to obtain the attractor corresponding to each speaker.

[0111] Step 508: Perform speaker recognition on the speech sequence embedding features of the speech frames based on the attractor to obtain speaker recognition results corresponding to the speech frames.

[0112] The attractor is combined with the speech sequence embedding feature of the speech frame to obtain the feature to be analyzed, and the probability prediction of the feature to be analyzed is performed to obtain the posterior probabilities corresponding to the speech frame and the speaker respectively, and the speaker recognition result corresponding to the speech frame is determined based on the posterior probabilities.

[0113] The attractors corresponding to the speaker embedding features are stacked to obtain stacked features, and the product of the stacked features and the speech sequence embedding features of the speech frame is used as the feature to be analyzed.

[0114] Optionally, a Sigmoid layer is used to calculate the posterior probabilities of the features to be analyzed corresponding to the current frame and each speaker, so as to obtain a speaker recognition result.

[0115] Optionally, when the posterior probability is expressed in the form of a hard label: in response to the posterior probability corresponding to the speech frame and the target speaker being a first value, it is determined that the speaker has made a pronunciation in the speech frame; in response to the posterior probability corresponding to the speech frame and the target speaker being a second value, it is determined that the speaker has not made a pronunciation in the speech frame.

[0116] Schematically, the calculation process of the posterior probability is shown in the following formula 8:

[0117] Formula 8: p t =σ(A T e t )∈(0,1) s

[0118] Among them, A represents the attractor after stacking, that is, the stacking feature, e t It is the hidden layer vector of the speech frame, and the posterior probability value includes 0 or 1. When the posterior probability calculation result is 0, it means that there is no content corresponding to the speaker in the speech frame. When the posterior probability calculation result is 1, it means that the speech frame includes content corresponding to the speaker.

[0119] Optionally, when the posterior probability is embodied in the form of a soft label: in response to the posterior probability corresponding to the speech frame and the target speaker belonging to a first value range, it is determined that the speaker has made a pronunciation in the speech frame; in response to the posterior probability corresponding to the speech frame and the target speaker belonging to a second value range, it is determined that the speaker has not made a pronunciation in the speech frame, wherein the first value range and the second value range have no intersection.

[0120] Illustratively, the first value range is [0, 0.6), and the second value range is [0.6, 1]. When the posterior probability that the first speech frame corresponds to speaker A is 0.8, it means that the first speech frame includes the utterance of speaker A.

[0121] The training process of the above speaker recognition model is explained as follows:

[0122] Since the mapping relationship between the decoded attractor and the actual labeled speaker ID cannot be determined during the decoding process, such as speaker ID A is labeled as speaker ID B during the decoding process, the model uses the Permutation Invariant Training (PIT) strategy during training. The specific loss can be expressed as the following formula 9:

[0123] Formula 9:

[0124] Among them, S represents the number of speakers, M represents the number of audio frames, is the pre-marked label, p m is the predicted posterior probability, and H is used to represent the distance between the calculated label and the posterior probability. The minimum value of the calculated result is taken as the calculated loss value.

[0125] In addition, the decoded attractor needs to correspond to the number of speakers, so a loss for determining the number of speakers is also required, so that the attractor corresponding to the number of speakers can be decoded according to the threshold during inference. Please refer to the following formula 10:

[0126] Formula 10:

[0127]

[0128] Among them, a s represents the attractor of the sth speaker, b exist The preset parameters corresponding to the number of speakers present, is a parameter corresponding to the number of speech frames and the number of speakers, l is the number of pre-annotated speakers, and q s is the number of speakers predicted.

[0129] The loss values ​​calculated by the above formula 9 and formula 10 are added together to obtain a total loss value, and the model parameters of the speaker recognition model are adjusted and trained according to the total loss value.

[0130] In summary, the method provided in this embodiment, in the process of performing speaker recognition on speech to generate a speaker log, obtains speaker embedding features corresponding to each speaker by recursively estimating the fused features, and then obtains the speaker recognition result through the attention mechanism based on the speaker embedding features, improves LSTM, avoids decoding with input zero vectors, so that all parameters can be used in the decoding process, improves the utilization efficiency of LSTM, and thus improves the accuracy of speaker recognition on speech.

[0131] The method provided in this embodiment improves the core LSTM module of the decoder (Decoder) of the encoder-decoder-attractor (EDA) module in the related art for predicting the number of speakers, and uses a method of recursively estimating mask features to estimate the speaker embedding features of a single speaker, thereby improving the accuracy of speaker number prediction and speaker recognition.

[0132] The method provided in this implementation introduces a permutation invariant training strategy to obtain the loss value of speaker prediction and introduces the predicted loss value of the number of speakers during the model training process, thereby obtaining a more comprehensive loss calculation and improving the comprehensiveness and accuracy of model training.

[0133] In an optional embodiment, after obtaining the speaker recognition result, a speaker log is generated. Figure 6 is a flow chart of a speech recognition method provided by another exemplary embodiment of the present application, and takes the method applied in a server as an example for explanation. Figure 6 As shown, the method includes:

[0134] Step 601: Acquire a voice signal stream.

[0135] The speech signal stream is speech to be subjected to speaker recognition, and the speech signal stream includes speech frames.

[0136] Step 602: extract the speech sequence embedding features of the speech frame.

[0137] The speech sequence embedding features are extracted from the trained encoder.

[0138] Optionally, the speech sequence embedding features of the speech frame are extracted through the SA-EEND module.

[0139] Step 603: fuse the speech sequence embedding features to obtain fused features.

[0140] In the embodiment of the present application, the second-order statistics in the hidden layer representation are counted by the statistical pooling layer SP as the fusion feature.

[0141] Optionally, a stacked sequence corresponding to the speech sequence embedding feature is generated, and a statistical pooling process is performed on the stacked sequence to obtain a fused feature. When the statistical pooling process is performed on the stacked sequence, a mean and a standard deviation corresponding to the stacked sequence are obtained, and the mean and the standard deviation are concatenated to obtain a fused feature.

[0142] Step 604 , recursively estimate the fused features to obtain speaker embedding features corresponding to the speakers.

[0143] In the embodiment of the present application, the above fusion features are recursively estimated to obtain speaker embedding features corresponding to each speaker.

[0144] Optionally, the fused features are recursively estimated through an LSTM network to obtain n layers of mask features, where n is a positive integer, and the speaker embedding features corresponding to the i-th speaker are determined based on the i-th layer of mask features and the i-1-th layer of fused features, where 0<i≤n.

[0145] Step 605: Perform speaker recognition on the speech sequence embedding features through an attention mechanism based on the speaker embedding features to obtain speaker recognition results corresponding to the speech frames.

[0146] The speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

[0147] Step 606: construct a speaker log corresponding to the speech signal stream based on the speaker recognition result, wherein the speaker log is marked with the occurrence stage of the speaker in the speech signal stream.

[0148] Optionally, the speaker recognition result of the speech signal stream includes a speaker identifier corresponding to the speech frame. Schematically, the speech signal stream includes speech frame 1, speech frame 2, speech frame 3, speech frame 4, speech frame 5,...; the speaker recognition result is marked with speech frame 1: speaker A (indicating that speech frame 1 includes the speech content of speaker A); speech frame 2: speaker A (indicating that speech frame 2 includes the speech content of speaker A); speech frame 3: speaker A, speaker B (indicating that speech frame 3 includes the speech content of speaker A and speaker B); speech frame 4: speaker A, speaker B (indicating that speech frame 4 includes the speech content of speaker A and speaker B); speech frame 5: speaker C (indicating that speech frame 5 includes the speech content of speaker C);...

[0149] According to the speaker recognition results, the part of each speaker's corresponding voice content in the speech frame is marked, so that the speaker's voice stage in the speech signal stream can be obtained, and the speaker log corresponding to the speech signal stream can be generated.

[0150] In summary, the method provided in this embodiment, in the process of performing speaker recognition on speech to generate a speaker log, obtains speaker embedding features corresponding to each speaker by recursively estimating the fused features, and then obtains the speaker recognition result through the attention mechanism based on the speaker embedding features, improves LSTM, avoids decoding with input zero vectors, so that all parameters can be used in the decoding process, improves the utilization efficiency of LSTM, and thus improves the accuracy of speaker recognition on speech.

[0151] The method provided in this embodiment marks the speaker's utterance stage in the speech signal stream according to the speaker corresponding to each speech frame in the speaker recognition result, and obtains the speaker log, thereby improving the generation efficiency and accuracy of the speaker log.

[0152] In order to verify the effectiveness of the method proposed in the embodiment of the present application, experiments were conducted on a preset speaker log dataset, and the experimental results under different configurations were compared. The specific results are shown in Table 1 below:

[0153] Table 1

[0154]

[0155] Among them, the basic structure has always remained the Transformer model. Experiments 1.1 and 1.2 use the same model structure and configuration, but there are certain differences in the organization of the input, which leads to changes in the length of the modeled speech and a large gap in the experimental results; Experiments 2.1 and 2.2 use the speaker recognition model provided in the embodiment of the present application, but there are certain differences in the organization of the input, so the length of the modeled speech changes and a large gap in the experimental results.

[0156] Among them, by comparing Experiment 1.1 with Experiment 2.1 horizontally, and by comparing Experiment 1.2 with Experiment 2.2 horizontally, it can be obtained that the speaker recognition model provided in the embodiment of the present application has a 51% improvement in effect, thereby improving the accuracy of the speaker log model.

[0157] Figure 7 is a structural block diagram of a speech recognition device provided by an exemplary embodiment of the present application, such as Figure 7 As shown, the device comprises:

[0158] An acquisition module 710 is used to acquire a speech signal stream, wherein the speech signal stream is speech to be speaker recognized, and the speech signal stream includes speech frames;

[0159] An extraction module 720, configured to extract a speech sequence embedding feature of the speech frame, wherein the speech sequence embedding feature is a feature extracted by a pre-trained encoder;

[0160] A fusion module 730 is used to fuse the speech sequence embedding features of the speech frame to obtain a fusion feature;

[0161] An estimation module 740, configured to recursively estimate the fused features to obtain speaker embedding features corresponding to the speakers respectively;

[0162] The recognition module 750 is used to perform speaker recognition on the speech sequence embedding features through an attention mechanism based on the speaker embedding features to obtain speaker recognition results corresponding to the speech frames respectively, and the speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

[0163] In an optional embodiment, if Figure 8 As shown, the fusion module 730 includes:

[0164] A generating unit 731 is used to generate a stacked sequence corresponding to the speech sequence embedding feature;

[0165] The pooling unit 732 is used to perform statistical pooling processing on the stacked sequence to obtain the fused feature.

[0166] In an optional embodiment, the pooling unit 732 is further used to obtain a mean and a standard deviation corresponding to the stacked sequence; and concatenate the mean and the standard deviation to obtain the fusion feature.

[0167] In an optional embodiment, the estimation module 740 is further used to recursively estimate the fused features through a long short-term memory network to obtain n layers of mask features, where n is a positive integer; and determine the speaker embedding features corresponding to the i-th speaker based on the i-th layer of mask features and the i-1-th layer of fused features, 0<i≤n.

[0168] In an optional embodiment, the estimation module 740 is further configured to determine the product of the i-th layer mask features and the i-1-th layer fusion features as the speaker embedding feature corresponding to the i-th speaker.

[0169] In an optional embodiment, the recognition module 750 is further used to query the speaker embedding features through the attention mechanism and the speech sequence embedding features to obtain attractors corresponding to the speaker embedding features respectively; perform speaker recognition on the speech sequence embedding features of the speech frames based on the attractors to obtain speaker recognition results corresponding to the speech frames respectively.

[0170] In an optional embodiment, the recognition module 750 is further used to project the stacked sequence to a key matrix and a value matrix; construct a query matrix according to the speaker embedding features; query the speaker embedding features according to the key matrix, the value matrix and the query matrix to obtain attractors corresponding to the speaker embedding features respectively.

[0171] In an optional embodiment, the recognition module 750 is further used to combine the attractor with the speech sequence embedding feature of the speech frame to obtain the feature to be analyzed; perform probability prediction on the feature to be analyzed to obtain the posterior probabilities corresponding to the speech frame and the speaker respectively; and determine the speaker recognition result corresponding to the speech frame based on the posterior probability.

[0172] In an optional embodiment, the recognition module 750 is further used to stack the attractors corresponding to the speaker embedding features to obtain stacked features; and use the product of the stacked features and the speech sequence embedding features of the speech frame as the feature to be analyzed.

[0173] In an optional embodiment, the recognition module 750 is further used to determine that the speaker has made a sound in the speech frame in response to the posterior probability corresponding to the speech frame and the target speaker being a first value; and to determine that the speaker has not made a sound in the speech frame in response to the posterior probability corresponding to the speech frame and the target speaker being a second value.

[0174] In an optional embodiment, the device further includes:

[0175] The construction module 760 is used to construct a speaker log corresponding to the speech signal stream based on the speaker recognition result, wherein the speaker log is marked with the speaker's utterance stage in the speech signal stream.

[0176] In summary, the device provided in this embodiment, in the process of performing speaker recognition on speech to generate a speaker log, obtains speaker embedding features corresponding to each speaker by recursively estimating the fused features, and then obtains the speaker recognition result through the attention mechanism based on the speaker embedding features, improves LSTM, avoids decoding with input zero vectors, so that all parameters can be used in the decoding process, improves the utilization efficiency of LSTM, and thus improves the accuracy of speaker recognition on speech.

[0177] It should be noted that the speech recognition device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the speech recognition device provided in the above embodiment and the speech recognition method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0178] Fig. 9The block diagram of the structure of a computer device 900 provided by an exemplary embodiment of the present application is shown. The computer device 900 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a laptop computer or a desktop computer. The computer device 900 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0179] Typically, the computer device 900 includes a processor 901 and a memory 902 .

[0180] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0181] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the speech recognition method provided in the method embodiment of the present application.

[0182] In some embodiments, the terminal 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a display screen 905, a camera 906, an audio circuit 907, a positioning component 908 and a power supply 909.

[0183] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0184] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0185] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, and the front panel of the terminal 900 is set; in other embodiments, the display screen 905 can be at least two, which are respectively set on different surfaces of the terminal 900 or are folded; in some other embodiments, the display screen 905 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 900. Even, the display screen 905 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0186] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0187] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0188] The positioning component 908 is used to locate the current geographical location of the terminal 900 to implement navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on the GPS (Global Positioning System), Beidou system or Galileo system.

[0189] The power supply 909 is used to power various components in the terminal 900. The power supply 909 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0190] In some embodiments, the terminal 900 further includes one or more sensors 910 , including but not limited to: an acceleration sensor 911 , a gyroscope sensor 912 , a pressure sensor 913 , a fingerprint sensor 914 , an optical sensor 915 , and a proximity sensor 916 .

[0191] The acceleration sensor 911 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 900. For example, the acceleration sensor 911 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the touch display screen 905 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used for collecting game or user motion data.

[0192] The gyro sensor 912 can detect the body direction and rotation angle of the terminal 900, and the gyro sensor 912 can cooperate with the acceleration sensor 911 to collect the user's 3D actions on the terminal 900. The processor 901 can implement the following functions based on the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0193] The pressure sensor 913 can be set on the side frame of the terminal 900 and / or the lower layer of the touch display screen 905. When the pressure sensor 913 is set on the side frame of the terminal 900, it can detect the user's holding signal of the terminal 900, and the processor 901 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is set on the lower layer of the touch display screen 905, the processor 901 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0194] The fingerprint sensor 914 is used to collect the user's fingerprint, and the processor 901 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 914 can be set on the front, back, or side of the terminal 900. When a physical button or a manufacturer logo is set on the terminal 900, the fingerprint sensor 914 can be integrated with the physical button or the manufacturer logo.

[0195] The optical sensor 915 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the touch display screen 905 according to the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 905 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera component 906 according to the ambient light intensity collected by the optical sensor 915.

[0196] The proximity sensor 916, also called a distance sensor, is usually arranged on the front panel of the terminal 900. The proximity sensor 916 is used to collect the distance between the user and the front of the terminal 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the terminal 900 is gradually decreasing, the processor 901 controls the touch display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 916 detects that the distance between the user and the front of the terminal 900 is gradually increasing, the processor 901 controls the touch display screen 905 to switch from the screen-off state to the screen-on state.

[0197] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal 900, and the terminal 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0198] Optionally, the computer readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0199] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0200] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A speech recognition method, characterized in that: The method comprises: Acquire a speech signal stream, wherein the speech signal stream is speech to be speaker recognized, and the speech signal stream includes speech frames; Extracting a speech sequence embedding feature of the speech frame, wherein the speech sequence embedding feature is a feature extracted by a pre-trained encoder; Generate a stacked sequence corresponding to the speech sequence embedding feature; Performing statistical pooling processing on the stacked sequence to obtain fusion features; Recursively estimating the fused features to obtain speaker embedding features corresponding to the speakers respectively; Based on the speaker embedding feature, speaker recognition is performed on the speech sequence embedding feature through an attention mechanism to obtain speaker recognition results corresponding to the speech frames respectively, and the speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

2. The method according to claim 1, characterized in that The performing statistical pooling processing on the stacked sequence to obtain the fusion feature includes: Obtaining the mean and standard deviation corresponding to the stacked sequence; The mean and the standard deviation are concatenated to obtain the fusion feature.

3. The method according to claim 1, characterized in that The recursive estimation of the fused features to obtain speaker embedding features corresponding to the speakers respectively includes: Recursively estimating the fusion features through a long short-term memory network to obtain n layers of mask features, where n is a positive integer; The speaker embedding feature corresponding to the i-th speaker is determined based on the i-th layer mask feature and the i-1-th layer fusion feature, 0<i≤n.

4. The method according to claim 3, characterized in that: The determining the speaker embedding feature corresponding to the i-th speaker based on the i-th layer mask feature and the i-1-th layer fusion feature includes: The product of the i-th layer mask feature and the i-1-th layer fusion feature is determined as the speaker embedding feature corresponding to the i-th speaker.

5. The method according to claim 4, characterized in that The speaker recognition is performed on the speech sequence embedding feature through an attention mechanism based on the speaker embedding feature to obtain speaker recognition results corresponding to the speech frames, including: The speaker embedding feature is queried through the attention mechanism and the speech sequence embedding feature to obtain attractors corresponding to the speaker embedding features respectively; Based on the attractor, speaker recognition is performed on the speech sequence embedding features of the speech frame to obtain speaker recognition results corresponding to the speech frames respectively.

6. The method according to claim 5, characterized in that The querying of the speaker embedding feature through the attention mechanism and the speech sequence embedding feature to obtain attractors corresponding to the speaker embedding features respectively includes: projecting the stacked sequence into a key matrix and a value matrix; constructing a query matrix based on the speaker embedding features; The speaker embedding features are queried according to the key matrix, the value matrix and the query matrix to obtain attractors corresponding to the speaker embedding features respectively.

7. The method according to claim 5, characterized in that The performing speaker recognition on the speech sequence embedding features of the speech frames based on the attractor to obtain speaker recognition results corresponding to the speech frames respectively includes: Combining the attractor with the speech sequence embedding feature of the speech frame to obtain a feature to be analyzed; Probabilistically predicting the features to be analyzed to obtain posterior probabilities corresponding to the speech frames and the speakers respectively; The speaker recognition result corresponding to the speech frame is determined based on the posterior probability.

8. The method according to claim 7, characterized in that The step of combining the attractor with the speech sequence embedding feature of the speech frame to obtain the feature to be analyzed includes: The attractors corresponding to the speaker embedding features are stacked to obtain stacked features; The product of the stacked feature and the speech sequence embedding feature of the speech frame is used as the feature to be analyzed.

9. The method according to claim 7, characterized in that: The determining the speaker recognition result corresponding to the speech frame based on the posterior probability includes: In response to the posterior probability corresponding to the speech frame and the target speaker being a first value, determining that the speaker has uttered an utterance in the speech frame; In response to the posterior probability corresponding to the speech frame and the target speaker being a second value, it is determined that the speaker does not make an utterance in the speech frame.

10. The method according to any one of claims 1 to 9, characterized in that: After obtaining the speaker recognition results corresponding to the speech frames respectively, the method further includes: A speaker log corresponding to the speech signal stream is constructed based on the speaker recognition result, wherein the speaker log is marked with the speaker's utterance stage in the speech signal stream.

11. A speech recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring a speech signal stream, wherein the speech signal stream is speech to be speaker recognized, and the speech signal stream includes speech frames; An extraction module, used for extracting speech sequence embedding features of the speech frame, wherein the speech sequence embedding features are features extracted by a pre-trained encoder; A generating unit, used to generate a stacked sequence corresponding to the speech sequence embedding feature; A pooling unit, used for performing statistical pooling processing on the stacked sequence to obtain fusion features; An estimation module, used for recursively estimating the fused features to obtain speaker embedding features corresponding to the speakers respectively; A recognition module is used to perform speaker recognition on the speech sequence embedding features through an attention mechanism based on the speaker embedding features to obtain speaker recognition results corresponding to the speech frames respectively, and the speaker recognition results corresponding to the speech frames constitute the speaker log corresponding to the speech signal stream.

12. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the speech recognition method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the speech recognition method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the speech recognition method according to any one of claims 1 to 10.

Images