Speech recognition method, device and related equipment

By performing feature extraction and sparse processing on speech data, and using ladder activation function and network model to segment speakers, the problem of low speech recognition accuracy in the prior art is solved, and a higher accuracy of speaker segmentation is achieved.

CN115083389BActive Publication Date: 2024-11-15CHINA MOBILE SHANGHAI ICT CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110259629.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-10
Publication Date
2024-11-15
Estimated Expiration
2041-03-10

AI Technical Summary

Technical Problem

In the prior art, the recognition accuracy of the segmentation results of the speaker in the voice is low.

Method used

By obtaining voice data, performing feature extraction processing, using step activation function to sparsely process the speech feature information, outputting the intimacy matrix, and inputting it into the second network model for speaker segmentation processing, so as to improve the accuracy of speaker segmentation results .

Benefits of technology

By sparselyzing the speech feature information, the accuracy of the speaker segmentation results is improved, and the accuracy of the output results of the second network model is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115083389B_ABST
    Figure CN115083389B_ABST
Patent Text Reader

Abstract

The present invention provides a speech recognition method, device and related equipment. The method includes: obtaining first speech data; performing feature extraction processing on the first speech data to obtain first speech feature information; inputting the first speech feature information into a first network model, performing sparse processing on the first speech feature information using a step activation function, and outputting an intimacy matrix; inputting the intimacy matrix into a second network model to perform speaker segmentation processing, and outputting a speaker segmentation result corresponding to the first speech data. In this way, the first speech feature information can be sparse, thereby improving the accuracy of the speaker segmentation result finally output by the second network model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of communication technology, and in particular to a speech recognition method, apparatus and related equipment. Background Art

[0002] With the development of electronic technology, speech recognition technology has played an increasingly important role in people's lives. For example, obtaining information from speech data is an important direction of modern machine learning research. The development direction of speech machine learning mainly includes automatic speech recognition (ASR), text-to-speech (TTS), speaker recognition (SR), etc. Speaker recognition based on dialogue scenarios also includes the sub-problem of speaker segmentation (SD). Speaker segmentation refers to the task of distinguishing the speaking time of different speakers and marking them in the collected speech signal stream.

[0003] During actual use, the inventors found that the prior art has the following problems: the recognition accuracy of the segmentation results of the speaker in the speech is low. Summary of the invention

[0004] The embodiments of the present invention provide a speech recognition method, an apparatus and related equipment to solve the problem of low recognition accuracy of speaker segmentation results in speech.

[0005] To solve the above problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a speech recognition method, the method comprising:

[0007] Acquire first voice data;

[0008] Performing feature extraction processing on the first voice data to obtain first voice feature information;

[0009] Inputting the first speech feature information into a first network model, performing sparse processing on the first speech feature information using a step activation function, and outputting an intimacy matrix;

[0010] The intimacy matrix is ​​input into the second network model for speaker segmentation processing, and the speaker segmentation result corresponding to the first speech data is output.

[0011] In a second aspect, an embodiment of the present invention provides a speech recognition device, including:

[0012] A first acquisition module, used to acquire first voice data;

[0013] A first processing module, used for performing feature extraction processing on the first voice data to obtain first voice feature information;

[0014] A second processing module, configured to input the first speech feature information into a first network model, perform sparse processing on the first speech feature information using a step activation function, and output an intimacy matrix;

[0015] The third processing module is used to input the intimacy matrix into the second network model for speaker segmentation processing, and output the speaker segmentation result corresponding to the first voice data.

[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is used to read the program in the memory to implement the steps in the method described in the first aspect above.

[0017] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium for storing a program, wherein the program, when executed by a processor, implements the steps in the method described in the first aspect.

[0018] In the embodiment of the present invention, first speech data is obtained; feature extraction processing is performed on the first speech data to obtain first speech feature information; the first speech feature information is input into the first network model, and the first speech feature information is sparsely processed using a step activation function to output an intimacy matrix; the intimacy matrix is ​​input into the second network model to perform speaker segmentation processing, and the speaker segmentation result corresponding to the first speech data is output. In this way, the first speech feature information can be sparsely processed, thereby improving the accuracy of the speaker segmentation result finally output by the second network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 is one of the flow charts of the speech recognition method provided by an embodiment of the present invention;

[0021] Figure 2 is a training schematic diagram of a first network model provided by an embodiment of the present invention;

[0022] Figure 3 is a structural diagram of a first network model provided by an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a step activation function provided by an embodiment of the present invention;

[0024] Figure 5 is a structural diagram of a speech recognition device provided by an embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of the structure of an electronic device provided by the implementation of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The terms "first", "second" etc. in the embodiments of the present invention are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of the connected objects, such as A and / or B and / or C, which represents 7 situations including single A, single B, single C, and A and B all exist, B and C all exist, A and C all exist, and A, B and C all exist.

[0028] See also Figure 1 , Figure 1 A flow chart of a speech recognition method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0029] Step 101: Acquire first voice data.

[0030] The first voice data may be voice data collected by an audio collection device such as a microphone, and because the first voice data is intended to segment the speakers in the first voice data, the first voice data may be voice data collected in a conversation scenario, that is, the first voice data may include voice data of at least two conversationalists. Therefore, the first voice data may also be understood as conversation scenario data or user voice data.

[0031] The above-mentioned dialogue scenario may include a dialogue scenario between people or a dialogue scenario between people and machines.

[0032] Step 102: Perform feature extraction processing on the first voice data to obtain first voice feature information.

[0033] Since the storage sampling rate of speech data is usually 16kHz, the short-term morpheme feature is considered to be a stable process within a range of more than 20ms. Therefore, it is necessary to perform frame division and windowing processing on the original audio data (i.e., the first speech data) to extract the spectral features of the corresponding phonemes (i.e., the first speech feature information).

[0034] For example, the first speech data can be processed by framing and windowing, and then a fast Fourier transform can be performed to extract the spectral features of the speech in a frame, so as to obtain a spectrogram in frames, and the spectrogram can be understood as the first speech feature information. In other words, as an optional implementation, the first speech feature information is a spectrogram. In this way, it is more convenient for the first network model to recognize the first speech feature information. Of course, as another optional implementation, the first speech feature information is a feature vector, for example, it can be a set of Mel-cepstral coefficients based on a Mel filter and combined features, etc.

[0035] Step 103: input the first speech feature information into the first network model, perform sparse processing on the first speech feature information using a step activation function, and output an intimacy matrix.

[0036] Among them, the main purpose of the step activation function is to sparse the vector space, which is essentially different from the purpose of the general activation function to produce nonlinear mapping relationships.

[0037] Among them, the above-mentioned intimacy matrix can also be understood as an intimacy matrix, and the intimacy matrix has a frame as the unit length.

[0038] Step 104: input the intimacy matrix into the second network model for speaker segmentation processing, and output the speaker segmentation result corresponding to the first voice data.

[0039] In this way, through steps 101 to 104, the first speech feature information can be made sparse, thereby improving the accuracy of the speaker segmentation result finally output by the second network model. It should be noted that since the voiceprint spectrum (which can also be understood as the first speech feature information, i.e., the spectrum graph) is like an image, there is a specific structure that can represent the different features of different speakers, and the spectral features on each frame of the voiceprint spectrum can be extracted to distinguish the speakers, that is, to achieve speaker segmentation processing and obtain the speaker segmentation result.

[0040] It should be noted that the better the sparsification effect of the first speech feature information is, the easier it is to achieve speaker segmentation, resulting in a higher accuracy of the speaker segmentation result ultimately output by the second network model.

[0041] In addition, the generalization ability of the first network model in the embodiment of the present invention is good, so that the result of the first network model can be well adapted to the second network model, and the second network model has good interpretability and can well fit the distribution of real data. At the same time, due to the application of the step activation function, the voiceprint vector of the first network model (see the subsequent description for details, that is, the intimacy matrix obtained by sparse processing of the first speech feature information in the first network model) no longer needs to exist in a high-dimensional space, which reduces the dimension of the voiceprint vector, thereby improving the accuracy of the speaker segmentation result output by the second network model.

[0042] Among them, the intimacy matrix is ​​input into the second network model, and the second network model can perform at least one of the processing methods such as regularization, matrix symmetry, Gaussian blur, threshold filtering and white noise normalization on the intimacy matrix to enhance the effect of the intimacy matrix, thereby obtaining an enhanced intimacy matrix.

[0043] In addition, the enhanced intimacy matrix can also be processed using an unsupervised learning method to obtain speaker segmentation results annotated with classification labels. The above-mentioned unsupervised learning method may include algorithms such as agglomerative hierarchical clustering or spectral clustering. Therefore, the accuracy of the speaker segmentation results output by the second network model using the above-mentioned unsupervised learning method mainly depends on the accuracy of the voiceprint information (that is, the intimacy matrix output by the first network model). Therefore, the extraction of voiceprint information is the key to improving the accuracy of speaker segmentation results.

[0044] The first network model is mainly responsible for the extraction of voiceprint information. The essence of voiceprint information is to construct an embedding vector in a high-dimensional space to extract feature information that can be distinguished from the original audio. In order to improve the accuracy of voiceprint information extraction, the first network model can be a Gaussian mixture model (GMM), a deep neural network (DNN), or a recurrent neural network (RNN).

[0045] For example, as an optional implementation, the first network model may be a deep learning neural network model, such as a ResNet34 model, or a ResNet34-VLAD model (which may be understood as a ResNet34 model connected to a VLAD layer).

[0046] It should be noted that the first network model can be called the front-end model, the second network model can be called the back-end model, and the first network model can be connected to the second network model, that is, the first network model outputs processed data, and the above data can be input into the second network model for further processing.

[0047] It should be noted that the specific application location of the step activation function in the first network model is not limited here. For example, the step activation function can be applied to the embedding layer in the first network model. Of course, the step activation function can also be applied to some convolutional layers in the first network model.

[0048] As an optional implementation, the first network model includes an embedding layer representing a vector, and the step activation function is applied to the embedding layer. In this way, the step activation function can participate in the training during the training of the first network model, but the training result has no effect on the step activation function, and the training result only affects other parts of the first network model except the embedding layer. Due to the participation of the step activation function in the training process, the training result of the first network model is better.

[0049] For example: See Figure 3 , Figure 3 is a schematic diagram of the structure of the first network model, wherein: Figure 3 The AI ​​model backbone area is used to represent the backbone area of ​​the first network model, and the step activation function area is used to represent the embedding layer. Figure 3 The conv in the code is used to represent the convolutional layer. For example, conv7*7*64 or conv7*7,64 can be used to represent the convolutional layer. Figure 3 The "x" in the above is equivalent to the "*". Figure 3 The max pooling in can represent the maximum pool.

[0050] For example, the processing process of the first voice data can be referred to the following description: The input spectrogram data (i.e., the first voice data) is connected to a max pooling layer after a 7*7*64 convolutional layer to extract spectral features (which can also be referred to as spectrogram, i.e., the first voice feature information). The subsequent network consists of 4 types of modules (blocks). The structure of the block is basically the same. Each block consists of 2 1*1 convolutional layers (used to expand the number of channels) and a 3*3 convolutional layer. The stride of the convolutional layer is 1. After each convolutional layer, a batch normalization layer and a relu activation layer are connected. In addition to the above layer connection structure, there is also a direct path that is not processed between the input and output of every three convolutional layers. This structure is repeated 2 groups. The first group contains a direct path from the input to the output, and there is a 1*1 convolutional layer on the direct path of the second group. The structures of the 4 blocks are basically the same. The difference lies in that the number of output channels increases from 96 channels in the first block to 128 channels in the second block, then to 256 channels in the third block, and 512 channels in the fourth block. The above structure is the same as the ResNet34 structure. The output of the Resnet structure is connected to a max pooling layer and a convolutional layer with a convolutional kernel of 7*1. The result is input into the NetVLAD layer (which can be understood as a pooling layer). NetVLAD is a capsule structure, and the NetVLAD layer can summarize the features extracted by the previous network. By comparing the inner product with the standard centroid structure, the features are purposefully classified to obtain the feature vector of the embedding layer. In this way, NetVLAD can summarize information like a pooling layer by calculating the cosine distance from the target center. After passing through the NetVLAD structure, a fully connected layer is connected to map the voiceprint features to an embedding layer of 512 (obtained through experiments). The specific process can be referred to Figure 3 as shown.

[0051] As an optional implementation manner, the step activation function is:

[0052] where a, c, and p are all adjustable hyperparameters, and p < 1, 1 < a < c, x is an n-dimensional vector on the rational number set, and ceil is the ceiling function.

[0053] For example: x ∈ R 1 , x ∈ (-1, 1), a = 10, c = 100, and the image can be referred to Figure 4 as shown, where Figure 4 the vertical axis in can represent the value of f(x), and the horizontal axis can represent the value of x, Figure 4 step activation in represents the activation function, that is, Figure 4 the image shown in.

[0054] In this way, the above-mentioned step activation function can make the calculation result more accurate and the calculation method simpler, so that the speaker segmentation result corresponding to the first voice data finally outputted is more accurate.

[0055] As another optional implementation, the difference from the above implementation is that the step activation function may further include a scene coefficient b, wherein in each scene, the value of the scene coefficient b is a constant, and the scene coefficients b corresponding to different scenes may be different. For example: the scene coefficient corresponding to the first scene is b1, and the scene coefficient corresponding to the second scene is b2, the values ​​of b1 and b2 may be different, but b1 and b2 may both be constants.

[0056] In addition, the scene coefficient b can be determined according to the decibel number of the speech data in each scene, for example, the larger the decibel number, the larger the value of the scene coefficient b. Of course, the method for determining the value of the scene coefficient b is not limited here.

[0057] It should be noted that ceil can also be understood as a rounding function. For example: ceil(x) = [x] + 1, x∈R.

[0058] In addition, the first-order gradient of the step activation function is: Since the activation function is applied in the embedding layer (also called the embedding layer), it is close to the starting point of the back propagation, so the constant gradient can ensure the transmission of the backward gradient. The above backward can be understood as getting the direction towards the second network model.

[0059] In addition, when x→∞ + and x→∞ - When the value is , the gradient is p, so the gradient vanishing situation cannot be avoided. Because the step activation function has a value when X<0, it can avoid bias shift and make the output gradient closer to the natural gradient, thereby accelerating the convergence of the model.

[0060] When x i1 <ceil(ax i1 )<x i2 When xi1, xi2 satisfy ceil(ax i1 )<x i1 <x i2 <ceil(ax i2), f(x) is non-expanding, and when p→0, the value range of f(x) is discontinuous. Therefore, the step activation function can approximately map x to a relatively sparse vector space. This makes the feature vector sparse. Using the feature vector of the comparison coefficient can obtain a better backend model in the backend model (i.e., the second network model), thereby improving the capability of the backend model.

[0061] It should be noted that the step activation function may also be a function shown in other formulas, which are not specifically limited here. For example, the activation function may also be a sigmoid function.

[0062] As an optional implementation manner, before inputting the first speech feature information into the first network model, the method further includes:

[0063] Acquiring second voice data;

[0064] Performing feature extraction processing on the second voice data to obtain second voice feature information;

[0065] Inputting the second speech feature information into the network model to be trained to obtain the first network model;

[0066] Wherein, the step activation function is applied in both the network model to be trained and the first network model.

[0067] Among them, the process of training the network model to be trained as the first network model can be a standard neural network training step, including: model construction (also known as artificial intelligence AI model backbone construction), data flow feedforward transmission, gradient backward propagation, using a multi-task training model, and updating the gradient by calculating the normalized exponential function amsoftmax loss and the loss function triplet loss. At the same time, an adaptive moment estimation (ADAM) optimizer can also be used.

[0068] For details, please refer to Figure 2 As shown in Figure 2 The original data in can be understood as the second voice data, and the second voice data can carry labels. The preprocessing of the voice data can be understood as the preset processing described later. The backbone model construction of the AI ​​algorithm model can be understood as the network model to be trained. According to the classification or correctness of the data classification of the network model to be trained, the loss function is iterated, and the loss function can be applied to the network model to be trained, and then optimized according to the adam optimizer. In this way, by repeatedly executing the above steps, the first network model can be finally trained. It should be noted that in the process of processing the first voice data, the steps corresponding to the original data and labels to the step activation function can also be executed.

[0069] In addition, during the training phase of the model, after obtaining the feature vector (i.e., the spectrogram, which is the second speech feature information), the feature vector needs to be connected to the fully connected layer (i.e., the FC layer) and the softmax activation function is added to obtain the logistic regression (logit) probability of the target category and then optimize the model by calculating the loss. During model inference, the feature vector is directly used as the output of the model.

[0070] The second voice data and the first voice data may both be voice data collected in a conversation scenario, that is, the second voice data and the first voice data may both include voice data of multiple interlocutors. In addition, the first voice data and the second voice data may both include annotation information.

[0071] As an optional implementation, the second speech feature information may be a spectrogram. The description of the second speech feature information may refer to the corresponding description of the first speech feature information, and will not be described in detail here.

[0072] In this way, the first network model can be trained through the second voice data, thereby making the training process of the first network model simpler and more efficient.

[0073] As an optional implementation manner, performing feature extraction processing on the second voice data to obtain second voice feature information includes:

[0074] performing preset processing on the second voice data;

[0075] The second voice data after the preset processing is subjected to feature extraction processing to obtain second voice feature information.

[0076] The preset processing may include: performing data enhancement processing on the second voice data, for example: the data enhancement processing may include: at least one of: acoustic echo canceller (AEC), automatic noise suppression (ANS), automatic gain compensation (AGC), and voice activity detection (VAD). In addition, the preset processing may also include: framing and feature extraction.

[0077] AEC is based on the correlation between the speaker signal and the multipath echo generated by it, and establishes a speech model of the far-end signal, using it to estimate the echo, and constantly modifying the coefficients of the filter to make the estimated value closer to the real echo. Then, the echo estimate is subtracted from the microphone input signal to achieve the purpose of echo elimination. AEC also compares the microphone input with the past value of the speaker to eliminate the acoustic echo of multiple reflections with extended delays. Based on the number of past speaker output values ​​stored in the memory, AEC can eliminate various delayed echoes.

[0078] ANS can detect fixed-frequency background noise and eliminate it, such as automatically filtering out the sound of fans and air conditioners, presenting the clear voices of participants.

[0079] AGC can automatically adjust the microphone's pickup volume so that participants receive a certain volume level and the sound will not fluctuate due to changes in the distance between the speaker and the microphone.

[0080] VAD is generally used to identify speech presence and speech absence in audio signals. The overall steps of VAD are as follows: divide the audio signal into frames; extract features from each frame of data; train a classifier on a set of data frames with known speech and silence signal areas; classify unknown framed data to determine whether it belongs to speech signals or silence signals.

[0081] In this way, by performing preset processing on the second voice data, the influence of noise, echo and background noise in the second voice data on the conversion result can be reduced, so that the accuracy of the obtained second voice feature information is higher.

[0082] It should be noted that the above preset processing can also be performed during the processing of the first voice data, thereby solving the problem of uneven distribution of different frequencies in the voice data and reducing the impact of noise on the accuracy of the processing result of the first voice data.

[0083] See also Figure 5 , Figure 5 FIG. 1 is one of the structural diagrams of the speech recognition device provided by the embodiment of the present invention. Figure 5 As shown, the speech recognition device 500 includes:

[0084] A first acquisition module 501, used to acquire first voice data;

[0085] A first processing module 502 is used to perform feature extraction processing on the first voice data to obtain first voice feature information;

[0086] The second processing module 503 is configured to input the first voice feature information into a first network model, perform sparse processing on the first voice feature information by using a step activation function, and output an intimacy matrix;

[0087] The third processing module 504 is configured to input the intimacy matrix into a second network model for speaker segmentation processing, and output a speaker segmentation result corresponding to the first voice data.

[0088] Optionally, the first network model includes an embedding layer for representing vectors, and the step activation function is applied to the embedding layer.

[0089] Optionally, the step activation function is:

[0090] where a, c, and p are all adjustable hyperparameters, and p < 1, 1 < a < c, x is an n-dimensional vector on the rational number set, and ceil is a rounding function.

[0091] Optionally, the voice recognition device 500 further includes:

[0092] A second acquisition module, configured to acquire second voice data;

[0093] A fourth processing module, configured to perform feature extraction processing on the second voice data to obtain second voice feature information;

[0094] A training module, configured to input the second voice feature information into a network model to be trained to train and obtain the first network model;

[0095] where the step activation function is applied to both the network model to be trained and the first network model.

[0096] Optionally, the fourth processing module includes:

[0097] A processing sub-module, configured to perform preset processing on the second voice data;

[0098] An extraction sub-module, configured to perform feature extraction processing on the second voice data after the preset processing to obtain second voice feature information.

[0099] The voice recognition device 500 can implement each process of the method embodiment in the embodiments of the present invention Figure 1 to Figure 2 shown in and achieve the same beneficial effects. To avoid repetition, details are not described herein again.

[0100] An embodiment of the present invention further provides an electronic device. Please refer to Figure 6, the electronic device may include a processor 601, a memory 602, and a program 6021 stored in the memory 602 and executable on the processor 601. When the program 6021 is executed by the processor 601, Figure 1 Any steps in the corresponding method embodiments and achieving the same beneficial effects will not be repeated here.

[0101] A person skilled in the art can understand that all or part of the steps of implementing the above-mentioned embodiment method can be completed by hardware related to program instructions, and the program can be stored in a readable medium. The embodiment of the present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 Any steps in the corresponding method embodiments can achieve the same technical effect and will not be repeated here to avoid repetition.

[0102] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0103] The above is a preferred implementation of the embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A speech recognition method, characterized in that: The method includes: Obtain first speech data; Perform feature extraction processing on the first speech data to obtain first speech feature information; Input the first speech feature information into a first network model, and use a step activation function to perform sparse processing on the first speech feature information, and output an affinity matrix; Input the affinity matrix into a second network model for speaker segmentation processing, and output a speaker segmentation result corresponding to the first speech data.

2. The method according to claim 1, characterized in that The first network model includes an embedding layer representing vectors, and the step activation function is applied to the embedding layer.

3. The method according to claim 1, characterized in that The step activation function is: where a, c, and p are all adjustable hyperparameters, and p < 1, 1 < a < c, x is an n-dimensional vector on the rational number set, and ceil is a rounding function.

4. The method according to claim 1, characterized in that Before inputting the first speech feature information into the first network model, the method further includes: Obtain second speech data; Perform feature extraction processing on the second speech data to obtain second speech feature information; Input the second speech feature information into a network model to be trained to train the first network model; where the step activation function is applied to both the network model to be trained and the first network model.

5. The method according to claim 4, characterized in that The performing feature extraction processing on the second speech data to obtain second speech feature information includes: Perform preset processing on the second speech data; Perform feature extraction processing on the second speech data after the preset processing to obtain second speech feature information.

6. The method according to claim 4, characterized in that Both the first speech feature information and the second speech feature information are spectrograms.

7. A speech recognition device, characterized in that: It includes: A first acquisition module for obtaining first speech data; A first processing module for performing feature extraction processing on the first speech data to obtain first speech feature information; A second processing module for inputting the first speech feature information into a first network model, and using a step activation function to perform sparse processing on the first speech feature information, and outputting an affinity matrix; A third processing module for inputting the affinity matrix into a second network model for speaker segmentation processing, and outputting a speaker segmentation result corresponding to the first speech data.

8. The device according to claim 7, characterized in that The first network model includes an embedding layer representing vectors, and the step activation function is applied to the embedding layer.

9. The device according to claim 7, characterized in that The step activation function is: where a, c, and p are all adjustable hyperparameters, and p < 1, 1 < a < c, x is an n-dimensional vector on the rational number set, and ceil is a rounding function.

10. The device according to claim 7, characterized in that It further includes: A second acquisition module for obtaining second speech data; A fourth processing module for performing feature extraction processing on the second speech data to obtain second speech feature information; A training module for inputting the second speech feature information into a network model to be trained to train the first network model; where the step activation function is applied to both the network model to be trained and the first network model.

11. The device according to claim 10, characterized in that The fourth processing module includes: A processing sub-module for performing preset processing on the second speech data; An extraction sub-module for performing feature extraction processing on the second speech data after the preset processing to obtain second speech feature information.

12. The device according to claim 10, characterized in that The first speech feature information and the second speech feature information are both spectrograms.

13. An electronic device comprising: A transceiver, a memory, a processor and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the speech recognition method as described in any one of claims 1 to 6.

14. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the speech recognition method according to any one of claims 1 to 6 are implemented.