Role recognition method of voice dialogue, electronic equipment and storage medium

Abstract

The invention provides a role recognition method for a voice dialogue, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-recognized voice dialogue; voice separation is carried out on the to-be-recognized voice dialogue, multiple pieces of single-channel voice corresponding to multiple speakers are obtained, and one piece of single-channel voice comprises the voice of one speaker; and performing role judgment on each piece of single-channel voice in the plurality of pieces of single-channel voice, and determining a role of a speaker corresponding to each piece of single-channel voice. According to the method, the voices of different speakers in the voice dialogue can be separated, the voice dialogue is split into a plurality of single-channel voices corresponding to a plurality of speakers, so that one single-channel voice only contains the voice of one speaker, then role recognition is performed according to the single-channel voices, and the role recognition efficiency is improved for the scene that a plurality of persons speak at the same time. And the role of the speaker can be identified more accurately.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a method for character recognition in voice conversation, an electronic device, and a storage medium. Background Art

[0002] With the continuous development of artificial intelligence, intelligent communication, Internet technology and 5G communication, speech recognition technology has been widely used in various scenarios in life, such as smart home, network calls and other intelligent voice applications. In the application of intelligent voice, the scenario of identifying the speaker in the voice conversation is very typical and common, especially the voice separation of customer service / customer and customer service quality inspection of intelligent customer service. At present, some mainstream role recognition methods in the industry directly establish voiceprint models based on data and perform recognition. However, under the rapid switching of ultra-short sentences, voiceprint recognition is prone to misjudgment and the misjudgment rate is very high. Some methods divide the voice conversation into multiple voice segments, and then use clustering methods such as k-means to cluster the voice segments belonging to the same speaker identity together, and obtain the role of each speaker in the voice conversation based on the clustering results. However, this segment segmentation method has a very low accuracy rate for role recognition in scenarios where multiple speakers speak at the same time. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a method, an electronic device and a storage medium for role recognition in voice conversations, which can more accurately identify the roles of the speakers in scenarios where multiple people are speaking at the same time.

[0004] A first aspect of an embodiment of the present application provides a method for role recognition of a voice conversation, comprising: obtaining a voice conversation to be recognized; performing voice separation on the voice conversation to be recognized to obtain multiple single-channel voices corresponding to multiple speakers, wherein one single-channel voice contains the voice of one speaker; performing role judgment on each of the multiple single-channel voices to determine the role of the speaker corresponding to each single-channel voice.

[0005] In a possible implementation, performing role judgment on each of the multiple single-channel voices and determining the role of the speaker corresponding to each single-channel voice includes: converting each of the multiple single-channel voices into text, obtaining the text corresponding to each single-channel voice, performing role judgment on the text corresponding to each single-channel voice, and determining the role of the speaker corresponding to each single-channel voice.

[0006] In a possible implementation, the step of performing role judgment on the text corresponding to each of the single-channel voices to determine the role of the speaker corresponding to each of the single-channel voices includes: taking the text corresponding to each of the single-channel voices as the target text, and obtaining the text keywords in the target text; comparing the text keywords in the target text with the keywords in the preset role dictionary, determining whether the text keywords in the target text hit the keywords in the preset role dictionary, obtaining the hit keywords of the target text, summing up the scores corresponding to all the hit keywords in the preset role dictionary, and obtaining the score value of the target text; comparing the score value of the target text with the preset score value, and if the score value of the target text reaches the preset score value, determining that the role corresponding to the speaker of the single-channel voice corresponding to the target text is the role corresponding to the preset role dictionary.

[0007] In a possible implementation, the step of performing role judgment on the text corresponding to each of the single-channel voices to determine the role of the speaker corresponding to each of the single-channel voices includes: inputting the text corresponding to each of the single-channel voices into a role prediction model to perform role prediction, and obtaining the probability that the speaker of each of the single-channel voices belongs to various set roles in the role preset model; comparing the probabilities that the speaker of each of the single-channel voices belongs to various set roles in the role preset model, and determining the role corresponding to the speaker of each of the single-channel voices.

[0008] In a possible implementation, before the step of inputting the text corresponding to each single-channel speech into the role prediction model for role prediction, it also includes: performing data compression processing on the text corresponding to each single-channel speech, wherein the data compression processing includes any one or both of data abbreviation processing and data truncation processing.

[0009] In a possible implementation, before the step of converting each of the multiple single-channel voices into text and obtaining the text corresponding to each single-channel voice, it also includes: performing silence detection on each of the single-channel voices and clearing the silence segments in each of the single-channel voices.

[0010] In a possible implementation, the step of performing speech separation on the speech dialogue to be recognized to obtain multiple single-channel speech corresponding to multiple speakers, wherein each single-channel speech contains the voice of one speaker, includes: performing speech separation processing on the speech dialogue to be recognized using an end-to-end speech separation model so that the end-to-end speech separation model outputs multiple single-channel speech, wherein the number of single-channel speech is equal to the number of speakers in the speech dialogue to be recognized, and one single-channel speech records the voice of one speaker.

[0011] In a possible implementation, the end-to-end speech separation model includes a coding network layer, a separation network layer, and a decoding network layer; the step of using the end-to-end speech separation model to perform speech separation processing on the speech dialogue to be recognized so that the end-to-end speech separation model outputs a corresponding number of single-channel voices according to the number of speakers of the speech dialogue to be recognized, includes: constructing a first feature matrix for characterizing the speech dialogue to be recognized according to the duration and sampling frequency of the speech dialogue to be recognized through the coding network layer, and performing feature dimension expansion processing on the first feature matrix according to a preset expansion matrix to obtain an expanded second feature matrix; performing masking processing on the second feature matrix according to the number of speakers of the speech dialogue to be recognized through the separation network layer to obtain a plurality of separated voices corresponding to the number of speakers; performing dimensionality reduction processing on the plurality of separated voices through the decoding network layer to obtain the single-channel voices corresponding to each of the plurality of separated voices.

[0012] A second aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the electronic device, wherein when the processor executes the computer program, each step of the method for character recognition in voice conversation provided in the first aspect is implemented.

[0013] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for character recognition in voice dialogue provided in the first aspect are implemented.

[0014] The embodiment of the present application provides a method for character recognition in voice dialogue, an electronic device, and a storage medium, which have the following beneficial effects:

[0015] This application obtains a voice conversation to be recognized; performs voice separation on the voice conversation to be recognized to obtain multiple single-channel voices corresponding to multiple speakers, wherein one single-channel voice contains the voice of one speaker; and performs role judgment on each of the multiple single-channel voices to determine the role of the speaker corresponding to each single-channel voice. Based on this method, the voices of different speakers in a mixed voice of multiple people speaking at the same time can be separated, and the mixed voice can be split into multiple single-channel voices corresponding to multiple speakers, so that one single-channel voice contains only the voice of one speaker. Then, role recognition is performed based on the single-channel voice, and the roles of the speakers can be more accurately identified for scenes where multiple people are speaking at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0017] Figure 1 A flowchart of a method for character recognition in a voice conversation provided in an embodiment of the present application;

[0018] Figure 2 A flowchart of a method for implementing role judgment in a method for identifying a role in a voice conversation provided in an embodiment of the present application;

[0019] Figure 3 A flowchart of another method for implementing role judgment in the method for role recognition of voice dialogue provided in an embodiment of the present application;

[0020] Figure 4 A flowchart of a method for implementing speech separation using an end-to-end speech separation model in a method for character recognition of speech dialogue provided in an embodiment of the present application;

[0021] Figure 5 A basic structural block diagram of a voice dialogue role recognition device provided in an embodiment of the present application;

[0022] Figure 6 A basic structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] See also Figure 1 , Figure 1 This is a flowchart of a method for character recognition in a voice conversation provided in an embodiment of the present application. Specifically, it may include the following steps S11 to S13.

[0025] S11: Acquire the voice dialogue to be recognized.

[0026] In this embodiment, voice dialogue can be understood as voice data generated during a conversation between two or more speakers. Specifically, in online consultation scenarios, online sales scenarios, etc., there will be voice dialogues between customers and customer service. In some actual application scenarios, such as customer service quality inspection, it is necessary to obtain voice dialogues between customer service and customers, identify roles by voice data generated by these voice dialogues, and then analyze the content of the dialogue between customers and customer service in depth based on the identified roles, detect whether the customer service speaks in a standardized manner, and then achieve the purpose of supervising the service quality of customer service personnel. Through quality inspection, the service quality of customer service can be optimized in a targeted manner, the service quality can be effectively improved, and the efficiency and effect of customer service improvement can be improved. In this embodiment, customer service mainly interacts with customers in the form of telephone or online voice, and the voice data generated during the interaction process is the voice dialogue to be identified.

[0027] S12: performing speech separation on the speech dialogue to be recognized to obtain a plurality of single-channel speech corresponding to a plurality of speakers, wherein a single-channel speech contains the voice of a single speaker.

[0028] In this embodiment, if there are multiple speakers speaking at the same time or obvious background noise in the voice dialogue used for customer service quality inspection, it will interfere with the quality inspection results and greatly reduce the accuracy of the quality inspection. In this embodiment, multiple single-channel voices with undetermined speaker roles can be obtained by performing voice separation on the voice dialogue to be recognized. It can be understood that when the voice dialogue to be recognized contains multiple speakers, multiple single-channel voices corresponding to the multiple speakers can be obtained, and one single-channel voice contains the voice of one speaker.

[0029] In some embodiments of the present application, an end-to-end speech separation model can be pre-trained. The end-to-end speech separation model specifically takes a speech conversation as input and a single-channel speech as output. The number of single-channel speech is equal to the number of speakers in the speech conversation to be recognized. It should be noted that the speech conversation contains speech data of multiple speakers, and the single-channel speech contains only speech data of one speaker. That is, a single-channel speech records the voice of one speaker, and the number of speakers included in the speech conversation is the number of single-channel speech output by the model. In this embodiment, the speech conversation to be recognized is separated by the above-mentioned end-to-end speech separation model, so that the end-to-end speech separation model outputs a corresponding number of single-channel speech according to the number of speakers in the speech conversation to be recognized, and multiple single-channel speech corresponding to multiple speakers can be obtained. Among them, one speaker corresponds to one single-channel speech. Exemplarily, taking the customer service scenario as an example, in order to train the end-to-end speech separation model, the voices of the two speakers of the customer service and the customer can be spliced ​​and then the spliced ​​sound can be mixed to construct a training sample. When training the end-to-end speech separation model, the number of speakers is pre-set to 2 according to the scenario. By setting the input of the end-to-end speech separation model to the training sample obtained by mixing the spliced ​​sounds, the spliced ​​sounds include the voices of two speakers, the customer service and the customer, and setting the output of the end-to-end speech separation model to two single-channel voices, each of which includes the voice of only one speaker, the end-to-end speech separation model is trained to a convergence state, so that the end-to-end speech separation model has the ability to separate the voice conversation containing two speakers into two single-channel voices. It can be understood that the end-to-end speech separation model does not need to distinguish which single-channel voice belongs to the customer service and which single-channel voice belongs to the customer.

[0030] S13: Performing role judgment on each of the multiple single-channel voices to determine the role of the speaker corresponding to each single-channel voice.

[0031] In this embodiment, for each single-channel speech, acoustic features can be extracted from the single-channel speech to obtain the acoustic features corresponding to each single-channel speech, and then role judgments can be performed based on the acoustic features corresponding to each single-channel speech to determine the role of the speaker corresponding to each single-channel speech.

[0032] In some embodiments of the present application, after obtaining multiple single-channel voices through an end-to-end voice separation model, each of the multiple single-channel voices can be converted into text to obtain the text corresponding to each single-channel voice. Then, the text corresponding to each single-channel voice is judged to determine the role of the speaker corresponding to each single-channel voice. Specifically, the speech recognition model can be used to realize speech recognition of the multiple single-channel voices one by one, and each single-channel voice is converted into a corresponding text. Exemplarily, the speech recognition model used in this embodiment includes four parts: feature extraction, acoustic model, language model and dictionary decoding. The feature extraction part converts the sound signal from the time domain to the frequency domain to provide the acoustic model with a feature vector in the speech. The acoustic model calculates the score of each feature vector on the acoustic feature according to the acoustic characteristics. The language model calculates the probability that the sound signal corresponds to a possible phrase sequence. The dictionary encoding decodes the phrase sequence according to the existing dictionary to obtain the text corresponding to the speech.

[0033] In this embodiment, the content in the text corresponding to each single-channel voice can be analyzed respectively to calculate the probability that the single-channel voice belongs to each role, and the role of the speaker corresponding to each single-channel voice can be determined by comparing the probability values. It can be understood that the probability value comparison can be a comparison between the probability value of the single-channel voice belonging to a certain role and the probability threshold, or it can be a comparison between the probability values ​​of the single-channel voice belonging to each role. Since the text corresponding to each single-channel voice only contains the voice data of one speaker, by judging the role of the speaker based on the text corresponding to the single-channel voice, when judging the role of a speaker, the interference of the voice data of other speakers in the voice conversation can be avoided, and the accuracy of the judgment of the speaker's role can be effectively improved.

[0034] As can be seen from the above, the role recognition method for voice dialogue provided in the embodiments of the present application can separate the voices of different speakers from the mixed voice of multiple people speaking at the same time, split the mixed voice into multiple single-channel voices corresponding to multiple speakers, so that a single-channel voice only contains the voice of one speaker, and then perform role recognition based on the single-channel voice. For scenes where multiple people are speaking at the same time, the roles of the speakers can be more accurately identified. In some embodiments of the present application, please refer to Figure 2 , Figure 2 A flowchart of a method for implementing role judgment in the method for identifying a role in a voice dialogue provided in an embodiment of the present application. Figure 2 As shown, it may specifically include steps S21 to S23.

[0035] S21: taking the text corresponding to each single-channel speech as the target text, and obtaining text keywords in the target text;

[0036] S22: Compare the text keywords in the target text with the keywords in the preset role dictionary, determine whether the text keywords in the target text hit the keywords in the preset role dictionary, obtain the hit keywords of the target text, sum the scores corresponding to all the hit keywords in the preset role dictionary, and obtain the score value of the target text;

[0037] S23: Compare the score of the target text with a preset score. If the score of the target text reaches the preset score, determine that the role of the speaker of the single-channel speech corresponding to the target text is the role corresponding to the preset role dictionary.

[0038] In this embodiment, when performing role judgment, the role recognition can be performed on the speaker corresponding to each single-channel voice according to the text corresponding to each single-channel voice. Specifically, the text corresponding to each single-channel voice can be used as the target text, and the text keywords in the target text can be obtained by performing keyword recognition on the text content. It can be understood that the target text can include one or more text keywords. In this embodiment, a corresponding preset role dictionary is pre-constructed for different dialogue roles. A preset role dictionary is constructed corresponding to a dialogue role. In this embodiment, based on each preset role dictionary, the text keywords in the target text can be compared with the keywords in the preset role dictionary to determine whether the text keywords in the target text hit the keywords in the preset role dictionary. If they hit, the text keywords are determined as the hit keywords of the target text. That is, the hit keywords are words that exist in both the target text and the preset role dictionary. The scores corresponding to all the hit keywords of the target text in the preset role dictionary are summed up to obtain the score value of the target text. Then, by comparing the score of the target text with the preset score set based on the preset role dictionary, if the score of the target text reaches the preset score, it is determined that the role of the speaker of the single-channel voice corresponding to the target text is the role corresponding to the preset role dictionary. Exemplarily, taking the customer service scenario as an example, there are two roles, customer service and customer, and corresponding preset role dictionaries can be constructed for the two roles respectively, such as the customer service role corresponds to the first preset role dictionary, and the customer role corresponds to the second preset role dictionary. The first preset role dictionary is formed by extracting keywords from the standard customer service speech collected based on the application scenario and storing the keywords in a database, and each keyword stored in the database is configured with a corresponding score. It should be noted that the score corresponding to each keyword in the first preset role dictionary can be used to measure the difficulty of distinguishing the role to which it belongs through the keyword. The larger the score corresponding to the keyword, the easier it is to distinguish the role to which it belongs through the keyword. On the contrary, it is more difficult to distinguish the role to which it belongs through the keyword. Specifically, based on the collected standard customer service speech data, the score of each keyword can be determined by counting the frequency of occurrence of each keyword in these standard customer service speech data. Among them, the higher the frequency of occurrence of a keyword, the higher its score; the lower the frequency of occurrence of a keyword, the lower its score. The second preset role dictionary is formed by extracting keywords from customer questions collected based on application scenarios and storing the keywords in a database, and each keyword stored in the database is configured with a corresponding score. Similarly, the score corresponding to each keyword in the second preset role dictionary can be used to measure the difficulty of distinguishing the role to which it belongs through the keyword. The larger the score corresponding to the keyword, the easier it is to distinguish the role to which it belongs through the keyword. Conversely, it is more difficult to distinguish the role to which it belongs through the keyword.Specifically, the score of each keyword can be determined based on the collected customer question data by counting the frequency of each keyword in these customer question data. Among them, the higher the frequency of occurrence of a keyword, the higher its score; the lower the frequency of occurrence of a keyword, the lower its score. In this embodiment, after obtaining the text keywords in the target text, the text keywords in the target text can be compared with the keywords in the first preset role dictionary to obtain all the hit keywords of the target text that hit the first preset role dictionary. According to the sum of the corresponding scores of these hit keywords in the first preset role dictionary, the first score value of the target text can be calculated, and the first score value is compared with the first preset score value. If the first score value reaches the first preset score value, it is determined that the role of the speaker of the single-channel voice corresponding to the target text is a customer service role. After obtaining the text keywords in the target text, the text keywords in the target text can also be compared with the keywords in the second preset role dictionary to obtain all the hit keywords in the target text that hit the second preset role dictionary. According to the scores corresponding to these hit keywords in the second preset role dictionary, the second score value of the target text can be calculated, and the second score value is compared with the second preset score value. If the second score value reaches the second preset score value, it is determined that the role of the speaker of the single-channel voice corresponding to the target text is the customer role. The above-mentioned role judgment method can be applied to quality inspection scenarios with relatively simple business and relatively standardized speech. For example, in scenarios where customer service has clear opening remarks, closing remarks and other standard speech, such as "Hello, I am xxx and I am happy to serve you", "May I ask if there is anything else I can help you with", etc., it is only necessary to collect standard speech in advance to formulate a role dictionary, score by whether the target text hits the keywords, sum the scores of the hit keywords, calculate the score value of the target text, and determine whether the speaker corresponding to the target text is a customer service role by judging whether the score value reaches the preset score.

[0039] In some embodiments of this application, please refer to Figure 3 , Figure 3 Another method for implementing role judgment in the role recognition method of voice dialogue provided in the embodiment of the present application is a flowchart. Figure 3 As shown, it may specifically include step S31 to step S32.

[0040] S31: inputting the text corresponding to each single-channel speech into the role prediction model to perform role prediction, and obtaining the probability that the speaker of each single-channel speech belongs to various set roles in the role preset model;

[0041] S32: comparing the probabilities that the speaker of each single-channel speech belongs to various set roles in the preset role model, and determining the role corresponding to the speaker of each single-channel speech.

[0042] In this embodiment, the role prediction of the speaker of the single-channel voice is performed by pre-training the role prediction model. Specifically, when training the role prediction model, various set roles are configured in the role prediction model based on the application scenario, so that after the role prediction model is trained to a convergent state, the probability that the speaker of the single-channel voice belongs to various set roles can be predicted based on the text corresponding to the single-channel voice. For example, in the customer service scenario, the role prediction model is configured with two set roles, customer service and customer. After the text corresponding to the single-channel voice is input into the role prediction model for role prediction, the first probability that the speaker of the single-channel voice belongs to the customer service role and the second probability that the speaker of the single-channel voice belongs to the customer role can be obtained. After the text corresponding to each single-channel voice is input into the role prediction model for role prediction, two probability values ​​can be obtained for each single-channel voice. In this embodiment, by comparing all the probability values ​​obtained for all single-channel voices with each other, it can be determined which text in the text corresponding to each single-channel voice is more likely to be spoken by the customer, thereby determining the role corresponding to the speaker of each single-channel voice. Exemplarily, assuming that the text corresponding to the first single-channel voice is input into the role prediction model, the probability value P11 that the speaker of the first single-channel voice belongs to the customer role and the probability value P12 that the speaker of the first single-channel voice belongs to the customer service role are obtained, and the text corresponding to the second single-channel voice is input into the role prediction model, and the probability value P21 that the speaker of the second single-channel voice belongs to the customer role and the probability value P22 that the speaker of the second single-channel voice belongs to the customer service role are obtained. If P12≥P22, it is determined that the role corresponding to the speaker of the first single-channel voice is the customer service role. If P22>P12, it is determined that the role corresponding to the speaker of the second single-channel voice is the customer service role. In some specific implementations, the role prediction model is obtained by training the neural network BERT model, which is suitable for complex scenarios without standard speech.

[0043] In some embodiments of the present application, in order to ensure the probability calculation speed of the role prediction model, the text input into the role prediction model can also be subjected to data compression processing, so as to achieve the purpose of reducing the amount of text data, thereby improving the calculation speed. In this embodiment, the data compression processing method includes any one or both of data abbreviation processing and data interception processing. Data abbreviation processing, also known as padding processing, refers to achieving the purpose of data compression by setting the inner margin attributes of all current or specified elements in the text. Data interception processing refers to intercepting part of the data from the text, and achieving the purpose of data compression by deleting the remaining data except the intercepted data.

[0044] In some embodiments of the present application, in order to improve the speed and accuracy of converting speech to text, before converting the single-channel speech to text, each single-channel speech that needs to be converted into text can be subjected to silence detection, and the silence segments in each single-channel speech can be identified, and then the silence segments in each single-channel speech can be removed. In this embodiment, a TDNN-LSTM model constructed by combining TDNN (Time Delay Neural Network) and LSTM (Long Short Term Memory) can be used to perform silence detection. Specifically, the input of the TDNN-LSTM model is the MFCC (Mel-scale Frequency Cepstral Coefficients) feature of the single-channel speech, and the output of the TDNN-LSTM model is two labels, namely, a silence label or a human voice label. In this way, after silence detection, the mfcc features corresponding to the silent labels can be determined, the speech segments corresponding to these mfcc features can be cleared, and the remaining speech segments after clearing can be spliced ​​to obtain single-channel speech for conversion into text.

[0045] In some embodiments of this application, please refer to Figure 4 , Figure 4 A flowchart of a method for implementing speech separation using an end-to-end speech separation model in the method for character recognition of speech dialogue provided in an embodiment of the present application. Figure 4 As shown, it may specifically include steps S41 to S43.

[0046] S41: constructing a first feature matrix for characterizing the voice conversation to be recognized according to the duration and sampling frequency of the voice conversation to be recognized through the coding network layer, and performing feature dimension expansion processing on the first feature matrix according to a preset expansion matrix to obtain an expanded second feature matrix;

[0047] S42: performing mask processing on the second feature matrix according to the number of speakers of the speech conversation to be recognized through a separation network layer to obtain a plurality of separated speech corresponding to the number of speakers;

[0048] S43: Performing dimensionality reduction processing on the plurality of separated speech pieces respectively through a decoding network layer to obtain single-channel speech pieces corresponding to the plurality of separated speech pieces respectively.

[0049] In this embodiment, the end-to-end speech separation model is obtained by pre-training using the TasNet-skip network. Specifically, the end-to-end speech separation model uses mixed speech data containing multiple speakers and the corresponding single-channel speech data of a single speaker as sample data for training, wherein the mixed speech data of multiple speakers in the sample data is used as the input of the end-to-end speech separation model, and the corresponding single-channel speech data of a single speaker in the sample data is output as the prediction result of the end-to-end speech separation model. The end-to-end speech separation model is trained to a convergence state by using a large amount of sample data, so that the end-to-end speech separation model has the ability to separate a mixed speech data containing multiple speakers into multiple single-channel speech data containing only one speaker. In this embodiment, the end-to-end speech separation model includes a coding network layer, a separation network layer, and a decoding network layer. In the coding network layer, for the speech dialogue to be recognized input to the end-to-end speech separation model, the speech dialogue to be recognized can be constructed into a 1×T first feature matrix according to the duration s and sampling frequency sr of the speech dialogue to be recognized, wherein T=s*sr. Then, the first feature matrix is ​​expanded in feature dimension according to the preset expansion matrix N, and after expansion, a second feature matrix of N×T is obtained. In this embodiment, the structure of the encoding network layer is a one-dimensional convolutional network (nn.Conv1d), which is used for text data, and only convolution is performed on the width, not on the height, and its calculation process is multiple inputs corresponding to one output. In the separation network layer, for the input second feature matrix, the second feature matrix can be masked according to the number of speakers of the voice dialogue to be recognized, and multiple separated voices corresponding to the number of speakers are obtained. In this embodiment, the separation network layer is composed of multiple convolutional modules stacked, each of which contains x hole convolution modules 1-D conv, and the convolution kernels of the x hole convolution modules are 1, 2, ..., 2X-1 respectively. Since the separation network layer realizes speech separation through a series of convolutional layers, that is, the high-dimensional data of each separated speech obtained by the decoding network layer, in the decoding network layer, for each separated speech obtained, the multiple separated speech can be respectively subjected to dimensionality reduction processing through the convolutional layer, so as to obtain the single-channel speech corresponding to each of the multiple separated speech. In this embodiment, the structure of the decoding network layer is a one-dimensional transposed convolutional network (nn.ConvTranspose1d), and the calculation process of the one-dimensional transposed convolutional network is the reverse process of the calculation process of the one-dimensional convolutional network, that is, one input corresponds to multiple outputs.

[0050] It is understandable that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0051] In some embodiments of this application, please refer to Figure 5 , Figure 5 This is a basic structural block diagram of a voice dialogue role recognition device provided in an embodiment of the present application. The various units included in the device in this embodiment are used to execute the various steps in the above method embodiment. For details, please refer to the relevant description in the above method embodiment. For ease of explanation, only the parts related to this embodiment are shown. Figure 5 As shown, the role recognition device for voice dialogue includes: an acquisition module 51, a separation module 52 and a judgment module 53. Among them: the acquisition module 51 is used to obtain the voice dialogue to be recognized. The separation module 52 is used to perform voice separation on the voice dialogue to be recognized, and obtain multiple single-channel voices corresponding to multiple speakers, wherein a single-channel voice contains the sound corresponding to a speaker. The judgment module 53 is used to perform role judgment on each single-channel voice in the multiple single-channel voices, and determine the role of the speaker corresponding to each single-channel voice.

[0052] It should be understood that the above-mentioned role recognition device for voice dialogue corresponds one-to-one with the above-mentioned role recognition method for voice dialogue, and will not be described in detail here.

[0053] In some embodiments of this application, please refer to Figure 6 , Figure 6 This is a basic structural block diagram of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61, such as a program of a method for character recognition of voice conversation. When the processor 61 executes the computer program 63, the steps in each embodiment of the method for character recognition of voice conversation described above are implemented. Alternatively, when the processor 61 executes the computer program 63, the functions of each module in the embodiment corresponding to the device for character recognition of voice conversation described above are implemented. Please refer to the relevant description in the embodiment for details, which will not be repeated here.

[0054] Exemplarily, the computer program 63 may be divided into one or more modules (units), which are stored in the memory 62 and executed by the processor 61 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 63 in the electronic device 6. For example, the computer program 63 may be divided into an acquisition module, a separation module, and a judgment module, and the specific functions of each module are as described above.

[0055] The electronic device may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0056] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0057] The memory 62 may be an internal storage unit of the electronic device 6, such as a hard disk or memory of the electronic device 6. The memory 62 may also be an external storage device of the electronic device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 62 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 62 is used to store the computer program and other programs and data required by the electronic device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0058] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0059] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented. In this embodiment, the computer-readable storage medium can be non-volatile or volatile.

[0060] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0061] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0062] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0063] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0064] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for character recognition in voice dialogue, characterized in that: include: Get the voice dialogue to be recognized; Performing voice separation on the voice dialogue to be recognized to obtain multiple single-channel voices corresponding to multiple speakers, wherein one single-channel voice contains the voice of one speaker; Perform role judgment on each of the multiple single-channel voices to determine the role of the speaker corresponding to each single-channel voice.

2. The method for character recognition in voice dialogue according to claim 1, characterized in that: The step of performing role judgment on each of the multiple single-channel voices to determine the role of the speaker corresponding to each single-channel voice comprises: Each of the multiple single-channel voices is converted into text to obtain the text corresponding to each single-channel voice, and role judgment is performed on the text corresponding to each single-channel voice to determine the role of the speaker corresponding to each single-channel voice.

3. The method for character recognition in voice dialogue according to claim 2, characterized in that: The step of performing role judgment on the text corresponding to each of the single-channel voices to determine the role of the speaker corresponding to each of the single-channel voices comprises: The text corresponding to each single-channel speech is used as the target text, and the text keywords in the target text are obtained; Compare the text keywords in the target text with the keywords in the preset role dictionary, determine whether the text keywords in the target text hit the keywords in the preset role dictionary, obtain the hit keywords of the target text, sum the scores corresponding to all the hit keywords in the preset role dictionary, and obtain the score value of the target text; The score of the target text is compared with a preset score. If the score of the target text reaches the preset score, it is determined that the role corresponding to the speaker of the single-channel speech corresponding to the target text is the role corresponding to the preset role dictionary.

4. The method for character recognition in voice dialogue according to claim 2, characterized in that: The step of performing role judgment on the text corresponding to each of the single-channel voices to determine the role of the speaker corresponding to each of the single-channel voices comprises: Inputting the text corresponding to each single-channel speech into the role prediction model to perform role prediction, and obtaining the probability that the speaker of each single-channel speech belongs to various set roles in the role preset model; The probabilities that the speaker of each single-channel voice belongs to various set roles in the role preset model are compared with each other to determine the role corresponding to the speaker of each single-channel voice.

5. The method for character recognition in voice dialogue according to claim 4, characterized in that: Before the step of inputting the text corresponding to each single-channel speech into the role prediction model for role prediction, the method further includes: Data compression processing is performed on the text corresponding to each single-channel voice, wherein the data compression processing includes any one or both of data abbreviation processing and data interception processing.

6. The method for character recognition in voice dialogue according to claim 2, characterized in that: Before the step of converting each of the multiple single-channel voices into text and obtaining the text corresponding to each single-channel voice, the method further includes: Silence detection is performed on each single-channel voice respectively, and silence segments in each single-channel voice are removed.

7. The method for character recognition in voice dialogue according to claim 1, characterized in that: The step of performing speech separation on the speech dialogue to be recognized to obtain multiple single-channel speech corresponding to multiple speakers, wherein each single-channel speech contains the voice of one speaker, comprises: An end-to-end speech separation model is used to perform speech separation processing on the speech dialogue to be recognized, so that the end-to-end speech separation model outputs multiple single-channel speech, wherein the number of single-channel speech is equal to the number of speakers in the speech dialogue to be recognized, and one single-channel speech records the voice of one speaker.

8. The method for character recognition in voice dialogue according to claim 7, characterized in that: The end-to-end speech separation model includes a coding network layer, a separation network layer and a decoding network layer; The step of using an end-to-end speech separation model to perform speech separation processing on the speech dialogue to be recognized so that the end-to-end speech separation model outputs a corresponding number of single-channel speech according to the number of speakers of the speech dialogue to be recognized includes: Constructing a first feature matrix for characterizing the voice conversation to be recognized according to the duration and sampling frequency of the voice conversation to be recognized through the coding network layer, and performing feature dimension expansion processing on the first feature matrix according to a preset expansion matrix to obtain an expanded second feature matrix; The second feature matrix is ​​masked by the separation network layer according to the number of speakers of the speech conversation to be recognized, so as to obtain a plurality of separated speech corresponding to the number of speakers; The multiple separated speech pieces are respectively subjected to dimensionality reduction processing through the decoding network layer to obtain the single-channel speech pieces corresponding to each of the multiple separated speech pieces.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images