Single-channel speech recognition method and device, and storage medium

By segmenting and extracting features from speech data, combined with processing model acceleration technology, the problems of computational time consumption and high hardware requirements in single-channel multi-role communication dialogues are solved, achieving fast and efficient speech recognition and separation effects.

CN119007709APending Publication Date: 2024-11-22CHINA MOBILE ONLINE SERVICES CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411323079.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-23
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

In existing technologies, speech recognition calculations for single-channel multi-role dialogues are time-consuming and require high hardware performance, making it difficult to achieve good recognition and separation performance under low hardware conditions.

Method used

By segmenting the speech data, multiple speech segments are obtained, and feature extraction and matching are performed in parallel. The pre-processed model is used to accelerate the model. Speech recognition is performed by combining identity information and speech start and end time information. Feedforward neural networks, time-delay neural networks and pooling layers are used for feature extraction and clustering. The model structure is simplified to improve processing speed and efficiency.

Benefits of technology

It achieves fast and efficient speech recognition and separation under low hardware conditions, reduces the requirements for hardware performance, and meets the accuracy and efficiency requirements in single-channel multi-role communication scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007709A_ABST
    Figure CN119007709A_ABST
Patent Text Reader

Abstract

The invention provides a single-channel voice recognition method and device and a storage medium, and relates to the technical field of voice recognition, and the method comprises the steps: obtaining voice data; segmenting the voice data to obtain a plurality of voice segment data; performing feature extraction processing on the multiple pieces of voice segment data to obtain acoustic robust feature information; inputting the acoustic robust feature information into a preset processing model for processing, and obtaining a human voice fragment embedding vector; according to each human voice fragment embedding vector, obtaining identity identification information and corresponding voice starting and ending time information; and obtaining voice recognition result information according to the identity identification information, the voice starting and ending time information and the voice fragment data. The voice segment data is obtained through segmentation processing, parallel processing is facilitated, the processing efficiency is improved, the processing model is obtained after model acceleration processing, the processing process is simplified, the processing speed is improved, the model performance level is maintained, the requirement for hardware performance is lowered, and the accuracy and efficiency requirements of voice recognition are met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a single-channel speech recognition method, device and storage medium. Background Art

[0002] During the communication operation process, voice quality inspection of customer service is required to ensure the service quality. During the customer service communication process, voice quality inspection can be easily performed using voice recognition. In the scene of communication dialogue with multiple roles, such as customer service and customer dialogue, the voices of customer service and customers are stored in the same channel at the same time. The voices of different roles need to be separated for subsequent quality inspection and other processing.

[0003] In the existing technology, the realization of single-channel speech recognition and separation depends on the accurate extraction of voiceprint features and effective state management strategies. The models trained by the commonly used mainstream voiceprint technology require a large number of parameters and calculations to achieve optimal performance, and the calculations take a long time. In the customer service field, there are requirements for the recognition accuracy and efficiency of single-channel multi-role communication dialogues, which leads to high performance requirements for hardware devices. Therefore, how to achieve good speech recognition and separation performance and efficiency in single-channel multi-role communication dialogue scenarios based on low hardware performance conditions is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a single-channel speech recognition method, device and storage medium, which are used to solve the defects of single-channel speech recognition in the prior art, such as long calculation time consumption and high hardware performance requirements.

[0005] The present invention provides a single-channel speech recognition method, comprising: Get voice data; Segmenting the voice data to obtain a plurality of voice segment data; Performing feature extraction processing on the plurality of speech segment data to obtain acoustic robust feature information; Inputting the acoustic robust feature information into a preset processing model for processing to obtain a vocal segment embedding vector; Performing clustering processing according to the embedding vectors of each of the human voice segments to obtain identity information and corresponding voice start and end time information; Acquire speech recognition result information according to the identity information, the speech start and end time information, and the speech segment data, wherein the speech recognition result information includes speech recognition results corresponding to at least two roles; The processing model is obtained by performing model acceleration processing on a trained model.

[0006] According to a single-channel speech recognition method provided by the present invention, segmenting the speech data to obtain a plurality of speech segment data includes: Determining a sliding window position according to the voice data; According to a preset sliding window length and a preset waveform offset length, at the sliding window position of the voice data, the voice sampling points corresponding to the sliding window length are divided according to the waveform offset length to obtain a preset number of divided groups; According to each of the divided groups, determining the divided group with the smallest volume as the target group; Calculate and obtain a waveform offset value according to the speech sampling point with the largest amplitude in the target group; For each of the division groups, subtract the amplitude mean and the waveform offset value from the amplitude of the speech sampling point to form a modified division group; Calculating the zero-crossing rate of each of the modified division groups, determining a segmentation position according to the modified division group with the largest zero-crossing rate, and recording the segmentation position; The segmentation position is used as a new sliding window position, and a new segmentation position is obtained until the sliding window position plus the sliding window length is greater than the end position of the voice data; The voice data is segmented according to each of the recorded segmentation positions to obtain a plurality of voice segment data.

[0007] According to a single-channel speech recognition method provided by the present invention, determining the sliding window position according to the speech data includes: Performing voice activity detection on the voice data to determine non-human voice segments; Removing the non-human voice segment from the speech data; The initial position of the speech data is used as the initial sliding window position.

[0008] According to a single-channel speech recognition method provided by the present invention, the feature extraction process is performed on the plurality of speech segment data to obtain acoustic robust feature information, including: Performing feature extraction processing on the plurality of speech segment data based on linear predictive coding to obtain acoustic feature information; According to the acoustic feature information, based on feature matching processing, determining the corresponding absolute frequency position code; Inputting the acoustic feature information and the absolute frequency position code into a preset convolution residual calculation module to obtain the acoustic robust feature information; Wherein, the absolute frequency position code is determined by pre-training.

[0009] According to a single-channel speech recognition method provided by the present invention, the processing model includes a feedforward neural network layer, a time-delay neural network layer and a pooling layer; the acoustic robust feature information is input into a preset processing model for processing to obtain a human voice segment embedding vector, including: Expanding the acoustic robust feature information based on channel and frequency dimensions to form input feature information; Inputting the input feature information into the feedforward neural network layer for processing to obtain hidden layer feature information; Inputting the hidden layer feature information into the time-delay neural network layer to extract time feature information; Inputting the hidden layer feature information into the pooling layer for processing to obtain global pooling feature information and segmented pooling feature information; Acquire mask information according to the global pooling feature information and the segmented pooling feature information; The embedding vector of the vocal segment is obtained by calculation according to the time feature information and the corresponding mask information.

[0010] According to a single-channel speech recognition method provided by the present invention, the processing model is obtained by the following method: Get the trained model; Convert the trained model into a first ONNX model in a preset format; According to the first ONNX model, determine the corresponding first computational graph information; According to the first computation graph information, replacing the output of the processing operation with a fixed output with a fixed constant to simplify the first computation graph information to form a second computation graph information; Adjust the first ONNX model according to the second computational graph information to obtain a second ONNX model; The second ONNX model is converted into an IR model, and the IR model is accelerated using a preset tool to obtain the processed model.

[0011] According to a single-channel speech recognition method provided by the present invention, the acquiring of speech recognition result information according to the identity information, the speech start and end time information and the speech segment data includes: Acquire text recognition information based on the speech start and end time information and the speech segment data based on speech recognition processing; Determine the role information corresponding to the identity information according to the identity information and the corresponding text recognition information; The speech recognition result information is generated according to the role information and the text recognition information.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a single-channel speech recognition method as described above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the single-channel speech recognition method as described in any one of the above is implemented.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the single-channel speech recognition method as described above is implemented.

[0015] The present invention provides a single-channel speech recognition method, device and storage medium, which have at least the following beneficial effects: By segmenting the voice data to form multiple voice segment data, it is convenient to perform parallel processing to improve processing efficiency. Feature extraction processing is performed on each voice segment data in parallel to obtain acoustic robust feature information, and then input into the preset processing model. Since the processing model is obtained after the trained model is subjected to model acceleration processing, compared with the trained model obtained after the original model training, it can simplify the processing process, improve the processing speed and maintain the model performance level, and then the processing model can quickly process and output the human voice segment embedding vector. According to the human voice segment embedding vector, the corresponding identity and voice start and end time information are determined, and voice recognition is performed in combination with the voice segment data to determine the role and the corresponding voice recognition result, so as to achieve the effect of separating the dialogue role and recognizing the voice content corresponding to the role. In this way, the voice segment data is segmented and processed to obtain multiple voice segment data, which is convenient for parallel processing to improve processing efficiency and shorten processing time. At the same time, the processing model is obtained after the trained model is subjected to model acceleration processing, which can simplify the processing process, improve the processing speed and maintain the model performance level, which is conducive to reducing the requirements for hardware performance and meeting the accuracy and efficiency requirements of voice recognition in the single-channel multi-role communication dialogue scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of a single-channel speech recognition method provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the processing process of a single-channel speech recognition method provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figure 1 and Figure 2 A single-channel speech recognition method of the present invention is described, comprising: S100: Acquire voice data; S200: Segment the voice data to obtain a plurality of voice segment data; S300: performing feature extraction processing on the plurality of speech segment data to obtain acoustic robust feature information; S400: inputting the acoustic robust feature information into a preset processing model for processing to obtain a vocal segment embedding vector; S500: performing clustering processing according to the embedding vectors of each of the human voice segments to obtain identity information and corresponding voice start and end time information; S600: Acquire speech recognition result information according to the identity information, the speech start and end time information, and the speech segment data, wherein the speech recognition result information includes speech recognition results corresponding to at least two roles; The processing model is obtained by performing model acceleration processing on a trained model.

[0022] By segmenting the speech data to form multiple speech segment data, parallel processing is facilitated to improve processing efficiency. Feature extraction is performed on each speech segment data in parallel to obtain acoustic robust feature information, which is then input into a preset processing model. Since the processing model is obtained after model acceleration processing of the trained model, compared with the original trained model obtained after model training, it can simplify the processing process, improve the processing speed and maintain the model performance level, so that the processing model can quickly process and output the vocal segment embedding vector. According to the vocal segment embedding vector, the corresponding identity and speech start and end time information are determined, and speech recognition is performed in combination with the speech segment data to determine the role and the corresponding speech recognition result, so as to achieve the effect of separating the dialogue roles and recognizing the speech content corresponding to the role.

[0023] In this way, the voice fragment data is segmented and processed to obtain multiple voice fragment data, which is convenient for parallel processing to improve processing efficiency and shorten processing time. At the same time, the processing model is obtained after model acceleration processing of the trained model, which can simplify the processing process, improve the processing speed and maintain the model performance level, which is conducive to reducing the requirements for hardware performance and meeting the accuracy and efficiency requirements of speech recognition in single-channel multi-role communication dialogue scenarios.

[0024] It is understandable that the voice data is generated by the dialogue process of at least two characters. The identity information is used to distinguish the objects to which different voices belong. The voice start and end time information represents the start and end time points of a voice segment. The voice data can be streaming data, that is, the input is voice stream data.

[0025] In some embodiments of the present invention, the voice data may be in a format with a sampling rate of 8000 Hz and a bit depth of 16 bits.

[0026] When there are multiple characters' voices in a single channel, there is a problem of different characters' voices sticking together and overlapping. For this problem, refer to Figure 2 In some embodiments of a single-channel speech recognition method of the present invention, S200 includes: Determining a sliding window position according to the voice data; According to a preset sliding window length and a preset waveform offset length, at the sliding window position of the voice data, the voice sampling points corresponding to the sliding window length are divided according to the waveform offset length to obtain a preset number of divided groups; According to each of the divided groups, determining the divided group with the smallest volume as the target group; Calculate and obtain a waveform offset value according to the speech sampling point with the largest amplitude in the target group; For each of the division groups, subtract the amplitude mean and the waveform offset value from the amplitude of the speech sampling point to form a modified division group; Calculating the zero-crossing rate of each of the modified division groups, determining a segmentation position according to the modified division group with the largest zero-crossing rate, and recording the segmentation position; The segmentation position is used as a new sliding window position, and a new segmentation position is obtained until the sliding window position plus the sliding window length is greater than the end position of the voice data; The voice data is segmented according to each of the recorded segmentation positions to obtain a plurality of voice segment data.

[0027] By dividing long segments of speech data into small segments of speech data, it is convenient for subsequent parallel processing. At the same time, in order to ensure the integrity of each speech segment data, the speech data is analyzed and processed locally by using a sliding window, combined with zero-crossing rate judgment to find a suitable segmentation position, to ensure the integrity of the speech segment and avoid over-segmentation, which is conducive to improving the integrity of the speech segment data and realizing adaptive segmentation of speech data.

[0028] The specific segmentation process is as follows: first, at the sliding window position, the local speech data corresponding to the sliding window length C is processed as the target data; then, according to the waveform offset length shiftsize, the speech sampling points included in the local speech data are divided to form k groups, and the value of k is determined by the sliding window length C and the waveform offset length shiftsize; the volume of each divided group is calculated, and the divided group with the smallest volume is taken as the target group, because the divided group with the smallest volume can easily distinguish between environmental noise and clear sound; according to the speech sampling point with the largest absolute amplitude value in the target group, the waveform offset value is calculated; for each divided group, the amplitude mean and waveform offset of each speech sampling point are subtracted from each other. value, forming a new amplitude of the speech sampling point, and the division group is also converted into a modified division group; the zero-crossing rate of each modified division group is calculated. Since the zero-crossing rate will increase significantly when the human voice appears, pauses, and disappears, the segmentation position is determined according to the modified division group with the largest zero-crossing rate, so that the segmentation can be accurately performed to ensure the integrity of the speech segment data after segmentation; after determining a segmentation position, the sliding window is moved to the segmentation position, that is, the segmentation position is used as the new sliding window position, and the above process is re-executed to obtain a new segmentation position, and multiple segmentation positions are repeatedly obtained until the sliding window moves to the end position of the speech data, and the acquisition of the segmentation position is completed; finally, the speech data is segmented according to each segmentation position to obtain the required speech segment data.

[0029] It should be emphasized that the calculation of the zero-crossing rate needs to subtract the DC offset, that is, the amplitude mean, so that the number of zero-crossing times can reflect the degree of change and frequency of the voice sampling point, and then the zero-crossing rate of the voice sampling point increases significantly when the human voice appears, pauses, and disappears, and the duration of the human voice part in the voice can be identified, so as to facilitate the determination of the segmentation position. Only by subtracting the DC offset, that is, the amplitude mean, it is impossible to distinguish between the unvoiced sound and the environmental noise, resulting in a decrease in the accuracy of recognition. To address this problem, the present invention calculates the waveform offset value according to the division group with the smallest volume, and corrects the voice sampling point based on the waveform offset value, so that when the segmentation position is identified based on the zero-crossing rate, the segmentation can be performed more accurately, so as to achieve the effect of distinguishing between the unvoiced sound and the environmental noise.

[0030] In some embodiments of the present invention, the volume calculation of each divided group may be calculated by the following expression:

[0031] in, The volume of the jth group; shiftsize is the waveform offset length; is the value of the i-th speech sampling point in the partition group.

[0032] In some embodiments of the present invention, the new amplitude of each speech sampling point in the modified divided group may be calculated by the following expression:

[0033] in, is the initial amplitude of the speech sampling point; is the mean amplitude of the i-th group, Belongs to the i-th group division group; is the middle amplitude; It is the new amplitude after the speech sampling point is corrected; shift is the waveform offset value.

[0034] In some embodiments of the present invention, the zero-crossing rate of each partition group may be calculated by the following expression:

[0035] Wherein, ZCR is the zero crossing rate; N is the number of speech sampling points in the divided group; The new amplitude after correction of the nth speech sampling point; is the new amplitude after correction of the n-1th speech sampling point; f[ ] is the indicator function. When the amplitude sign of the adjacent speech sampling point changes, that is, The result is 1 when , otherwise it is 0.

[0036] refer to Figure 2In some embodiments of a single-channel speech recognition method of the present invention, S210 includes: Performing voice activity detection on the voice data to determine non-human voice segments; Removing the non-human voice segment from the speech data; The initial position of the speech data is used as the initial sliding window position.

[0037] By performing voice activity detection on voice data to identify non-human voice segments for removal, the proportion of valid data can be increased, the amount of data in subsequent segmentation processing can be reduced, and the processing efficiency of the segmentation processing can be improved.

[0038] It is understandable that after removing the non-voice segment, the remaining vocal part still includes a part of the speech before the vocal appears and after the vocal disappears, so as to avoid removing effective vocal information and facilitate the subsequent calculation of the zero-crossing rate and segmentation based on the zero-crossing rate.

[0039] In some embodiments of the present invention, the voice data may be input into a VAD (Voice Activity Detection) module for processing to remove non-human voice segments.

[0040] refer to Figure 2 In some embodiments of a single-channel speech recognition method of the present invention, the S300 includes: Performing feature extraction processing on the plurality of speech segment data based on linear predictive coding to obtain acoustic feature information; According to the acoustic feature information, based on feature matching processing, determining the corresponding absolute frequency position code; Inputting the acoustic feature information and the absolute frequency position code into a preset convolution residual calculation module to obtain the acoustic robust feature information; Wherein, the absolute frequency position code is determined by pre-training.

[0041] Based on linear prediction coding, feature extraction processing is performed on each speech segment data to obtain the acoustic feature information corresponding to each speech segment data. Based on the acoustic feature information, the corresponding absolute frequency position code is determined, and the acoustic feature information and the corresponding absolute frequency position code are input into the convolution residual calculation module to obtain the acoustic robust feature information corresponding to the domain speech segment data. In this way, the absolute frequency position code is obtained by training the learnable vector. By adding the corresponding absolute frequency position code, that is, the absolute frequency position of the speech when the acoustic feature information is input into the convolution residual calculation module, the frequency domain features can be extracted more stably and reliably, so that the obtained acoustic robust feature information can more accurately reflect the acoustic features of the speech segment data, which is conducive to improving the accuracy of subsequent speech recognition processing.

[0042] In audio, different frequencies represent different speech features, such as pitch, etc. Therefore, the position information of the speech in the frequency dimension is crucial. Absolute frequency position encoding can help the model clearly know the frequency dimension position of the speech, rather than relying solely on the convolution operation or feature extraction of the input data itself. Therefore, adding absolute frequency position encoding is conducive to more accurate, stable and reliable feature extraction.

[0043] In some embodiments of the present invention, a plurality of convolution residual calculation modules may be connected in series, and each convolution residual calculation module corresponds to an absolute frequency position encoding obtained by input training. For example, three convolution residual calculation modules may be connected in series.

[0044] In some embodiments of the present invention, the acoustic feature information may be an Fbank acoustic feature. The acoustic feature information and the acoustic robust feature information may be represented in the form of a feature map.

[0045] refer to Figure 2 In some embodiments of a single-channel speech recognition method of the present invention, the processing model includes a feedforward neural network layer, a time-delay neural network layer and a pooling layer; S400 includes: Expanding the acoustic robust feature information based on channel and frequency dimensions to form input feature information; Inputting the input feature information into the feedforward neural network layer for processing to obtain hidden layer feature information; Inputting the hidden layer feature information into the time-delay neural network layer to extract time feature information; Inputting the hidden layer feature information into the pooling layer for processing to obtain global pooling feature information and segmented pooling feature information; Acquire mask information according to the global pooling feature information and the segmented pooling feature information; The embedding vector of the vocal segment is obtained by calculation according to the time feature information and the corresponding mask information.

[0046] The acoustic robust feature information is represented in the form of a feature graph, including channel dimension, frequency dimension and time dimension. The acoustic robust feature information is expanded along the channel and frequency dimensions to form input feature information and input into the feedforward neural network layer for processing to obtain hidden layer feature information. The hidden layer feature information is input into the time delay neural network layer (TDNN) to extract time feature information. However, since the time feature information is obtained based on the acoustic robust feature information, the time feature information obtained here is a description of the local features of the segment, which is not optimal for the role separation task. Therefore, the hidden layer feature information is input into the pooling layer for processing. The pooling layer includes a global pooling layer and a segmented pooling layer to obtain global pooling feature information and segmented pooling feature information. The global pooling feature information and segmented pooling feature information are input into the two-level feedforward neural network for processing. The feedforward neural network is combined with the ReLU function. After processing, the mask information is obtained. The mask information includes the speaker of interest and its voice features. According to the time feature information and the corresponding mask information, the embedding vector of the human voice segment is obtained, which is convenient for subsequent tasks such as role separation.

[0047] refer to Figure 2 In some embodiments of the present invention, the processing model may connect in series a plurality of processing modules formed by a feedforward neural layer, a TDNN, a pooling layer, and two connected feedforward neural networks combined with a ReLU function to further improve the accuracy of the embedding vector of the vocal segment.

[0048] In some embodiments of the present invention, the acquisition of mask information may be calculated by the following expression:

[0049] in, It is the global pooling feature information; is the tth hidden layer feature information; T is the total number of hidden layer feature information, corresponding to the number of speech segment data; is the segmented pooling feature information of the kth segment; is the starting position of the kth segment; is the mask information; and is the weight matrix; and is the bias vector; σ( ) is the sigmoid function; δ( ) is the ReLU function.

[0050] It can be understood that the starting position of the kth segment corresponds to the starting position of the kth segment of voice fragment data.

[0051] In some embodiments of the present invention, the vocal segment embedding vector may be obtained by performing a Hadamard product multiplication operation on the hidden layer feature information and the corresponding mask information.

[0052] refer to Figure 2 In some embodiments of a single-channel speech recognition method of the present invention, the processing model is obtained by: S010: Get the trained model; S020: Convert the trained model into a first ONNX model in a preset format; S030: Determine corresponding first computational graph information according to the first ONNX model; S040: According to the first computation graph information, replace the output of the processing operation with a fixed output with a fixed constant to simplify the first computation graph information to form second computation graph information; S050: Adjust the first ONNX model according to the second computational graph information to obtain a second ONNX model; S060: Convert the second ONNX model into an IR model, use a preset tool to accelerate the processing of the IR model, and obtain the processing model.

[0053] The processing of S400 can be performed through model processing, for example, after training through a Pytorch model. In order to improve the speed of model processing, the trained model, such as the Pytorch model, is converted into an open format ONNX model, i.e., the first ONNX model, and then the ONNX Runtime engine is used to infer the calculation graph of the first ONNX model, and then constant output is used to replace redundant calculation operations to obtain a simplified model structure diagram, i.e., the second calculation graph information, and then the first ONNX model is simplified to obtain the second ONNX model, and then the second ONNX model is converted into an IR (intermediate representation) model, and the IR model is accelerated using preset tools, such as the OpenVino tool, to finally obtain a processing model that meets the requirements.

[0054] In this way, after the model structure is simplified and accelerated, the processing model can be lightweight and the reasoning speed can be improved, the demand for hardware performance can be reduced and the processing efficiency can be improved, meeting the requirements of speech recognition accuracy and speech recognition efficiency.

[0055] In some embodiments of the present invention, the processing of S300 can also be implemented by a processing model, that is, the trained model that implements steps S300 and S400 is simplified and accelerated through the above-mentioned structure to form a processing module, and the multiple speech segment data after segmentation are input into the processing model to obtain the corresponding vocal segment embedding vector.

[0056] In some embodiments of a single-channel speech recognition method of the present invention, the acquiring speech recognition result information according to the identity information, the speech start and end time information, and the speech segment data includes: Acquire text recognition information based on the speech start and end time information and the speech segment data based on speech recognition processing; Determine the role information corresponding to the identity information according to the identity information and the corresponding text recognition information; The speech recognition result information is generated according to the role information and the text recognition information.

[0057] The embedding vectors of each vocal segment are clustered. Since different roles have different speech features, clusters corresponding to the roles are formed after clustering. Identity information is assigned to different clusters, and the speech start and end time information is determined based on the embedding vectors of the vocal segments included in the corresponding clusters. According to the speech start and end time information and the corresponding voice segment data, the speech is converted into text based on speech recognition processing to obtain text recognition information, which corresponds to the identity information. According to the content of the text recognition information, the role information is determined, that is, the text recognition information corresponds to the customer service role or the customer role. Finally, based on the role information and the corresponding text recognition information, the speech recognition result information is generated to achieve the effect of speech content recognition and role separation.

[0058] It should be noted that identity identification information is used to identify different role identities, but cannot determine the specific role. For example, the identity identification information is A, but it cannot be known whether A is a customer service representative or a customer. It is necessary to subsequently combine text recognition information to determine whether it is a customer service representative or a customer, that is, the role information.

[0059] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the above-mentioned single-channel speech recognition method.

[0060] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0061] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute a single-channel speech recognition method provided by the above methods.

[0062] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute a single-channel speech recognition method provided by the above methods.

[0063] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0064] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0065] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0066] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0067] All actions to obtain signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located, and with the authorization given by the owner of the corresponding device.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A single-channel speech recognition method, characterized in that: include: Get voice data; Segmenting the voice data to obtain a plurality of voice segment data; Performing feature extraction processing on the plurality of speech segment data to obtain acoustic robust feature information; Inputting the acoustic robust feature information into a preset processing model for processing to obtain a vocal segment embedding vector; Performing clustering processing according to the embedding vectors of each of the human voice segments to obtain identity information and corresponding voice start and end time information; Acquire speech recognition result information according to the identity information, the speech start and end time information, and the speech segment data, wherein the speech recognition result information includes speech recognition results corresponding to at least two roles; The processing model is obtained by performing model acceleration processing on a trained model.

2. A single-channel speech recognition method according to claim 1, characterized in that: The segmenting process of the voice data to obtain a plurality of voice segment data includes: Determining a sliding window position according to the voice data; According to a preset sliding window length and a preset waveform offset length, at the sliding window position of the voice data, the voice sampling points corresponding to the sliding window length are divided according to the waveform offset length to obtain a preset number of divided groups; According to each of the divided groups, determining the divided group with the smallest volume as the target group; Calculate and obtain a waveform offset value according to the speech sampling point with the largest amplitude in the target group; For each of the division groups, subtract the amplitude mean and the waveform offset value from the amplitude of the speech sampling point to form a modified division group; Calculating the zero-crossing rate of each of the modified division groups, determining a segmentation position according to the modified division group with the largest zero-crossing rate, and recording the segmentation position; The segmentation position is used as a new sliding window position, and a new segmentation position is obtained until the sliding window position plus the sliding window length is greater than the end position of the voice data; The voice data is segmented according to each of the recorded segmentation positions to obtain a plurality of voice segment data.

3. A single-channel speech recognition method according to claim 2, characterized in that: The step of determining the sliding window position according to the voice data comprises: Performing voice activity detection on the voice data to determine non-human voice segments; Removing the non-human voice segment from the speech data; The initial position of the speech data is used as the initial sliding window position.

4. A single-channel speech recognition method according to claim 1, characterized in that: The step of performing feature extraction processing on the plurality of speech segment data to obtain acoustic robust feature information includes: Performing feature extraction processing on the plurality of speech segment data based on linear predictive coding to obtain acoustic feature information; According to the acoustic feature information, based on feature matching processing, determining the corresponding absolute frequency position code; Inputting the acoustic feature information and the absolute frequency position code into a preset convolution residual calculation module to obtain the acoustic robust feature information; Wherein, the absolute frequency position code is determined by pre-training.

5. A single-channel speech recognition method according to claim 1, characterized in that: The processing model includes a feedforward neural network layer, a time-delay neural network layer, and a pooling layer; the step of inputting the acoustic robust feature information into a preset processing model for processing to obtain a human voice segment embedding vector includes: Expanding the acoustic robust feature information based on channel and frequency dimensions to form input feature information; Inputting the input feature information into the feedforward neural network layer for processing to obtain hidden layer feature information; Inputting the hidden layer feature information into the time-delay neural network layer to extract time feature information; Inputting the hidden layer feature information into the pooling layer for processing to obtain global pooling feature information and segmented pooling feature information; Acquire mask information according to the global pooling feature information and the segmented pooling feature information; The embedding vector of the vocal segment is obtained by calculation according to the time feature information and the corresponding mask information.

6. A single-channel speech recognition method according to claim 1, characterized in that: The processing model is obtained by: Get the trained model; Convert the trained model into a first ONNX model in a preset format; According to the first ONNX model, determine the corresponding first computational graph information; According to the first computation graph information, replacing the output of the processing operation with a fixed output with a fixed constant to simplify the first computation graph information to form a second computation graph information; Adjust the first ONNX model according to the second computational graph information to obtain a second ONNX model; The second ONNX model is converted into an IR model, and the IR model is accelerated using a preset tool to obtain the processed model.

7. A single-channel speech recognition method according to claim 1, characterized in that: The acquiring of speech recognition result information according to the identity information, the speech start and end time information and the speech segment data includes: Acquire text recognition information based on the speech start and end time information and the speech segment data based on speech recognition processing; Determine the role information corresponding to the identity information according to the identity information and the corresponding text recognition information; The speech recognition result information is generated according to the role information and the text recognition information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the single-channel speech recognition method as described in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a single-channel speech recognition method as claimed in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, a single-channel speech recognition method as claimed in any one of claims 1 to 7 is implemented.