Speech recognition method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides a voice recognition method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: carrying out integrity detection on received voice data until complete voice data is detected, and determining whether the mute time length reaches a mute time length threshold value or not; under the condition that the mute duration reaches a mute duration threshold value, determining first voice data; and performing voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data. According to the invention, the accuracy of speech recognition can be improved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a speech recognition method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, Voice Activity Detection (VAD) technology determines that the user has finished speaking if the silence duration exceeds the preset limit, triggers the VAD end, and sends the received voice data to the speech recognition module for speech recognition. However, if the user hesitates for more than the preset silence duration while issuing a voice command, the speech will be truncated, leading to speech recognition errors.

[0003] To address the aforementioned issues, relevant technologies typically trigger a VAD (Voice Over Receive) to stop voice reception when the silence duration times out. Then, they assess the completeness of the truncated voice data to determine if the user's voice command has been fully input. If incomplete, they continue receiving voice data for a period, finally merging and analyzing the segments of voice data from several truncated segments for recognition. However, this method may lose some voice data between the truncated segments, resulting in incomplete voice data even after merging. This leads to low accuracy in voice recognition and consequently reduces the overall accuracy of speech recognition. Summary of the Invention

[0004] This application provides a speech recognition method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the accuracy of speech recognition.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a speech recognition method, the method comprising:

[0007] The integrity of the voice data received from the start of the voice activity is checked. If complete voice data is detected, it is determined whether the silence duration has reached the silence duration threshold.

[0008] If the silence duration reaches the silence duration threshold, the first voice data is determined;

[0009] The first speech data is subjected to speech recognition to obtain the recognition result corresponding to the first speech data.

[0010] This application provides a voice recognition device, including:

[0011] The voice detection module is used to perform integrity detection on the voice data received from the start of the voice activity. If complete voice data is detected, it determines whether the silence duration has reached the silence duration threshold.

[0012] Voice determination module: used to determine the first voice data when the silence duration reaches the silence duration threshold;

[0013] The speech recognition module is used to perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0014] Optionally, the voice detection module is further configured to perform integrity detection on the received voice data, and if no complete voice data is detected, continue receiving voice data, and after a first time interval, perform integrity detection on the voice data received from the start of the voice activity.

[0015] Optionally, the voice detection module is further configured to stop receiving voice data when the silence duration reaches the silence duration threshold, and to use the voice data received from the start of reception to the stop of reception as the first voice data.

[0016] Optionally, the voice determination module is further configured to, before performing voice recognition on the first voice data and obtaining the recognition result corresponding to the first voice data, stop receiving if the silence duration within the first time interval is greater than the silence duration threshold, and use the voice data received from the start of receiving to the stop of receiving as the first voice data.

[0017] Optionally, the speech recognition module is further configured to determine whether the semantic intent of the first speech data is clear; if the semantic intent of the first speech data is clear, perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0018] Optionally, the speech recognition module is further configured to: after determining whether the semantic intent of the first speech data is clear, if the semantic intent of the first speech data is unclear, detect whether speech activity occurs within a first duration; if speech activity is detected within the first duration, determine the second speech data corresponding to the speech activity occurring within the first duration; concatenate the first speech data and the second speech data to obtain third speech data; and perform speech recognition on the third speech data to obtain the recognition result corresponding to the third speech data.

[0019] Optionally, the speech recognition module is further configured to detect whether speech activity has occurred within a first duration, and if no speech activity is detected within the first duration, perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0020] Optionally, the voice detection module is further configured to determine that the voice activity has not ended if voice data is received when the silence duration has not reached the silence duration threshold.

[0021] Optionally, the speech detection module is further configured to: after determining whether the semantic intent of the first speech data is clear, if the semantic intent of the first speech data is clear, detect whether speech activity occurs during a second duration; if speech activity is detected during the second duration, determine the fourth speech data corresponding to the speech activity during the second duration; determine the correlation between the first speech data and the fourth speech data; if the correlation is greater than or equal to a correlation threshold, concatenate the first speech data and the fourth speech data to obtain fifth speech data; and perform speech recognition on the fifth speech data to obtain the recognition result corresponding to the fifth speech data.

[0022] Optionally, the voice determination module is further configured to perform integrity detection on the voice data received from the voice activities that occur within the first duration. If the silence duration after detecting complete voice data reaches the silence duration threshold, the voice data received from the start of the voice activities within the first duration to the stop of reception is determined as the second voice data.

[0023] Alternatively, the voice determination module is further configured to perform integrity detection on the voice data received from the voice activities occurring within the second duration. If the silence duration after detecting complete voice data reaches the silence duration threshold, then the voice data received from the start to the stop of the voice activities within the second duration is determined as the fourth voice data.

[0024] Optionally, the speech detection module is further configured to extract acoustic features and text features from the speech data received from the start of the speech activity; and to perform integrity detection on the speech data received from the start of the speech activity based on the acoustic features and the text features.

[0025] This application provides an electronic device, including:

[0026] Memory is used to store executable instructions for a computer;

[0027] The processor, when executing computer-executable instructions stored in the memory, implements the speech recognition method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the speech recognition method provided in this application when executed by a processor.

[0029] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the speech recognition method provided in this application.

[0030] The embodiments of this application have the following beneficial effects:

[0031] During voice data reception, the integrity of the voice data is checked until complete voice data is detected, and then the silence duration is determined. If the silence duration reaches a silence duration threshold, the first voice data is selected. This achieves the goal of first obtaining complete voice data based on integrity detection, and then truncating the received voice data by judging the silence duration. In this way, the selected first voice data satisfies both the integrity detection requirements and the silence duration requirements, thereby improving the integrity of the first voice data and thus improving the accuracy of speech recognition based on the first voice data. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of an optional flow of the speech recognition method provided in the embodiments of this application;

[0033] Figure 2 This is a schematic diagram of an optional flow of the speech recognition method provided in the embodiments of this application;

[0034] Figure 3 This is a schematic diagram of an optional flow of the speech recognition method provided in the embodiments of this application;

[0035] Figure 4 This is a schematic diagram of an optional flow of the speech recognition method provided in the embodiments of this application;

[0036] Figure 5 This is a flowchart illustrating the application of the speech recognition method provided in this application to a real-world scenario;

[0037] Figure 6 This is a schematic diagram of the structure of the speech recognition device provided in the embodiments of this application;

[0038] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0042] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0043] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0044] Currently, to ensure complete speech is captured for speech recognition, related technologies typically use Voice Activity Detection (VAD) to detect user speech activity and determine if the user has finished speaking. For example, after detecting that the user has started speaking, the VAD is activated, and the system continuously receives the user's speech data. If the user is silent (i.e., no longer speaking), a preset time period (silence duration, usually a few milliseconds) is waited. If the user remains silent, it is considered that the user has finished speaking, and the currently received audio is sent to the speech recognition module for recognition. However, during actual voice interaction, users may hesitate or pause while issuing voice commands, causing the silence time to exceed the silence duration. This results in the speech being truncated, and incomplete speech data is sent to the speech recognition module for recognition. For example, if a user says, "I want to hear... Chopin's violin," and the pause exceeds the silence duration, only the "I want to hear" part before the pause will be truncated for speech recognition. Alternatively, even if the user speaks a complete sentence, if the meaning of the sentence is unclear, it may also affect the recognition of the user's intent, leading to inaccurate speech recognition results.

[0045] To address the aforementioned issues, current technologies typically trigger a VAD (Voice Over Receive) to stop voice reception after the silence period expires. Then, they assess the completeness of the truncated voice data to determine if the user's voice command has been fully input. If incomplete, they continue receiving voice data for a period, finally merging and analyzing the segments of voice data from several truncated segments for recognition. However, this method may lose some voice data between the truncated segments, resulting in incomplete voice data even after merging. This leads to low accuracy in voice recognition and consequently reduces the overall accuracy of speech recognition. Furthermore, current technologies lack solutions for situations where the user speaks a complete sentence, but the meaning of the sentence is unclear, resulting in inaccurate speech recognition.

[0046] This application provides a speech recognition method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can improve the accuracy of speech recognition. The method of this application can be applied to electronic devices. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server. In some embodiments, the electronic device may include a vehicle control unit (VCU), vehicle computer, and other in-vehicle terminals, depending on the actual situation; this application does not limit the specific choice.

[0047] See Figure 1 , Figure 1 This is a first flowchart illustrating the speech recognition method provided in an embodiment of this application, as shown below. Figure 1 As shown, including S101-S103, as follows:

[0048] S101. Perform integrity detection on the voice data received from the start of the voice activity. If complete voice data is detected, determine whether the silence duration has reached the silence duration threshold.

[0049] In S101, voice detection is performed based on VAD technology. Upon detecting the start of voice activity, the start time of the voice activity, i.e., VAD start, is recorded, and voice data is continuously received in real time. The integrity of the received voice data is then checked to ensure its completeness. For example, the voice data includes voice commands issued by the user.

[0050] For example, detecting human voice activity in a silent environment can determine the start of speech activity.

[0051] In some embodiments, when the start of voice activity is detected, it is detected whether the received voice data is complete, and until the voice data is detected to be complete, it is determined whether the silence duration has reached a silence duration threshold.

[0052] In some embodiments, the data received from the voice activity can be periodically detected according to a preset time interval, such as a first time interval. It is understood that, as the voice activity continuously receives new voice data, the voice data for which the first network model needs to perform integrity checks each time may include the voice data from the start of the voice activity to the current detection time.

[0053] In step S101, if complete voice data is detected, it indicates that the user may have finished speaking a complete sentence. The electronic device can then begin recording the silence duration to determine if the silence duration has reached a preset silence duration threshold. It should be noted that the silence duration begins accumulating from the moment the silence state of the detected voice activity is detected.

[0054] In some embodiments, if complete voice data is not detected, i.e., if the voice data received since the start of the voice activity is incomplete, it is determined that the voice activity has not ended, and voice data for the voice activity continues to be received. After a first time interval, the voice data received since the start of the voice activity is detected, and this periodic detection is performed until complete voice data is detected. The completeness detection performed after the first time interval covers all voice data received since the start of the voice activity, including all voice data received during the first time interval.

[0055] It should be noted that the first time interval can be a preset fixed time interval, or a first time interval of variable duration can be used according to the actual situation. For example, the specific duration of the first time interval can be determined according to the text content or acoustic characteristics of the received voice data, etc. The specific selection is made according to the actual situation, and this application embodiment does not limit it.

[0056] S102. If the silence duration reaches the silence duration threshold, determine the first voice data.

[0057] Here, since S101 has already determined the integrity of the voice data, if the silence duration reaches the silence duration threshold, it means that the user paused for the threshold duration after inputting a complete sentence. This situation usually indicates that the user has completed all voice data input. Therefore, the electronic device determines that the voice activity has ended, triggers the VAD end in the VAD detection, stops receiving voice data, records the end time of the voice activity, and uses the voice data received from the start to the end of the voice activity as the first voice data.

[0058] As can be seen, compared to the related technologies that initiate VAD start upon detecting voice activity and trigger VAD end after a fixed silence duration, the VAD end in this embodiment can be dynamically postponed based on the completeness of the currently received user voice data. That is, if incomplete user voice data is detected, the silence duration is not determined, effectively postponing the VAD end until complete voice data is detected, at which point the silence duration is determined, thus establishing the VAD end. This ensures that the first voice data captured between VAD start and VAD end is complete, thereby avoiding truncating voice data when it is incomplete, reducing data loss, and improving the accuracy of speech recognition of the first voice data.

[0059] In some embodiments, if voice data is received before the silence duration reaches the silence duration threshold, it is determined that the voice activity has not ended, and the silence duration is reset until the silence duration reaches the silence duration threshold, at which point the voice activity is determined to have ended.

[0060] For example, if a voice activity is detected to have entered a silent state, but the silence duration has not reached the silence duration threshold, and new voice data is received, indicating that the user has started voice input again, in order to ensure the integrity of the voice data, it is determined that the voice activity has not ended, and the VAD end is not triggered. At the same time, the silence duration is reset, such as clearing the accumulated silence duration to zero, until a voice activity is detected to have entered a silent state again, and the silence duration is accumulated from zero until the silence duration reaches the silence duration threshold, at which point the voice activity is determined to have ended.

[0061] S103. Perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0062] In step S103, once the first speech data is determined, speech recognition is performed on the first speech data that has passed the integrity check to obtain the recognition result corresponding to the first speech data.

[0063] For example, the recognition result corresponding to the first voice data may include the recognized user operation command, query request or device interaction request, etc. The specific selection is made according to the actual situation, and this application embodiment does not limit it.

[0064] Understandably, during the voice data reception process, the integrity of the voice data is checked until complete voice data is detected, and then the silence duration is determined. If the silence duration reaches a silence duration threshold, the first voice data is selected. This achieves the goal of first obtaining complete voice data based on integrity detection, and then truncating the received voice data by judging the silence duration. In this way, the selected first voice data satisfies both the integrity detection requirements and the silence duration requirements, thereby improving the integrity of the first voice data and consequently improving the accuracy of speech recognition based on the first voice data.

[0065] In some embodiments, during the detection of continuously received voice data, if incomplete voice data is detected, it is determined that the voice activity has not ended, and voice data reception continues. After a first time interval, the next integrity check is initiated, and so on. Specifically, if the voice activity becomes silent during the first time interval, and the silence duration within the first time interval exceeds a silence duration threshold, it is determined that the voice activity has ended, triggering VADend and stopping voice data reception. It can be seen that in this case, the VADend time point represents the moment when the silence duration threshold is extended from the start of the first time interval timing. The voice data received from the start of reception (VAD start time point) to the stop of reception (VAD end time point, i.e., the moment when the silence duration threshold is extended from the start of the first time interval timing) is used as the first voice data, and then speech recognition is performed on the first voice data to obtain the recognition result corresponding to the first voice data.

[0066] In other words, during the process of dynamically delaying the VAD end based on voice integrity detection, if the silence duration exceeds the silence duration threshold, it indicates that the user may want to end the current voice input. At this point, the voice activity is determined to end, and the voice data is intercepted and sent for inspection. This can avoid waiting for the user to input for a long time and improve the response speed of voice interaction.

[0067] In some embodiments, acoustic features and text features can be extracted from the speech data received from the start of the speech activity; and integrity checks can be performed on the speech data received from the start of the speech activity based on the acoustic features and text features.

[0068] For example, the integrity of speech data can be detected using a pre-trained first network model. The first network model can extract acoustic features and / or text features from the currently received speech data, and determine whether the currently received speech data is complete based on the extracted acoustic features and / or text features.

[0069] For example, using a first network model, acoustic and textual features are extracted from the received speech data from the start of the speech activity. Based on the acoustic and textual features, it is determined whether the received speech data is complete. For instance, based on the acoustic features, it can be determined whether the sound features of the speech data are complete. For example, complete sound features can characterize whether the speech data conforms to the sound characteristics of the beginning and end of normal speech data, and does not conform to the sound characteristics of abrupt interruptions or truncations. Based on the textual features, it can be determined whether the text content of the speech data is complete. If both the sound features and text content of the speech data are complete, the speech data is determined to be complete through integrity detection.

[0070] For example, the first network model may include, but is not limited to, any one of Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), or Text Recognition Neural Network, or a combination of two or more of them. The specific selection is made according to the actual situation, and the embodiments of this application do not limit it.

[0071] Understandably, by combining the acoustic features of speech data with text features for integrity detection, the accuracy of integrity detection is improved, which in turn improves the accuracy of speech recognition on the first speech data obtained based on integrity detection.

[0072] In some embodiments, based on Figure 1 ,like Figure 2 As shown, S103 can be achieved by executing the processes S1031-S1032, as follows:

[0073] S1031. Determine whether the semantic intent of the first speech data is clear.

[0074] In this embodiment, based on the first speech data determined through integrity detection, it is possible to further determine whether the semantic intent of the first speech data is clear before performing speech recognition on the first speech data. In some embodiments, the first speech data is input into a second network model, and the second network model performs semantic intent recognition on the first speech data to determine whether the semantic intent of the first speech data is clear.

[0075] For example, the second network model can determine whether the semantic intent of the first speech data is clear by converting the first speech data into speech-to-text and then using the converted text.

[0076] For example, the second network model may include a Natural Language Processing (NLP) model, or other types of neural network models for analyzing semantic intent. The specific model may be selected according to the actual situation, and this application embodiment does not limit it.

[0077] S1032. If the semantic intent of the first speech data is clear, perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0078] In this embodiment of the application, if the semantic intent of the first voice data is clear, it means that the data integrity and intent clarity of the first voice data have passed the check. The first voice data can be sent to the speech recognition model to perform speech recognition on the first voice data and obtain the recognition result corresponding to the first voice data, which is used as the recognition result of the voice activity.

[0079] Understandably, by identifying whether the semantic intent of the first speech data is clear, the final speech recognition is triggered when the semantic intent is clear, thus obtaining the recognition result. In this way, speech recognition is achieved based on complete and clearly defined speech data, reducing the impact of incomplete or ambiguous speech data on the final recognition result and improving the accuracy of speech recognition.

[0080] In some embodiments, such as Figure 3 As shown, after determining whether the semantic intent of the first speech data is clear through S1031, S201-S204 can also be executed, as follows:

[0081] S201. If the semantic intent of the first speech data is unclear, detect whether speech activity has occurred within the first duration.

[0082] In step S201, if the semantic intent of the first voice data is unclear, it indicates that the user may need to supplement the previous voice. The electronic device waits for a preset first duration and detects whether voice activity will occur within the first duration. Here, the first duration can be set according to the actual situation, and this embodiment does not limit it.

[0083] S202. If voice activity is detected within a first duration, determine the second voice data corresponding to the voice activity that occurred within the first duration.

[0084] In S202, if voice activity is detected within the first duration, it means that the user started voice input again when the semantic intent of the previously input first voice data was unclear, and the correlation between the two voice inputs is high. The electronic device will use the voice data received through the voice activity that occurred within the first duration as the corresponding second voice data.

[0085] In some embodiments, the voice data received from voice activities occurring within a first duration can be subjected to integrity detection. If the silence duration after detecting complete voice data reaches a silence duration threshold, then the voice data received from the start of the voice activity within the first duration to the stop of reception is determined as the second voice data.

[0086] Alternatively, voice data reception can begin when voice activity is detected within a first duration, and stop when the silence duration of the voice activity within the first duration reaches a silence duration threshold. The voice data received between the start and stop of the voice activity within the first duration can then be identified as the second voice data.

[0087] In other words, for voice activities occurring within the first duration, the VAD start time of the voice activity is within the first duration, and the VAD end time of the voice activity can be determined based on whether the silence duration is greater than a preset silence duration threshold. Alternatively, the same method as S101-S102 described above can be used to perform integrity checks on the voice data of the voice activities occurring within the first duration. If the voice data of the voice activities occurring within the first duration is complete, it is determined whether the silence duration has reached the silence duration threshold. If the silence duration has reached the silence duration threshold, it is determined that the voice activity occurring within the first duration has ended. Thus, the voice data received between the start and end of the voice activity occurring within the first duration is used as the second voice data.

[0088] S203. Combine the first speech data with the second speech data to obtain the third speech data.

[0089] S204. Perform speech recognition on the third speech data to obtain the recognition result corresponding to the third speech data.

[0090] In this embodiment, since the second voice data is highly correlated with the first voice data, the second voice data can be concatenated after the first voice data to obtain the third voice data. This ensures that the third voice data contains the complete user-input voice. Speech recognition is then performed on the third voice data to obtain the corresponding recognition result.

[0091] It is understandable that by receiving the second voice data corresponding to the voice activity that occurred within the first duration when the semantic intent of the first voice data is unclear, and concatenating the first voice data with the second voice data to obtain the third voice data, and recognizing the recognition result corresponding to the third voice data, the integrity of the voice data is further improved, thereby improving the accuracy of voice recognition based on voice data.

[0092] In some embodiments, after detecting whether voice activity has occurred within a first duration, if no voice activity is detected within the first duration, it indicates that the user is no longer providing voice input. Therefore, the system stops waiting, performs voice recognition on the first voice data, and obtains the recognition result corresponding to the first voice data. This improves the response speed to user voice commands.

[0093] In some embodiments, after determining whether the semantic intent of the first voice data is clear in S1031, further steps may be taken as follows: Figure 4 As shown, execute S301-S305 as follows:

[0094] S301. If the semantic intent of the first speech data is clear, detect whether speech activity occurs during the second duration.

[0095] S302. If speech activity is detected within the second duration, determine the fourth speech data corresponding to the speech activity that occurred within the second duration.

[0096] In some embodiments, the voice data received from voice activities occurring within a second duration can be subjected to integrity detection. If the silence duration after detecting complete voice data reaches a silence duration threshold, the voice data received from the start of reception to the stop of reception within the second duration is determined as the fourth voice data.

[0097] Alternatively, voice data reception can begin when voice activity is detected within the second duration, and stop when the silence duration of the detected voice activity within the second duration reaches the silence duration threshold. The voice data received between the start and stop of the voice activity within the second duration can then be defined as the fourth voice data.

[0098] In other words, for voice activities occurring within the second duration, the VAD start time of the voice activity is within the second duration, and the VAD end time of the voice activity can be determined based on whether the silence duration is greater than a preset silence duration threshold. Alternatively, the same method as S101-S102 described above can be used to perform integrity checks on the voice data of voice activities occurring within the second duration. If the voice data of voice activities occurring within the second duration is complete, it is determined whether the silence duration reaches the silence duration threshold. If the silence duration reaches the silence duration threshold, it is determined that the voice activity occurring within the second duration has ended. Thus, the voice data received between the start and end of the voice activity occurring within the second duration is used as the fourth voice data.

[0099] S303. Determine the correlation between the first speech data and the fourth speech data.

[0100] S304. If the correlation is greater than or equal to the correlation threshold, the first speech data and the fourth speech data are concatenated to obtain the fifth speech data.

[0101] S305. Perform speech recognition on the fifth speech data to obtain the recognition result corresponding to the fifth speech data.

[0102] Here, even when the semantic intent of the first speech data is clear, a preset second duration is waited for to determine whether speech activity occurs during that second duration. The second duration may be the same as or different from the first duration. If speech activity is detected within the second duration, the speech data corresponding to the speech activity within the second duration is determined as the fourth speech data through a process similar to that described above, which is used to determine the second speech data corresponding to the speech activity within the first duration. Since the fourth speech data is obtained when the semantic intent of the first speech data is clear, the correlation between the first and fourth speech data needs to be further verified. If the correlation is greater than or equal to a correlation threshold, the first and fourth speech data are concatenated. For example, the fourth speech data is concatenated after the first speech data to obtain the fifth speech data. Then, speech recognition is performed on the fifth speech data to obtain the recognition result corresponding to the fifth speech data.

[0103] In some embodiments, the correlation between the first speech data and the fourth speech data can be determined using a third network model.

[0104] Thus, for users who input complete and clearly intended voice data but subsequently supplement it—for example, a user first says "Please draw a flower pattern," pauses for more than a preset silence duration threshold, and then speaks again within a second duration, saying "The flower is red"—in this case, although the initial voice input "Please draw a flower pattern" is complete and clear, it needs to be combined with the highly relevant subsequent input "The flower is red" for speech recognition to fully identify the user's intent.

[0105] It is understood that the method in this application embodiment can merge two highly relevant voice data input by the user before and after each other for speech recognition, which greatly improves the accuracy of speech recognition.

[0106] It should be noted that if the relevance is less than the relevance threshold, it means that the fourth voice data input later is unrelated to the first voice data. Therefore, no concatenation is performed, and the first voice data is sent to the voice recognition model to perform voice recognition and obtain the recognition result corresponding to the first voice data.

[0107] The speech recognition method of this application embodiment will be described below in conjunction with a practical scenario, such as... Figure 5 As shown below:

[0108] In practical applications, when a user speaks, VAD start (voice activity start, voice activity start time T-min) is triggered. Every T-max milliseconds (the first time interval), a detection is performed to check whether the user has finished speaking (to check whether the received voice data is complete). The ability to make this judgment is provided by model A (the first network model). Model A will determine whether the user has finished speaking based on the text converted from the currently received voice data and the acoustic features of the currently received voice data.

[0109] If Model A determines that the user has finished speaking, that is, if it detects that the voice data received since the start of the voice activity is complete, then when the silence duration reaches the silence duration threshold (e.g., 400ms), it triggers VADend (determining the end of the voice activity), and simultaneously performs a semantic intent determination on all currently received voice data (the first voice data). This determination is provided by Model B (the second network model). If the determination result indicates that the semantic intent of the first voice data is clear, then speech recognition is performed on the first voice data to obtain the corresponding recognition result.

[0110] If model A determines that the user has not finished speaking, the VAD end is forcibly extended to wait for T-max milliseconds. If the user continues speaking within T-max milliseconds, the corresponding voice data is received and the next round of integrity detection continues. If the user does not continue speaking within T-max milliseconds, the VAD end is triggered, and speech recognition is performed on all voice data received since the start of the voice activity.

[0111] If model A determines that the user has finished speaking, and model B determines that the semantic intent of the first speech data is unclear, then if a speech activity (second speech activity) occurs within the next Xms (first duration), the first speech data is concatenated with the second speech data received within the subsequent Xms to obtain the third speech data. Then, speech recognition is performed on the third speech data to obtain the recognition result corresponding to the third speech data.

[0112] It is understood that the embodiments of this application can intelligently determine whether the user has finished speaking, rather than waiting for a fixed silence duration as is common in current related technologies. This reduces the impact of users hesitating to speak or truncating questions while thinking, thereby improving the accuracy of speech recognition.

[0113] This application provides a voice recognition device 1, such as... Figure 6 As shown, it includes:

[0114] The voice detection module 11 is used to perform integrity detection on the voice data received from the start of the voice activity. If complete voice data is detected, it determines whether the silence duration has reached the silence duration threshold.

[0115] The voice determination module 12 is used to determine the first voice data when the silence duration reaches the silence duration threshold.

[0116] The speech recognition module 13 is used to perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0117] In some embodiments, the voice detection module 11 is further configured to perform integrity detection on the received voice data, and if no complete voice data is detected, continue receiving voice data, and perform integrity detection on the voice data received from the start of the voice activity after a first time interval.

[0118] In some embodiments, the voice detection module 11 is further configured to stop receiving voice data when the silence duration reaches the silence duration threshold, and to use the voice data received from the start of reception to the stop of reception as the first voice data.

[0119] In some embodiments, the voice determination module 12 is further configured to, before performing voice recognition on the first voice data and obtaining the recognition result corresponding to the first voice data, stop receiving if the silence duration within the first time interval is greater than the silence duration threshold, and use the voice data received from the start of receiving to the stop of receiving as the first voice data.

[0120] In some embodiments, the speech recognition module 13 is further configured to determine whether the semantic intent of the first speech data is clear; if the semantic intent of the first speech data is clear, perform speech recognition on the first speech data to obtain the recognition result corresponding to the first speech data.

[0121] In some embodiments, the speech recognition module 13 is further configured to: after determining whether the semantic intent of the first speech data is clear, if the semantic intent of the first speech data is unclear, detect whether speech activity occurs within a first duration; if speech activity is detected within the first duration, determine the second speech data corresponding to the speech activity occurring within the first duration; concatenate the first speech data and the second speech data to obtain third speech data; and perform speech recognition on the third speech data to obtain the recognition result corresponding to the third speech data.

[0122] In some embodiments, the speech recognition module 13 is further configured to detect whether speech activity has occurred within a first duration, and if no speech activity is detected within the first duration, to perform speech recognition on the first speech data to obtain a recognition result corresponding to the first speech data.

[0123] In some embodiments, the voice detection module 11 is further configured to determine that the voice activity has not ended if voice data is received when the silence duration has not reached the silence duration threshold.

[0124] In some embodiments, the speech detection module 11 is further configured to: after determining whether the semantic intent of the first speech data is clear, if the semantic intent of the first speech data is clear, detect whether speech activity occurs during a second duration; if speech activity is detected during the second duration, determine the fourth speech data corresponding to the speech activity during the second duration; determine the correlation between the first speech data and the fourth speech data; if the correlation is greater than or equal to a correlation threshold, concatenate the first speech data and the fourth speech data to obtain fifth speech data; and perform speech recognition on the fifth speech data to obtain the recognition result corresponding to the fifth speech data.

[0125] In some embodiments, the voice determination module 12 is further configured to perform integrity detection on the voice data received from the voice activities that occur within the first duration. If the silence duration after detecting complete voice data reaches the silence duration threshold, the voice data received from the start of the voice activities within the first duration to the stop of reception is determined as the second voice data.

[0126] Alternatively, the voice determination module 12 is further configured to perform integrity detection on the voice data received from the voice activities occurring within the second duration. If the silence duration after detecting complete voice data reaches the silence duration threshold, the voice data received from the start to the stop of the voice activities within the second duration is determined as the fourth voice data.

[0127] In some embodiments, the speech detection module 11 is further configured to extract acoustic features and text features from the speech data received from the start of the speech activity; and to perform integrity detection on the speech data received from the start of the speech activity based on the acoustic features and the text features.

[0128] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0129] This application also provides an electronic device. Figure 7 This is a schematic diagram of an optional structure of the electronic device 3 provided in an embodiment of this application. For example... Figure 7 As shown, the electronic device 3 includes a memory 32 and a processor 33. The memory 32 and the processor 33 are connected via a communication bus 34. The memory 32 is used to store executable instructions. The processor 33 is used to implement the speech recognition method provided in this application embodiment when executing the executable instructions stored in the memory 32.

[0130] This application embodiment provides a vehicle, the vehicle comprising: as shown in the example Figure 6 The speech recognition device shown, or including, for example, Figure 7 The electronic device shown.

[0131] This application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by the processor, the processor will execute the speech recognition method provided in this application.

[0132] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0133] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0134] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A voice recognition method, characterized by, The method comprises: performing integrity detection on voice data received from the start of voice activity, and determining whether the length of silence reaches a silence length threshold if complete voice data is detected; determining first voice data if the length of silence reaches the silence length threshold. performing voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data.

2. The method of claim 1, wherein, After the integrity detection on the voice data received from the start of voice activity, the method further comprises: if complete voice data is not detected, continuing to receive voice data, and performing integrity detection on the voice data received from the start of voice activity after a first time interval.

3. The method according to claim 1 or 2, characterized in that, If the length of silence reaches the silence length threshold, the method further comprises: stopping the reception of voice data, and taking the voice data received from the start of reception to the stop of reception as the first voice data.

4. The method of claim 2, wherein, Before the voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data, the method further comprises: if the length of silence in the first time interval is greater than the silence length threshold, stopping the reception of voice data, and taking the voice data received from the start of reception to the stop of reception as the first voice data.

5. The method of any one of claims 1, 2, or 4, wherein, The voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data comprises: determining whether the semantic intent of the first voice data is clear; if the semantic intent of the first voice data is clear, performing voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data.

6. The method of claim 5, wherein, After the determination of whether the semantic intent of the first voice data is clear, the method further comprises: if the semantic intent of the first voice data is not clear, detecting whether voice activity occurs within a first time length; if it is detected that voice activity occurs within the first time length, determining second voice data corresponding to the voice activity occurring within the first time length; splicing the first voice data and the second voice data to obtain third voice data; performing voice recognition on the third voice data to obtain a recognition result corresponding to the third voice data.

7. The method of claim 6, wherein, After the detection of whether voice activity occurs within the first time length, the method further comprises: if voice activity is not detected within the first time length, performing voice recognition on the first voice data to obtain a recognition result corresponding to the first voice data.

8. The method of claim 5, wherein, After the determination of whether the semantic intent of the first voice data is clear, the method further comprises: if the semantic intent of the first voice data is clear, detecting whether voice activity occurs within a second time length; if it is detected that voice activity occurs within the second time length, determining fourth voice data corresponding to the voice activity occurring within the second time length; determining the relevance of the first voice data and the fourth voice data; if the relevance is greater than or equal to a relevance threshold, splicing the first voice data and the fourth voice data to obtain fifth voice data; perform speech recognition on the fifth speech data to obtain a recognition result corresponding to the fifth speech data.

9. The method according to claim 6 or 8, characterized in that, The second speech data is determined by: performing integrity detection on speech data received during speech activity occurring in a first time length, and if a silence time length after detecting complete speech data reaches the silence time length threshold, determining the speech data received between the start of receiving and the stop of receiving of speech activity in the first time length as the second speech data. The fourth speech data is determined by: performing integrity detection on speech data received during speech activity occurring in a second time length, and if a silence time length after detecting complete speech data reaches the silence time length threshold, determining the speech data received between the start of receiving and the stop of receiving of speech activity in the second time length as the fourth speech data.

10. The method of any one of claims 1, 2, 4, or 6-8, wherein, The integrity detection on the speech data received from the start of speech activity includes: extracting acoustic features and text features from the speech data received from the start of speech activity; performing integrity detection on the speech data received from the start of speech activity according to the acoustic features and the text features.

11. A speech recognition apparatus, characterized by comprising: The apparatus includes: a speech detection module configured to perform integrity detection on speech data received from the start of speech activity, and if complete speech data is detected, determine whether a silence time length reaches a silence time length threshold; a speech determination module configured to determine first speech data if the silence time length reaches the silence time length threshold; a speech recognition module configured to perform speech recognition on the first speech data to obtain a recognition result corresponding to the first speech data.

12. An electronic device, comprising: The electronic device includes: a memory configured to store computer executable instructions; a processor configured to execute the computer executable instructions stored in the memory to implement the method of any one of claims 1 to 10.

13. A computer-readable storage medium storing computer-executable instructions, wherein execution of the computer-executable instructions by one or more processors of a computing system causes the one or more processors to perform operations comprising: The computer executable instructions or computer program, when executed by the processor, implement the method of any one of claims 1 to 10.

14. A computer program product comprising computer executable instructions, characterised in that, The computer executable instructions or computer program, when executed by the processor, implement the method of any one of claims 1 to 10.

15. A vehicle characterized by comprising: The vehicle includes: the speech recognition apparatus of claim 11, or the electronic device of claim 12.

Images