Integrated methods, systems, and electronic devices for speech recognition and semantic understanding

Abstract

This invention provides an integrated method, system, and electronic device for speech recognition and semantic understanding. The method includes: performing speech recognition on real-time detected user speech to obtain real-time recognized text; inputting the real-time recognized text to a semantic completeness determination module; if the real-time recognized text has complete semantics, performing semantic analysis on the real-time recognized text to obtain a real-time semantic analysis result corresponding to the real-time recognized text; when the user speech stops, obtaining the final recognized text through speech recognition, comparing the final recognized text with the real-time recognized text, and if they are the same, determining the real-time semantic analysis result corresponding to the real-time recognized text as the final semantic analysis result of the user speech. This invention saves processing time in semantic understanding, ensuring that semantics are requested as early as possible, reducing overall process time, and also ensuring the completeness of the semantic results, making them accurate and effective semantic results.

Description

Technical Field

[0001] This invention relates to the field of intelligent speech, and more particularly to an integrated method, system, and electronic device for speech recognition and semantic understanding. Background Technology

[0002] In spoken dialogue interaction systems, the speech recognition module and the semantic understanding module are independent and sequential. Intelligent devices equipped with both modules acquire speech signals in real time. When a user inputs speech, the system first uses Voice Activity Detection (VAD) to determine the start and end of valid speech segments. Then, the speech recognition module converts these valid speech segments into text. Finally, the semantic understanding module converts this text into a semantically understood result (containing intent, slot information, etc.). This semantically understood result is then used for downstream tasks or directly output.

[0003] The specific oral dialogue interaction process is as follows:

[0004] The user's voice input is processed by the voice endpoint detection module to determine the start and end of the valid voice segment. If the start signal of valid voice is detected, the valid voice signal is input to the subsequent recognition module; if the end signal of voice input is detected, an end signal is sent to the subsequent recognition module, thereby ending the voice recognition.

[0005] The voice signal is input into the voice recognition engine, which converts the voice signal into the corresponding text;

[0006] The identified text is input into the semantic understanding engine, which aims to understand the meaning of what the user is saying and convert the text into corresponding semantic information, such as intent and slots.

[0007] The semantic parsing results are input into downstream tasks, such as dialogue management tasks, to complete dialogue interaction.

[0008] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0009] Since the above interaction process is serial, the voice signal first needs to pass through the endpoint detection module to detect the start point of valid audio, then be processed by the speech recognition engine to obtain the recognized text, and then input the recognized text into the semantic understanding engine to obtain the semantic parsing result for downstream tasks. At this time, the overall latency of the link is equal to the processing latency of speech recognition plus the processing latency of semantic understanding, so the latency is relatively high.

[0010] Furthermore, the endpoint detection module determines the end time of valid speech after detecting the end of the user's voice, but this time point has a certain lag. This lag is due to the inherent limitations of the endpoint detection algorithm, which further exacerbates the delay in the entire dialogue process and affects the user's voice interaction experience. Summary of the Invention

[0011] To at least address the overall latency issue in existing voice dialogue technologies, in a first aspect, embodiments of the present invention provide an integrated method for speech recognition and semantic understanding, comprising:

[0012] Speech recognition is performed on the user's voice detected in real time to obtain the real-time recognized text of the user's voice;

[0013] The real-time recognized text is input into the semantic completeness determination module to determine whether the real-time recognized text has complete semantics;

[0014] If the real-time recognized text has complete semantics, perform semantic parsing on the real-time recognized text to obtain the real-time semantic parsing result corresponding to the real-time recognized text;

[0015] When the user's speech stops, the final recognized text is obtained through speech recognition. The final recognized text is compared with the real-time recognized text. If the comparison results are the same, the real-time semantic parsing result corresponding to the real-time recognized text is determined as the final semantic parsing result of the user's speech.

[0016] Secondly, embodiments of the present invention provide an integrated system for speech recognition and semantic understanding, comprising:

[0017] The speech recognition program module is used to perform speech recognition on the user's voice detected in real time, and obtain the real-time recognized text of the user's voice;

[0018] The completeness judgment module is used to input the real-time recognized text into the semantic completeness judgment module to determine whether the real-time recognized text has complete semantics;

[0019] The semantic parsing module is used to perform semantic parsing on the real-time recognized text if the real-time recognized text has complete semantics, and obtain the real-time semantic parsing result corresponding to the real-time recognized text.

[0020] The semantic determination module is used to obtain the final recognized text through speech recognition when the user's speech is detected to have stopped, compare the final recognized text with the real-time recognized text, and if the comparison results are the same, determine the real-time semantic parsing result corresponding to the real-time recognized text as the final semantic parsing result of the user's speech.

[0021] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the integrated method for speech recognition and semantic understanding according to any embodiment of the present invention.

[0022] Fourthly, embodiments of the present invention provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the integrated method for speech recognition and semantic understanding according to any embodiment of the present invention.

[0023] The beneficial effects of this invention are as follows: during speech recognition, the semantic understanding module is requested in advance, saving processing time for semantic understanding. Furthermore, the trigger point for requesting the semantic understanding module in advance is calculated by the semantic integrity judgment module of this method, ensuring that semantics are requested as early as possible, minimizing overall processing time, and guaranteeing the integrity of the semantic result, which can then be used as the final semantic result. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of an integrated method for speech recognition and semantic understanding provided in an embodiment of the present invention;

[0026] Figure 2 This is a flowchart illustrating a VAD detection method, an example of an integrated method for speech recognition and semantic understanding provided in an embodiment of the present invention.

[0027] Figure 3 This is a flowchart illustrating the overall structure of an integrated method for speech recognition and semantic understanding provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram of the structure of an integrated speech recognition and semantic understanding system provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of an embodiment of an electronic device integrating speech recognition and semantic understanding, provided by an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] like Figure 1 The diagram shown is a flowchart of an integrated method for speech recognition and semantic understanding provided by an embodiment of the present invention, including the following steps:

[0032] S11: Perform speech recognition on the user's voice detected in real time to obtain the real-time recognized text of the user's voice;

[0033] S12: Input the real-time recognized text into the semantic integrity judgment module to determine whether the real-time recognized text has complete semantics;

[0034] S13: If the real-time recognized text has complete semantics, perform semantic parsing on the real-time recognized text to obtain the real-time semantic parsing result corresponding to the real-time recognized text;

[0035] S14: When the user's speech stops, the final recognized text is obtained through the speech recognition. The final recognized text is compared with the real-time recognized text. If the comparison result is the same, the real-time semantic parsing result corresponding to the real-time recognized text is determined as the final semantic parsing result of the user's speech.

[0036] In this embodiment, it was found that the speech recognition (including speech endpoint detection) and semantic understanding modules accounted for most of the total time consumption of the entire process. Therefore, it was considered to jointly optimize these two modules to reduce the overall dialogue time.

[0037] In existing technologies, speech recognition and semantic understanding are performed sequentially. VAD (Voice Activity Detection) detects the end of speech, speech recognition is completed, and then semantic understanding is performed. The overall latency of this process equals the processing latency of speech recognition plus the processing latency of semantic understanding, resulting in high latency. Figure 2The diagram illustrates the VAD detection process in existing technologies. A pulse represents a valid speech signal. "VAD begin" is the start time point when the VAD module detects valid speech, and "VAD end" is the end time point. It can be seen that this time point lags behind the falling edge of the valid speech signal; the lag time is represented by the "VAD pause time" in the diagram. This time is determined by the VAD module algorithm itself. Typical settings include 300ms, 500ms, and 800ms. This time increases the response speed of the link. Those skilled in the art would typically consider setting this delay lower to reduce it. However, simply reducing the value can introduce new problems. For example, setting it to 100ms in a dialogue interaction scenario can easily truncate the user's speech input, causing the system to respond before the user finishes speaking, significantly impacting the user experience. Therefore, this method integrates cross-modal and cross-domain speech recognition and semantic understanding without adjusting the delay value.

[0038] For step S11, in actual use, the smart speaker collects audio signals in real time and detects user voice by performing voice endpoint detection on the audio signals. After detecting the user's voice, the voice is sent to the voice recognition module, which recognizes the input voice signal and decodes it to output the corresponding text. This process is real-time output, that is, word-by-word output. For example, the user says "Help me turn on the air conditioner." As the user continues to say it, the voice recognition module gradually recognizes it, and the recognized text is gradually determined over time as follows: "help" at the first time, "help me" at the second time, "help me turn it on" at the third time, "help me turn on the air conditioner" at the fourth time, and the fifth time is the end point when the VAD module algorithm detects the user's valid voice.

[0039] For step S12, we will take "Help me" determined in the second time and "Help me turn on the air conditioner" determined in the fourth time as examples. For instance, in the second time, the determined "Help me" is input into the semantic completeness judgment module to determine whether "Help me" has complete semantics. In the fourth time, the determined "Help me turn on the air conditioner" is input into the semantic completeness judgment module to determine whether "Help me turn on the air conditioner" has complete semantics.

[0040] As one implementation, the semantic completeness determination module includes: a binary classification machine learning model trained by dialogue training data, wherein the binary classification machine learning model includes a machine learning model, a deep learning model, and a data-based pre-trained model.

[0041] In this implementation, the model can be trained using real-world business data (such as user-input dialogue) to achieve binary classification. The machine learning model can be a traditional machine learning model, such as SVM (Support Vector Machine), gradient boosting decision trees (e.g., XGBoost), or a deep learning model, such as LSTM (Long Short-Term Memory), CNN (Convolutional Neural Networks), or a Transformer model based on self-attention mechanisms. It can also be a pre-trained model based on a large amount of data, such as BERT (Bidirectional Encoder Representations from Transformers).

[0042] As one implementation, the semantic integrity determination module includes: a semantic resource library generated from the user's request data and the user's custom semantic data;

[0043] The step of inputting the real-time recognized text into the semantic integrity determination module includes:

[0044] The real-time recognized text is matched with the semantic resource library. If the match is successful, it is determined that the real-time recognized text has complete semantics.

[0045] In this implementation, a semantic resource library can be generated based on actual business data (i.e., actual user request data) and user-defined semantic data (user-written text regular expressions, which are expanded into text and corresponding semantic results when used). When a recognition result is generated, the recognition result is matched with the text in the semantic resource library. If a match is found, it indicates that the recognized text has a complete semantic result; otherwise, it is considered that the recognized text has no complete semantics.

[0046] The two semantic integrity judgment modules mentioned above are simple to judge and have relatively low processing volume, and can obtain the judgment result of whether the semantics are complete within 10ms.

[0047] Furthermore, as one implementation, the semantic completeness determination module includes: a binary classification machine learning model trained with dialogue training data and a semantic resource library generated from the user's request data and the user's custom semantic data;

[0048] The step of inputting the real-time recognized text into the semantic integrity determination module includes:

[0049] The first judgment result of the real-time recognized text is determined based on the binary classification machine learning model.

[0050] A second judgment result is obtained by matching the real-time recognized text based on the semantic resource library;

[0051] Based on the first and second judgment results, it is determined whether the real-time recognized text has complete semantics.

[0052] In this embodiment, considering that the semantic completeness determination result affects whether semantic understanding is required in subsequent steps, the semantic completeness determination module can simultaneously use a binary classification machine learning model and a semantic resource library for joint judgment. The semantic completeness of the real-time recognized text is determined by the joint judgment results of these two semantic completeness determination modules.

[0053] For example, at the second time point, inputting "help me" into the semantic completeness judgment module yields a result indicating incomplete semantics. If it is determined that the semantics are incomplete, no semantic parsing is performed. At the third time point, inputting "help me open" into the semantic completeness judgment module again yields a result indicating incomplete semantics. Semantic parsing is still not performed. At the fourth time point, inputting "help me turn on the air conditioner" into the semantic completeness judgment module yields a result indicating complete semantics.

[0054] For step S13, for example, if "Help me turn on the air conditioner" has complete semantics, then the semantic parsing module is immediately requested to obtain the semantic parsing result nlu_1, and this result and the corresponding recognized text text_1 (Help me turn on the air conditioner) are cached; otherwise, no operation is performed and the recognition result is read again. The advantage of this setting is that it is equivalent to performing semantic parsing in parallel while still in the process of speech recognition, and this latency is less than that of the traditional serial method.

[0055] In step S14, when the VAD module generates a VAD end signal, which is the fifth time point, it determines that the user's speech has stopped. At this time, speech recognition ends, and the final recognized text text_2 is generated (for example, the final recognized text is "Help me turn on the air conditioner"). The final recognized text text_2 is compared with the recognized text text_1 determined in step S13. At this time, both are "Help me turn on the air conditioner", so the semantic parsing result nlu_1 obtained in step S13 is determined as the final semantic parsing result of the user's speech.

[0056] As can be seen from this implementation method, requesting the semantic understanding module in advance during speech recognition saves processing time for semantic understanding. Furthermore, the trigger point for requesting the semantic understanding module in advance is calculated by the semantic integrity judgment module of this method. This ensures that semantics are requested as early as possible, minimizing the overall processing time, while also guaranteeing the integrity of the semantic result, making it suitable as the final semantic result.

[0057] As another implementation, comparing the final recognized text with the real-time recognized text includes:

[0058] If the comparison results are different, semantic analysis is performed on the final recognized text to obtain the final semantic analysis result of the user's speech. In this embodiment, special cases of user input speech are considered, such as "Play me a song, something cheerful." When the speech recognition detects "Play me a song," the text has complete semantics, and the first semantic analysis result is obtained. When the user speaks the complete sentence, "Play me a song, something cheerful," the cached "Play me a song" is not consistent with the final "Play me a song, something cheerful." Therefore, semantic analysis is performed on "Play me a song, something cheerful," and the semantic analysis result of "Play me a song, something cheerful" is used as the final analysis result. The overall process is as follows: Figure 3 As shown, this approach ensures both improved interaction latency during normal speech-to-speech conversations and accurate understanding of the semantics of this reverse-order supplementary dialogue.

[0059] like Figure 4 The diagram shown is a structural schematic of an integrated speech recognition and semantic understanding system provided in an embodiment of the present invention. The system can execute the integrated speech recognition and semantic understanding method described in any of the above embodiments and is configured in a terminal.

[0060] This embodiment provides an integrated system 10 for speech recognition and semantic understanding, which includes: a speech recognition program module 11, a completeness judgment program module 12, a semantic parsing program module 13, and a semantic determination program module 14.

[0061] The speech recognition module 11 is used to perform speech recognition on the user's real-time detected speech to obtain the real-time recognized text of the user's speech; the completeness judgment module 12 is used to input the real-time recognized text to the semantic completeness judgment module to determine whether the real-time recognized text has complete semantics; the semantic parsing module 13 is used to perform semantic parsing on the real-time recognized text if the real-time recognized text has complete semantics to obtain the real-time semantic parsing result corresponding to the real-time recognized text; the semantic determination module 14 is used to obtain the final recognized text through the speech recognition when the user's speech is detected to have stopped, compare the final recognized text with the real-time recognized text, and if the comparison result is the same, determine the real-time semantic parsing result corresponding to the real-time recognized text as the final semantic parsing result of the user's speech.

[0062] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the integrated speech recognition and semantic understanding method in any of the above method embodiments.

[0063] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0064] Speech recognition is performed on the user's voice detected in real time to obtain the real-time recognized text of the user's voice;

[0065] The real-time recognized text is input into the semantic completeness determination module to determine whether the real-time recognized text has complete semantics;

[0066] If the real-time recognized text has complete semantics, perform semantic parsing on the real-time recognized text to obtain the real-time semantic parsing result corresponding to the real-time recognized text;

[0067] When the user's speech stops, the final recognized text is obtained through speech recognition. The final recognized text is compared with the real-time recognized text. If the comparison results are the same, the real-time semantic parsing result corresponding to the real-time recognized text is determined as the final semantic parsing result of the user's speech.

[0068] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, they perform the integrated method of speech recognition and semantic understanding in any of the above method embodiments.

[0069] Figure 5 This is a schematic diagram of the hardware structure of an electronic device for an integrated method of speech recognition and semantic understanding provided in another embodiment of this application, as shown below. Figure 5 As shown, the device includes:

[0070] One or more processors 510 and memory 520, Figure 5 Taking a processor 510 as an example, the device for the integrated method of speech recognition and semantic understanding may also include an input device 530 and an output device 540.

[0071] The processor 510, memory 520, input device 530, and output device 540 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0072] The memory 520, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the integrated speech recognition and semantic understanding method in the embodiments of this application. The processor 510 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 520, thereby realizing the integrated speech recognition and semantic understanding method of the above-described method embodiments.

[0073] The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data, etc. Furthermore, the memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may optionally include memory remotely located relative to the processor 510, and these remote memories may be connected to the mobile device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0074] Input device 530 can receive input numerical or character information. Output device 540 may include display devices such as a display screen.

[0075] The one or more modules are stored in the memory 520, and when executed by the one or more processors 510, they perform the integrated method of speech recognition and semantic understanding in any of the above method embodiments.

[0076] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0077] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0078] This invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the integrated speech recognition and semantic understanding method of any embodiment of this invention.

[0079] The electronic devices described in this application exist in various forms, including but not limited to:

[0080] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0081] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as tablet computers.

[0082] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0083] (4) Other electronic devices with data processing functions.

[0084] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An integrated method for speech recognition and semantic understanding, comprising: Speech recognition is performed on the user's voice detected in real time to obtain the real-time recognized text of the user's voice; The real-time recognized text is input to the semantic completeness judgment module to determine whether the real-time recognized text has complete semantics. The semantic completeness judgment module includes: a binary classification machine learning model trained by dialogue training data and a semantic resource library generated by user request data and user-defined semantic data. The binary classification machine learning model includes a machine learning model, a deep learning model, and a data-based pre-trained model. A first judgment result of the real-time recognized text is determined based on the binary classification machine learning model. A second judgment result of the real-time recognized text is matched based on the semantic resource library. Whether the real-time recognized text has complete semantics is determined based on the first judgment result and the second judgment result. If the real-time recognized text has complete semantics, determine the trigger time point for requesting the semantic understanding module in advance, perform semantic parsing on the real-time recognized text, and obtain the real-time semantic parsing result corresponding to the real-time recognized text; When the user's speech stops, the final recognized text is obtained through speech recognition. The final recognized text is compared with the real-time recognized text. If the comparison results are the same, the real-time semantic parsing result corresponding to the real-time recognized text is determined as the final semantic parsing result of the user's speech.

2. The method according to claim 1, wherein, The semantic integrity determination module includes: a semantic resource library generated from the user's request data and the user's custom semantic data; The step of inputting the real-time recognized text into the semantic integrity determination module includes: The real-time recognized text is matched with the semantic resource library. If the match is successful, it is determined that the real-time recognized text has complete semantics.

3. The method according to claim 1, wherein, The method further includes: if the real-time recognized text does not have complete semantics, then semantic parsing is not performed.

4. The method according to claim 1, wherein, The step of comparing the final recognized text with the real-time recognized text includes: If the comparison results are different, semantic analysis is performed on the final recognized text to obtain the final semantic analysis result of the user's speech.

5. The method according to claim 1, wherein, The real-time detected user voice is determined by the voice endpoint detection method.

6. An integrated system for speech recognition and semantic understanding, comprising: The speech recognition program module is used to perform speech recognition on the user's voice detected in real time, and obtain the real-time recognized text of the user's voice; A completeness judgment module is used to input the real-time recognized text into a semantic completeness judgment module to determine whether the real-time recognized text has complete semantics. The semantic completeness judgment module includes: a binary classification machine learning model trained by dialogue training data and a semantic resource library generated by user request data and user-defined semantic data. The binary classification machine learning model includes a machine learning model, a deep learning model, and a data-based pre-trained model. A first judgment result of the real-time recognized text is determined based on the binary classification machine learning model. A second judgment result of the real-time recognized text is matched based on the semantic resource library. Whether the real-time recognized text has complete semantics is determined based on the first judgment result and the second judgment result. The semantic parsing module is used to determine the trigger point for requesting the semantic understanding module in advance if the real-time recognized text has complete semantics, perform semantic parsing on the real-time recognized text, and obtain the real-time semantic parsing result corresponding to the real-time recognized text. The semantic determination module is used to obtain the final recognized text through speech recognition when the user's speech is detected to have stopped, compare the final recognized text with the real-time recognized text, and if the comparison results are the same, determine the real-time semantic parsing result corresponding to the real-time recognized text as the final semantic parsing result of the user's speech.

7. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-5.

Images