Real-time semantic understanding methods, systems, and electronic devices for spoken dialogue

Abstract

This invention provides a method, system, and electronic device for real-time semantic understanding of spoken dialogue. The method includes: inputting detected user speech into a streaming speech recognition engine in real time, and outputting streaming speech recognition results; receiving the streaming speech recognition results using the streaming semantic understanding engine, and querying the engine state corresponding to a speech request identifier within the streaming speech recognition results; the streaming semantic understanding engine inferring from the streaming speech recognition results based on the engine state to obtain an inferred semantic parsing result, and updating the engine state; if the user's speech ends, determining the inferred semantic parsing result as the final semantic parsing result of the user's speech. This invention utilizes a streaming speech parsing engine, eliminating the need to wait for the user to finish speaking before performing speech recognition, and avoiding the computational burden required for real-time semantic understanding. Without increasing computational load, it reduces user waiting time and improves user experience.

Description

Technical Field

[0001] This invention relates to the field of intelligent speech, and more particularly to a real-time semantic understanding method, system, and electronic device for spoken dialogue. Background Technology

[0002] To improve the user's voice interaction experience, it's necessary not only to enhance the accuracy of speech recognition but also to perform accurate and efficient semantic understanding of the recognition results. Existing semantic understanding technologies require inputting the complete text and then parsing it to obtain the corresponding semantic analysis result. For example, in a spoken conversation, after the user finishes speaking, the speech recognition result is obtained, and then input into the semantic understanding engine to obtain the corresponding semantic analysis result. This results in a sequential process between speech recognition and semantic understanding in spoken conversations: after the user speaks a sentence, the entire sentence's speech data is sent to the speech recognition module for speech recognition; only after the entire sentence has been recognized is the recognition result sent to the semantic understanding module for semantic understanding to obtain the complete sentence analysis result.

[0003] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0004] During spoken dialogue with users, it's necessary to wait for the user to finish speaking a complete sentence. The speech recognition engine then decodes and extracts the complete text before semantic understanding is performed. This entire process includes the time spent on speech recognition plus the time spent on semantic understanding. In other words, to obtain the correct semantic parsing result from the user's input, the system needs to wait for the user to finish speaking before inputting the complete sentence into the semantic parsing engine. Therefore, it's crucial to wait for the speech recognition engine to output the complete decoding result before semantic parsing can begin, which increases the overall processing time. This results in a relatively long overall time, requiring users to wait for feedback from the interactive system, leading to a poor user experience. Summary of the Invention

[0005] To address the problem that existing technologies require waiting for the user to speak a complete sentence for semantic parsing, resulting in long overall processing times and a poor user experience, this invention provides, in a first aspect, a real-time semantic understanding method for spoken dialogue, comprising:

[0006] The detected user voice is input to the streaming speech recognition engine in real time, and the streaming speech recognition result is output. The speech recognition result includes a voice request identifier used to determine whether it is the same sentence.

[0007] The streaming semantic understanding engine receives the speech recognition results of the streaming stream and queries the engine status corresponding to the speech request identifier within the speech recognition results of the streaming stream.

[0008] The streaming semantic understanding engine infers the streaming speech recognition results based on the engine state to obtain the inferred semantic parsing results, and updates the engine state accordingly.

[0009] If the user's speech ends, the inference semantic parsing result is determined as the final semantic parsing result of the user's speech.

[0010] Secondly, embodiments of the present invention provide a real-time semantic understanding system for spoken dialogue, comprising:

[0011] The speech recognition program module is used to input the detected user speech into the streaming speech recognition engine in real time and output the streaming speech recognition result, wherein the speech recognition result includes a speech request identifier used to determine whether it is the same sentence;

[0012] The engine status determination module is used to receive the speech recognition results of the streaming semantic understanding engine and query the engine status corresponding to the speech request identifier in the speech recognition results of the streaming speech.

[0013] A semantic reasoning program module is used by the streaming semantic understanding engine to reason about the streaming speech recognition results based on the engine state, obtain the reasoning semantic parsing results, and update the engine state;

[0014] The semantic understanding program module is used to determine the inference semantic parsing result as the final semantic parsing result of the user's speech if the user's speech is detected to have ended.

[0015] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of a real-time semantic understanding method for spoken dialogue according to any embodiment of the present invention.

[0016] Fourthly, embodiments of the present invention provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of the real-time semantic understanding method for spoken dialogue according to any embodiment of the present invention.

[0017] The beneficial effects of this invention are as follows: Utilizing a streaming speech parsing engine, speech recognition is not required after the user finishes speaking, thus avoiding the exponentially increased computational load required for real-time semantic understanding. The semantic parsing time of this method is the last frame delay (the time from when the effective audio detection module detects the user's finished speaking to when the final recognition result is output). When the recognition module outputs the recognition result in real time, it is input into the streaming semantic parsing engine in real time, resulting in the real-time output of the semantic parsing result. Therefore, the semantic parsing engine's time is only the parsing time of the last word, which is significantly reduced. This reduces user waiting time and improves the user experience without increasing computational load. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a real-time semantic understanding method for spoken dialogue provided in an embodiment of the present invention;

[0020] Figure 2 This is an internal flowchart of a streaming semantic understanding engine for a real-time semantic understanding method for spoken dialogue, provided in an embodiment of the present invention.

[0021] Figure 3 This is a flowchart of the overall interactive process of a real-time semantic understanding method for spoken dialogue provided by an embodiment of the present invention.

[0022] Figure 4 This is a schematic diagram of the structure of a real-time semantic understanding system for spoken dialogue provided in an embodiment of the present invention;

[0023] Figure 5 This is a schematic diagram of an embodiment of an electronic device for real-time semantic understanding of spoken dialogue, provided by an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] like Figure 1 The diagram shows a flowchart of a real-time semantic understanding method for spoken dialogue provided by an embodiment of the present invention, including the following steps:

[0026] S11: Input the detected user voice into the streaming speech recognition engine in real time and output the streaming speech recognition result, wherein the speech recognition result includes a voice request identifier for determining whether it is the same sentence;

[0027] S12: Receive the speech recognition result of the streaming semantic understanding engine and query the engine status corresponding to the speech request identifier in the speech recognition result of the streaming speech;

[0028] S13: The streaming semantic understanding engine infers the streaming speech recognition results based on the engine state to obtain the inference semantic parsing results, and updates the engine state;

[0029] S14: If the user's speech ends, the inference semantic parsing result is determined as the final semantic parsing result of the user's speech.

[0030] In this embodiment, the method can be applied to servers that serve smart voice devices. For example, when smart speakers, smartphones, and other smart voice devices make voice service requests, the server equipped with this method can provide them with voice equipment, semantic parsing, and other services.

[0031] When smart voice devices interact with users, users directly input spoken language, for example, saying "I want to listen to Liu Moumou's songs." With existing technology, the system would wait for the user to finish speaking the entire sentence, request a voice service from the server, determine the speech recognition result, and then perform semantic understanding of the entire sentence. This would make the entire semantic understanding process time-consuming.

[0032] To avoid this situation, those skilled in the art may perform real-time semantic understanding, for example, by performing semantic understanding on the real-time recognition results and pre-calculating the semantics. However, there are many real-time recognition results for a single sentence, for example (Var represents the output of the streaming speech recognition engine):

[0033] Var: I think / / Here we perform semantic understanding of "I think";

[0034] Var: I want to hear / / At this point, semantic understanding of "I want to hear" is performed;

[0035] Var: I want to hear Liu / / At this point, the semantic understanding of "I want to hear Liu" is performed;

[0036] Var: I want to hear Liu Moumou / / At this point, semantic understanding of "I want to hear Liu Moumou" is performed;

[0037] Var: I want to listen to Liu Moumou's songs / / At this point, semantic understanding of "I want to listen to Liu Moumou's songs" is performed, and finally the semantic parsing result is obtained.

[0038] In other words, when a user inputs a sentence, the semantic parsing is performed five times during internal processing. Although this method may seem to improve the overall response speed, each recognition result is sent to the semantic parsing engine that provides services in the cloud, which puts a lot of computational pressure on the server that provides semantic parsing and consumes a large amount of computing resources.

[0039] From the perspective of a single user, only five semantic parsing operations are performed. While the computational load on the server providing semantic parsing is significantly increased, it improves the overall response speed of user spoken interaction. However, the server does not serve just one user; it needs to provide semantic understanding services for a large number of users. In real-world scenarios, if the computational load for semantic parsing for each user increases exponentially, the pressure on the server becomes enormous. It is highly likely that the server will become overwhelmed by the large number of semantic parsing requests from users, making it difficult to handle and thus exacerbating the time-consuming process of semantic understanding. This method addresses the shortcomings of existing technologies and the limitations of real-time semantic understanding by optimizing the overall semantic understanding steps.

[0040] For step S11, for example, in practical use, when a user engages in spoken dialogue with a smart speaker, the smart speaker performs VAD (Voice Activity Detection, also known as voice activity detection) in real time. When the smart speaker detects that the user has spoken, it takes the speech containing the user's speech as valid audio and streams the valid audio into the speech recognition module in real time. Here, "streaming" means that the recognition result can be returned in real time during the processing of the audio stream. Streaming is better suited for scenarios that require real-time acquisition of recognition results, such as real-time voice interaction and real-time conferencing.

[0041] When the user says the voice of "I want", the smart speaker recognizes the user's valid voice and streams it in real time to the streaming speech recognition engine. The streaming speech recognition engine outputs the streaming speech recognition result. Considering that in the subsequent steps, the streaming semantic understanding engine of this method needs to call the corresponding engine state for reasoning, therefore, it is necessary to assign a corresponding speech request identifier (Record ID) to the streaming speech recognition result in the streaming speech recognition engine in advance to determine whether the continuously input words are the same sentence. For example, after the speech recognition of "I want", the assigned Record ID is: 00SP057; as the user continuously inputs spoken language, for example, what the user wants to say is "I want to listen to the songs of Liu XX, and忘情水 will do", after the speech recognition of "listen", the assigned Record ID is still: 00SP057; after the speech recognition of "Liu XX", the assigned Record ID is still: 00SP057; after the speech recognition of "的歌", the assigned Record ID is still: 00SP057. After the speech recognition of "忘情水", the assigned Record ID is: 00SP058; after the speech recognition of "就可以", the assigned Record ID is: 00SP058.

[0042] For step S12, use the streaming semantic understanding engine of this method to receive the streaming speech recognition result with the speech request identifier determined in step S11, and query the engine state of the streaming semantic understanding engine corresponding to the speech recognition result.

[0043] Specifically, as an implementation manner, the using the streaming semantic understanding engine to receive the streaming speech recognition result and query the engine state corresponding to the speech request identifier in the streaming speech recognition result includes:

[0044] If the speech request identifier does not exist, set the streaming semantic understanding engine to the initial state;

[0045] If the speech request identifier exists, read the engine state corresponding to the speech request identifier.

[0046] In this implementation manner, when the streaming semantic understanding engine receives "I want", the corresponding Record ID is 00SP057. Since "I want" is the first sentence input and the Record ID cannot be queried at this time, the engine state of the streaming semantic understanding engine is set to the initial state. The initial state of the streaming semantic understanding engine is used to perform semantic understanding on "I want" to obtain a temporary inference semantic parsing result. At this time, the speech request identifier 00SP057 is stored for subsequent semantic understanding calls.

[0047] The streaming semantic understanding engine receives "listen" again, with the corresponding Record ID being 00SP057. At this time, the engine state of the streaming semantic understanding engine corresponding to 00SP057 is queried. Then, the engine state after semantic understanding of "I want" is called to perform semantic understanding on "listen", and a temporary inference semantic parsing result is obtained.

[0048] Continue to input "Liu Moumou" and "songs" into the streaming semantic understanding engine in sequence in the above example manner. When the streaming semantic understanding engine receives "Liu Moumou", the engine state after semantic understanding of "listen" is called at this time to perform semantic understanding on "Liu Moumou"; when the streaming semantic understanding engine receives "songs", the engine state after semantic understanding of "Liu Moumou" is called at this time to perform semantic understanding on "songs", and an inference semantic parsing result is obtained, which will not be elaborated here.

[0049] When the streaming semantic understanding engine receives "Love Song", with the corresponding Record ID being 00SP058, and this Record ID cannot be queried at this time, the engine state of the streaming semantic understanding engine is set to the initial state to perform semantic understanding of the next sentence. Such a cycle will not be elaborated, and the overall process is as Figure 2 shown.

[0050] For step S13, as described in S12, when the streaming semantic understanding engine receives "I want", the engine state of the semantic understanding engine is the initial state at this time. The semantic understanding engine in the initial state is used to perform inference on "I want" to obtain a temporary inference semantic parsing result. That is to say, the streaming semantic parsing engine of this method has state memory, stores the internal structure and state of the current parsing, and ensures that the next input will continue to parse based on the result of the previous parsing, so as to achieve incremental parsing.

[0051] When the streaming semantic understanding engine receives "listen", the engine state of the semantic understanding engine is the engine state after inference on "I want" at this time. This state is used to continue to perform inference on "listen" to obtain a further inference semantic parsing result.

[0052] As an implementation manner, if it is detected that the user's speech has not ended, the method further includes:

[0053] Continuously input the detected user speech into the streaming speech recognition engine in real time, and output a streaming speech recognition result;

[0054] Use the streaming semantic understanding engine to receive the streaming speech recognition result, and query the engine state corresponding to the speech request identifier in the streaming speech recognition result;

[0055] The streaming semantic understanding engine continuously performs inference on the streaming speech recognition result according to the engine state, obtains an inference semantic parsing result, and continuously updates the engine state.

[0056] In this embodiment, the VAD detects that the user's speech has not ended, and continuously inputs the user's speech into the streaming speech recognition engine, outputting a streaming speech recognition result. For example, it gradually outputs "Liu XX" and "of the song".

[0057] The streaming semantic understanding engine receives "Liu XX", queries the engine state of the streaming semantic understanding engine corresponding to Record ID 00SP057. At this time, the queried engine state of the streaming semantic understanding engine is: the engine state after inferring "I want to listen to". The streaming semantic understanding engine performs inference on "Liu XX" in this state, obtaining a temporary inference semantic parsing result. During the inference process, the engine state of the streaming semantic understanding engine is continuously updated.

[0058] That is to say, the streaming semantic understanding engine of this method does not need to obtain the complete speech recognition result before performing semantic understanding processing. The streaming semantic understanding engine of this method can obtain text input in a streaming manner and output parsing results in real time.

[0059] For step S14, the streaming semantic understanding engine receives "of the song", queries the engine state of the streaming semantic understanding engine corresponding to Record ID 00SP057. At this time, the queried engine state of the streaming semantic understanding engine is: the engine state after inferring "I want to listen to Liu XX". The streaming semantic understanding engine performs inference on "of the song" in this state, obtaining a temporary inference semantic parsing result.

[0060] At this time, the VAD detects that the user's speech pauses, the speech recognition determines that this sentence is finished, and determines that the current sentence "I want to listen to Liu XX's song" has ended. The inference semantic parsing result obtained by the streaming semantic understanding engine after inferring "of the song" from the inference of "I want to listen to Liu XX" is determined as the final semantic parsing result of the user's sentence.

[0061] From the above example, it can be seen that the time consumption of the streaming semantic parsing engine of this method is greatly reduced. After the user finishes saying "of the song", the user only needs to wait for the semantic parsing time of "of the song", which overall improves the response speed of the spoken dialogue system.

[0062] When the speech recognition engine receives "Love Me Tender" in the next sentence, by querying the voice request identifier 00SP058, it is not found at this time. The streaming semantic understanding engine will perform inference on "Love Me Tender" using the initial state. In subsequent steps, when the streaming semantic understanding engine receives the streaming speech recognition result, the steps are as shown above and will not be elaborated.

[0063] Taking a user's spoken input of "I want to listen to Liu Moumou's songs" as an example, after the streaming semantic understanding engine determines the final semantic parsing result of the user's speech, the method further includes:

[0064] The final semantic parsing result is sent to the dialogue management engine, which generates interactive dialogue voice to respond to the user based on the final semantic parsing result.

[0065] In this embodiment, the final semantic parsing result is sent to the dialogue management engine, which generates corresponding operation instructions and interactive dialogue voice for user feedback, such as "Playing 'Wishing you prosperity' to complete the interaction with the user." The overall process is as follows: Figure 3 As shown.

[0066] This implementation demonstrates that by utilizing a streaming speech parsing engine, speech recognition can be performed without waiting for the user to finish speaking, thus avoiding the exponentially increased computational load required for real-time semantic understanding. The semantic parsing time of this method is the last frame delay (the time from when the effective audio detection module detects the user's finished speaking to when the final recognition result is output). When the recognition module outputs its recognition result in real time, this result is input into the streaming semantic parsing engine in real time, resulting in the real-time output of the semantic parsing result. Therefore, the semantic parsing engine's time is only the parsing time of the last word, which is significantly reduced. This reduces user waiting time and improves the user experience without substantially increasing computational load. Furthermore, based on the semantic understanding of this method, real-time semantic interruption can also be achieved. When the system plays the synthesized audio, if the playback is too long and the user doesn't want to wait for it to finish, they might interrupt the playback. In this case, the user's speech is recognized as text by streaming speech recognition, and the semantic results are parsed by the streaming semantic understanding module. If the semantic results match the settings, the playback of the synthesized audio is interrupted. This avoids accidental interruptions and allows for the fastest possible interruption of the synthesized audio playback.

[0067] like Figure 4 The diagram shown is a schematic diagram of a real-time semantic understanding system for spoken dialogue provided by an embodiment of the present invention. The system can execute the real-time semantic understanding method for spoken dialogue described in any of the above embodiments and is configured in a terminal.

[0068] This embodiment provides a real-time semantic understanding system 10 for spoken dialogue, which includes: a speech recognition program module 11, an engine state determination program module 12, a semantic reasoning program module 13, and a semantic understanding program module 14.

[0069] The speech recognition module 11 is used to input the detected user speech into the streaming speech recognition engine in real time and output the streaming speech recognition result, wherein the speech recognition result includes a voice request identifier for determining whether it is the same sentence; the engine state determination module 12 is used to receive the streaming speech recognition result using the streaming semantic understanding engine and query the engine state corresponding to the voice request identifier in the streaming speech recognition result; the semantic reasoning module 13 is used by the streaming semantic understanding engine to reason about the streaming speech recognition result based on the engine state, obtain the reasoned semantic parsing result, and update the engine state; the semantic understanding module 14 is used to determine the reasoned semantic parsing result as the final semantic parsing result of the user speech if the user speech is detected to have ended.

[0070] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the real-time semantic understanding method for spoken dialogue in any of the above method embodiments.

[0071] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0072] The detected user voice is input to the streaming speech recognition engine in real time, and the streaming speech recognition result is output. The speech recognition result includes a voice request identifier used to determine whether it is the same sentence.

[0073] The streaming semantic understanding engine receives the speech recognition results of the streaming stream and queries the engine status corresponding to the speech request identifier within the speech recognition results of the streaming stream.

[0074] The streaming semantic understanding engine infers the streaming speech recognition results based on the engine state to obtain the inferred semantic parsing results, and updates the engine state accordingly.

[0075] If the user's speech ends, the inference semantic parsing result is determined as the final semantic parsing result of the user's speech.

[0076] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, they perform the real-time semantic understanding method for spoken dialogue in any of the above method embodiments.

[0077] Figure 5This is a schematic diagram of the hardware structure of an electronic device for a real-time semantic understanding method for spoken dialogue, provided in another embodiment of this application. Figure 5 As shown, the device includes:

[0078] One or more processors 510 and memory 520, Figure 5 Taking a processor 510 as an example, the device for a real-time semantic understanding method for spoken dialogue may further include an input device 530 and an output device 540.

[0079] The processor 510, memory 520, input device 530, and output device 540 can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0080] The memory 520, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the real-time semantic understanding method for spoken dialogue in the embodiments of this application. The processor 510 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 520, thereby implementing the real-time semantic understanding method for spoken dialogue in the above-described method embodiments.

[0081] The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data, etc. Furthermore, the memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 520 may optionally include memory remotely located relative to the processor 510, and these remote memories may be connected to the mobile device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] Input device 530 can receive input numerical or character information. Output device 540 may include display devices such as a display screen.

[0083] The one or more modules are stored in the memory 520, and when executed by the one or more processors 510, they execute the real-time semantic understanding method for spoken dialogue in any of the above method embodiments.

[0084] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0085] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0086] This invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the real-time semantic understanding method for spoken dialogue according to any embodiment of this invention.

[0087] The electronic devices described in this application exist in various forms, including but not limited to:

[0088] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0089] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as tablet computers.

[0090] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0091] (4) Other electronic devices with data processing functions.

[0092] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0094] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time semantic understanding method for spoken dialogue, comprising: The detected user voice is input to the streaming speech recognition engine in real time, and the streaming speech recognition result is output. The speech recognition result includes a voice request identifier used to determine whether it is the same sentence. The streaming semantic understanding engine with state memory is used to receive the speech recognition results of the streaming stream and query the engine state corresponding to the voice request identifier in the speech recognition results of the streaming stream. If the voice request identifier does not exist, the streaming semantic understanding engine is set to its initial state. The initial state of the streaming semantic understanding engine is used to perform semantic understanding on the streaming speech recognition results, obtaining a temporary inference semantic parsing result. The current parsing is stored using the streaming semantic understanding engine with state memory. If the voice request identifier exists, the streaming semantic understanding engine continues parsing from the previously stored parsing, performing incremental parsing to obtain the inference semantic parsing result, and updating the engine state. If the user's speech ends, the inference semantic parsing result is determined as the final semantic parsing result of the user's speech.

2. The method according to claim 1, wherein, If the user's voice is detected to be not finished, the method further includes: The detected user voice is continuously input into the streaming speech recognition engine in real time, and the streaming speech recognition results are output. The streaming semantic understanding engine receives the speech recognition results of the streaming stream and queries the engine status corresponding to the speech request identifier within the speech recognition results of the streaming stream. The streaming semantic understanding engine continuously infers the streaming speech recognition results based on the engine state to obtain inference semantic parsing results, and continuously updates the engine state.

3. The method according to claim 1, wherein, After determining the inference semantic parsing result as the final semantic parsing result of the user's speech, the method further includes: The final semantic parsing result is sent to the dialogue management engine, which generates interactive dialogue voice to respond to the user based on the final semantic parsing result.

4. A real-time semantic understanding system for spoken dialogue, comprising: The speech recognition program module is used to input the detected user speech into the streaming speech recognition engine in real time and output the streaming speech recognition result, wherein the speech recognition result includes a speech request identifier used to determine whether it is the same sentence; The engine state determination module is used to receive the speech recognition results of the streaming semantic understanding engine with state memory and query the engine state corresponding to the voice request identifier in the speech recognition results of the streaming engine. The semantic reasoning module is used to set the streaming semantic understanding engine to an initial state if the voice request identifier does not exist. It then uses this initial state to perform semantic understanding on the streaming speech recognition results, obtaining a temporary semantic reasoning parsing result. Finally, it stores the current parsing result using a streaming semantic understanding engine with state memory. If the voice request identifier exists, the streaming semantic understanding engine continues parsing from the previously stored parsing, performing incremental parsing to obtain the inference semantic parsing result, and updating the engine state. The semantic understanding program module is used to determine the inference semantic parsing result as the final semantic parsing result of the user's speech if the user's speech is detected to have ended.

5. The system according to claim 4, wherein, The semantic understanding program module is used for: If the system detects that the user's speech has not ended, it continuously inputs the detected user speech into the streaming speech recognition engine in real time and outputs the streaming speech recognition result. The streaming semantic understanding engine receives the speech recognition results of the streaming stream and queries the engine status corresponding to the speech request identifier within the speech recognition results of the streaming stream. The streaming semantic understanding engine continuously infers the streaming speech recognition results based on the engine state to obtain inference semantic parsing results, and continuously updates the engine state.

6. The system according to claim 4, wherein, The system also includes an interactive program module for: The final semantic parsing result is sent to the dialogue management engine, which generates interactive dialogue voice to respond to the user based on the final semantic parsing result.

7. An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-3.

8. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-3.

Images