Dialogue system, dialogue processing method, translation device and translation method

Abstract

The present invention relates to a dialogue system, a dialogue processing method, a translation device and a translation method. The dialogue system includes: a speech recognizer, a dialogue manager and a result processor, wherein the speech recognizer is configured to generate an input sentence by converting a user's speech into text; the dialogue manager is configured to generate a meaning representation of the input sentence; and the result processor is configured to generate a plurality of output sentences corresponding to the meaning representation. The dialogue manager generates a meaning representation for each of the plurality of output sentences. The result processor generates a system response based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

Description

Technical Field

[0001] Embodiments of the present invention relate to a dialogue system, a dialogue processing method, a translation device and a translation method, which are capable of outputting input sentences by translating the input sentences into an electronic device in another language that can communicate with the user. Background Art

[0002] A dialog system is a system that can recognize a user's voice and provide a service corresponding to the recognized voice.

[0003] The dialog system can output a system response to provide a service corresponding to the recognized voice. The system response may include an answer to the user's question, and may include a question for confirming the user's voice, or may include a guide to the service to be provided.

[0004] The system response output from the dialogue system makes the user feel as if the user is having a real conversation and speaks naturally, as if the user is having a conversation with someone. Therefore, when evaluating the performance of the dialogue system, an important factor may be the matching of the system response output from the dialogue system with the user's voice. Summary of the invention

[0005] The disclosed embodiments provide a dialog system and a dialog processing method that can improve the accuracy of system responses and user convenience by generating system responses using a semantic analysis model for analyzing the meaning of user speech.

[0006] In addition, the disclosed embodiments provide a translation apparatus and a translation method that can improve the accuracy of translation by using a semantic analysis model for analyzing the meaning of a user's utterance when selecting a final output sentence from a plurality of output sentences translated from an input language into another language.

[0007] According to an embodiment of the present invention, a dialogue system includes: a speech recognizer, a dialogue manager, and a result processor, wherein the speech recognizer is configured to generate an input sentence by converting a user's speech into text; the dialogue manager is configured to generate a meaning representation of the input sentence; and the result processor is configured to generate a plurality of output sentences corresponding to the meaning representation. The dialogue manager can generate a meaning representation for each of the plurality of output sentences. The result processor can generate a system response based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0008] Further, the result processor may use an N-best algorithm to determine the order of the multiple output statements.

[0009] Further, the result processor may determine again the ranking of the plurality of output sentences based on the similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0010] Further, the result processor may use an N-best algorithm to assign a confidence score to each of the plurality of output statements.

[0011] Further, the result processor may assign a similarity score to each of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0012] Further, the result processor may generate a system response based on a total score that adds the confidence score and the similarity score.

[0013] According to an embodiment of the present invention, a translation device includes: a translator, a first dialogue manager, a second dialogue manager, and a determiner, wherein the translator is configured to translate an input sentence in a first language into a second language to generate a plurality of output sentences; the first dialogue manager is configured to generate a meaning representation of the input sentence in the first language; the second dialogue manager is configured to generate a meaning representation of each of the plurality of output sentences in the second language; and the determiner is configured to determine a final output sentence among the plurality of output sentences in the second language based on the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0014] Further, the translator may use an N-best algorithm to determine the ranking of the plurality of output sentences in the second language.

[0015] Further, the determiner may determine again the ranking of the plurality of output sentences based on the similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0016] Further, the translator may assign a confidence score to each of the plurality of output sentences in the second language using an N-best algorithm.

[0017] Further, the determiner may assign a similarity score to each of the plurality of output sentences in the second language based on a similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0018] Further, the determiner may determine the final output sentence based on a total score that adds the confidence score and the similarity score.

[0019] According to an embodiment of the present invention, a dialogue processing method includes: generating an input sentence by converting a user's voice into text; generating a meaning representation of the input sentence; generating multiple output sentences corresponding to the meaning representation of the input sentence; generating a meaning representation of each of the multiple output sentences; and generating a system response based on the meaning representation of the input sentence and the meaning representation of each of the multiple output sentences.

[0020] Further, generating a plurality of output sentences corresponding to the meaning representation may include: determining the ranking of the plurality of output sentences using an N-best algorithm.

[0021] Further, generating the system response may include: re-determining the ranking of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0022] Further, generating a plurality of output sentences corresponding to the meaning representations may include: assigning a confidence score to each of the plurality of output sentences using an N-best algorithm.

[0023] Further, generating the system response may include assigning a similarity score to each of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0024] Further, generating the system response may include generating the system response based on a total score obtained by adding the confidence score and the similarity score.

[0025] According to an embodiment of the present invention, a translation method includes: generating a plurality of output sentences by translating an input sentence in a first language into a second language; generating a meaning representation of the input sentence in the first language; generating a meaning representation of each of the plurality of output sentences in the second language; and determining a final output sentence among the plurality of output sentences in the second language based on the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0026] Further, generating a plurality of output sentences in the second language may include: determining the ranking of the plurality of output sentences in the second language using an N-best algorithm.

[0027] Further, determining the final output sentence may include: determining again the ranking of the plurality of output sentences based on a similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0028] Further, generating a plurality of output sentences in the second language may include: assigning a confidence score to each of the plurality of output sentences in the second language using an N-best algorithm.

[0029] Further, determining the final output sentence may include assigning a similarity score to each of the plurality of output sentences in the second language based on a similarity between a meaning representation of the input sentence in the first language and a meaning representation of each of the plurality of output sentences in the second language.

[0030] Furthermore, determining the final output sentence may further include: determining the final output sentence based on a total score obtained by adding the confidence score and the similarity score. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] These and / or other aspects of the present invention will become more apparent and more readily appreciated through the following detailed description of the embodiments presented in conjunction with the accompanying drawings, in which:

[0032] Figure 1 is a control block diagram of a dialog system according to an implementation scheme.

[0033] Figure 2 is a diagram showing an example of a result processor of a dialog system according to an embodiment.

[0034] Figures 3 to 6 is a schematic diagram showing a detailed process of outputting a system response when a user's utterance is input to a dialog system according to an embodiment.

[0035] Figure 7 is a control block diagram of a translation device according to an embodiment.

[0036] Figures 8 to 10 is a schematic diagram showing a specific process of outputting a translation sentence when an input sentence is input to the translation apparatus according to the embodiment.

[0037] Fig.11 is a flowchart of a conversation processing method according to an implementation scheme.

[0038] Fig.12 is a flow chart of a translation method according to an embodiment.

[0039] Description of Reference Numerals

[0040] 100: Dialogue System

[0041] 110: Speech Recognizer

[0042] 120: Dialogue Manager

[0043] 130: Result Processor

[0044] 200: Translation device

[0045] 210: Translator

[0046] 220: First semantic analyzer

[0047] 230: Second semantic analyzer

[0048] 240: Determinator. DETAILED DESCRIPTION

[0049] The embodiments described herein and the configurations shown in the drawings are examples of the present invention. At the time of filing this application, various modifications that can replace the embodiments and drawings of this specification may be made.

[0050] In addition, the terms used in this article are only used to describe specific embodiments and are not intended to limit the present invention. Unless the context clearly indicates otherwise, the singular includes the plural. As used herein, the term "including" or "having" is intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described in the specification. The term "including" or "having" does not exclude the pre-existence or addition of one or more other features or numbers, steps, operations, components, parts or combinations thereof.

[0051] In addition, terms such as "-component", "-group", "-block", "-member", and "-module" may also refer to a unit for processing at least one function or operation. For example, these terms may refer to at least one hardware processed by at least one hardware, such as a field programmable gate array (FPGA) / application specific integrated circuit (ASIC), at least one software stored in a memory, or a processor.

[0052] In addition, ordinal numbers such as "first" and "second" used before the components described in this document are also only used to distinguish the components from each other. The connection order between these components and the order of use do not have different semantics such as priority.

[0053] The reference numerals of these steps are used to identify the steps. These reference numerals do not indicate the order between the steps. Unless the context clearly indicates a specific order, each step can be performed in an order different from the order described.

[0054] On the other hand, the disclosed embodiments can be implemented in the form of a recording medium for storing instructions that can be executed by a computer. The instructions can be stored in the form of program code, and when executed by a processor, a program module can be generated to perform the operations of the disclosed embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0055] The computer-readable recording medium may include various recording media storing instructions that can be read by a computer, such as a read-only memory (ROM), a random access memory (RAM), a magnetic tape, a magnetic disk, a flash memory, an optical data storage device, etc.

[0056] Hereinafter, implementations of a dialog system, a dialog processing method, and an electronic device according to an aspect are described in detail below with reference to the accompanying drawings.

[0057] The dialogue system according to the embodiment is a system that can recognize the user's voice and provide services corresponding to the recognized voice. The dialogue system can output a system response to provide services corresponding to the recognized voice. The system response may include answers to user questions, and may include questions for confirming the user's voice, or may include guidance for services to be provided.

[0058] Figure 1 is a control block diagram of a dialog system according to an implementation scheme.

[0059] refer to Figure 1 According to the implementation scheme, the dialogue system 100 includes: a speech recognizer 110, a dialogue manager 120 and a result processor 130, wherein the speech recognizer 110 converts the user's speech into text and generates an input sentence; the dialogue manager 120 generates a meaning representation for the input sentence; and the result processor 130 generates a plurality of output sentences corresponding to the meaning representation.

[0060] According to one embodiment, the dialog system 100 may include at least one memory for storing programs and related data, and at least one processor for executing the stored programs.

[0061] In this embodiment, components are categorized not based on physical location or configuration but rather based on operation. The speech recognizer 110, the dialog manager 120, and the result processor 130 may each use separate memory and processors, or may also partially or fully share memory and processors.

[0062] According to one embodiment, the dialogue system 100 may be implemented as a server. In this case, the speech recognizer 110, the dialogue manager 120, and / or the result processor 130 of the dialogue system 100 may be provided in the server. However, some components such as the speech recognizer 110, the dialogue manager 120, and / or the result processor 130 of the dialogue system 100 may be provided in a user terminal that connects the dialogue system 100 with the user.

[0063] For example, when the user terminal is a vehicle, some components of the dialogue system 100 may be set in the vehicle. When the user terminal is a mobile device such as a smart phone, an artificial intelligence (AI) speaker, or a personal computer (PC), some components of the dialogue system 100 may be set in the mobile device, the AI ​​speaker, or the PC.

[0064] Alternatively, the user may download and use a program for executing some functions of the components of the dialog system 100 on the user terminal.

[0065] The microphone and the speaker may be provided in a user terminal such as a vehicle, a mobile device, or a PC. The user terminal may be connected to the dialogue system 100 via wireless communication. Figure 1 As shown in the control block diagram, the dialogue system 100 may further include a wireless communication module that can exchange data with the user terminal through wireless communication.

[0066] The user voice input through the microphone may be transmitted to the wireless communication module of the dialogue system 100 .

[0067] The speech recognizer 110 may convert the speech into text by applying a speech recognition algorithm or speech to text (STT) to the user's speech. For example, by applying a feature vector extraction technique such as Cepstrum, Linear Predictive Coefficient (LPC), Mel Frequency Cepstral Coefficient (MFCC) or Filter Bank Energy, the speech recognizer 110 extracts a feature vector of a speech signal corresponding to the user's speech that may be extracted.

[0068] The recognition result can be obtained by comparing the extracted feature vector with the trained reference pattern. To this end, an acoustic model for modeling and comparing the signal characteristics of the speech and a language model for modeling the linguistic order relationship such as words or syllables corresponding to the recognized vocabulary can be used.

[0069] The acoustic model can be further divided into a direct comparison method for setting the recognition object as a feature vector model and comparing it with the feature vector of the speech data, and a statistical method for performing statistical processing on the feature vector of the recognition object.

[0070] The direct comparison method is a method of setting the unit of the word, phoneme, etc. to be recognized as a feature vector model and comparing the similarity of the input speech. According to the vector quantization method, the feature vector of the input speech data is mapped with the codebook as a reference model and encoded as a representative value to compare the code value.

[0071] The statistical model method is a method of constructing the unit of the recognition object into a state sequence and using the relationship between the state columns. The state column can include multiple nodes. The method of using the relationship between the state columns again includes dynamic time warping (DTW), hidden Markov models (HMM) and neural networks.

[0072] Dynamic Time Warping is a technique that compensates for differences in the time axis while taking into account the dynamic nature of speech, where the length of speech varies over time, even when the same person makes the same utterance. Hidden Markov Models, or HMMs for short, assume speech to be a Markov process with state transition probabilities and observation probabilities of nodes (output symbols) in each state. HMMs then estimate state transition probabilities and observation probabilities of nodes based on training data. In the estimated model, HMMs calculate the probability that the input speech will occur as a recognition technique.

[0073] On the other hand, a language model for modeling linguistic order relations such as words or syllables can apply the acoustic relations between units constituting the language to the units obtained in speech recognition, thereby reducing acoustic ambiguity and reducing recognition errors. The language model is a model based on a statistical language model and a finite state automaton (FSA), and the statistical language model uses chain probabilities of words such as Unigram, Bigram, and Trigram.

[0074] The speech recognizer 110 may use any of the above methods when recognizing a speech signal corresponding to the user's speech and converting it into text. For example, an acoustic model using a hidden Markov model may be used, or a model integrating an acoustic model and a language model may be used.

[0075] The speech recognizer 110 converts the user's speech into text and inputs it into the dialog manager 120. In the following embodiments, the user's speech converted into text is referred to as an input sentence.

[0076] The dialog manager 120 may apply natural language understanding to input sentences to determine the user's intent included in the user's speech. Entity names are proper nouns such as names of people, places, organizations, time, date, currency, etc. Entity name recognition is the task of recognizing entity names in sentences and determining the type of the recognized entity names. By recognizing individual names, we can extract important keywords from sentences to understand the meaning of the sentences.

[0077] The dialogue manager 120 may extract a domain from the user's speech. The domain may identify the subject of the language spoken by the user. For example, various domains may be predetermined, such as text, navigation, schedule, weather, traffic information, vehicle control, and home appliance control. The dialogue manager 120 may determine which predetermined domain the domain of the user's speech is.

[0078] The dialogue manager 120 can analyze the voice behavior of the user's voice. Voice behavior analysis is used to analyze the intention of the user's voice and grasp the voice intention such as whether the user asks a question, makes a request, responds, or expresses simple emotions.

[0079] The dialog manager 120 may grasp the user's intention based on information about domains, entity names, dialog actions, etc. corresponding to the user's voice, and extract actions corresponding to the user's intention. Actions may be defined by objects and operators.

[0080] For example, when the identified user intention is "run the air conditioner", the corresponding action can be defined as "air conditioner (object)_on (operator)", and when the identified user intention is "send text messages", the corresponding action can be defined as "text messages (object)_send (operator)".

[0081] In addition, the dialog manager 120 may extract factors related to performing an action. The factors related to performing an action may be valid factors directly required to perform the action, or may be invalid factors used to extract such valid factors.

[0082] For example, if the text output generated by the speech recognizer 110 is "Send a text message to Jidong", the dialog manager 120 determines that the domain corresponding to the user's speech is "SMS", the action is "SMS send", and the speech behavior corresponds to "request".

[0083] The object name "Jidong" corresponds to {factor 1: recipient} related to performing the action. However, in order to actually send the text message, {factor 2: specific message content} is required.

[0084] In this case, the dialog system 100 may output a system response “Please tell me the message to be sent” to obtain specific message content from the user.

[0085] Based on the above text (input sentence) analysis results, the dialogue manager 120 generates a meaning representation in the result processor 130 for generating a system response corresponding to the user's intention or providing a service corresponding to the user's intention. The meaning representation in the dialogue processing can be the result of natural language understanding or the input of natural language generation. For example, the dialogue manager 120 can analyze the user's voice to generate a meaning representation that expresses the user's intention, and generate a meaning representation corresponding to the next system voice taking into account the dialogue flow and context.

[0086] The result processor 130 may generate a sentence (hereinafter referred to as an output sentence) to be output as a system response based on the meaning representation output from the dialogue manager 120. The generated sentence may be synthesized into a voice signal by a text-to-speech (TTS) engine provided in the result processor 130 and output through a speaker provided in the user terminal.

[0087] The result processor 130 may generate an output sentence by applying a natural language generation technique, and may generate a plurality of output sentences by applying an N-best algorithm.

[0088] When the result processor 130 generates three output sentences (output sentence 1, output sentence 2, and output sentence 3) based on the meaning representation of the input sentence, the three output sentences are input to the dialogue manager 120. The dialogue manager 120 may generate the meaning representation for each of the plurality of output sentences.

[0089] As described above, the dialog manager 120 may apply natural language understanding techniques to analyze the meaning of the input sentence. The dialog manager 120 may generate a meaning representation by analyzing the output sentence generated by the result processor 130 using the same techniques.

[0090] The dialog manager 120 may generate meaning representation 1 for output sentence 1, meaning representation 2 for output sentence 2, and meaning representation 3 for output sentence 3, respectively, and send the results to the result processor 130.

[0091] The result processor 130 may generate a system response based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0092] In the following, reference is made to Figure 2 The process of generating a system response by the result processor is described in more detail.

[0093] Figure 2 is a schematic diagram showing a specific example of a result processor of the dialog system according to the embodiment.

[0094] refer to Figure 2 , the result processor 130 of the dialog system 100 according to the embodiment may include a sentence generator 131 and a determiner 132. The sentence generator 131 and the determiner 132 are also classified based on the operation to be performed. The sentence generator 131 and the determiner 132 may share: a memory or a computer-readable medium in which a program including computer-executable instructions and related data is stored; and a processor that executes the program.

[0095] The sentence generator 131 may generate a plurality of output sentences by applying a natural language generation technique to the meaning representation of the input sentence generated by the dialog manager 120 .

[0096] The sentence generator 131 can use the N-best algorithm to calculate the confidence score for each of the multiple output sentences, and output the N output sentences in the order of confidence scores from high to low. In other words, the multiple output sentences can be sorted using the N-best algorithm. The confidence score is an indicator of the reliability of the result of the natural language generation result, which can be expressed as a value between 0 and 1.

[0097] As in Figure 2 As shown in the example of , the sentence generator 131 can output three output sentences in the order of confidence scores from high to low. The three output sentences are input to the dialogue manager 120 again.

[0098] The dialog manager 120 may apply natural language understanding technology to generate meaning representations for each of the three output sentences. Meaning representation 1 of output sentence 1, meaning representation 2 of output sentence 2, and meaning representation 3 of output sentence 3 may be generated and output respectively. The output meaning representations are input to the result processor 130.

[0099] The determiner 132 of the result processor 130 may reorder the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

[0100] Since the meaning representation of the input sentence and the meaning representation of the output sentence are similar, the accuracy of the output sentence is improved. Accordingly, the determiner 132 can assign a similarity score to each of the plurality of output sentences based on the similarity between the meaning representation of the input sentence and the output sentence. The higher the similarity, the higher the similarity score.

[0101] The determiner 132 may calculate a total score obtained by adding the reliability score and the similarity score of each of the plurality of output statements, and re-determine the ranking of the plurality of output statements based on the total score. The ranking may be re-ranked in descending order of the total score, and finally, the output statement with the highest total score may be selected as the system response.

[0102] In the control block diagram, only the sentence generator 131 and the determiner 132 are shown in the result processor 130, but the result processor 130 may have a text-to-speech (TTS) engine for synthesizing the final output sentence output from the determiner 132 and a control signal generator for generating various control signals to provide services corresponding to the user's intention.

[0103] According to the above operation, the system response output from the result processor 130 may be output as a voice through a speaker provided in the user terminal.

[0104] In the following, reference is made to Figures 3 to 6 A specific example of a system response output from the dialogue system 100 according to the embodiment is described.

[0105] Figures 3 to 6 is a schematic diagram showing a detailed process of outputting a system response when a user's utterance is input to a dialog system according to an embodiment.

[0106] refer to Figure 3 , when the user's voice "Send a text message to Hong Gil-dong" is input into the dialogue system 100, the voice recognizer 110 converts the user's voice into text to generate an input sentence.

[0107] The input sentence is input to the dialogue manager 120 , and a natural language understanding technology may be applied to analyze the meaning of the input sentence “send a text message to Hong Gil-dong” and generate a meaning representation based on the analysis result.

[0108] The dialog manager 120 may determine that the action of the input sentence "send a text message to Hong Gil-dong" is "text message_send", the voice behavior is "request", and the factor_recipient is "Hong Gil-dong". The dialog manager 120 may generate a meaning representation including the information. The generated meaning representation is input to the result processor 130.

[0109] The result processor 130 may generate multiple output sentences based on the meaning representation of the input sentence. As described above, confidence scores may be calculated for the multiple output sentences, and the N output sentences may be sorted in descending order of confidence scores. Figure 4 , the result processor 130 can generate output sentence 1: "Is the text you sent to Hong Gil-dong correct?", output sentence 2: "Please tell me the content of the text message you want to send to Hong Gil-dong", and output sentence 3: "Please tell me the content of the text message you want to send" based on the meaning representation of the input sentence. The result processor 130 can sort the three sentences based on the confidence score. In this example, the order is determined to be in the order of output sentence 1 (confidence score: 0.7), output sentence 2 (confidence score: 0.6), and output sentence 3 (confidence: 0.3).

[0110] A plurality of output sentences generated based on the meaning representation of the input sentence are input to the dialog manager 120 .

[0111] The dialog manager 120 may: analyze the meaning by applying a natural language understanding technique to each of the plurality of output sentences in the same manner as analyzing the meaning of the input sentence; and generate a meaning representation.

[0112] refer to Figure 5, as a result of semantic analysis of output sentence 1, it can be determined that the action is "SMS_Send", the voice behavior is "Confirm", the factor_recipient is "Hong Gil-dong", and a meaning representation 1 including the information is generated.

[0113] As a result of the semantic analysis of output sentence 2, it can be determined that the action is "SMS_Send", the voice behavior is "Request", the factor_recipient is "Hong Gil-dong", and a meaning representation 2 including this information is generated.

[0114] As a result of the semantic analysis of the output sentence 3, it can be determined that the action is "SMS_Send", the voice behavior is "Request", and a meaning representation 3 including this information is generated.

[0115] The meaning representation of each of the plurality of output sentences generated by the dialog manager 120 is input to the result processor 130 .

[0116] The result processor 130 may assign a similarity score to each of the plurality of output sentences by comparing the meaning representation of each of the plurality of output sentences with the meaning representation of the input sentence. As described above, the more similar the meaning representations are, the higher the similarity score may be given.

[0117] refer to Figure 6 ,Based on the comparison results of meaning representation, the similarity score of output sentence 1 is 0.66, the similarity score of output sentence 2 is 1.0, and the similarity score of output sentence 3 is 0.66.

[0118] The result processor 130 can calculate a total score by adding the confidence score and the similarity score of each of the multiple output sentences. The total score of output sentence 1 is 0.7+0.66=1.36. The total score of output sentence 2 is 0.6+1.0=1.6. The total score of output sentence 3 is 0.3+0.66=0.96.

[0119] The result processor 130 reorders the plurality of output statements based on the total scores. If the order is reordered from high to low in terms of the total scores, output statement 2 is ranked first, output statement 1 is ranked second, and output statement 3 is ranked third.

[0120] Therefore, the result processor 130 can convert the output sentence 2 of the first priority level “Please tell me the content of the SMS you want to send” into a voice signal, and output the result as a system response.

[0121] As described above, the higher the similarity between the meaning representation of the input sentence and the meaning representation of the output sentence, the more accurate the output sentence can be. The same principle can be applied to the translation device to improve the accuracy of the translation result. Hereinafter, the translation device according to the embodiment is described below.

[0122] Figure 7 is a control block diagram of a translation device according to an embodiment.

[0123] refer to Figure 7 According to the embodiment, the translator 200 may include: a translator 210, a first meaning analyzer 220, a second meaning analyzer 230, and a determiner 240, wherein the translator 210 is used to translate an input sentence in a first language into a second language and generate a plurality of output sentences; the first meaning analyzer 220 is used to generate a meaning representation for the input sentence in the first language; the second meaning analyzer 230 is used to generate a meaning representation for each of the plurality of output sentences in the second language; the determiner 240 determines the final output sentence in the plurality of output sentences in the second language based on the meaning representation of the input sentence in the first language and the meaning representation of the plurality of output sentences in the second language. The first meaning analyzer 220 may be referred to as a first semantic analyzer or a first dialog manager. The second meaning analyzer 230 may be referred to as a second semantic analyzer or a second dialog manager.

[0124] The translation device 200 according to the embodiment can be a device for translating an input sentence into another language by performing machine translation. The user can input the input sentence in the form of text, or can input it in the form of a voice signal by voice. When input in the form of a voice signal, a speech recognizer can convert the voice signal into text and input it into the translation device 200, or the translation device 200 can include a speech recognizer. In either case, it is assumed that the input sentence input to the translator 210 is text.

[0125] The translation device 200 according to the embodiment may include: at least one memory or computer-readable medium and at least one processor, wherein the at least one memory or computer-readable medium is used to store a program containing computer-executable instructions and related data for performing the operations described below; and the at least one processor is used to execute the stored program.

[0126] The components in this embodiment are classified based on operation rather than physical location or composition, and may include a translator 210, a first meaning analyzer 220, a second meaning analyzer 230, etc. The determiner 240 may use a separate memory and processor, and some or all of them may share a memory and processor.

[0127] The translation device 200 according to the embodiment may be implemented as a server, and in this case, the components 210, 220, 230, and 240 of the translation device 200 may be provided in the server. However, some of the components 210, 220, 230, and 240 of the translation device 200 may be provided in a user terminal connecting the user and the translation device 200.

[0128] Alternatively, the user may download and use a program for executing some functions of the components of the dialog system 100 on the user terminal.

[0129] An input device and an output device are provided in a user terminal such as a vehicle, a mobile device or a PC. The input device is, for example, a microphone, a touch screen and a keyboard, into which a sentence in the first language is input; and the output device is, for example, a display and a speaker, into which a sentence translated into the second language is output. The user terminal can be connected to the translation apparatus 200 via wireless communication. Figure 7 As shown in the control block diagram, the translation device 200 may further include a wireless communication module, which can exchange data with the user terminal through wireless communication.

[0130] Translator 210 can generate multiple output sentences by translating an input sentence in a first language into a second language using one of various machine translation models. The first language and the second language only need to be different from each other. The types of the first language and the second language can include, but are not limited to, Korean, English, Chinese, Japanese, Spanish, Portuguese, German, French, and Hindi.

[0131] When the translator 210 generates multiple output sentences, the N-best algorithm can be used. As described above, in the embodiment of the dialogue system 100, the translator 210 can sort the multiple output sentences in the second language. The sorting of the multiple output sentences in the second language can be determined according to the order of confidence scores from high to low.

[0132] The first meaning analyzer 220 can apply natural language understanding technology to the input sentence in the first language to analyze the meaning of the input sentence and generate a meaning representation of the input sentence. The description of the meaning analysis and the meaning representation is as described above in the embodiment of the dialogue system 100.

[0133] The second meaning analyzer 230 may analyze the meaning of each of the plurality of output sentences and generate a meaning representation by applying a natural language understanding technique to the plurality of output sentences in the second language.

[0134] like Figure 7 As shown, when translator 210 generates output statement 1, output statement 2 and output statement 3, second meaning analyzer 230 can generate and input meaning representation 1 of output statement 1, meaning representation 2 of output statement 2 and meaning representation 3 of output statement 3 to determiner 240.

[0135] The first meaning analyzer 220 and the second meaning analyzer 230 differ only in the language of the sentences to be analyzed, and they use the same method of analyzing the meaning by applying the natural language understanding technology.

[0136] The determiner 240 determines again the ranking of the plurality of output sentences in the second language based on the similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

[0137] In detail, as the similarity between the meaning expressions increases, the determiner 240 may assign a higher similarity score. The determiner 240 may calculate a total score obtained by adding the confidence score and the similarity score for each of the plurality of output sentences. The determiner 240 may sort the plurality of output sentences in order of increasing total scores.

[0138] The determiner 240 may determine the output sentence with the highest total score as the final output sentence. The translation apparatus 200 may output the final output sentence as a translation sentence of the input sentence. The final output sentence may be output as a text, or may be synthesized into a speech signal and output as a system response.

[0139] In the following, reference is made to Figures 8 to 10 The operation of the translation device 200 according to the embodiment is described in detail.

[0140] Figures 8 to 10 2 is a schematic diagram showing a specific process of outputting a translation sentence when an input sentence is input to the translation apparatus 200 according to the embodiment.

[0141] In this example, the first language is English and the second language is Korean. Figure 8 As shown, when an input sentence "Trump met Moon" in a first language is input to the translation device 200, the translator 210 translates the input sentence into Korean in a second language, and converts and generates a plurality of output sentences.

[0142] For example, you can generate output statement 1 Output statement 2 and output statement 3

[0143] Translator 210 may assign a confidence score to each of the plurality of output sentences by applying an N-best algorithm. For example, output sentence 1 may have a confidence score of 0.7, output sentence 2 may have a confidence score of 0.6, and output sentence 3 may have a confidence score of 0.4.

[0144] Therefore, according to the N-best algorithm, multiple output statements are sorted in the order of output statement 1 → output statement 2 → output statement 3.

[0145] The output sentence 1, the output sentence 2, and the output sentence 3 may be input to the second meaning analyzer 230. The second meaning analyzer 230 may generate a meaning representation by analyzing the meaning of Korean as the second language.

[0146] refer to Fig. 9 , the second meaning analyzer 230 can analyze the meaning of the output sentence 1 to determine that the voice behavior is "information" and the object name is " / person's name", and a meaning representation 1 including the speech behavior and the object name can be generated.

[0147] The second meaning analyzer 230 may analyze the meaning of the output sentence 2 to determine that the voice behavior is "notification" and the object name is " / name"," / person's name", and a meaning representation 2 including the speech behavior and the object name can be generated.

[0148] The second meaning analyzer 230 may analyze the meaning of the output sentence 3 to determine that the voice behavior is "notification" and the object name is " / name"," / person's name", and can generate a meaning representation 3 including the speech behavior and the object name.

[0149] The meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences are input to the determiner 240. The determiner 240 compares the meaning representation of the input sentence with the meaning representation of each of the plurality of output sentences to determine a final output sentence.

[0150] refer to Fig.10 , determiner 240: compares the meaning representation 1 of the input sentence and the meaning representation 1 of the output sentence 1, and calculates a similarity score of 0.66; compares the meaning representation of the input sentence and the meaning representation 2 of the output sentence 2, and calculates a similarity score of 1.0; compares the meaning representation of the input sentence and the meaning representation 3 of the output sentence 3, and calculates a similarity score of 1.0.

[0151] The determiner 240 may calculate a total score obtained by adding the confidence score and the similarity score of each of the plurality of output sentences, and re-determine the ranking based on the total score.

[0152] According to the total score, the order becomes output statement 2→output statement 3→output statement 1. The determiner 240 may determine output statement 2 having the highest total score as the final output statement and output it.

[0153] In this way, when the similarity between the meaning representation of the input sentence and the meaning representation of the output sentence into which the input sentence is translated is considered, a translated sentence having a meaning similar to that of the input sentence can be output to the user.

[0154] Hereinafter, a dialog processing method and a translation method according to an embodiment are described below.

[0155] When implementing the dialogue processing method according to the embodiment, the above-mentioned dialogue system 100 may be used. When implementing the translation method according to the embodiment, the above-mentioned translation apparatus 200 may be used. Therefore, unless otherwise specified, the foregoing description of the dialogue system 100 and the translation apparatus 200 may be equally applied to the dialogue processing method and the translation method.

[0156] Fig.11 is a flowchart of a conversation processing method according to an implementation scheme.

[0157] according to Fig.11 The dialog processing method shown generates an input sentence (310) by converting the user's speech into text, and generates a meaning representation of the input sentence (320). The speech recognizer 110 can use a speech recognition algorithm to convert the user's speech into text. The dialog manager 120 can use natural language understanding technology to generate the meaning representation of the input sentence.

[0158] Generate multiple output sentences corresponding to the meaning representation (330). When the meaning representation of the input sentence is input to the result processor 130, the result processor 130 generates multiple output sentences corresponding to the meaning representation of the input sentence using the N-best algorithm. The ranking can be determined based on the confidence score of each output sentence.

[0159] Generate a meaning representation for each of the plurality of output sentences (340). The plurality of output sentences generated by the result processor 130 are input to the dialog manager 120. The dialog manager 120 analyzes the meaning of each of the plurality of output sentences by applying natural language understanding technology in the same manner as the input sentences and generates a meaning representation.

[0160] A system response is generated based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences (350). The meaning representation of each of the plurality of output sentences is input back to the result processor 130. The result processor 130 may assign a similarity score to each of the plurality of output sentences based on the similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences. The higher the similarity between the meaning representations, the higher the similarity score.

[0161] The result processor 130 may calculate a total score obtained by adding the confidence score and the similarity score of each of the plurality of output sentences. The result processor 130 may re-determine the ranking according to the total score. The result processor 130 may determine the output sentence with the highest total score as the system response.

[0162] Fig.12 is a flow chart of a translation method according to an embodiment.

[0163] according to Fig.12 The method for translating a first language input sentence into a second language generates a plurality of output sentences (410). When the translator 210 receives the input sentence in the first language, the translator 210 generates a plurality of output sentences in the second language using an N-best algorithm, calculates a confidence score for each output sentence, and determines a ranking of the plurality of output sentences based on the confidence score.

[0164] A meaning representation of the input sentence in the first language may be generated (420), and a meaning representation of each of a plurality of output sentences in the second language may be generated (430). The first meaning analyzer 220 may analyze the meaning by applying the natural language understanding technology performed by the dialogue system 100 to the input sentence in the first language, and generate the meaning representation. The second meaning analyzer 230 may analyze the meaning by applying the natural language understanding technology performed by the dialogue system 100 to the plurality of output sentences in the second language, and generate the meaning representation.

[0165] Based on the meaning representation of the input sentence in the first language and the meaning representation of the plurality of output sentences in the second language, a final output sentence is determined among the plurality of output sentences in the second language (440). The meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language are input to the determiner 240. The determiner 240 may reorder the plurality of output sentences in the second language based on the similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language. The higher the degree of similarity with the meaning representation of the input sentence, the higher the similarity score assigned to the output sentence in the second language, and the sum of the confidence score and the similarity score of each of the plurality of output sentences may be calculated. The determiner 240 may re-determine the ranking of the plurality of output sentences based on the total score, and the higher the total score, the higher the ranking may be. The determiner 240 may again determine the output sentence with the highest ranking as the final output sentence according to the determined ranking.

[0166] According to the above-described embodiment, when generating an output sentence of an input sentence in a dialogue system or a translation device, the accuracy of the output sentence can be improved by considering the similarity between the meaning representation of the input sentence and the meaning representation of the output sentence.

[0167] As described above, the disclosed embodiments have been described with reference to the accompanying drawings. It should be understood by those skilled in the art that the present invention may be implemented in a form different from the disclosed embodiments without changing the technical spirit or basic features of the present invention. The disclosed embodiments are examples and should not be interpreted as limiting.

[0168] One aspect of the present invention is directed to providing a dialog system and a dialog processing method, which can improve the accuracy of system responses and user convenience by generating system responses using a semantic analysis model for analyzing the meaning of user speech.

[0169] In addition, one aspect of the present invention is directed to providing a translation device and a translation method, which can: when selecting a final output sentence from a plurality of output sentences translated from an input language into another language, improve the accuracy of translation by using a semantic analysis model for analyzing the meaning of user utterances.

Claims

1. A dialogue system, comprising: a speech recognizer configured to generate an input sentence by converting a user's speech into text; a dialog manager configured to generate a meaning representation of an input sentence; a result processor configured to generate a plurality of output sentences corresponding to the meaning representations of the input sentences; wherein the dialog manager generates a meaning representation of each of a plurality of output sentences; The result processor generates a system response based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences; The dialog manager is configured to: analyze the voice behavior of the user's voice based on an input sentence, extract actions corresponding to the user's intentions, and extract factors related to the actions; generate a meaning representation of the input sentence including the voice behavior, actions, and factors; and generate a meaning representation of each of multiple output sentences including the voice behavior, actions, and factors based on each of multiple output sentences.

2. The dialogue system according to claim 1, wherein: The result processor determines the order of the plurality of output statements using an N-best algorithm.

3. The dialogue system according to claim 2, wherein: The result processor re-determines the ranking of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

4. The dialogue system according to claim 1, wherein: The result processor assigns a confidence score to each of the plurality of output sentences using an N-best algorithm.

5. The dialogue system according to claim 4, wherein: The result processor assigns a similarity score to each of the plurality of output sentences based on a similarity between a meaning representation of the input sentence and a meaning representation of each of the plurality of output sentences.

6. The dialogue system according to claim 5, wherein: The result processor generates a system response based on a total score that adds the confidence score and the similarity score.

7. A translation device, comprising: a translator configured to translate an input sentence in a first language into a second language to generate a plurality of output sentences; a first dialog manager configured to generate a meaning representation of an input sentence in a first language; a second dialog manager configured to generate a meaning representation for each of a plurality of output sentences in a second language; as well as a determiner configured to determine a final output sentence among the plurality of output sentences in the second language based on the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language; The first dialog manager is configured to analyze the voice behavior of the user's voice based on the input sentence in the first language, extract the action corresponding to the user's intention, and extract the factors related to the action; generate the meaning representation of the input sentence in the first language including the voice behavior, the action, and the factors; The second dialog manager is configured to generate a meaning representation of each of the plurality of output sentences in the second language including a speech act, an action, and a factor based on each of the plurality of output sentences in the second language.

8. The translation device according to claim 7, wherein: The translator determines an ordering of a plurality of output sentences in a second language using an N-best algorithm.

9. The translation device according to claim 8, wherein: The determiner re-determines the ranking of the plurality of output sentences based on a similarity between a meaning representation of the input sentence in the first language and a meaning representation of each of the plurality of output sentences in the second language.

10. The translation device according to claim 7, wherein: The translator assigns a confidence score to each of a plurality of output sentences in the second language using an N-best algorithm.

11. The translation device according to claim 10, wherein: The determiner assigns a similarity score to each of the plurality of output sentences in the second language based on a similarity between a meaning representation of the input sentence in the first language and a meaning representation of each of the plurality of output sentences in the second language.

12. The translation device according to claim 11, wherein: The determiner determines a final output sentence based on a total score obtained by adding the confidence score and the similarity score.

13. A method for processing a conversation, the method comprising: Generate input sentences by converting user speech into text; Analyzing a voice behavior of a user's voice based on an input sentence, extracting an action corresponding to the user's intention, and extracting factors related to the action; generating a meaning representation of the input sentence including the speech acts, actions, and factors; generating a plurality of output sentences corresponding to the meaning representations of the input sentences; generating a meaning representation of each of the plurality of output sentences including speech acts, actions, and factors based on each of the plurality of output sentences; A system response is generated based on the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

14. The method according to claim 13, wherein: Generating a plurality of output sentences corresponding to the meaning representation of the input sentence includes: determining an order of the plurality of output sentences using an N-best algorithm.

15. The method according to claim 14, wherein: Generating the system response includes re-determining an ordering of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

16. The method according to claim 13, wherein: Generating a plurality of output sentences corresponding to the meaning representation of the input sentence includes assigning a confidence score to each of the plurality of output sentences using an N-best algorithm.

17. The method according to claim 16, wherein: Generating the system response includes assigning a similarity score to each of the plurality of output sentences based on a similarity between the meaning representation of the input sentence and the meaning representation of each of the plurality of output sentences.

18. The method according to claim 17, wherein: Generating a system response includes generating a system response based on a total score that adds the confidence score and the similarity score.

19. A translation method, the method comprising: generating a plurality of output sentences by translating input sentences in a first language into a second language; analyzing a speech behavior of a user's speech based on an input sentence in a first language, extracting an action corresponding to the user's intention, and extracting factors related to the action; generating a meaning representation of an input sentence in the first language including the speech acts, actions, and factors; generating a plurality of output sentences in a second language corresponding to the meaning representation of the input sentence in the first language; generating a meaning representation of each of the plurality of output sentences in the second language including speech acts, actions, and factors based on each of the plurality of output sentences in the second language; A final output sentence among the plurality of output sentences in the second language is determined based on the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

20. The method according to claim 19, wherein: Generating a plurality of output sentences includes: determining the ranking of the plurality of output sentences in the second language using an N-best algorithm.

21. The method according to claim 20, wherein: Determining the final output sentence includes re-determining the ranking of the plurality of output sentences based on the similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

22. The method according to claim 19, wherein: Generating the plurality of output sentences includes assigning a confidence score to each of the plurality of output sentences in the second language using an N-best algorithm.

23. The method according to claim 22, wherein: Determining the final output sentence includes assigning a similarity score to each of the plurality of output sentences in the second language based on a similarity between the meaning representation of the input sentence in the first language and the meaning representation of each of the plurality of output sentences in the second language.

24. The method according to claim 23, wherein: Determining the final output sentence further includes determining the final output sentence based on a total score obtained by adding the confidence score and the similarity score.

Images