Electronic device and control method thereof

Abstract

An electronic device is disclosed. The electronic device includes a communication interface, a memory storing at least one instruction, and one or more processors, the one or more processors are configured to, by executing the at least one instruction, generate response information based on dialog content received through the communication interface: confirm a current request and a previous request of a user within the dialog content to identify a request of the user; identifying whether the request of the user is a multi-round request or a single-round request based on the received dialog content; if the request of the user is a single-round request, generating a confirmation request for confirming the estimated user request; and if the user's request is a multi-round request, generating a response including information corresponding to the estimated user's request and omitting the generation of the acknowledgement request.

Description

Technical Field

[0001] Devices and methods consistent with the present disclosure relate to an electronic device and a control method thereof, and more particularly, to an electronic device for performing voice recognition using a user's conversation history and a control method thereof. Background Art

[0002] Recently, user requests or commands are confirmed through voice or text input into a messaging application (or instant messaging application, chat application) or a voice recognition application, and services corresponding to the confirmed user requests (e.g., schedule management services, reservation services, shopping services, etc.) are provided.

[0003] To ensure accurate responses to user requests, it is required to accurately understand the user's intentions and commands from the text or voice input by the user. Summary of the invention

[0004] Solution to the problem

[0005] According to one aspect of the present disclosure, an electronic device may include: a communication interface; a memory configured to store at least one instruction; and one or more processors configured to generate response information based on a conversation content received through the communication interface by running the at least one instruction to perform the following operations: identifying a user request by confirming a current request and a previous request of the user in the conversation content; identifying whether the user request is a multi-round request or a single-round request based on the conversation content; generating a confirmation request to request the user to confirm the user request based on the user request being identified as a single-round request; and omitting the generation of the confirmation request and generating a response corresponding to the user request based on the user request being identified as a multi-round request.

[0006] The one or more processors are also configured to execute the at least one instruction to perform the following operations: when the same type of request is identified a predetermined number of times or more within a predetermined time based on the conversation content, identify that the user request is a multi-round request, and generate the response corresponding to the user request based on the multi-round request.

[0007] The same type of requests correspond to requests for information belonging to a common category and are directed to a targeted entity or target.

[0008] When multiple multi-round requests are identified, the one or more processors are also configured to execute the at least one instruction to perform the following operations: use the received conversation order within the conversation content to identify one of the identified multiple multi-round requests, and generate the response corresponding to the user request based on the identified one multi-round request.

[0009] When multiple user requests are confirmed, the one or more processors are further configured to execute the at least one instruction to perform the following operations: estimating a user request among the multiple user requests based on a reliability value corresponding to each of the multiple user requests. When the reliability value of the user request is greater than or equal to a threshold, generating the response based on the user request.

[0010] The one or more processors are further configured to execute the at least one instruction to generate the confirmation request or the response based on at least one of a user preference for the confirmation request and a user preference for the response.

[0011] The one or more processors are also configured to execute the at least one instruction to perform the following operations: obtaining a user preference value for the confirmation request based on conversation history information including user feedback information, and when the obtained preference value is identified as being less than a threshold, generating the response based on the identified request, and the user feedback information includes at least one of feedback information about the response and feedback information about the confirmation request.

[0012] The memory also includes user preference information corresponding to different types of requests, respectively. The one or more processors are further configured to execute the at least one instruction to perform the following operations: identifying a user preference value for a response corresponding to the user request based on the user preference information stored in the memory, and generating the response based on the user request when the user preference value is identified as being greater than or equal to a threshold.

[0013] The memory stores a trained first neural network model, which is trained to output the user request when conversation content, response information and feedback information for the response are input as training data, and the one or more processors are also configured to execute the at least one instruction to perform the following operations: input the conversation content received through the communication interface to the trained first neural network model to estimate the user request.

[0014] The user requests include a first user request and a second user request, and the one or more processors are also configured to execute the at least one instruction to perform the following operations: estimating the first user request based on the current request of the user within the received conversation content, estimating the second user request based on the current request and a previous request of the user, and when the first user request and the second user request are identified as different types, generating the confirmation request for identifying the first user request or the second user request.

[0015] According to another aspect of the present disclosure, a method for controlling an electronic device may include: obtaining conversation content from a user; identifying a user request by confirming a current request and a previous request of the user in the conversation content; identifying whether the user request is a multi-round request or a single-round request based on the conversation content; based on the user request being identified as a single-round request, generating a confirmation request to request the user to confirm the user request; and based on the user request being identified as a multi-round request, omitting the generation of the confirmation request and generating a response corresponding to the user request.

[0016] The method may further include: identifying that the user request is a multi-round request when the same type of request is identified a predetermined number of times or more within a predetermined time based on the conversation content; and generating a response corresponding to the user request based on the multi-round request.

[0017] The same type of requests may correspond to requests for information belonging to a common category and directed to different entities or targets.

[0018] The method may further include: when multiple multi-round requests are identified, using the conversation order within the conversation content to identify one of the identified multiple multi-round requests, and the step of generating the response may include: generating a response based on the one multi-round request.

[0019] The estimating step may include: when multiple user requests are confirmed, estimating one user request among the multiple user requests based on a reliability value corresponding to each of the multiple user requests, and the generating the response step may include: when the reliability value of the user request is greater than or equal to a threshold, generating the response based on the user request. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a diagram illustrating a dialog system according to one or more embodiments of the present disclosure;

[0021] Figure 2 is a sequence diagram illustrating the operation of a dialog system according to an embodiment of the present disclosure;

[0022] Figure 3 is a block diagram showing a configuration of an electronic device according to one or more embodiments;

[0023] Figure 4 is a flowchart for describing a method of controlling an electronic device according to one or more embodiments;

[0024] Figure 5a is a diagram for describing a multi-round request according to one or more embodiments;

[0025] Figure 5b is a diagram for describing a single round of requests according to one or more embodiments;

[0026] Figure 6 is a diagram for describing a method of estimating a user request according to one or more embodiments;

[0027] Figure 7 is a diagram for describing a control method when a multi-round request is recognized according to one or more embodiments;

[0028] Figure 8 is a diagram for describing a control method when a multi-round request is recognized according to one or more embodiments;

[0029] Fig. 9 is a diagram for describing a method of generating response information in consideration of user preferences according to one or more embodiments;

[0030] Fig.10 is a diagram for describing a method of generating response information in consideration of user preferences according to one or more embodiments;

[0031] Fig.11 is a diagram for describing a method of estimating a user request according to one or more embodiments;

[0032] Fig.12 is a diagram for describing different types of user requests according to one or more embodiments; and

[0033] Fig.13 is a block diagram illustrating a detailed configuration of an electronic device according to one or more embodiments. DETAILED DESCRIPTION

[0034] Hereinafter, various exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that the technology mentioned in the present disclosure is not limited to a specific exemplary embodiment, but includes all modifications, equivalents and replacements according to the exemplary embodiments of the present disclosure. Throughout the accompanying drawings, similar components will be represented by similar reference numerals.

[0035] In the present disclosure, the expressions “having”, “may have”, “including”, “may include”, etc. indicate the existence of corresponding features (e.g., values, functions, operations, components (such as parts), etc.), and do not exclude the existence of additional features.

[0036] In the present disclosure, the expressions "A or B", "at least one of A and / or B", "one or more of A and / or B", etc. may include all possible combinations of the items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may indicate all of the following: 1) the case of including at least one A, 2) the case of including at least one B, or 3) the case of including both at least one A and at least one B.

[0037] The expressions "first", "second", "1st" or "2nd" etc. used in the present disclosure may indicate various components regardless of the order and / or importance of the components, and will only be used to distinguish one component from other components without limiting the corresponding components.

[0038] When it is mentioned that any component (e.g., a first component) is (operably or communicatively) coupled to / coupled to another component (e.g., a second component) or connected to another component (e.g., a second component), it should be understood that any component is directly coupled to another component or can be coupled to another component through other components (e.g., a third component). On the other hand, when it is mentioned that any component (e.g., a first component) is "directly coupled" or "directly connected" to another component (e.g., a second component), it should be understood that there is no other component (e.g., a third component) between any component and the other component.

[0039] The expression "configured (or set) to" used in the present disclosure may be replaced by the expression "suitable for", "having the ability to", "designed to", "adapted to", "manufactured to", or "capable of", as appropriate. The term "configured (or set) to" may not necessarily mean "specially designed to" in hardware. On the contrary, the expression "a device configured to" may mean that the device is "capable of" together with other devices or components. For example, "a sub-processor configured (or set) to perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) for performing the corresponding operations, or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can perform the corresponding operations by running one or more software programs stored in a memory device.

[0040] The term "turn" in the context of a voice assistant may refer to an exchange or interaction between a user and a voice assistant (e.g., an artificial intelligence (AI) system employed in the voice assistant). In a conversation setting between two participants, a turn may represent a particular unit of communication in which one participant speaks or interacts and then the other participant responds. When a user asks a query or provides a command, it constitutes the user's turn. The voice assistant may process the user's turn and may generate a response, which constitutes the voice assistant's turn.

[0041] In addition, the dialogue history information disclosed herein refers to text information composed of the dialogue window and the user's response in the electronic device. Such dialogue history information can be a dialogue history between a user and a response robot (response bot), or a dialogue history between multiple users and a response robot. Here, the response robot is an automated machine program that provides specific services based on user requests. In addition, such dialogue history information may include not only text information directly received from the user, but also voice signals or text, video, emoticons, etc. that are speech recognition results for voice signals.

[0042] In the present disclosure, text information is information having at least one user request and / or slot information corresponding to the user request. Such text information may be referred to as sentence information, command information, request information, query, etc. The text information may consist of one sentence, or may consist of multiple sentences. In the context of a voice assistant, the term "slot information" may refer to a specific data segment or parameter that the voice assistant system expects to receive from a user during a conversation or interaction. Slots represent variables or placeholders for different types of information, such as names, dates, locations, numbers, or any other relevant data required to satisfy a user's request.

[0043] A user request is a command requesting execution of a specific service, and slot information may be an object or additional explanation information for specifying the corresponding request. For example, in the sentence "TV is turned on", "turn on" may be a user request, and TV may be slot information. This division is an example and may be implemented in various ways according to the system implementation method. That is, in the case of classifying user requests in more detail, the above-mentioned "TV is turned on" may be a user request.

[0044] In the present disclosure, a service is not only a function that can be executed by driving an application installed in a corresponding device, but also a function that can be executed by using or controlling another device, a function that can be executed by using or controlling a plurality of devices, and the like.

[0045] In the present disclosure, character data means data that does not include numerical data (such as real numbers, imaginary numbers), and includes not only data composed of text, but also all types of data that can convert specific information into numerical values ​​(for example, image data, voice data, etc.).

[0046] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0047] Figure 1 is a diagram illustrating a dialog system according to one or more embodiments of the present disclosure.

[0048] Reference Figure 1, the dialogue system 1000 may include a user terminal device 100 and a server 200 .

[0049] The user terminal device 100 may receive the conversation content from the user 10 and transmit the received conversation content to the server 200. Here, the conversation content may be text directly input by the user 10 or a speech recognition result obtained by converting a speech uttered by the user into text via speech recognition. However, the present disclosure is not limited thereto, and according to an example, the conversation content may include text (specifically, a sentence), voice, video, emoticons, etc.

[0050] The server 200 may receive the conversation content from the user terminal device 100 , generate response information corresponding to the received conversation content, and transmit the generated response information to the user terminal device 100 .

[0051] In this case, the server 200 may generate response information using only the current conversation content, using only the text included in the conversation history information of the current user 10, or using text included in the conversation history information of other users and text included in the conversation history information of the current user.

[0052] For example, when the received conversation content includes the text "Tell me the population of Greece", the server 200 can determine that the user's current request (intention) is a search command and confirm "the population of Greece" (slot), and therefore, can generate response information that it is necessary to search for the population of Greece via the search application.

[0053] In addition, the current conversation content refers to the conversation content corresponding to the continuous conversation received from the time when the voice wake-up operation is performed to the time when the conversation ends. Therefore, at least one user request (turn) may be included in the current conversation content. Figure 4 , Figure 5a , Figure 5b This is described in detail.

[0054] In addition, when executing a service corresponding to a user request, it may be difficult to execute the specific service when it is difficult to execute the specific service using only a text corresponding to the user's current request or when there are many ways to execute the corresponding request.

[0055] For example, when the current text included in the received conversation content includes the text "How about Morocco?", it is difficult to understand the user request, and thus, it may be difficult to perform a specific service using only the above content.

[0056] In this case, the user request may be determined based on the current conversation content including the user's previous request.

[0057] For example, when a past request with the text "Tell me the population of Greece" is recognized before the conversation "How about Morocco?" is received, that is, when the user has previously requested a separate search, the server 200 can generate response information indicating that it is necessary to search for the population of Morocco via the search application by considering not only the currently input conversation content but also the previously input conversation content to confirm that the user request (intention) is a search command and confirming "Population of Morocco" (slot).

[0058] In this manner, when information for specifying a user request (ie, slot information) is included in the conversation content of the user, the server 200 can generate response information by using more accurate response information by using the corresponding information.

[0059] In addition, as described above, when the user request is understood using the conversation content including the user's past requests and the text corresponding to the user's current request, the server 200 may generate response information for requesting user confirmation so as to ensure the stability and reliability of the user. For example, in the case of the above example, before generating the response information requiring the search for the population of Morocco, a response information requesting user confirmation such as "Can I tell you the population of Morocco?" may be generated.

[0060] In this way, when it is difficult to perform a specific service using only the text corresponding to the user's current request, or when there are many ways to perform the request, the server 200 can generate response information for requesting user confirmation. However, when such user confirmation is repeated, the user's fatigue may increase and the user satisfaction may decrease.

[0061] Therefore, various embodiments will be described below in which response information for requesting user confirmation is generated only under specific circumstances by using conversation history information including the user's current conversation content, thereby being able to reduce reconfirmation request responses.

[0062] Figure 2 is a sequence diagram illustrating the operation of a dialog system according to an embodiment of the present disclosure. Specifically, Figure 2 is a diagram illustrating an example of a dialogue system based on user utterances.

[0063] Reference Figure 2 , when receiving a user utterance from a user ( 201 ), the terminal device 100 may perform voice recognition on the received utterance and generate dialogue information (ie, text) corresponding to the user utterance ( 203 ).

[0064] When the utterance for the corresponding service is continuously performed, the terminal device 100 may generate a conversation content including the corresponding text and a response from the previous server 200. In addition, such a conversation content may be generated on the server 200 side instead of the terminal device 100 side.

[0065] When the text is generated, the terminal device 100 may transmit the conversation content to the server 200 ( 205 ).

[0066] When receiving the corresponding dialogue content, the server 200 generates response content corresponding to the corresponding content (207). Specifically, the server 200 can apply the corresponding dialogue content to the natural language understanding algorithm to confirm the user's intention and slot, and generate response content corresponding to the confirmed intention and slot.

[0067] The server 200 may transmit the generated response content to the terminal device 100 (209). Such response content may consist of text only, or may include text or content information corresponding to the user request.

[0068] In addition, the terminal device 100 that has received the response content may execute the corresponding response content.

[0069] For example, when the user says "Search for contact A", the terminal device 100 may generate text corresponding to the input utterance and transmit the text to the server 200. Accordingly, the server 200 may generate response content for searching for contact "A" in the contact application of the terminal device 100 and provide the generated response content to the terminal device 100.

[0070] The terminal device 100 receiving such response content may drive a contact application and provide the contact A to the user by using the corresponding contact application.

[0071] In addition, Figure 2 In the present invention, an operation example of a dialogue system based on speech has been shown and described, but in practice, the same operation can be performed even when receiving text directly from a user. That is, the present disclosure can be applied to a dialogue system based on a dialogue window (or chat window).

[0072] In addition, Figure 1 and Figure 2 In the illustration and description of the present invention, in order to perform the conversation service, the user terminal device 100 and the server 200 are shown and described as operating in coordination, but the above functions may be performed in a single device.

[0073] Figure 3 is a block diagram illustrating a configuration of an electronic device according to one or more embodiments.

[0074] Specifically, in Figure 1In the system, the device that directly generates response information (such as a server), the device that collects conversation messages or voices and generates necessary conversation information (such as a user terminal device), etc. can be called an electronic device. That is, Figure 3 Electronic devices can be used as Figure 1 A user terminal device or Figure 1 In addition, it is also possible to execute Figure 1 The functions of the user terminal device and the server.

[0075] Reference Figure 3 According to one or more embodiments, the electronic device 300 may be various devices such as a personal computer (PC), a laptop computer, a smart phone, a tablet computer, and a server. Figure 3 , the electronic device 300 may include a communication interface 310 , a memory 320 , and a processor 330 .

[0076] The communication interface 310 is formed to connect the electronic device 300 to an external device (not shown), and can be connected to the external device through a local area network (LAN) and an Internet network, or connected to a terminal device through a USB port or a wireless communication (e.g., wireless fidelity (WiFi), 802.11a / b / g / n, near field communication (NFC) or Bluetooth) port. Such a communication interface 310 may also be referred to as a transceiver.

[0077] The communication interface 310 can receive the content of the conversation (or conversation information) from another device or send the content of the conversation to another device. Here, the content of the conversation may include a user message or a message input through several turns. Here, the user message may be a text directly typed by the user, a message obtained by converting the voice uttered by the user into text through voice recognition, or voice data corresponding to the voice uttered by the user.

[0078] The communication interface 310 may send response content (or response information) corresponding to the conversation content to another device that sent the conversation content, or may receive response content corresponding to the conversation content. Such response content may be information including command information for executing a specific service, content corresponding to information requested by a user, guidance information on the execution result of the service requested by the user, etc.

[0079] For example, when the dialog content includes a message that the user requests to execute a specific application, a response content including command information for executing the specific application may be received. However, the above example of the response content is not an example, and various user requests may be generated in the dialog system and various responses may be made accordingly.

[0080] The memory 320 is a component for storing an O / S, various software, data, etc. for driving the electronic device 300. The memory 320 may be implemented in various forms such as RAM, ROM, flash memory, HDD, external memory, and memory card, but is not limited to any one.

[0081] The memory 320 may store at least one instruction. These instructions may include instructions for running an application necessary for executing the above-mentioned dialogue system, an application for voice recognition, and various applications for running services corresponding to response information.

[0082] The conversation history information may be stored in the memory 320. Here, the conversation history information may include not only the current conversation history of the user, but also the conversation history of other users. In addition, the conversation history information may include the current conversation content of the user and the past conversation content of the user.

[0083] One or more processors 330 control each component in the electronic device 300. The processor 330 may be composed of a single device such as a central processing unit (CPU) and an application specific integrated circuit (ASIC), or may be composed of a plurality of devices such as a CPU and a graphic processing unit (GPU).

[0084] According to one or more embodiments, when one or more processors 330 receive text input from a user or a voice uttered by a user, one or more processors 330 may control the communication interface 310 to generate conversation content using the input text, and control the generated conversation content to be sent to an external device.

[0085] According to one or more embodiments, one or more processors 330 (hereinafter referred to as processor) may generate response information using the conversation content received through the communication interface 310 by executing at least one instruction.

[0086] According to an example, when receiving the conversation content, the processor 330 may use the received conversation content to identify a user request (intent), slot information, etc. For example, the processor 330 may identify a user request and / or slot information, etc. by analyzing the text included in the conversation content using a natural language recognition algorithm.

[0087] In this case, according to an example, the processor 330 may identify additional information from the received conversation content. For example, the additional information may be time information, conversation frequency, and user emotion information. In addition, the time information may be time information of the conversation content, the conversation frequency may be the number of conversation turns in which a response is performed according to a user request, etc., and the user emotion information may be information inferred through various information such as the voice tone or emoticons of the user included in the conversation content, and the relationship with the subject included in the conversation content (boss, husband, and wife). In the above, the above information is referred to as additional information, but such additional information may be used as the above slot information.

[0088] According to one or more embodiments, the processor 330 may estimate the user request by confirming the current request and the previous request of the user in the received conversation content.

[0089] According to the example, when it is necessary to confirm a user request corresponding to a current text, such as, when the reliability of a current request corresponding to a current text included in a conversation content is less than a predetermined value, or when multiple different types of user requests are confirmed based on the current text, etc., the processor 330 may estimate the user request by confirming the current request and previous requests of the user in the received conversation content.

[0090] In addition, according to an example, the processor 330 may identify a reliability value corresponding to the confirmed user request based on the conversation history information, wherein the conversation history information includes not only the conversation content of the current user but also the conversation content of other users, which will be referred to as Figure 6 Detailed description.

[0091] According to an example, the processor 330 may estimate the user request by confirming the user's previous request (or past request). When executing a service corresponding to the user request, there may be a case where it is difficult to execute a specific service using only a text corresponding to the user's current request.

[0092] For example, when it is difficult to provide a specific service based on the current request, such as when the current text included in the received conversation content includes the text "How about Morocco?" (or when the reliability value of the current request corresponding to the current text is less than a threshold value), the processor 330 can confirm the previous request of the user included in the current conversation content. When the previous text "Tell me the population of Greece" is included in the current conversation content, the processor 330 will identify the search command of the search application as a user request included in the previous text, and the user request corresponding to the current text can be estimated as a search command based on the recognized command. However, the previous request is confirmed only when it is difficult to perform a specific service based on the current request, and the operation of confirming the current request and the operation of confirming the previous request can be performed independently.

[0093] In addition, it may be difficult to execute a specific service not only when it is difficult to execute a specific service using only a text corresponding to a user's current request but also when there are many ways to execute the corresponding request.

[0094] According to an example, the processor 330 may estimate the user request by respectively confirming a current request and a previous request of the user.

[0095] For example, when the current text included in the received conversation content includes the text "Search A", the processor 330 may confirm the search command via the search application as the current request. In addition, when the previous text "Search for contact B" is included in the current conversation content, the processor 330 may confirm the search command via the contact application as the previous request based on the search command included in the previous text. The processor 330 may estimate the current request or the previous request as the user request.

[0096] Optionally, according to an example, when a plurality of current requests are confirmed, the processor 330 may estimate one of the plurality of current requests as a user request.

[0097] For example, when the current text included in the received conversation content includes the text "Search for A", the processor 330 may confirm the search command via the search application as the current request, or the processor 330 may confirm the search command via the contact application as the current request. That is, the user request may be confirmed by A's Internet search and A's contact search. In this case, the user request may be estimated based on the preference of the confirmed request. Fig. 9 This is described in detail.

[0098] In this way, when a specific service may not be provided by the current request confirmed based on the current text, or when multiple user requests are identified, according to an example, the processor 330 may identify the current request or the previous request and estimate the identified one request as the user request. Figure 6 This is described in detail.

[0099] In addition, according to one or more embodiments, the processor 330 may use the received conversation content to identify whether the estimated user request is a multi-round request or a single-round request. The terms "multi-round request" and "single-round request" are also referred to as multi-round commands and single-round commands, respectively. In the context of a voice assistant, single-round requests and multi-round requests refer to different types of interactions between a user and an assistant.

[0100] A multi-turn request refers to a type of user request that is identified and estimated through the current conversation content and (one or more) additional subsequent conversations. A multi-turn request involves a series of interactions between the user and the voice assistant to complete a specific task or collect more information, and a conversation with multiple back-and-forth exchanges may be requested. The assistant may need to refer to previous questions, or may need to ask follow-up questions or seek clarification to meet the user's request. According to an example, when the same type of user request is estimated a predetermined number of times or more within the current conversation content, the processor 330 may identify the estimated user request as a multi-turn request. In this case, since the estimated user request is provided to the user through a confirmation request response, the multi-turn request may be a type of user request that provides one or more confirmation request responses within the current conversation content.

[0101] On the other hand, a single-turn request refers to a type of user request that is estimated or identified within the current conversation content without reference to previous conversations and without the need for subsequent conversations. A single-turn request may be an independent, self-contained request or question issued by a user in a single utterance or turn, and may involve a straightforward request or query that can be answered or executed in a single interaction or in a single turn without the need for any additional back-and-forth conversations.

[0102] Here, a type of user request means excluding a type of control command for specifying slot information of the user request, and the type of control command includes a type of application driven to perform an operation corresponding to the corresponding command.

[0103] For example, "a search command via a search application" and "a search command via a contact application" may be different types of user requests. That is, "a search command via a search application" and "a search command via a contact application" are different types of applications (a search application and a contact application) driven to perform operations corresponding to the corresponding commands, and therefore, may be different types of user requests.

[0104] Optionally, for example, "search command for A via search application" and "command search for B via search application" each include the same type of control command except for the fact that the slot information (A and B) (i.e., only the search target) used to specify the user request is different, and thus "search command for A via search application" and "command search for B via search application" may be the same type of user requests.

[0105] In addition, we will refer to Figure 5a and 5b Describes multi-round requests and single-round requests.

[0106] According to one or more embodiments, when the estimated user request is a single-round request, the processor 330 may generate confirmation request response information to confirm the estimated user request, and when the estimated user request is a multi-round request, the processor 330 may generate response information including information corresponding to the estimated user request.

[0107] For example, assume that the user request is estimated to be a search command for contact A via a contact application. When the estimated user request is identified as a single-round request based on the received current conversation content, the processor 330 generates confirmation request response information for confirming the estimated user request, such as "Do you want to search for contact A?". Optionally, according to the example, when the estimated user request is identified as a multi-round request, the processor 330 generates response information including a search command for contact A via the contact application without generating separate confirmation request response information. When the user request is identified as a multi-round request, the processor 330 may choose to bypass the confirmation step, wherein the confirmation step is used to ask the user to confirm the accuracy of the estimate of the user request.

[0108] Figure 4 is a flowchart for describing a method of controlling an electronic device according to one or more embodiments.

[0109] Reference Figure 4 According to one or more embodiments, the control method may include estimating or identifying a user request by confirming a current request and a previous request of the user in the received conversation content (S410).

[0110] For example, the processor 330 assumes that "How is Company C?" is received as the current text. When the text "Tell me the salary of Company A" has been previously received based on the current conversation content, the processor can recognize the slot information corresponding to "the annual salary of Company A" and the user's previous request corresponding to the search command via the search application based on the received text, and can estimate the current request corresponding to the current text as a search command for the annual salary of Company C via the search application based on the recognition.

[0111] Subsequently, according to one or more embodiments, the control method may include identifying whether the estimated user request is a multi-round request or a single-round request using the received conversation content ( S420 ).

[0112] For example, when the processor 330 receives "How about Company B?" as previous text based on the conversation history information including the received current conversation content, and thus recognizes that the search command for the annual salary of Company B via the search application is estimated to be a previous user request, the processor may identify the estimated "search command for the annual salary of Company C via the search application" as a multi-round request.

[0113] For example, whether the estimated user request is a multi-round request may be identified based on a previous user request, wherein the user request may be a user request excluding slot information. That is, in the above example, in order to determine whether the estimated user request is a multi-round request, the processor 330 may only compare the “search command via the search application” excluding slot information among the previously estimated user request with the current estimated user request, and determine whether the current estimated user request is a multi-round request.

[0114] However, it is not limited thereto, and according to an example, the processor 330 may determine whether the currently estimated user request is a multi-round request based on the user request including the slot information.

[0115] Subsequently, according to one or more embodiments, when the estimated user request is identified as a single-round request (S430), the control method may generate confirmation request response information for confirming the estimated user request (S450). This will refer to Figure 5b Detailed description.

[0116] In addition, according to one or more embodiments, when the estimated user request is identified as a multi-round request ( S440 ), the control method may generate response information including information corresponding to the estimated user request ( S460 ).

[0117] For example, when the estimated “search command for annual salary of company C via search application” is identified as a multi-round request, the processor 330 may generate response information including information corresponding to the search command for annual salary of company C via search application.

[0118] According to the above example, when the estimated user request is a previously repeated request, a response corresponding to the user request can be provided immediately without going through a separate user reconfirmation process. As a result, fatigue caused by user reconfirmation can be prevented and user satisfaction can be improved.

[0119] Figure 5a is a diagram for describing a multi-round request according to one or more embodiments.

[0120] Reference Figure 5a According to one or more embodiments, the processor 330 of the electronic device 500 may identify whether the user request is a multi-round request or a single-round request, and generate response information based on the identification. Here, for ease of description, the electronic device 500 is shown as a user terminal device, but is not limited thereto, and according to an example, the electronic device 500 may be implemented as a server.

[0121] For example, the processor 330 may first estimate the user request using the received current conversation contents 510 to 516 .

[0122] For example, when the current request corresponding to the received current text 516 "How about Mexico?" is not confirmed, the processor 330 may confirm the search command via the search application as a previous request based on the previous text 510 "Tell me the GDP of Greece", and estimate the user request corresponding to the current text 516 as "a search command for the GDP of Mexico via the search application".

[0123] Subsequently, when the same type of user request (e.g., a request for GDP information) is estimated a predetermined number of times or more within the current conversation content, or when the current request (e.g., "How about Morocco") is identified by reference to another conversation (e.g., a previous conversation (such as "Tell me the GDP of Greece") or a subsequent conversation), the processor 330 may identify the estimated user request as a multi-round request. For example, the predetermined number of times may be one, but is not limited thereto, and may be other numbers (e.g., two or more times). The same type of request may refer to a request for information belonging to a common category (e.g., a request for GDP information), but directed to different entities or targets (e.g., Greece and Mexico).

[0124] For example, when a "search command for GDP of Morocco via search application" is identified as a user request of the same type as the user request currently estimated through some conversation contents 512 to 514 in the current conversation content that was previously estimated, the processor may identify the currently estimated user request as a multi-round request.

[0125] Subsequently, according to one or more embodiments, when the estimated user request is a multi-round request, the processor 330 may generate response information including information corresponding to the estimated user request.

[0126] For example, when a "search command for Mexico's GDP via a search application" as an estimated user request is identified as a multi-round request, the processor 330 may generate response information corresponding to the estimated user request and provide a response 517 to the user based on the generated response information.

[0127] Figure 5b is a diagram for describing a single round of requests according to one or more embodiments.

[0128] Reference Figure 5b According to one or more embodiments, the processor 330 of the electronic device 500 may identify whether the user request estimated using the received conversation content is a multi-round request or a single-round request, and generate confirmation request response information based on the identification.

[0129] For example, the processor 330 may first estimate the user request using the received current conversation contents 520 to 522 .

[0130] For example, a current request corresponding to “search for B” as the received current text 522 may be confirmed as “a search command for B via a search application” or “a search command for B via a contact application”, and a previous request may be confirmed as “a search command for A via a contact application” based on “search for A in contacts” as the previous text 520. In this case, the processor 330 may estimate the current user request as “a search command for B via a contact application” which is a request of the same type as the previous request among the multiple current requests.

[0131] Then, when a user request of the same type as the estimated user request is estimated or identified within the current conversation content without referring to the previous conversation and without requiring a subsequent conversation, the processor 330 may identify the estimated user request as a single-turn request.

[0132] For example, when identifying that the same type of user request as the user request currently estimated through the current conversation contents 520 to 522 has not been estimated, the processor 330 may identify the estimated “search command for B via the contact application” as a single-round request.

[0133] Subsequently, according to one or more embodiments, when the estimated user request is a single-round request, the processor 330 may generate confirmation request response information for confirming the estimated user request.

[0134] For example, when the "search command for B via the contacts application" as an estimated user request is identified as a single-round request, the processor 330 may generate a confirmation request response message for confirming the estimated user request, and may provide "Do you want to search for B in the contacts?" as a response 517 to the user based on the generated response information.

[0135] Figure 6 is a diagram for describing a method of estimating a user request according to one or more embodiments.

[0136] According to one or more embodiments, when multiple user requests are confirmed, the processor 330 may estimate the user request based on the reliability value corresponding to the user request. That is, when it is necessary to confirm the user request, such as when multiple user requests are confirmed for the current text, the processor 330 may estimate any one user request based on the reliability value corresponding to each of the multiple user requests.

[0137] Reference Figure 6 According to one or more embodiments, the control method may first determine whether a plurality of user requests are confirmed (S610).

[0138] According to one example, when the current text included in the received conversation content includes the text "Search A", the processor 330 may confirm the "search command via the search application" as the first current request, or the processor 330 may confirm the "search command via the contacts application" as the second current request.

[0139] However, it is not limited thereto, and it goes without saying that each of at least one current request and at least one previous request may be confirmed.

[0140] Subsequently, according to one or more embodiments, the control method may estimate one of the plurality of user requests based on the reliability value corresponding to each of the plurality of user requests ( S620 ).

[0141] According to an example, the processor 330 may identify a reliability value corresponding to each of the plurality of confirmed user requests based on conversation history information, wherein the conversation history information includes not only the conversation content of the current user but also the conversation content of other users.

[0142] For example, the processor 330 can identify the reliability of a "search command via a search application" (a first current request) and a "search command via a contact application" (a second current request) based on conversation history information, wherein the conversation history information includes not only the conversation content of the current user but also the conversation content of other users.

[0143] Optionally, the processor 330 may input the conversation history information into a trained neural network model to identify a reliability value corresponding to the user request.

[0144] Here, according to an example, the feedback information of the user may include information on a probability that the user provides a positive response when the first current request is provided to the user, but is not limited thereto.

[0145] According to an example, when the reliability value corresponding to each of the plurality of user requests is identified, the processor 330 may estimate one user request having a maximum reliability value among the confirmed user requests as the user request.

[0146] For example, when the reliabilities corresponding to each of the "search command via search application" (first current request) and the "search command via contacts application" (second current request) are respectively identified as 0.8 and 0.9 based on the conversation history information, the processor can estimate a user request with a relatively large reliability value for providing the user with the "search command via contacts application" (second current request).

[0147] Then, according to one or more embodiments, the control method may identify whether the estimated reliability value of the user request is greater than or equal to a threshold value (S630). Here, the threshold value may be a value stored in the memory 120 at the time of initial setting, but is not limited thereto, and may be changed based on user input.

[0148] Subsequently, according to one or more embodiments, when the estimated reliability value of the user request is greater than or equal to the threshold value (Y), the control method may generate response information based on the estimated user request (S640). According to an example, when the reliability value corresponding to the "search command via the contact application" (the second current request) is identified as being greater than or equal to the threshold value 0.9, the processor 330 may generate response information corresponding to the "search command via the contact application" (the second current request).

[0149] Therefore, even when a plurality of user requests are confirmed, the electronic device 300 can estimate that a user request with high reliability can be provided to the user, thereby improving the user's satisfaction with the provided service.

[0150] Figure 7 is a diagram for describing a control method when a multi-round request is recognized according to one or more embodiments.

[0151] according to Figure 7 According to one or more embodiments, the control method may first identify whether the confirmation request response information is recognized a predetermined number of times or more within a predetermined time (S710). Here, according to an example, the predetermined time may be the time from the time when the current conversation content is provided, that is, the time when the voice wake-up operation is performed to the time when the conversation ends. In addition, the predetermined number of times may be, for example, once, but is not limited thereto.

[0152] According to an example, when the user request is estimated, the processor 330 may identify whether the confirmation request response information is recognized one or more times within the current conversation content.

[0153] For example, when a "search command for A via a search application" is estimated as a user request, the processor 330 may identify whether confirmation request response information has been previously generated within the received current conversation content.

[0154] In this case, according to an example, the processor 330 analyzes the text of the current conversation content to identify text corresponding to the confirmation request response, for example, an inquiry text such as “Can I tell you contact B?”, thereby identifying whether the confirmation request response information is recognized.

[0155] Alternatively, according to an example, the processor 330 may identify the confirmation request response information based on history information about previously generated response information.

[0156] On the other hand, in this case, the user request corresponding to the identified confirmation request response information can be of the same type as the estimated user request, but is not limited to this, and it can also be identified whether the confirmation request response information is identified one or more times within the current conversation content, wherein the confirmation request response information corresponds to a user request having a similarity with the estimated user request greater than or equal to a predetermined value.

[0157] In this case, according to an example, similarity values ​​corresponding to each of multiple user requests may be pre-stored in the memory 320, and the processor 330 may identify confirmation request response information corresponding to a user request having a similarity greater than or equal to a predetermined value with the estimated user request based on the information stored in the memory 320.

[0158] Subsequently, according to one or more embodiments, when the estimated user request is identified as a multi-round request ( S720 ), the control method may generate response information based on the identified multi-round request ( S730 ).

[0159] According to an example, when the confirmation request response is identified as being provided in the current conversation content, the processor 330 may identify, based on the identification, that the "search command via the search application" as the estimated user request is a multi-round request. Subsequently, when the estimated user request is identified as a multi-round request, the processor 330 may generate a "search command for A via the search application" as response information.

[0160] Therefore, when the confirmation request response has been provided to the user within the conversation content, the electronic device 300 can provide the response to the user without going through a separate re-confirmation process, thereby improving user satisfaction.

[0161] Figure 8 is a diagram for describing a control method when a multi-round request is recognized according to one or more embodiments.

[0162] Reference Figure 8 According to one or more embodiments, the control method may first identify whether the confirmation request response information is recognized a predetermined number of times or more within a predetermined time (S810). Here, according to an example, the predetermined time may be the time from the time when the current conversation content is provided, that is, the time when the voice wake-up operation is performed to the time when the conversation ends. In addition, the predetermined number of times may be, for example, once, but is not limited thereto.

[0163] According to an example, when the user request is estimated, the processor 330 may identify whether confirmation request response information corresponding to the same type of user request is estimated one or more times within the current conversation content.

[0164] For example, when a “search command for A via a search application” is estimated as a user request, the processor 330 may identify whether the “search command via a search application” has been previously estimated based on the received current conversation content.

[0165] In this case, according to an example, the processor 330 may identify whether the “search command via the search application” has been previously estimated based on whether the confirmation request response is provided (or whether the confirmation request response information is generated) to provide the “search command via the search application”. For example, when a confirmation request response including the text “Do you want to search for B on the Internet?” is provided within the current conversation content, the processor 330 may identify that the “search command via the search application” has been previously estimated.

[0166] Subsequently, according to one or more embodiments, when the estimated user request is identified as a multi-round request ( S820 ), the control method may generate response information based on the identified multi-round request ( S830 ).

[0167] According to an example, when a confirmation request response corresponding to a user request of the same type as the "search command via the search application" as the estimated user request is identified as being provided in the current conversation content, the processor 330 may identify the "search command via the search application" as the estimated user request as a multi-round request based on the identification. Subsequently, when the estimated user request is identified as a multi-round request, the processor 330 may generate a "search command for A via the search application" as the response information.

[0168] However, not limited thereto, and according to an example, the processor 330 determines that when a confirmation request response corresponding to the same type of user request is recognized two or more times within the current conversation content, the estimated user request may be recognized as a multi-round request.

[0169] Therefore, when a confirmation request response to the same type of user request has been provided to the user within the conversation content, the electronic device 300 can provide the response to the user without going through a separate reconfirmation process, thereby improving user satisfaction.

[0170] Fig. 9 is a diagram for describing a method of generating response information in consideration of user preference according to one or more embodiments.

[0171] According to one or more embodiments, the processor 330 may identify a user preference for a confirmation request response based on the conversation history information including the user feedback information. Here, the user preference for the confirmation request response means the user preference of the current user. According to an example, the conversation history information including the user feedback information may be stored in the memory 320.

[0172] Here, the user feedback information is information about the user response to the response provided by the electronic device 300, and according to an example, the user feedback information may include information about the positive feedback or negative feedback of the user for the response corresponding to the user request. Optionally, according to an example, the user feedback information may include information about the positive feedback or negative feedback of the user for the confirmation request response.

[0173] Reference Fig. 9 According to one or more embodiments, the control method may first acquire a user preference value for a confirmation request response based on dialog history information including user feedback information (S910).

[0174] According to an example, the processor 330 may calculate a ratio of positive feedback among user feedbacks for the confirmation request response based on the information stored in the memory 320, and identify the calculated ratio as a user preference value for the confirmation request response.

[0175] Optionally, according to an example, the processor 330 may identify a user preference value for the confirmation request response through a trained neural network model.

[0176] For example, the trained second neural network model may be stored in the memory 320, and the trained second neural network model may be trained to output user preference information for the confirmation request response when the conversation history information including the user feedback information is input as the training data. The processor 330 may input the conversation history information to the trained second neural network model to identify the user preference value for the confirmation request response.

[0177] Subsequently, according to one or more embodiments, the control method may identify whether the acquired user preference value is less than a threshold value ( S920 ).

[0178] Then, according to one or more embodiments, when the user preference value for the confirmation request response is identified as being less than the threshold value (Y), the control method may generate response information based on the identified request ( S930 ).

[0179] According to the example, it is assumed that the estimated user request is identified as a multi-round request based on the received conversation content. In this case, when the current user preference for confirming the request response is identified as less than a threshold, the processor 330 may generate response information including information corresponding to the estimated user request.

[0180] In addition, according to an example, when the current user preference for the confirmation request response is identified as being greater than or equal to the threshold, the processor 330 may generate confirmation request response information for confirming the estimated user request.

[0181] That is, in the case of multiple rounds of requests, only when the user's preference for the confirmation request response is less than a predetermined value, the electronic device 300 may provide a response including information corresponding to the user request considering the user's preference for the confirmation request response.

[0182] In addition, according to the example, it is assumed that the estimated user request is identified as a multi-round request based on the received conversation content. In this case, when the current user preference for the confirmation request response is identified as less than a threshold, the processor 330 may generate a response message including information corresponding to the user request, rather than the confirmation request response message.

[0183] That is, even in the case of a single-round request, the electronic device 300 may provide a response including information corresponding to the user request in consideration of the user's preference for the confirmation request response.

[0184] According to the above example, a response may be provided to the user in consideration of a user preference corresponding to each user for a confirmation request response, thereby improving the accuracy of the provided response.

[0185] In addition, return Figure 3 According to one or more embodiments, the processor 330 may generate response information based on user preferences corresponding to a specific user request.

[0186] According to one or more embodiments, user preference values ​​corresponding to different types of requests may be stored in the memory 320. Here, the user preference value corresponding to each of the different types of requests is information about preferences of the current user and other users for the user request. According to one example, the user preference value corresponding to "search command via search application" may be stored as 0.9 in the memory 320, but this is merely an example, and values ​​different therefrom may be stored.

[0187] In addition, according to one or more embodiments, the processor 330 may identify a user preference value corresponding to the identified request based on the conversation history information of the current user and other users. In this case, according to an example, the conversation history information may include feedback information about the identified request of the current user and other users.

[0188] According to an example, it is assumed that "A" is identified as a user request. According to an example, the processor 330 may calculate the ratio of positive feedback among the user feedback for the identified "A" based on the information stored in the memory 320, and identify the calculated ratio as the user preference value for the identified user request "A".

[0189] Optionally, according to an example, the processor 330 may identify a user preference value for the identified user request through a trained neural network model.

[0190] For example, the trained third neural network model may be stored in the memory 320, and the trained third neural network model may be trained to output user preference information for the identified user request when the conversation history information of the current user and the conversation history information of other users including the user feedback information are input as training data. The processor 330 may input the conversation history information of the current user and the conversation history information of other users into the trained third neural network model to identify the user preference value for the identified user request.

[0191] Thereafter, according to one or more embodiments, when the identified user preference value is identified as being greater than or equal to the threshold, the processor 330 may generate response information based on the identified request. Fig.10 This is described in detail.

[0192] Fig.10 is a diagram for describing a method of generating response information in consideration of user preference according to one or more embodiments.

[0193] Reference Fig.10 According to one or more embodiments, the control method may first identify a user preference value for a response corresponding to an identified request based on user preference information corresponding to each of different types of requests (S1010).

[0194] According to an example, it is assumed that the identified request is request “A.” The processor 330 may identify a user preference value for a response corresponding to request “A” (or a user preference value for request “A”) based on information stored in the memory 320 .

[0195] Optionally, according to an example, the processor 330 may identify a user preference value for a response corresponding to request "A" based on conversation history information, or the processor 330 may input conversation history information of the current user and other users into a trained third neural network model to identify a user preference value for a response corresponding to request "A".

[0196] Subsequently, according to one or more embodiments, the control method may identify whether the identified user preference value is greater than or equal to a threshold value ( S1020 ).

[0197] Subsequently, when the preference value for the identified user is greater than or equal to the threshold value (Y), the control method according to one or more embodiments may generate response information based on the identified user request ( S1030 ).

[0198] According to one example, when the user preference value for the identified user request “A” is greater than or equal to a threshold value (eg, 0.9), the processor 330 may generate response information including information corresponding to the identified request “A”.

[0199] In addition, in this case, according to an example, the identified user request may be a multi-round request. For example, when the identified user request is a multi-round request, the processor 330 may identify a user preference value for the identified multi-round request, and may generate response information including information corresponding to the estimated user request only when the identified user preference value is greater than or equal to a threshold.

[0200] On the other hand, reliability is a value for determining whether confirmation of a user request is necessary, and a user preference (a user preference for a response corresponding to an identified request or a user preference for a confirmation request response) is a value for identifying the type of response information. For example, the user preference is a value used by the processor 330 to identify confirmation request response information or response information.

[0201] According to the above example, a response may be provided to the user taking into account the user's preference for the type of identified request, thereby ensuring user reliability.

[0202] Fig.11 is a diagram for describing a method of estimating a user request according to one or more embodiments.

[0203] Reference Fig.11 According to one or more embodiments, the processor 330 may estimate the user request 1120 using the received conversation content 1110. According to an example, the processor 330 may estimate the user request 1120 using the trained first neural network model 1100.

[0204] According to an example, the trained first neural network model may be stored in the memory 320. The trained first neural network model 1100 may be trained to output a user request when a conversation content, response information, and feedback information for the response are input as training data.

[0205] Here, according to an example, the conversation content may be the current conversation content. According to an example, the response information may include response information and confirmation request response information generated by the processor 330. That is, the response information input to the trained first neural network model as training data refers to information about different types of responses output by the electronic device 300 and provided to the user.

[0206] On the other hand, the feedback information about the response is information about user feedback corresponding to the response provided to the user, and according to an example, the feedback information about the response may include information about positive feedback or negative feedback for the provided response.

[0207] According to an example, the processor 330 may input the conversation content 1110 received through the communication interface 310 into the trained first neural network model 1100 to estimate the user request 1120.

[0208] For example, processor 330 may Figure 5a The conversation contents 510 to 570 shown in FIG. 1 are input to the trained first neural network model 1100 to estimate the user request corresponding to the “search command for Mexico's GDP via the search application”.

[0209] Fig.12 is a diagram for describing different types of user requests according to one or more embodiments.

[0210] Here, for convenience of description, the electronic device 1200 is illustrated as a user terminal device, but is not limited thereto, and according to an example, the electronic device 1200 may be implemented as a server.

[0211] Reference Fig.12 According to one or more embodiments, the processor 330 may first estimate a first user request based on a current request 1230 of a user in the received conversation contents 1210 to 1230, and estimate a second user request based on the current request 1230 and the previous request 1210 of the user. Here, the user request includes the first user request and the second user request.

[0212] According to an example, the processor 330 may estimate “a search command for B via a search application” as a first user request based on “search for B” as text corresponding to the user's current request. The “a search command for B via a contact application” may be estimated as a second user request based on the text “search for B” corresponding to the user's current request 1230 and the text “search for A in contacts” as text corresponding to the previous request 1310.

[0213] Subsequently, according to an embodiment, when the first user request and the second user request are identified as different types, the processor 330 may generate confirmation request response information for confirming the first user request or the second user request.

[0214] According to an example, when the "search command for B via the search application" estimated as the first user request and the "search command for B via the contact application" estimated as the second user request are identified as different types of user requests, the processor 330 may generate confirmation request response information for confirming the first user request or the second user request. For example, the processor 330 may generate confirmation request information corresponding to "Do you want to search for B in the contacts?", which is text for confirming the second user request.

[0215] In addition, return Figure 3 According to an embodiment, when a plurality of current requests are identified, the processor 330 may estimate one of the plurality of current requests based on the type of the previous request.

[0216] According to an example, in the case where "search for B" is the received current text, the corresponding current request may be confirmed as "a search command for B via a search application" or "a search command for B via a contact application", and the previous request may be confirmed as "a search command for A via a contact application" based on "search for A in contacts". In this case, the processor 330 may estimate the current user request as "a search command for B via a contact application" as a request of the same type as the previous request among the multiple current requests. The same type of request may refer to a request for information belonging to a common category (e.g., "a request to search a contact application"), but to different entities or targets (e.g., "A" and "B").

[0217] In this way, when multiple requests for a user utterance are confirmed, the order of the candidate groups may be distinguished taking into account the previous requests.

[0218] In addition, according to one or more embodiments, when multiple multi-round requests are identified, the processor 330 may use the conversation order within the received conversation content to identify one of the multiple identified multi-round requests, and generate response information based on the identified multi-round request. Here, the conversation order means the order in which the text included in the conversation content is generated over time. For example, in Figure 5a In the case of "tell me the GDP of Greece", "the GDP of Greece is xx", "how about Morocco?" ... "how about Mexico?" The order of dialogue is generated.

[0219] According to the example, when there are multiple types of user requests estimated once or more times in the current conversation content, multiple multi-round requests can be identified. In this case, the processor 330 can identify the multi-round requests corresponding to the latest user request in the generation order by using the conversation sequence in the conversation content received.

[0220] For example, when the user request "A" and the user request "B" are each evaluated once within the current conversation content, that is, when the confirmation request response corresponding to the user request "A" and the confirmation request response corresponding to the user request "B" are each provided once, a plurality of multi-round requests "A" and "B" may be identified. In this case, the processor 330 may identify the multi-round request corresponding to the user request "B" which is relatively late in the generation order by using the conversation order.

[0221] Then, according to an example, the processor 330 may generate response information based on the identified one of the plurality of multi-round requests.

[0222] Therefore, even when a plurality of multi-round requests are identified in consideration of the conversation context or the conversation order, the electronic device 300 can provide a response that satisfies the user's intention, thereby improving user satisfaction.

[0223] In addition, when a plurality of user requests are recognized according to an embodiment, the processor 330 may generate each response information including information corresponding to the plurality of user requests.

[0224] According to an example, when multiple multi-round requests are identified, the processor 330 may generate a first response information including information corresponding to any one of the identified multiple multi-round requests, and provide the generated first response information to the user in a first method, and generate a second response information including information corresponding to another multi-round request in the identified multiple multi-round requests, and provide the generated second response information to the user in a second method. Here, according to an example, the first method may be a voice output method through a speaker (not shown), and the second method may be a text output method through a display (not shown).

[0225] For example, the processor 330 may generate a first response information including information corresponding to any one of the identified multiple multi-round requests, and the user terminal device 100 may use the generated first response information to output a voice corresponding to the first response information through a speaker (not shown). In addition, the processor 330 may generate a second response information including information corresponding to another of the identified multiple multi-round requests, and the user terminal device 100 may use the generated second response information to display a response corresponding to the second response information together with the current conversation content through a display (not shown).

[0226] In this case, according to an example, the processor 330 may generate UI information such that a response (or text) corresponding to the second response information is displayed at the bottom of the screen, and transmit the generated UI information to the user terminal device 100 .

[0227] In addition, according to an example, the first response information and the second response information may be identified based on reliability corresponding to each response information and a user preference with respect to a user request corresponding to each response information.

[0228] For example, the processor 330 may identify response information corresponding to a user request having a relatively large reliability value among a plurality of user requests as first response information, and identify response information corresponding to a relatively small user request as second response information.

[0229] Alternatively, for example, the processor 330 may identify response information corresponding to the estimated first user request as the first response information, and identify response information corresponding to the estimated second user request as the second response information.

[0230] Optionally, for example, the processor 330 may identify response information corresponding to a user request having a relatively large user preference value corresponding to each of multiple user requests as first response information, and identify response information corresponding to a user request having a relatively small user preference value as second response information.

[0231] Fig.13 is a block diagram illustrating a detailed configuration of an electronic device according to one or more embodiments.

[0232] Reference Fig.13 According to an embodiment, the electronic device 300' may include a communication interface 310, a memory 320, a processor 330, a display 340, a speaker 350, an input interface 360, and a microphone 370. Figure 3 The components shown in the figure repeat the description of the components.

[0233] The display 340 displays a user interface window for selecting functions supported by the electronic device 300'. Specifically, the display 340 may display a user interface window for selecting various functions provided by the electronic device 300'. The display 340 may be a monitor such as a liquid crystal display (LCD) and an organic light emitting diode (OLED), and may be implemented as a touch screen capable of simultaneously executing some functions of the input interface 360 ​​to be described later.

[0234] The display 340 may display content corresponding to the response information or the current conversation history information.

[0235] Alternatively, according to an example, the display 340 may display a UI including a text corresponding to the second response information.

[0236] The speaker 350 outputs sound. The speaker 350 may output response information corresponding to the received response message as sound.

[0237] For example, when the electronic device 300' is implemented as an AI speaker, the processor 330 may convert the voice uttered by the user input through the microphone into text and transmit the converted text to the external server 200. When response information is received from the external server 200, the text corresponding to the received response information may be output through the speaker 350 using the TTS function.

[0238] The input interface 360 ​​may select a function of the electronic device 300' and receive a control command for the corresponding function from the user. The input interface 360 ​​may be a keyboard, a mouse, a touch pad, etc., and may include a microphone for receiving a user's voice.

[0239] In addition, the electronic device 300 ′ continuously monitors whether a voice of a predetermined trigger command is input, and when it is determined that the input voice uttered by the user includes a predetermined trigger command, the processor 330 may perform the conversation service as described above.

[0240] In addition, the electronic device 300' may receive a similar trigger name similar to the currently set trigger command from an external device, and determine whether to start the conversation service using the trigger name set by the user and the similar trigger name when determining whether the trigger command is included.

[0241] For example, the current command for starting (or waking up) the dialogue system of the notification electronic device 300' is "Hi Bixby", but when other users use "Hi Big~" and "Hi Bix~" with a higher frequency, the server 200 can provide the terminal device 100 with information about the corresponding similar names, so that the dialogue service can be triggered even in a specific user terminal device 100, even in user utterances corresponding to "Hi Big~" or "Hi Bix~".

[0242] The electronic device 300 ′ receiving such similar trigger information may store the similar trigger information in the memory 320 , and use not only the trigger command set by the user but also the similar trigger command when determining whether to start the speech service.

[0243] The microphone 160 may refer to a module that acquires sound and converts the acquired sound into an electrical signal, and may be a condenser microphone, a ribbon microphone, a dynamic microphone, a piezoelectric element microphone, a carbon microphone, or a micro-electromechanical system (MEMS) microphone. In addition, it may be implemented in a non-directional, bidirectional, unidirectional, sub-cardioid, super-cardioid, and hyper-cardioid method.

[0244] According to the above example, when the estimated user request is a previously repeated request, a response corresponding to the user request can be provided immediately without going through a separate user reconfirmation process. As a result, fatigue caused by user reconfirmation can be prevented and user satisfaction can be improved.

[0245] In addition, the term "processor / module" or "module" used in the present disclosure may include a unit configured by hardware, software or firmware, and may be used compatibly with terms such as, for example, logic, logic block, component, circuit, etc. A "unit" or "module" may be a component or minimum unit of an overall configuration that performs one or more functions or a part thereof. For example, a module may be configured by an application specific integrated circuit (ASIC).

[0246] Various embodiments of the present disclosure may be implemented by software including instructions stored in a machine-readable storage medium (e.g., a computer-readable storage medium). A machine is a device that can call stored instructions from a storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device 300) of the disclosed embodiments. In the case where the processor executes the above-mentioned commands, the processor may directly execute the functions corresponding to the commands, or other components may execute the functions corresponding to the above-mentioned commands under the control of the processor. The commands may include code created or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means that the storage medium is tangible and does not include signals, and does not distinguish whether the data is stored in the storage medium semi-permanently or temporarily.

[0247] Although not limited thereto, the example embodiments may be embodied as computer-readable codes on a computer-readable recording medium. A computer-readable recording medium is any data storage device that can store data that can be read by a computer system thereafter. Examples of computer-readable recording media include read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage devices. The computer-readable recording medium may also be distributed on a network-integrated computer system so that the computer-readable code is stored and executed in a distributed manner. In addition, the example embodiments may be written as a computer program transmitted by a computer-readable transmission medium (such as a carrier wave), and received and implemented in a general-purpose or special-purpose digital computer that executes the program. In addition, it should be understood that in the example embodiments, one or more units of the above-mentioned devices and apparatus may include circuits, processors, microprocessors, etc., and may execute a computer program stored in a computer-readable medium.

[0248] The foregoing exemplary embodiments are merely exemplary and should not be construed as limiting. The present teachings can be easily applied to other types of devices. In addition, the description of the exemplary embodiments is intended to be illustrative rather than limiting the scope of the claims, and many substitutions, modifications, and variations will be apparent to those skilled in the art.

Claims

1. An electronic device, comprising: Communication interface; a memory configured to store at least one instruction; as well as One or more processors are configured to generate response information based on the conversation content received through the communication interface by executing the at least one instruction to perform the following operations: Identifying a user request by identifying a current request and a previous request of the user in the conversation content; Identifying, based on the conversation content, whether the user request is a multi-round request or a single-round request; Based on the user request being identified as a single-round request, generating a confirmation request to request the user to confirm the user request; as well as Based on the user request being identified as a multi-round request, generation of the confirmation request is omitted, and a response corresponding to the user request is generated.

2. The electronic device according to claim 1, wherein: The one or more processors are further configured to execute the at least one instruction to perform the following operations: When the same type of request is identified a predetermined number of times or more within a predetermined time based on the conversation content, identifying the user request as a multi-round request, and The response corresponding to the user request is generated based on multiple rounds of requests.

3. The electronic device according to claim 2, wherein: The same type of requests correspond to requests for information belonging to a common category and are directed to different entities or targets.

4. The electronic device according to claim 1, wherein: When a plurality of multi-round requests are identified, the one or more processors are further configured to execute the at least one instruction to perform the following operations: identifying a multi-round request among the identified plurality of multi-round requests using a received conversation order within the conversation content, and The response corresponding to the user request is generated based on the identified multiple rounds of requests.

5. The electronic device according to claim 1, wherein: When the plurality of user requests are confirmed, the one or more processors are further configured to execute the at least one instruction to perform the following operations: estimating a user request among the plurality of user requests based on a reliability value corresponding to each of the plurality of user requests, and When the reliability value of the user request is greater than or equal to a threshold, the response is generated based on the user request.

6. The electronic device according to claim 1, wherein: The one or more processors are further configured to execute the at least one instruction to perform the following operations: The confirmation request or the response is generated based on at least one of a user preference for the confirmation request and a user preference for the response.

7. The electronic device according to claim 6, wherein: The one or more processors are further configured to execute the at least one instruction to perform the following operations: acquiring a user preference value for the confirmation request based on the conversation history information including the user feedback information, When the obtained preference value is identified as being less than a threshold value, generating the response based on the identified request, and The user feedback information includes at least one of feedback information about the response and feedback information about the confirmation request.

8. The electronic device according to claim 6, wherein: The memory also includes user preference information corresponding to different types of requests, and The one or more processors are further configured to execute the at least one instruction to: identify a user preference value for a response corresponding to the user request based on the user preference information stored in the memory, and When the user preference value is identified as being greater than or equal to a threshold, the response is generated based on the user request.

9. The electronic device according to claim 1, wherein: The memory stores a trained first neural network model, The trained first neural network model is trained to output the user request when the conversation content, the response information, and the feedback information for the response are input as training data, and The one or more processors are further configured to execute the at least one instruction to perform the following operations: input the conversation content received through the communication interface to a trained first neural network model to estimate the user request.

10. The electronic device according to claim 1, wherein: The user request includes a first user request and a second user request, The one or more processors are further configured to execute the at least one instruction to perform the following operations: estimating a first user request based on a current request of the user within the received conversation content, estimating a second user request based on the current request and a previous request of the user, and When the first user request and the second user request are identified as different types, the confirmation request for identifying the first user request or the second user request is generated.

11. A method for controlling an electronic device, the method comprising: Obtaining conversation content from the user; Identifying a user request by identifying a current request and a previous request of the user in the conversation content; Identifying, based on the conversation content, whether the user request is a multi-round request or a single-round request; Based on the user request being identified as a single-round request, generating a confirmation request to request the user to confirm the user request; as well as Based on the user request being identified as a multi-round request, generation of the confirmation request is omitted, and a response corresponding to the user request is generated.

12. The method of claim 11, further comprising: When the same type of request is identified a predetermined number of times or more within a predetermined time based on the conversation content, identifying the user request as a multi-round request; as well as The response corresponding to the user request is generated based on multiple rounds of requests.

13. The method of claim 12, wherein: The same type of requests correspond to requests for information belonging to a common category and are directed to different entities or targets.

14. The method of claim 11, further comprising: When multiple multi-round requests are identified, using the dialog order within the dialog content to identify a multi-round request among the identified multiple multi-round requests, and The step of generating the response includes: generating the response based on the one multi-round request.

15. The method of claim 11, wherein: The estimating steps include: When a plurality of user requests are confirmed, estimating a user request among the plurality of user requests based on a reliability value corresponding to each of the plurality of user requests, and The step of generating the response includes: when the reliability value of the user request is greater than or equal to a threshold, generating the response based on the user request.

Images