Electronic device and method of processing user utterance

Abstract

An electronic device according to an embodiment is provided. An electronic device may include a memory to store one or more computer programs. An electronic device may include one or more processors communicatively coupled to a memory. The one or more computer programs may include computer executable instructions. Computer executable instructions may be executed by the one or more processors to instruct the electronic device to receive a user's utterance. Computer executable instructions may be executed by the one or more processors to instruct the electronic device to analyze a bias of the utterance or a bias of the user. The computer executable instructions may be executed by the one or more processors to instruct the electronic device to generate a response including the bias information based on a result of analyzing the bias of the utterance or a result of analyzing the bias of the user. The computer executable instructions may be executed by the one or more processors to instruct the electronic device to provide a response to the user.

Description

Technical Field

[0001] This disclosure relates to electronic devices and methods for processing user speech. Background Technology

[0002] Electronic devices are provided in various ways, equipped with voice assistant functionality that provides services based on user speech. These devices can use artificial intelligence (AI) servers to recognize user speech and analyze its meaning and intent. The AI ​​server can infer the user's intent by interpreting their speech and perform tasks based on that inferred intent. The AI ​​server can perform tasks based on the user's intent as expressed through natural language interactions between the user and the AI ​​server.

[0003] When AI servers provide responses to users based on voice assistant functions, the potential bias and discrimination included in those responses are sensitive topics. Therefore, most language model-based question-answering systems are designed to eliminate any bias that might be included in the answers.

[0004] The above information is presented as background information only to aid in understanding this disclosure. No determination or assertion is made regarding whether any of the above content can be used as prior art with respect to this disclosure. Summary of the Invention

[0005] Technical solution

[0006] The aspects of this disclosure will at least address the aforementioned problems and / or disadvantages, and provide at least the following advantages. Therefore, one aspect of this disclosure is to provide an electronic device and method for processing user speech.

[0007] Other aspects will be set forth in part in the description which follows, and in part will be apparent from the description, or may be learned by practice of the embodiments presented.

[0008] According to one aspect of this disclosure, an electronic device is provided. The electronic device includes a memory storing one or more computer programs. The electronic device includes one or more processors communicatively coupled to the memory. The one or more computer programs include computer-executable instructions. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to receive speech from a user. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to analyze the bias of the speech or the user's bias. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to generate a response including biased information based on the analysis results of the speech bias or the user bias. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to provide a response to the user.

[0009] According to another aspect of this disclosure, a method for operating an electronic device is provided. The method includes receiving a user's speech. The method includes analyzing the bias of the speech or the user's bias. The method includes generating a response including bias information based on the analysis results of the speech bias or the user bias. The method includes providing the response to the user.

[0010] According to another aspect of this disclosure, an electronic device is provided. The electronic device includes a memory storing one or more computer programs. The electronic device includes one or more processors communicatively coupled to the memory. The one or more computer programs include computer-executable instructions. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to receive a user's speech. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to generate a response, including bias information, based on the speech. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to generate bias information about the response based on the response. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to provide the response and the bias information together to the user.

[0011] According to another aspect of this disclosure, a method for operating an electronic device is provided. The method includes receiving a user's speech. The method includes generating a response including bias information based on the speech. The method includes generating bias information about the response based on the response. The method includes providing the response and the bias information together to the user.

[0012] According to another aspect of this disclosure, an electronic device is provided. The electronic device includes a memory storing one or more computer programs. The electronic device includes one or more processors communicatively coupled to the memory. The one or more computer programs include computer-executable instructions. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to receive a user's speech. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to obtain first information about the bias of the speech or second information about the user's bias. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to generate a prompt based on the first information or the second information. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to generate a response including bias information from the prompt based on a general language model or a biased language model. When executed by the one or more processors, the computer-executable instructions can cause the electronic device to provide a response to the user.

[0013] According to another aspect of this disclosure, a method for operating an electronic device is provided. The method includes receiving a user's utterance. The method includes obtaining first information about the bias of the utterance or second information about the user's bias. The method includes generating a prompt based on the first or second information. The method includes generating a response including bias information from the prompt based on a general language model or a biased language model. The method includes providing the response to the user.

[0014] According to another aspect of this disclosure, one or more non-transitory computer-readable storage media are provided, storing one or more computer programs including computer-executable instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform operations. The operations include receiving user utterances. The operations include analyzing the bias of the utterances or the user's bias. The operations include generating a response including bias information based on the analysis results of the utterance bias or the user's bias. The operations include providing the response to the user.

[0015] Other aspects, advantages, and distinctive features of this disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the disclosure taken in conjunction with the accompanying drawings. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure;

[0017] Figure 2 This is a block diagram illustrating an integrated intelligent system according to an embodiment of the present disclosure;

[0018] Figure 3 This is a diagram illustrating the relationship information between concepts and actions according to embodiments of the present disclosure, stored in a database (DB).

[0019] Figure 4 This is a diagram illustrating the screen of an electronic device that processes received voice input via a smart application according to an embodiment of the present disclosure;

[0020] Figure 5 This is a diagram illustrating the operation of an electronic device processing a user's speech according to an embodiment of the present disclosure;

[0021] Figure 6 This is a block diagram schematically illustrating an electronic device according to an embodiment of the present disclosure;

[0022] Figure 7 This is a diagram illustrating information about bias according to an embodiment of the present disclosure;

[0023] Figure 8 , Figure 9 , Figure 10 , Figure 11 and Figure 12 The diagram illustrates examples of operations of an electronic device processing user speech according to various embodiments of the present disclosure; and

[0024] Figure 13 This is a flowchart illustrating a method of operating an electronic device according to an embodiment.

[0025] Throughout the accompanying drawings, the same reference numerals are used to denote the same elements. Detailed Implementation

[0026] The following description with reference to the accompanying drawings is intended to aid in a full understanding of the various embodiments of this disclosure as defined by the claims and their equivalents. It includes various specific details to aid understanding, but these are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the various embodiments described herein without departing from the scope and spirit of this disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and structures may be omitted.

[0027] The terms and words used in the following description and claims are not limited to their literal meaning, but are used by the inventors only to enable a clear and consistent understanding of this disclosure. Therefore, those skilled in the art will understand that the following description of various embodiments of this disclosure is for illustrative purposes only and is not intended to limit the purpose of this disclosure as defined by the appended claims and their equivalents.

[0028] It should be understood that, unless the context clearly specifies otherwise, the singular forms of “a,” “an,” and “the” include plural indicators. Thus, for example, a reference to “component surface” includes a reference to one or more such surfaces.

[0029] It should be understood that the boxes in each flowchart and the combination of flowcharts can be executed by one or more computer programs including instructions. The entirety of one or more computer programs can be stored in a single memory device, or one or more computer programs can be divided into different parts stored in multiple different memory devices.

[0030] Any function or operation described herein may be processed by a processor or a combination of processors. A processor or combination of processors is circuitry that performs processing and includes circuitry such as an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphics processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a Wi-Fi chip, a Bluetooth® chip, a global positioning system (GPS) chip, a near field communication (NFC) chip, a connectivity chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display driver integrated circuit (IC), an audio codec (CODEC) chip, a universal serial bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system-on-a-chip (SoC), an IC, etc.

[0031] Figure 1 This is a block diagram illustrating an electronic device in a network environment according to an embodiment of the present disclosure. Reference Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of these components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some components (e.g., sensor module 176, camera module 180, or antenna module 197) may be integrated into a single component (e.g., display module 160).

[0032] Processor 120 can execute, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and can perform various data processing or calculations. According to another embodiment, as at least part of data processing or calculation, processor 120 can store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to another embodiment, processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted specifically for a designated function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may be implemented as part of the main processor 121.

[0033] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one of the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., executing an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one of the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to another embodiment, the auxiliary processor 123 (e.g., ISP or CP) can be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to another embodiment, the auxiliary processor 123 (e.g., NPU) can include hardware architecture specified for artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. This learning can be performed, for example, by an electronic device 101 that executes the artificial intelligence model, or via a separate server (e.g., server 108). The learning algorithm can include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model can include multiple layers of artificial neural networks. Artificial neural networks can include, for example, deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), deep Q-networks, or combinations of two or more of these, but are not limited thereto. The artificial intelligence model can additionally or alternatively include software structures in addition to hardware structures.

[0034] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, input or output data of software (e.g., program 140) and associated commands. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0035] Program 140 may be stored as software in memory 130 and may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0036] Input module 150 can receive commands or data from outside electronic device 101 (e.g., a user) to be used by another component of electronic device 101 (e.g., processor 120). Input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0037] The audio output module 155 can output audio signals to the outside of the electronic device 101. The audio output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as playing multimedia or playing recordings. The receiver can be used to receive incoming calls. According to another embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0038] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display module 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to another embodiment, display module 160 may include a touch sensor adapted to sense touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0039] The audio module 170 can convert sound into electrical signals and vice versa. According to another embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or an external electronic device (e.g., electronic device 102, such as a speaker or headphones) directly or wirelessly connected to the electronic device 101.

[0040] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and generate electrical signals or data values ​​corresponding to the detected state. According to another embodiment, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0041] Interface 177 may support one or more specific protocols for direct (e.g., wired) or wireless connection between electronic device 101 and external electronic device (e.g., electronic device 102). According to another embodiment, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0042] Connection terminal 178 may include a connector via which electronic device 101 can be physically connected to an external electronic device (e.g., electronic device 102). According to another embodiment, connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0043] The tactile module 179 can convert electrical signals into mechanical stimulation (e.g., vibration or motion) or electrical stimulation that can be recognized by a user through his or her touch or kinesthesia. According to another embodiment, the tactile module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0044] Camera module 180 can capture still images and moving images. According to another embodiment, camera module 180 may include one or more lenses, an image sensor, an ISP, or a flash.

[0045] The power management module 188 can manage the power supplied to the electronic device 101. According to the example, the power management module 188 can be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0046] Battery 189 can power at least one component of electronic device 101. According to another embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0047] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more CPs that can operate independently of processor 120 (e.g., AP) and support direct (e.g., wired) or wireless communication. According to another embodiment, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). A corresponding one of these communication modules can communicate via a first network 198 (e.g., a short-range communication network, such as Bluetooth). TM The communication module 192 can communicate with external electronic devices 104 via a Wi-Fi Direct or Infrared Data Association (IrDA) network or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a fifth-generation (5G) network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can use subscriber information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the SIM 196 to identify and authenticate electronic devices 101 in the communication network (such as a first network 198 or a second network 199).

[0048] Wireless communication module 192 can support 5G networks beyond fourth-generation (4G) networks, as well as next-generation communication technologies such as New Radio (NR) access technology. NR access technology can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance on high-frequency bands, such as beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive MIMO. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic device (e.g., electronic device 104), or network system (e.g., second network 199). According to another embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or higher) for implementing eMBB, loss coverage (e.g., 164 dB or lower) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less for round trip) for implementing URLLC.

[0049] Antenna module 197 can transmit or receive signals or power to or from the outside of electronic device 101 (e.g., external electronic device). According to another embodiment, antenna module 197 may include an antenna comprising a radiating element comprising conductive material or conductive patterns formed in or on a substrate (e.g., a printed circuit board (PCB)). According to yet another embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190. Signals or power can be transmitted or received between communication module 190 and external electronic device via at least one selected antenna. According to yet another embodiment, another component besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0050] According to another embodiment, antenna module 197 can form a millimeter-wave (mmWave) antenna module. According to yet another embodiment, the millimeter-wave antenna module may include: a PCB; an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the PCB and capable of supporting a specified high-frequency band (e.g., millimeter-wave band); and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top or side surface) of the PCB and capable of transmitting or receiving signals in a specified high-frequency band.

[0051] At least some of the aforementioned components may be coupled to each other and communicate signals (e.g., commands or data) between them via peripheral communication schemes (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0052] According to another embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 coupled to a second network 199. Each of external electronic devices 102 or 104 can be a device of the same or different type as electronic device 101. According to another embodiment, all or some of the operations to be performed at electronic device 101 can be performed at one or more external electronic devices (e.g., external electronic devices 102 and 104, or server 108). For example, if electronic device 101 needs to perform a function or service automatically or in response to a request from a user or another device, it can either perform that function or service automatically or, in addition to performing that function or service, request one or more external electronic devices to perform at least a portion of that function or service. Upon receiving the request, one or more external electronic devices can perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and can transmit the result of the performance to electronic device 101. Electronic device 101 can provide the result, with or without further processing, as at least part of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technologies can be used, for example. Electronic device 101 can use, for example, distributed computing or MEC to provide ultra-low latency services. In one embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to another embodiment, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 can be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0053] Figure 2 This is a block diagram illustrating an integrated intelligent system according to an embodiment of the present disclosure.

[0054] refer to Figure 2 The integrated intelligent system 20 according to the embodiment may include an electronic device 201 (e.g., Figure 1 Electronic devices 101), intelligent servers 200 (e.g., Figure 1 Server 108) and server 300 (for example, Figure 1 Server 108).

[0055] The electronic device 201 according to the embodiment may be an internet-connected terminal device (or electronic device), and may be, for example, a mobile phone, smartphone, personal digital assistant (PDA), laptop computer, television (TV), white goods, wearable device, head-mounted display (HMD), or smart speaker.

[0056] According to the illustrated embodiment, electronic device 201 may include communication interface 202 (e.g., Figure 1 Interface 177), microphone 206 (e.g., Figure 1 Input module 150), speaker 205 (e.g., Figure 1 (e.g., sound output module 155), display module 204) Figure 1 The display module 160), and the memory 207 (e.g., Figure 1 The memory 130) or processor 203 (e.g., Figure 1 (Processor 120). The components listed above can be operatively or electrically connected to each other.

[0057] The communication interface 202 according to an embodiment can be connected to an external device and is configured to send data to and receive data from the external device. The microphone 206 according to an embodiment can receive sound (e.g., user speech) and convert the sound into an electrical signal. The speaker 205 according to an embodiment can output the electrical signal as sound (e.g., speech).

[0058] The display module 204 according to an embodiment can be configured to display images or videos. The display module 204 according to an embodiment can also display the graphical user interface (GUI) of a running application (app). The display module 204 according to an embodiment can receive touch input via a touch sensor. For example, the display module 204 can receive text input via a touch sensor in the keyboard area of ​​the screen displayed in the display module 204.

[0059] The memory 207 according to the embodiment may store a client module 209, a software development kit (SDK) 208, and multiple applications 211. The client module 209 and the SDK 208 may be configured to perform a framework (or solution program) for general functions. In addition, the client module 209 or the SDK 208 may be configured to handle user input (e.g., voice input, text input, or touch input).

[0060] According to an embodiment, the multiple applications stored in memory 207 may be programs for performing specified functions. According to another embodiment, the multiple applications may include a first application 211_1 and a second application 211_2. According to yet another embodiment, each of the multiple applications may include multiple actions for performing the specified function. For example, the applications may include an alarm clock application, a messaging application, and / or a scheduling application. According to yet another embodiment, the multiple applications may be executed by processor 203 to sequentially perform at least a portion of the multiple actions.

[0061] According to the embodiment, the processor 203 can control the overall operation of the electronic device 201. For example, the processor 203 can be electrically connected to the communication interface 202, microphone 206, speaker 205, and display module 204 to perform specified operations.

[0062] According to the embodiment, the processor 203 can also perform specified functions by executing a program stored in the memory 207. For example, the processor 203 can execute at least one of the client module 209 or the SDK 208 to perform the following operations for processing user input. The processor 203 can control the actions of multiple applications through, for example, the SDK 208. The following operations, which are operations of the client module 209 or the SDK 208, can be performed by the processor 203.

[0063] According to an embodiment, client module 209 can receive user input. For example, client module 209 can receive voice signals corresponding to user speech sensed by microphone 206. As another example, client module 209 can receive touch input sensed by display module 204. As yet another example, client module 209 can receive text input sensed by keyboard or on-screen keyboard. Furthermore, client module 209 can receive various types of user input sensed by input modules included in or connected to electronic device 201. Client module 209 can send the received user input to intelligent server 200. Client module 209 can also send the status information of electronic device 201 along with the received user input to intelligent server 200. The status information may be, for example, application execution status information.

[0064] According to the embodiment, the client module 209 can receive a result corresponding to the received user input. For example, when the intelligent server 200 is able to calculate a result corresponding to the received user input, the client module 209 can receive the result corresponding to the received user input. The client module 209 can display the received result on the display module 204. In addition, the client module 209 can output the received result in audio form through the speaker 205.

[0065] According to the embodiment, the client module 209 can receive a plan corresponding to the received user input. The client module 209 can display the results of executing multiple actions of the application according to the plan on the display module 204. For example, the client module 209 can sequentially display the results of executing multiple actions on the display module 204 and output the results in audio form via the speaker 205. As another example, the electronic device 201 can display only a portion of the results of executing multiple actions (e.g., the result of the last action) on the display module 204 and output that portion of the result in audio form via the speaker 205.

[0066] According to yet another embodiment, the client module 209 can receive a request from the smart server 200 for obtaining information needed to calculate a result corresponding to the user input. According to yet another embodiment, the client module 209 can send the necessary information to the smart server 200 in response to the request.

[0067] According to an embodiment, the client module 209 can send information about the results of performing multiple actions according to a plan to the intelligent server 200. The intelligent server 200 can use the information about the results to confirm that the received user input has been processed correctly.

[0068] According to an embodiment, client module 209 may include a speech recognition module. According to yet another embodiment, client module 209 may use the speech recognition module to recognize voice input for performing limited functions. For example, client module 209 may execute a smart application for processing voice input to perform organic operations upon specified input (e.g., wake-up!).

[0069] According to one embodiment, the intelligent server 200 can receive information related to user voice input from the electronic device 201 via a communication network. According to another embodiment, the intelligent server 200 can convert the data related to the received voice input into text data. According to yet another embodiment, the intelligent server 200 can generate a plan based on the text data for performing tasks corresponding to the user's voice input.

[0070] According to another embodiment, the plan can be generated by an artificial intelligence (AI) system. The AI ​​system can be a rule-based system or a neural network-based system (e.g., a feedforward neural network (FNN) or a recurrent neural network (RNN)). Alternatively, the AI ​​system can be a combination of the above-described systems or other AI systems. According to yet another embodiment, a plan can be selected from a set of predefined plans, or a plan can be generated in real time in response to a user request. For example, the AI ​​system can select at least one plan from predefined plans.

[0071] According to one embodiment, the intelligent server 200 can send the results of the generated plan to the electronic device 201, or send the generated plan to the electronic device 201. According to another embodiment, the electronic device 201 can display the results of the plan on the display module 204. According to yet another embodiment, the electronic device 201 can display the results of the actions performed according to the plan on the display module 204.

[0072] The intelligent server 200 according to the embodiment may include a front-end 215, a natural language platform 220, a capsule database (DB) 230, an execution engine 240, a terminal user interface 250, a management platform 260, a big data platform 270, or an analysis platform 280.

[0073] According to an embodiment, the front end 215 can receive received user input from the electronic device 201. The front end 215 can send a response corresponding to the user input.

[0074] According to yet another embodiment, the natural language platform 220 may include an automatic speech recognition (ASR) module 221, a natural language understanding (NLU) module 223, a planner module 225, a natural language generator (NLG) module 227, or a text-to-speech (TTS) module 229.

[0075] According to an embodiment, the ASR module 221 can convert data related to voice input received from the electronic device 201 into text data. According to an embodiment, the NLU module 223 can determine a domain (and / or intent information) corresponding to the voice input (e.g., user utterance) based on the text data of the voice input. A domain can correspond to a category (or service) associated with an action (or function) the user intends to perform using the device. Domains can be categorized based on text-related services (e.g., applications). For example, the Gracenote domain can correspond to a music search service (e.g., the Gracenote™ service). The Melon domain can correspond to a music streaming service (e.g., the Melon™ service). Domains can be associated with intent information corresponding to text. According to an embodiment, the NLU module 223 can use the text data of the voice input to discern user intent. For example, the NLU module 223 can discern user intent by performing syntactic or semantic analysis on the user input in text data form. According to an embodiment, the NLU module 223 can use linguistic features (e.g., grammatical elements) of morphemes or phrases to discern the meaning of words extracted from the user input and can determine the user intent by matching the discerned meaning of the words with the intent. NLU module 223 can obtain intent information corresponding to a user's utterance. The intent information can be information indicating the user's intent determined through analysis of text data. The intent information may include information indicating the user's intention to perform an action or function using the device. A slot can be detailed information associated with the intent information. Slots can be obtained based on a domain corresponding to the utterance. A slot can be variable information necessary to perform an action. In an embodiment, the variable information constituting a slot may include named entities.

[0076] According to one embodiment, the planner module 225 can generate a plan using parameters and intents determined by the NLU module 223. According to another embodiment, the planner module 225 can determine multiple domains required to perform a task based on the determined intent. The planner module 225 can determine multiple actions included in each of the multiple domains determined based on the intent. According to yet another embodiment, the planner module 225 can determine the parameters required to perform the determined multiple actions, or the result values ​​output by performing the multiple actions. Parameters and result values ​​can be defined as concepts of a specified form (or class). Accordingly, the plan can include multiple actions and multiple concepts determined by the user's intent. The planner module 225 can determine the relationships between the multiple actions and the multiple concepts step-by-step (or hierarchically). For example, the planner module 225 can determine the execution order of the multiple actions determined based on the user's intent based on the multiple concepts. In other words, the planner module 225 can determine the execution order of the multiple actions based on the parameters required to perform the multiple actions and the results output by performing the multiple actions. Accordingly, the planner module 225 can generate a plan that includes connection information (e.g., ontology) regarding the connections between the multiple actions and the multiple concepts. The planner module 225 can generate a plan using information stored in a DB 230, which encapsulates a set of relationships between stored concepts and actions.

[0077] According to an embodiment, NLG module 227 can convert specified information into text form. The information converted into text form can be in the form of natural language speech. According to an embodiment, TTS module 229 can convert information in text form into information in speech form.

[0078] According to yet another embodiment, some or all of the functions of the natural language platform 220 may also be implemented in the electronic device 201.

[0079] Encapsulation DB 230 can store information about relationships between multiple concepts and actions corresponding to multiple domains. According to an embodiment, an encapsulation may include multiple action objects (or action information) and concept objects (or concept information) included in a plan. According to yet another embodiment, encapsulation DB 230 may store multiple encapsulations in the form of a Concept Action Network (CAN). According to yet another embodiment, multiple encapsulations may be stored in a function registry included in encapsulation DB 230.

[0080] The encapsulation DB 230 may include a policy registry storing policy information required to determine a plan corresponding to the voice input. The policy information may include reference information for determining a plan when multiple plans exist corresponding to the voice input. According to another embodiment, the encapsulation DB 230 may include a follow-up registry storing information about subsequent actions for suggesting follow-up actions to the user in specified situations. Follow-up actions may include, for example, subsequent utterances. According to another embodiment, the encapsulation DB 230 may include a layout registry storing layout information output by the electronic device 201. According to another embodiment, the encapsulation DB 230 may include a vocabulary registry storing vocabulary information included in the encapsulation information. According to another embodiment, the encapsulation DB 230 may include a dialogue registry storing information about a dialogue (or interaction) with the user. The encapsulation DB 230 can update stored objects through developer tools. Developer tools may include, for example, a function editor for updating action objects or concept objects. Developer tools may include a vocabulary editor for updating vocabulary. Developer tools may include a policy editor for generating and registering policies for determining plans. Developer tools may include a dialogue editor for generating dialogues with the user. Developer tools may include a follow-up editor capable of activating follow-up goals and editing follow-up statements that provide prompts. The follow-up goal may be determined based on the currently configured goal, user preferences, or environmental conditions. In this embodiment, the encapsulated DB 230 may also be implemented in the electronic device 201.

[0081] According to an embodiment, the execution engine 240 can use the generated plan to calculate results. The terminal user interface 250 can send the calculated results to the electronic device 201. Accordingly, the electronic device 201 can receive the results and provide them to the user. According to an embodiment, the management platform 260 can manage the information used in the intelligent server 200. According to an embodiment, the big data platform 270 can collect user data. According to an embodiment, the analysis platform 280 can manage the quality of service (QoS) of the intelligent server 200. For example, the analysis platform 280 can manage the components and processing speed (or efficiency) of the intelligent server 200.

[0082] According to one embodiment, the service server 300 can provide specified services (e.g., food orders or hotel reservations) to the electronic device 201. According to yet another embodiment, the service server 300 can be a server operated by a third party. Services of the business server 300 (such as CP service A 301 and CP service B 302) can interact with the front end 215 of the intelligent server 200. According to one embodiment, the service server 300 can provide the intelligent server 200 with information for generating plans corresponding to received user input. The provided information can be stored in the encapsulated database 230. Furthermore, the service server 300 can provide the intelligent server 200 with result information based on the plans.

[0083] In the aforementioned integrated intelligent system 20, the electronic device 201 can provide various intelligent services to the user in response to user input. User input may include, for example, input via physical buttons, touch input, or voice input.

[0084] In this embodiment, the electronic device 201 can provide voice recognition services through a smart application (or voice recognition application) stored therein. For example, the electronic device 201 can recognize user speech or voice input received through a microphone and provide the user with services corresponding to the recognized voice input.

[0085] In this embodiment, the electronic device 201 can perform a specified action based on the received voice input, either alone or in conjunction with the intelligent server 200 and / or the service server 300. For example, the electronic device 201 can execute an application corresponding to the received voice input and perform the specified action through the executed application.

[0086] In this embodiment, when the electronic device 201 provides services together with the intelligent server 200 and / or the service server 300, the electronic device 201 can use the microphone 206 to detect user speech and generate a signal (or voice data) corresponding to the detected user speech. The electronic device 201 can use the communication interface 202 to send the voice data to the intelligent server 200.

[0087] In response to voice input received from electronic device 201, the intelligent server 200 according to an embodiment can generate a plan for performing a task corresponding to the voice input or the result of performing actions according to the plan. The plan may include, for example, multiple actions for performing the task corresponding to the user's voice input, and multiple concepts associated with the multiple actions. A concept may be defined as a parameter input for performing the multiple actions or a result value output by performing the multiple actions. The plan may include connection information regarding the connections between the multiple actions and the multiple concepts.

[0088] According to the embodiment, the electronic device 201 can receive a response using the communication interface 202. The electronic device 201 can use the speaker 205 to output voice signals generated internally to the outside, or it can use the display module 204 to output images generated internally to the outside.

[0089] Figure 3 This is a diagram illustrating the form in which relational information about the relationship between concepts and actions is stored in a database according to an embodiment of this disclosure.

[0090] Intelligent servers (e.g., Figure 2 The encapsulated DB of the intelligent server 200 (e.g., Figure 2 The encapsulated DB 230 can be stored in CAN 400 format. The encapsulated DB can store the actions and parameters required for processing tasks corresponding to the user's voice input in CAN format.

[0091] The encapsulation DB can store multiple encapsulations (encapsulation A 401 and encapsulation B 404) corresponding to multiple domains respectively. According to another embodiment, an encapsulation (e.g., encapsulation A 401) can correspond to a domain (e.g., location (geography) or application). Furthermore, an encapsulation can correspond to at least one service provider (e.g., CP1 402 or CP2 403) for performing functions associated with the domain. According to yet another embodiment, an encapsulation can include at least one action 410 and at least one concept 420 to perform a specified function. CAN 400 can store other information, such as CP 3 406. Additionally, encapsulation B 404 can correspond to a service provider (e.g., CP 4 405).

[0092] Natural language platforms (e.g., Figure 2 The natural language platform 220 can use encapsulations stored in the encapsulation DB to generate a plan for performing a task corresponding to the received speech input. For example, the planner module of the natural language platform (e.g., Figure 2 The planner module 225 can use the encapsulations stored in the encapsulation DB to generate a plan. For example, a plan 407 can be generated using actions 4011 and 4013 of encapsulation A 401 and concepts 4012 and 4014, and actions 4041 and concepts 4042 of encapsulation B 404.

[0093] Figure 4 This is a diagram illustrating the screen of an electronic device that processes received voice input via a smart application according to an embodiment of the present disclosure.

[0094] Electronic device 201 can execute intelligent applications to access intelligent servers (e.g., Figure 2The intelligent server 200 processes user input.

[0095] According to another embodiment, on screen 310, when a specified voice input (e.g., wake-up!) is recognized or input is received via a hardware key (e.g., a dedicated hardware key), electronic device 201 can execute a smart application for processing voice input. Electronic device 201 can execute the smart application, for example, while executing a scheduling application. According to another embodiment, electronic device 201 can be displayed on display module 204 (e.g., ...). Figure 1 The display module 160 displays an object (e.g., an icon) 311 corresponding to the smart application. According to another embodiment, the electronic device 201 can receive voice input via user speech. For example, the electronic device 201 can receive voice input such as "Let me know this week's schedule!". According to yet another embodiment, the electronic device 201 can display a user interface (UI) 313 (e.g., an input window) of the smart application on the display module 204, in which text data of the received voice input is displayed.

[0096] According to another embodiment, on screen 320, electronic device 201 can display the result corresponding to the received voice input on display module 204. For example, electronic device 201 can receive a schedule corresponding to the received user input and display "this week's schedule" on display module 204 according to the schedule.

[0097] Figure 5 This is a diagram illustrating the operation of an electronic device processing a user's speech according to an embodiment of the present disclosure.

[0098] refer to Figure 5 Electronic device 501 may include reference Figure 1 The described electronic device 101 and reference Figure 2 The described electronic device 201 includes at least some components. The intelligent server 601 may include reference... Figure 2 At least some components of the intelligent server 200 are described. References to electronic device 501 and intelligent server 601 are omitted. Figures 1 to 4 The provided description is duplicated.

[0099] Electronic device 501 (e.g., Figure 1 Electronic devices 101 or Figure 2 Electronic device 201) can be connected to intelligent server 601 (e.g., via LAN, WAN, value-added network (VAN), mobile radio communication network, satellite communication network, or any combination thereof) Figure 2The intelligent server 200. Electronic device 501 and intelligent server 601 can communicate with each other via wired or wireless communication methods (e.g., wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, ZigBee, Wi-Fi Direct (WFD), Ultra Wideband (UWB), Infrared Data Association (IrDA), and Near Field Communication (NFC)).

[0100] According to another embodiment, the electronic device 501 may be implemented as at least one of a smartphone, a tablet PC, a mobile phone, a speaker (e.g., an AI speaker), a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device.

[0101] According to another embodiment, electronic device 501 can acquire a voice signal corresponding to a user's speech and can send the voice signal to intelligent server 601. Intelligent server 601 can acquire text data corresponding to the user's speech based on the voice signal. The text data can be obtained by performing ASR on the voice signal to convert the speech portion into computer-readable text. Intelligent server 601 can use the text data to analyze the user's speech. Intelligent server 601 can use the analysis results (e.g., intent information, entities, and / or encapsulation) to perform desired functions, or can provide a response (e.g., questions and answers) to be provided to the user to the device (e.g., electronic device 501). Intelligent server 601 can be implemented as software. Some or all of the intelligent server 601 can be implemented in electronic device 501 and / or intelligent server 601 (e.g., Figure 2 The AI ​​is implemented in the intelligent server 200. The AI, used to process speech without communicating with the intelligent server 601, can be installed in the electronic device 501. (See reference...) Figures 2 to 4 The components of the described natural language platform 220 can be implemented in electronic device 501.

[0102] According to yet another embodiment, including Figure 2 The ASR module 221 in the Natural Language Platform 220 can convert user speech into text data. The NLU module 223 included in the Natural Language Platform 220 can determine the domain (and / or intent information) corresponding to the user speech based on the text data corresponding to the user speech. The electronic device 501 can provide a response corresponding to the user speech based on the intent information.

[0103] According to yet another embodiment, electronic device 501 can provide a user with a response that includes biased information. By providing a biased response, electronic device 501 can interact with the user in relation to a variety of questions. Electronic device 501 providing a biased response can enhance the user experience in terms of diversity.

[0104] According to yet another embodiment, electronic device 501 may also provide information about the bias of the response when providing it. This information about the bias of the response (and / or about the bias of the entire session) may be provided to the user along with the response, allowing the user to perceive that the response of electronic device 501 is biased. Electronic device 501 can ensure the objectivity of the response.

[0105] According to another embodiment, biased information can be information influenced by personal opinions, prejudices, or a deliberate intent to promote a predetermined viewpoint. Biased information can include distorted content. Biased information can deviate from a fair and balanced representation of facts or perspectives. Bias in information can be expressed in various forms, such as political, ideological, cultural, or personal biases. According to embodiments of this disclosure, the term "bias" is not limited to its dictionary meaning. Bias can correspond to features including inflammatory rhetoric, violence, and hate speech. In other words, biased information can correspond to information about a target that needs to be filtered but has not been filtered. According to embodiments of this disclosure, the term "bias" can encompass representations equivalent to those described above. The degree of bias included in a response may not deviate from typical ethical values. For example, if a friendly setting is implemented for a particular country, a response may be provided to the user (e.g., "This particular country has better scenery than other countries, therefore it is a good place to live"); however, a response may not be provided to the user (e.g., a response including biases related to social issues (e.g., "People from this particular country are a respectable and excellent nation.")). Bias can include statistical results that are systematically distorted due to unacceptable factors in the derivation of statistical results. Bias can be allowed to operate within a predetermined range. Bias can include a preference for or against a topic compared to another topic, person, or group.

[0106] refer to Figure 5Electronic device 501 can receive utterances from a user (e.g., “Do you think we (country A) will win the match between country A and country B?”). The utterances (e.g., “Do you think we (country A) will win the match between country A and country B?”) can be biased (e.g., “we (country A)” corresponding to the user’s nationality). Electronic device 501 can generate a response based on the utterances that includes biased information (e.g., “Of course, I think we (country A) will win”). Electronic device 501 can provide the user with a response that includes biased information (e.g., “Of course, I think we (country A) will win”). When providing a response (e.g., “Of course, I think we (country A) will win”), electronic device 501 can also provide the user with information about the bias in the response (e.g., see [link to relevant documentation]). Figure 8 Electronic device 501 can enhance the user experience in terms of diversity.

[0107] According to yet another embodiment, at least a portion of the operations performed by electronic device 501 may be performed by electronic device 501 and / or intelligent server 601. Hereinafter, the description is provided based on the assumption that electronic device 501 performs the operations.

[0108] Figure 6 This is a block diagram schematically illustrating an electronic device according to an embodiment of the present disclosure, and Figure 7 This is a diagram illustrating information about bias according to an embodiment of the present disclosure.

[0109] refer to Figure 6 Electronic device 501 may include the above reference Figure 1 The described electronic device 101 and the above reference Figure 2 The described electronic device 201 is configured in at least a portion thereof. As described above, it is used in situations where it is not connected to a smart server (e.g., Figure 2 Intelligent Server 200 or Figure 5 In the case of communication with the intelligent server 601, AI can be installed in the electronic device 501 to process speech. In other words, referring to... Figures 2 to 4 The described natural language platform 220 can be implemented in electronic device 501. References to electronic device 501 are omitted. Figures 1 to 4 The provided description is duplicated.

[0110] According to yet another embodiment, electronic device 501 may include wireless communication module 540 (e.g., Figure 1 The wireless communication module 192). Electronic device 501 may include processor 520 (e.g., wireless communication module 192). Figure 1 Processor 120 and Figure 2 The processor 203). Electronic device 501 may include memory 530 (e.g., processor 203). Figure 1The memory 130 and Figure 2 The memory 530 (207) is used by the processor 520 (e.g., the AP). The processor 520 can execute instructions by accessing the memory 530. The processor 520 can enable the electronic device 501 to provide a response to the user. The memory 530 can store various data used by at least one component of the electronic device 501 (e.g., the processor 520). The memory 530 can store a general language model 531 and a biased language model 532.

[0111] According to yet another embodiment, electronic device 501 can receive a user's speech. The user can execute biased speech. The user may expect to receive a biased response in response to his or her speech, and may also execute user settings associated with the bias.

[0112] According to another embodiment, electronic device 501 can use bias checking module 521-1 to obtain analysis results of utterance bias (e.g., first information about utterance bias). Electronic device 501 can use bias checking module 521-2 to obtain analysis results of user bias (e.g., second information about user bias). The first information about utterance bias may include the utterance bias category and / or the utterance bias degree. The second information about user bias may correspond to user settings associated with the bias. The bias setting UI (e.g., based on the bias displayed by electronic device 501) can be used. Figure 9 The bias settings (UI 921) are used to obtain secondary information about the user's biases.

[0113] According to another embodiment, the bias checking module 521-1 can be trained based on a corpus labeled with bias categories and bias degrees. The bias checking module 521-1 can analyze (e.g., determine) the bias of utterances based on the corpus as training data. The corpus can be a criterion used to determine the bias of utterances. The bias checking module 521-1 can analyze (e.g., determine) the bias of utterances based on the association (e.g., similarity) between the corpus and the utterances. The bias checking module 521-1 can generate analysis results of the bias of the utterances (e.g., first information about the bias of the utterances).

[0114] refer to Figure 7 Examples of first information that can identify bias in a discourse. Information 701 about bias can correspond to discourse 702. Categories of bias can include demographic bias (e.g., racial bias, gender bias, age bias, and national bias), malice (e.g., potentially inappropriate or offensive words, phrases, and sentences), privacy protection (e.g., privacy issues), robustness, fairness (e.g., equal opportunity and equal treatment for different groups), sensitive topics, controversial content and model vulnerabilities (e.g., aggressive input to a voice assistant model), user experience, and others. (See above for reference.) Figure 5 According to embodiments of this disclosure, the term "bias" is not limited to its dictionary meaning. In other words, bias information can correspond to information about targets that need to be filtered but have not been filtered. According to embodiments of this disclosure, the term "bias" can encompass representations equivalent to those described above.

[0115] According to yet another embodiment, discourse 702 may include demographic bias (e.g., national bias) and sensitive topic (e.g., rogue issue). Information 701 regarding bias may include the degree of bias for each bias category (e.g., demographic bias "0.6" and sensitive topic "0.2").

[0116] According to another embodiment, the bias checking module 521-2 can invoke user-defined values ​​(e.g., the analysis results of the user's bias (e.g., second information about the user's bias)). The second information about the user's bias may correspond to a value set by the user before or after receiving a utterance. The user's settings associated with the bias may include settings for the bias category (e.g., see...). Figure 7 Settings for bias and / or the degree of bias (e.g., a value between "0" and "1"). User settings associated with bias may include settings for the direction of the bias (e.g., indicating whether the user is from country A or country B). If the user sets electronic device 501 to child mode, electronic device 501 may be configured not to provide a response including bias information. Secondary information about the user's bias may also include user input obtained during the discourse (e.g., "Please allow future conversations to be biased towards politics").

[0117] According to yet another embodiment, electronic device 501 may use prompt generation module 522 to generate prompts from first information (e.g., first information about the bias of the discourse) and / or second information (e.g., second information about the user's bias). Prompts can be used to reduce the amount of time consumed in retraining and fine-tuning large-scale language models (e.g., general language model 531 and / or biased language model 532).

[0118] According to another embodiment, electronic device 501 can generate a response including bias information from prompts based on a general language model 531 and / or a biased language model 532. The biased language model 532 can be trained based on a corpus labeled with bias categories and degrees of bias. Electronic device 501 can preprocess the corpus used to train the biased language model 532. Electronic device 501 can label the bias categories and degrees of bias using the corpus. Electronic device 501 can train the biased language model 532 based on the corpus labeled with bias categories and degrees of bias. The corpus used to train the biased language model 532 can be substantially similar to the corpus used to train the bias checking module 521-1. The biased language model 532 can generate a response including bias information. Figure 6 In this embodiment, the general language model 531 and the biased language model 532 are represented as separate models; however, the embodiments are not limited thereto. The general language model 531 and the biased language model 532 may also be implemented as a single model.

[0119] According to yet another embodiment, electronic device 501 can generate information about the bias of a response (e.g., a response including bias information). The information about the bias of the response can be generated by bias checking modules 521-3. The information about the bias of the response can be configured in a format similar to the format of the first information about the bias of the utterance (e.g., see...). Figure 7 Information about the bias in the response can be displayed visually (e.g., see...). Figure 8 Information about the bias of the response (512). A bias checking module 521-3 can be trained to generate information about the bias of the response based on a corpus labeled with bias categories and bias degrees. The corpus used to train the bias checking module 521-3 can be substantially similar to the corpus used to train the bias checking module 521-1 and / or the corpus used to train the biased language model 532.

[0120] According to another embodiment, bias checking modules 521-1 and 521-3 can distinguish between profanity and criminal content, as well as various types of bias. A typical bias checking module can perform operations to remove bias from responses generated by a language model (e.g., debiased corpus). Bias checking modules 521-1 and 521-3 can be used to collect biased corpora (e.g., corpora labeled with bias categories and degrees) and utilize biased corpora.

[0121] According to another embodiment, electronic device 501 can provide a user with a response including bias information and information about the bias of the response. The degree of bias of the information to be included in the response, or the bias of the information to be included in the response, can be changed based on the user's settings (e.g., the results of utterance bias analysis) or the results of utterance bias analysis.

[0122] According to another embodiment, if a user desires a biased conversation on a predetermined topic, electronic device 501 can organize opinions for and against the predetermined topic or "N" sub-topics. In this process, argumentative essay evaluation techniques can be utilized. Electronic device 501 can assess the clarity of arguments and evidence. For example, if two locations on a topic are analyzed, electronic device 501 can mark representative documents most frequently cited or viewed for each location as references and can briefly describe these representative documents. The user can select one of the two locations and engage in a conversation. Based on the user-selected location, electronic device 501 can generate responses to arguments and evidence based on collected documents. Electronic device 501 can provide links via a UI for accessing references that form the basis of the aforementioned conversation and can organize and display the conversation content in a summarized form.

[0123] Figures 8 to 12 This is a diagram illustrating examples of how an electronic device processes user speech according to various embodiments of the present disclosure.

[0124] refer to Figure 8 In scenario 801, electronic device 800 may receive utterances from the user (e.g., “Do you think we (country A) will win the match between country A and country B?”). The utterances (e.g., “Do you think we (country A) will win the match between country A and country B?”) may be biased (e.g., “we (country A)” corresponding to the user’s nationality). Electronic device 800, designed to eliminate bias, may inevitably provide the user with only typical responses corresponding to biased utterances (e.g., “Both countries are very interested in the match between country A and country B. Nobody knows who will win.”).

[0125] According to another embodiment, in scenario 802, electronic device 501 can receive utterances from a user (e.g., “Do you think we (country A) will win the match between country A and country B?”). The utterances (e.g., “Do you think we (country A) will win the match between country A and country B?”) can be biased (e.g., “we (country A)” corresponding to the user’s nationality). Electronic device 501 can generate a response including biased information (e.g., “Of course, I think we (country A) will win”) based on the utterances. Electronic device 501 can provide the user with a response 511 including biased information (e.g., “Of course, I think we (country A) will win”). Electronic device 501 can also provide the user with information 512 about the bias of the response when providing response 511. The information 512 about the bias of the response can be visually displayed on screen 510. Electronic device 501 can enhance the user experience in terms of diversity.

[0126] refer to Figure 9 Electronic device 501 can receive utterances from a user (e.g., “Who do you think will win the match between country A and country B?”). Before receiving the utterances, the user can preset settings associated with their bias. These bias-associated user settings can include settings for bias categories (e.g., see...). Figure 7 The settings for bias can include the degree of bias (e.g., a value between "0" and "1"). User settings associated with bias can include settings for the direction of the bias (e.g., indicating whether the user is from country A or country B within the country category). User settings associated with bias can be executed even after a utterance. For example, in response to receiving a utterance (e.g., "Who do you think will win the match between country A and country B?"), electronic device 501 can provide a UI (e.g., a bias setting UI (e.g., bias setting UI 921)) to allow the user to select the direction of bias (e.g., country A or country B within the country category) via screen 510. Electronic device 501 can receive touch input from the user via screen 510 (e.g., a display). Subsequent responses of electronic device 501 can vary based on the user's touch input.

[0127] According to another embodiment, in scenario 901, electronic device 501 can receive touch input from a user who has selected country A using the bias setting UI 921. Electronic device 501 can generate a response 910 including information biased towards country A (e.g., "Country A is leading in the recent score. I think country A will win"). Electronic device 501 can provide the user with the response 910 including information biased towards country A (e.g., "Country A is leading in the recent score. I think country A will win"). When providing response 910, electronic device 501 can also provide the user with information 911 regarding the bias of response 910 (e.g., the flag of country A). The information 911 regarding the bias of response 910 can be visually displayed on screen 510.

[0128] According to another embodiment, in scenario 902, electronic device 501 can receive touch input from a user who has selected country B using the bias setting UI 921. Electronic device 501 can generate a response including information 913 biased towards country B (e.g., "Country B is the strongest footballing nation in Asia. I think country B will win"). Electronic device 501 can provide the user with information 913 biased towards country B (e.g., "Country B is the strongest footballing nation in Asia. I think country B will win"). When providing response 913, electronic device 501 can also provide the user with information 914 regarding the bias of response 913 (e.g., the flag of country B). The information 914 regarding the bias of response 913 can be visually displayed on screen 510. Information 911 and information 914 regarding the bias of responses 910 and 913 can be provided to the user together, allowing the user to perceive that the response of electronic device 501 is biased. Electronic device 501 can ensure the objectivity of the response.

[0129] refer to Figure 10 Electronic device 501 can receive utterances from a user (e.g., "We (Country A) will win the match between Country A and Country B. Tell me from both teams' perspectives"). The utterances (e.g., "We (Country A) will win the match between Country A and Country B. Tell me from both teams' perspectives") can be utterances from which the user expects responses to be received from various perspectives. Electronic device 501 can generate multiple responses 1001, 1002, and 1003 with different biases based on the utterances. Responses 1001 and 1003 can be biased towards Country A. Response 1002 can be biased towards Country B. The multiple responses 1001, 1002, and 1003 can be configured to enable mutual dialogue. The multiple responses 1001, 1002, and 1003 can be presented to the user via screen 510.

[0130] refer to Figure 11 Electronic device 501 can receive utterances from a user (e.g., "I think country A is the best country in the world"). The utterances (e.g., "I think country A is the best country in the world") can include biased statements. Electronic device 501 can generate multiple responses 1101, 1102, and 1103 with different biases based on the utterances. These multiple responses 1101, 1102, and 1103 can be presented to the user via screen 510. The multiple responses 1101, 1102, and 1103 can include three or more responses. Electronic device 501 can enhance the user experience in terms of diversity.

[0131] refer to Figure 12Electronic device 501 can provide users with responses biased to varying degrees based on the analysis results of the discourse bias. In scenario 1201, electronic device 501 can receive (e.g., compared to scenario 1202) a relatively less biased discourse (“Are we (Country A) not good at football in East Asia?”). In scenario 1201, electronic device 501 can provide users with (e.g., compared to scenario 1202) a relatively less biased response (e.g., “I think we (Country A) are one of the best football teams in East Asia”). In scenario 1202, electronic device 501 can receive (e.g., compared to scenario 1201) a relatively more biased discourse (“We (Country A) are the best football team in East Asia. Aren’t we?”). In scenario 1202, electronic device 501 can provide users with (e.g., compared to scenario 1201) a relatively more biased response (e.g., “Of course, we (Country A) are the best football team in East Asia!”).

[0132] Figure 13 This is a flowchart illustrating a method of operating an electronic device according to an embodiment of the present disclosure.

[0133] Operations 1310 through 1340 may be executed sequentially, but not necessarily sequentially. For example, the order of operations 1310 through 1340 can be changed, and at least two of 1310 through 1340 can be executed in parallel.

[0134] According to yet another embodiment, it can be understood that operations 1310 to 1340 can be performed by an electronic device (e.g., Figure 6 The processor of the electronic device 501 (e.g., Figure 6 The processor 520 executes the commands.

[0135] In operation 1310, the electronic device according to the embodiment (e.g., Figure 5 The electronic device 501 can receive the user's words.

[0136] In action 1320, the electronic device 501 according to the embodiment can generate a response including bias information based on the speech.

[0137] In operation 1330, the electronic device 501 according to the embodiment can generate information about the bias of the response based on the response.

[0138] In operation 1340, the electronic device 501 according to the embodiment can provide the user with information about the bias and the response together.

[0139] An electronic device according to an embodiment (e.g., Figure 1 Electronic devices 101 Figure 2 Electronic devices 201 and Figure 5The electronic device 501 may include a memory storing one or more computer programs (e.g., Figure 1 Memory 130, Figure 2 memory 207 and Figure 6 The memory 530). The electronic device may include one or more processors (e.g., memory 530) communicatively coupled to the memory. Figure 1 Processor 120 Figure 2 processor 203 and Figure 6 (Processor 520). One or more computer programs may include computer-executable instructions. When executed by one or more processors, the computer-executable instructions may enable the electronic device to receive user speech. When executed by the processor, the computer-executable instructions may enable the electronic device to analyze the bias of the speech or the user's bias. When executed by one or more processors, the computer-executable instructions may enable the electronic device to generate a response including bias information based on the analysis results of the speech bias or the user bias. When executed by one or more processors, the computer-executable instructions may enable the electronic device to provide a response to the user.

[0140] According to yet another embodiment, the analysis results of discourse bias may include at least one of discourse bias category or discourse bias degree.

[0141] According to another embodiment, the analysis results of a user's bias can correspond to the user's settings associated with that bias. These settings can be obtained based on a bias setting UI displayed on the electronic device.

[0142] According to yet another embodiment, in order to analyze a user's bias, one or more computer programs may further include computer-executable instructions. When executed by one or more processors, the computer-executable instructions can cause an electronic device to determine multiple bias directions based on utterance. When executed by one or more processors, the computer-executable instructions can cause the electronic device to provide a bias setting user interface to the user, including multiple bias directions. When executed by one or more processors, the computer-executable instructions can cause the electronic device to receive a bias setting based on a bias direction selected from the multiple bias directions.

[0143] According to yet another embodiment, in order to provide a response to a user, one or more computer programs may further include computer-executable instructions. When executed by one or more processors, the computer-executable instructions can cause the electronic device to visually display the bias setting along with the response.

[0144] According to yet another embodiment, one or more computer programs may further include computer-executable instructions. When executed by one or more processors, the computer-executable instructions can cause an electronic device to generate biased information about a response based on that response. When executed by a processor, the computer-executable instructions can cause the electronic device to visually display the biased information about the response.

[0145] According to another embodiment, the degree of bias of the information to be included in the response, or the bias of the information to be included in the response, can be changed based on the analysis results of the utterance bias or the analysis results of the user's bias.

[0146] An electronic device according to an embodiment (e.g., Figure 1 Electronic devices 101 Figure 2 Electronic devices 201 and Figure 5 The electronic device 501 may include a memory storing one or more computer programs (e.g., Figure 1 Memory 130, Figure 2 memory 207 and Figure 6 The memory 530). The electronic device may include one or more processors (e.g., memory 530) communicatively coupled to the memory. Figure 1 Processor 120 Figure 2 processor 203 and Figure 6 (Processor 520). One or more computer programs may include computer-executable instructions. When executed by one or more processors, the computer-executable instructions may enable an electronic device to receive a user's speech. When executed by one or more processors, the computer-executable instructions may enable an electronic device to generate a response, including bias information, based on the speech. When executed by one or more processors, the computer-executable instructions may enable an electronic device to generate bias information about the response based on the response. When executed by one or more processors, the computer-executable instructions may enable an electronic device to provide the response and the bias information to the user.

[0147] According to yet another embodiment, the information about bias may include at least one of the bias category of the response or the degree of bias of the response.

[0148] According to yet another embodiment, it can be based on a general language model (e.g., Figure 6 The general language model (531) and the biased language model (e.g., Figure 6 The biased language model (532) is used to generate responses.

[0149] According to yet another embodiment, a biased language model can be trained based on a corpus labeled with bias categories and bias degrees.

[0150] According to yet another embodiment, the degree of bias of the information to be included in the response, or the bias of the information to be included in the response, can be changed based on the user's settings or the analysis results of the bias of the utterance.

[0151] According to yet another embodiment, user settings can be obtained based on a bias setting UI displayed by an electronic device.

[0152] According to yet another embodiment, the response may include multiple responses with different biases.

[0153] According to yet another embodiment, information about the bias can be displayed visually.

[0154] According to yet another embodiment, one or more computer programs may further include computer-executable instructions. When executed by one or more processors, these computer-executable instructions can enable an electronic device to provide information about the biases of the entire session, including utterances and responses.

[0155] An electronic device according to an embodiment (e.g., Figure 1 Electronic devices 101 Figure 2 Electronic devices 201 and Figure 5 The electronic device 501 may include a memory storing one or more computer programs (e.g., Figure 1 Memory 130, Figure 2 memory 207 and Figure 6 The memory 530). The electronic device may include one or more processors (e.g., memory 530) communicatively coupled to the memory. Figure 1 Processor 120 Figure 2 processor 203 and Figure 6 The processor 520. One or more computer programs include computer-executable instructions. When executed by one or more processors, the computer-executable instructions can cause the electronic device to receive a user's speech. When executed by one or more processors, the computer-executable instructions can cause the electronic device to obtain first information about the bias of the speech or second information about the user's bias. When executed by one or more processors, the computer-executable instructions can cause the electronic device to generate a prompt based on the first or second information. When executed by one or more processors, the computer-executable instructions can cause the electronic device to generate a response including bias information from the prompt based on a general language model or a biased language model. When executed by one or more processors, the computer-executable instructions can cause the electronic device to provide a response to the user.

[0156] According to yet another embodiment, the first information may include at least one of the bias category of the discourse or the degree of bias of the discourse.

[0157] According to yet another embodiment, the second information may correspond to the settings of a user associated with a bias.

[0158] According to yet another embodiment, a biased language model can be trained based on a corpus labeled with bias categories and bias degrees.

[0159] According to yet another embodiment, one or more computer programs may further include computer-executable instructions. When executed by a processor, the computer-executable instructions can cause the electronic device to generate biased information about the response based on the response. When executed by one or more processors, the computer-executable instructions can cause the electronic device to provide the response and the biased information together to a user.

[0160] According to yet another embodiment, the degree of bias of the information to be included in the response, or the bias of the information to be included in the response, can be changed based on the user's settings or the analysis results of the bias of the utterance.

[0161] According to another aspect of this disclosure, one or more non-transitory computer-readable storage media are provided for storing one or more computer programs including computer-executable instructions, which are executed by an electronic device (e.g., Figure 1 Electronic devices 101 Figure 2 Electronic devices 201 and Figure 5 One or more processors (e.g., electronic device 501) Figure 1 Processor 120 Figure 2 processor 203 and Figure 6 When the processor 520 executes, it causes the electronic device to perform operations. These operations include receiving user speech; analyzing the bias of the speech or the user's bias; generating a response including bias information based on the analysis results of the speech bias or the user bias; and providing the response to the user.

[0162] The electronic device according to various embodiments can be one of a variety of types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0163] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the particular embodiments, but rather to include various changes, equivalents, or substitutions for the respective embodiments. In conjunction with the description of the accompanying drawings, the same reference numerals may be used for similar or related components. As used herein, “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one of the items listed together in a corresponding phrase, or all possible combinations thereof. Terms such as “first” and “second” or “first” and “second” may be used simply to distinguish the respective components from other components and do not limit the components in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if a component (e.g., a first component) is referred to as “coupled to another component (e.g., a second component),” “coupled to another component (e.g., a second component),” “connected to another component (e.g., a second component),” or “connected to another component (e.g., a second component)”, then the component may be directly (e.g., via a wire), wirelessly coupled to the other component, or coupled to the other component via a third component.

[0164] As used in conjunction with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "component," or "circuit." A module may be a single integrated component adapted to perform one or more functions, or its smallest unit or portion. For example, according to an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0165] The various embodiments set forth herein can be implemented as software (e.g., program 140) including one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke and execute at least one of the one or more instructions stored in the storage medium. This allows the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored in the storage medium.

[0166] According to yet another embodiment, methods according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TM The computer program product may be distributed online (e.g., downloaded or uploaded) or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as the memory of a manufacturer's server, an app store's server, or a relay server.

[0167] According to various embodiments, each of the above components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be arranged separately in different components. According to various embodiments, one or more of the above components or operations may be omitted, or one or more other components or operations may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as the corresponding components in the multiple components before integration. According to various embodiments, operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more operations may be performed in a different order or omitted, or one or more other operations may be added.

[0168] While this disclosure has been shown and described with reference to various embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims and their equivalents.

Claims

1. An electronic device (101; 201; 501), including: Memory (130; 207; 530) for storing one or more computer programs; and One or more processors (120; 203; 530) are communicatively coupled to the memory (130; 207; 530). Wherein, the one or more computer programs include computer-executable instructions that, when executed by the one or more processors (120; 203; 530), cause the electronic device (101; 201; 501): Receive user input. Analyze the bias of the discourse or the bias of the user. Based on the analysis results of the discourse bias or the user bias, a response including bias information is generated. The response is provided to the user.

2. The electronic device (101; 201; 501) according to claim 1, wherein, The analysis results of the bias of the discourse include at least one of the bias category of the discourse or the bias degree of the discourse.

3. The electronic device (101; 201; 501) according to any one of claims 1 to 2. in, The analysis results of the user's bias correspond to the user's settings associated with the bias, and The settings are obtained based on a bias setting user interface displayed by the electronic device.

4. The electronic device (101; 201; 501) according to any one of claims 1 to 3, wherein, To analyze the user's preferences, the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors (120; 203; 530), cause the electronic device (101; 201; 501) to: Multiple bias directions are determined based on the aforementioned discourse. Provide the user with a bias setting user interface that includes the plurality of bias directions, and Receive a bias setting based on the bias direction selected from the plurality of bias directions.

5. The electronic device (101; 201; 501) according to any one of claims 1 to 4, wherein, In order to provide the response to the user, the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors (120; 203; 530), cause the electronic device (101; 201; 501) to: The bias setting is visually displayed along with the response.

6. The electronic device (101; 201; 501) according to any one of claims 1 to 5, wherein, The one or more computer programs further include computer-executable instructions that, when executed by the one or more processors (120; 203; 530), cause the electronic device (101; 201; 501) to: Based on the response, information about the bias of the response is generated, and Information about the bias of the response is displayed visually.

7. The electronic device (101; 201; 501) according to any one of claims 1 to 6, wherein, The degree of bias in the information to be included in the response, or the bias of the information to be included in the response, may change based on the analysis results of the bias of the discourse or the analysis results of the user's bias.

8. An electronic device (101; 201; 501), including: Memory (130; 207; 530) for storing one or more computer programs; and One or more processors (120; 203; 530) are communicatively coupled to the memory (130; 207; 530). Wherein, the one or more computer programs include computer-executable instructions that, when executed by the one or more processors (120; 203; 530), cause the electronic device (101; 201; 501): Receive user input. Generate a response that includes biased information based on the discourse. Based on the response, information about the bias of the response is generated, and The response and information about the bias are provided to the user.

9. The electronic device (101; 201; 501) according to claim 8, wherein, Information regarding the bias includes at least one of the bias category of the response or the degree of bias of the response.

10. The electronic device (101; 201; 501) according to any one of claims 8 to 9, wherein, The response is generated based on a general language model and a biased language model.

11. The electronic device (101; 201; 501) according to any one of claims 8 to 10, wherein, The biased language model is trained on a corpus labeled with bias categories and bias degrees.

12. The electronic device (101; 201; 501) according to any one of claims 8 to 11, wherein, The degree of bias in the information to be included in the response, or the bias of the information to be included in the response, changes based on the user's settings or the analysis results of the bias of the utterance.

13. The electronic device (101; 201; 501) according to any one of claims 8 to 12, wherein, The user's settings are obtained based on a biased settings user interface displayed by the electronic device (101; 201; 501).

14. The electronic device (101; 201; 501) according to any one of claims 8 to 13, wherein, The response includes multiple responses with different biases.

15. The electronic device (101; 201; 501) according to any one of claims 8 to 14, wherein, Information about the bias is displayed visually.

Images