Electronic device, method, and non-transitory computer-readable recording medium for generating content
By integrating a generative AI model for text generation with customizable attributes, electronic devices can create content with enhanced user interaction and versatility, addressing limitations in existing content generation methods.
Patent Information
- Application Number
- PCT/KR2025/006803
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-09
- Filing Date
- 2025-05-19
- Publication Date
- 2026-01-15
AI Technical Summary
Existing electronic devices lack efficient methods for generating content with customizable attributes such as language, tone, and format, limiting user interaction and content generation capabilities.
Incorporating a generative artificial intelligence model, such as a large language model, to process user inputs and generate text data based on attribute data, allowing for customizable content creation through an input interface with separate areas for text input and attribute setting.
Enables dynamic and user-friendly content generation with customizable attributes, enhancing user interaction and improving the versatility of content creation on electronic devices.
Smart Images

Figure KR2025006803_15012026_PF_FP_ABST
Abstract
Description
Electronic device, method, and non-transitory computer-readable recording medium for generating content
[0001] The descriptions below relate to electronic devices, methods, and non-transitory computer-readable recording media for generating content.
[0002] Electronic devices can provide various types of content (e.g., text, images). These contents can be received from other electronic devices, or they can be generated by the electronic device and then provided to the user or transmitted to another electronic device. For example, a user can write text using an electronic device. The electronic device can display sentences corresponding to user input received through an input interface (e.g., keyboard, pen input, touch input).
[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.
[0004] In embodiments, an electronic device is provided. The electronic device includes a display; at least one processor; and a memory storing instructions, wherein the instructions, when executed by the at least one processor, collectively or individually cause the electronic device to display a user interface including a text input area through the display while an application is executed, display an input interface through the display in response to a user input while the user interface is displayed, generate text data through the input interface, obtain at least one sentence generated based on the text data and attribute data related to the sentence to be generated, and display the at least one sentence in the text input area through the display. The attribute data may be set based on the application.
[0005] In embodiments, the electronic device may include at least one processor; and a memory storing instructions. The instructions, when executed by the at least one processor, may collectively or individually cause the electronic device to display a user interface including a text input area through a touch screen, and, based on at least a portion of a user input related to the text input area, display an input interface including a first area corresponding to a keyboard and a second area for setting one or more properties for text input into the text input area, receive at least one word from a user through the first area, obtain at least one sentence generated using the at least one word and the one or more properties, and display the at least one sentence in the text input area.
[0006] In embodiments, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium may include a memory storing instructions. The instructions, when executed by at least one processor, may collectively or individually cause an electronic device to display a user interface including a text input area while an application is running on the electronic device, display the input interface in response to a user input while the user interface is displayed, generate text data through the input interface, obtain at least one sentence generated based on the text data and attribute data associated with the sentence to be generated, and display the at least one sentence in the text input area. The attribute data may be set based on the application.
[0007] In embodiments, a method performed by an electronic device is provided. The method may include: displaying a user interface including a text input area while an application is running on the electronic device; displaying an input interface in response to a user input while the user interface is displayed; generating text data through the input interface; obtaining at least one sentence generated based on the text data and attribute data related to the sentence to be generated; and displaying the at least one sentence in the text input area. The attribute data may be set based on the application.
[0008] Figure 1 is a block diagram of an electronic device within a network environment.
[0009] Figure 2 illustrates an example of functional components for generating content through an input interface on an application.
[0010] Figure 3 illustrates an operation flow of an electronic device for displaying content in a text input area.
[0011] Figures 4a to 4f illustrate examples of user interfaces for generating at least one content.
[0012] Figures 5a and 5b illustrate examples of a user interface for entering at least one content into a text area.
[0013] Figure 6 shows an example of functional components for generating content using a context extraction module.
[0014] Figure 7 illustrates an operation flow of an electronic device for displaying content in a text input area based on an application.
[0015] Figure 8 shows examples of properties related to content to be generated.
[0016] Figure 9 illustrates examples of functional components for generating content using multiple artificial intelligence models.
[0017] Figures 10a to 10e illustrate examples of user interfaces for generating content based on images.
[0018] Figure 11 shows an example of a user interface for setting the length of content to be generated.
[0019] Figures 12a to 12c illustrate examples of user interfaces for creating content in a shopping service.
[0020] Figures 13a to 13c illustrate examples of user interfaces for generating content based on alternative information.
[0021] Figure 14 is a schematic diagram of an exemplary AI system.
[0022] The terms used in this disclosure are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this disclosure. Terms defined in general dictionaries among the terms used in this disclosure may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this disclosure. In some cases, even if a term is defined in this disclosure, it cannot be interpreted to exclude embodiments of the present disclosure.
[0023] The various embodiments of the present disclosure described below illustrate a hardware-based approach as an example. However, since the various embodiments of the present disclosure include techniques utilizing both hardware and software, the various embodiments of the present disclosure do not exclude a software-based approach.
[0024] In the following description, terms referring to input data (e.g., signal, information, input data, text, text data, input text, input object), terms referring to characteristics (e.g., attribute, feature, type, attribute data, feature data, type data, attribute information, feature information, type information), information for indicating input to an artificial intelligence model (e.g., prompt, candidate prompt, prompt text, prompt part, prompt area, prompt target, candidate prompt part, prompt object, attribute indicator, attribute indicator information, attribute input information), terms referring to components of a device, etc. are examples for convenience of description. Therefore, the present disclosure is not limited to the terms described below, and other terms having equivalent technical meanings may be used. In addition, terms such as '... part', '... machine', '... thing', '... body', etc. used below may mean at least one shape structure or a unit that processes a function.
[0025] In addition, in the present disclosure, expressions such as "more than" or "less than" may be used to determine whether a specific condition is satisfied or fulfilled, but this is merely a description for expressing an example and does not exclude descriptions such as "more than" or "less than." A condition described as "more than" may be replaced with "more than," a condition described as "less than" may be replaced with "less than," and a condition described as "more than and less than" may be replaced with "more than and less than." In addition, hereinafter, "A" to "B" mean at least one of elements from A (including A) to B (including B). hereinafter, "C" and / or "D" mean at least one of "C" or "D," that is, including {"C", "D", "C" and "D"}.
[0026] Figure 1 is a block diagram of an electronic device within a network environment.
[0027] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).
[0028] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (120) may store a command or data received from another component (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the command or data stored in the volatile memory (132), and store the resulting data in a non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or a secondary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor)) that can operate independently or together therewith. For example, if the electronic device (101) includes a main processor (121) and a secondary processor (123), the secondary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a specified function. The secondary processor (123) may be implemented separately from the main processor (121) or as a part thereof.
[0029] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.
[0030] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto. The memory (130) can include volatile memory (132) or non-volatile memory (134).
[0031] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).
[0032] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).
[0033] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.
[0034] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.
[0035] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).
[0036] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.
[0037] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.
[0038] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).
[0039] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.
[0040] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.
[0041] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).
[0042] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.
[0043] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).
[0044] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.
[0045] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).
[0046] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.
[0047] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).
[0048] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.
[0049] Figure 2 illustrates examples of functional components for generating content (e.g., sentences, images, videos, audio, and / or combinations thereof) via an input interface on an application.
[0050] Referring to FIG. 2, an electronic device (101) (e.g., a processor (120)) can execute an application (210). While the application (210) is executed, the electronic device (101) can utilize an input interface (220). For example, the electronic device (101) can display a user interface of the application (210). The user interface can include a text input area (e.g., an address window of an Internet browser application). The electronic device (101) can receive a user input (e.g., a touch input) for the text input area. The electronic device (101) can call the input interface (220) in response to the user input. The input interface (220) can represent a means for obtaining a user input (e.g., a keyboard interface, a voice recognition module). The electronic device (101) can receive a user input through the input interface (220). The electronic device (101) can generate text data corresponding to the user input.
[0051] An electronic device (101) according to embodiments of the present disclosure can write text based on an artificial intelligence model (230). The artificial intelligence model (230) may represent a generative artificial intelligence model. For example, the artificial intelligence model (230) may be a large language model (LLM). For example, the artificial intelligence model (230) may be a large multimodal model (LMM). The artificial intelligence model (230) used in the present disclosure may include a machine learning model trained to determine the properties (e.g., content, format, tone, length, language) of a recommended sentence based on a user's usage pattern. For example, the artificial intelligence model (230) may be trained or learned to associate context information (e.g., a specific application, conversation context, word, situation) with at least one specific sentence through training data (e.g., a user's selection history, correction history, sharing history, deletion history) for a sentence to be generated. In the electronic device (101), user operations related to the functions of the present disclosure (e.g., functions repeatedly performed in a consistent order) may be monitored to train an artificial intelligence model (230). The artificial intelligence model (230) may include various transformer models.
[0052] The artificial intelligence model (230) used in the embodiments of the present disclosure may include a learning and inference process that finds patterns in data, stores them as a model, which is a generalized rule, and inputs new data into the learned model to obtain results. The learning of the artificial intelligence model (230) may correspond to initial learning or may correspond to re-learning. For example, the learning process may include a forward propagation method and / or a backward propagation method. As a non-limiting example, algorithms such as regression, decision trees, neural networks, and k-nearest neighbors may be used for learning. Multiple different machine learning models may be used depending on the target task. Hereinafter, LLM is exemplified as the artificial intelligence model (230), but the artificial intelligence model (230) for the embodiments of the present disclosure may include not only a language model, but also various foundation models such as a code model and an image model, and / or separate artificial intelligence neural network models.
[0053] The LLM mentioned in this disclosure may refer to an artificial neural network-based language model that has learned a large amount of text data through pre-training. The LLM may include relatively more parameters (e.g., more than 10 billion) than existing general language models. The LLM may use a transformer artificial neural network structure based on an attention mechanism. The attention mechanism is a technology that helps an artificial intelligence model focus (attention) on important parts of input data. The attention mechanism can be utilized to predict output data by predicting the degree to which a portion of time-series input data (e.g., input data such as voice or video, or input data of some layers of a neural network) contributes to the intermediate or final output of the neural network. A recurrent neural network (RNN) structure that sequentially processes each element of a sequence has poor prediction performance when there is information dependency between long time series distances, but an attention mechanism can consider information dependency between long time series distances by controlling the degree of weight concentration (attention) within the context of the entire (or part of) input data. A transformer may be configured as an encoder-decoder structure. The encoder processes input data to output compressed information (e.g., contextual representation), and the decoder may process the compressed information to output output data in token units. Each of the encoder and the decoder may include an independent attention network. The transformer may include a cross-attention network that connects the encoder and the decoder.
[0054] LLM training may involve pre-training and / or fine-tuning. Pre-training involves training the LLM to acquire general linguistic knowledge using large amounts of text data. For example, this may involve self-supervised learning, where the LLM predicts the next word based on the previous word sequence in a text string. Fine-tuning involves training the LLM to be suitable for a specific domain (e.g., chatbot, translation, summarization, question and answer (Q&A)) or task. The LLM may undergo additional supervised learning (or adaptive learning) based on the pre-trained model using a dataset tailored to the domain's purpose. The LLM may perform tasks based on text input containing natural language, called prompts. As a non-limiting example, fine-tuning may be omitted during LLM training. The prompts provided to the LLM may be controlled to enhance the performance of the user's desired task. Prompts can be supplemented with examples of tasks and / or guidance on how to perform them, similar to in-context learning or zero-shot / few-shot learning. Publicly available LLMs include BERT (Bidirectional Encoder Representations from Transformer) and GPT (generative pre-trained transformer).
[0055] The artificial intelligence model (230) according to embodiments of the present disclosure may represent not only the LLM itself, i.e., the language neural network model, but also a model of an LLM-based application (e.g., chatbot, translation, summary, text classification, sentence generation). For example, an LLM-based chatbot such as Chat GPT or an LLM-based translator may also be referred to as the artificial intelligence model (230). The artificial intelligence model (230) according to embodiments of the present disclosure may include an inference engine using the LLM neural network model. For example, “inputting an input prompt to the artificial intelligence model (230)” may indicate “inputting the input prompt to the inference engine based on the artificial intelligence model (230).” For example, “output of the artificial intelligence model (230) for the input prompt” may indicate the output information (or output information modified through additional processing) of the last neural network layer of the artificial intelligence model (230) obtained when the input prompt is input to the inference engine based on the artificial intelligence model (230).
[0056] An electronic device (101) according to embodiments of the present disclosure can write text using an artificial intelligence model (230). The electronic device (101) can call an input interface (220) while an application (210) is running. The electronic device (101) can receive user input through the input interface (220). The electronic device (101) can generate text data corresponding to the user input. The electronic device (101) can set attribute data through the input interface (220). The attribute data can indicate attributes related to sentences to be generated. For example, the attributes can indicate the language of the sentences to be generated (e.g., English, Spanish, Korean, Chinese), the length of the sentences to be generated (or the number of sentences to be generated), the tone of the sentences to be generated, and / or the format of the sentences to be generated (e.g., email, SNS, comment). For example, the input interface (220) can be a soft keyboard displayed on a display. The input interface (220) may include a first area (e.g., a keyboard interface, a handwriting interface) for obtaining text input by a user and a second area (e.g., an icon, a button, a field) for specifying attributes of a sentence to be generated. The first area may be used to generate text data. The second area may be used to set attribute data. For example, the second area may be used to display a setting interface for changing at least one attribute. As another example, the input interface (220) may be a voice recognition module for obtaining a user's voice. The electronic device (101) may obtain text data corresponding to the user's voice through the voice recognition module (e.g., an automatic speech recognition (ASR) module). The electronic device (101) may obtain attribute data corresponding to the user's command through the voice recognition module.Hereinafter, in the present disclosure, a soft keyboard of the keyboard type displayed on the display is exemplified as an example of an input interface (220), but any input means for specifying properties can be understood as an embodiment of the present disclosure.
[0057] According to one embodiment, one or more properties of the attribute data may be set based on information (211) provided from the application (210) (e.g., properties of the application, screens and / or content displayed in the application).
[0058] The electronic device (101) may obtain at least one sentence generated based on the attribute data and the text data. According to one embodiment, the electronic device (101) may generate request information (221) based on data obtained through the input interface (220). The electronic device (101) may transmit the request information (221) to an artificial intelligence model (230). The request information (221) may be used to request the artificial intelligence model (230) to generate at least one sentence based on the attribute data and the text data. For example, the request information (221) may include the attribute data and the text data. For example, the request information (221) may include a prompt generated based on the attribute data and the text data. For example, the prompt may include a text prompt composed of text input by a user and text for describing an attribute. For example, the prompt may include a text prompt for displaying a current screen. The artificial intelligence model (230) can generate at least one sentence based on the attribute data and the text data. The electronic device (101) can obtain the at least one sentence, which is an output of the artificial intelligence model (230), through the input interface (220). In the present disclosure, inputting the attribute data and the text data to the artificial intelligence model (230) may include not only inputting the attribute data and the text data to the artificial intelligence model (230), but also inputting a prompt generated based on the attribute data and the text data to the artificial intelligence model (230). In the present disclosure, the prompt may include a type of data other than text, such as a sentence. In one embodiment, the prompt may be a multi-modal prompt. For example, the prompt may include an image prompt for the current screen.For example, the prompt may include a prompt for multimedia (e.g., images, video, audio) being provided through the application or used by a user of the application.
[0059] The artificial intelligence model (230) can output at least one sentence. Information (e.g., a prompt) based on the attribute data and the text data can be input to the artificial intelligence model (230). For example, the artificial intelligence model (230) can be located outside the electronic device (101). The artificial intelligence model (230) can be included in an external electronic device (e.g., the electronic device (102), the electronic device (104), the server (108)). The electronic device (101) can provide a request message including information based on the attribute data and the text data to the external electronic device. As another example, the artificial intelligence model (230) can be located inside the electronic device (101). The artificial intelligence model (230) can be located in a storage space of the electronic device (101). In terms of being located inside the electronic device (101), the artificial intelligence model (230) can be referred to as an on-device AI model. The electronic device (101) can input the attribute data and the text data acquired through the input interface (220) into the artificial intelligence model (230), or input information edited through the attribute data and the text data into the artificial intelligence model (230). The artificial intelligence model (230) can provide response information (231) including at least one sentence to the input interface (220). The electronic device (101) can acquire the at least one sentence through the input interface (220). For example, the electronic device (101) can receive a response message corresponding to the request message from an external electronic device. The response message can include information about at least one sentence generated based on the attribute data and the text data. As another example, the electronic device (101) can acquire at least one sentence corresponding to the output of the internal artificial intelligence model (230).The electronic device (101) can display at least one sentence (241) obtained through the input interface (220) on the user interface of the application (210) according to a user input. For example, the electronic device (101) can display at least one sentence (241) in the text input area of the user interface.
[0060] An electronic device (101) according to embodiments of the present disclosure may obtain at least one sentence based on text data and attribute data obtained through an application (210) and / or an input interface (220). By generating the at least one sentence or instructing an external language model to generate the at least one sentence, the electronic device (101) may obtain the at least one intended sentence. To obtain the at least one intended sentence, the electronic device (101) may use attribute data related to the sentence to be generated. The attribute data may indicate attributes related to one or more sentences to be generated. The attributes may include format, language, sentence length, paragraph length, and / or tone. For example, a sentence in an email may differ in format, length, and / or tone from a sentence in a comment. When writing a text using an artificial intelligence model (230), attribute data may be used to write a text that is more appropriate for the situation. According to one embodiment, the attribute data may be set based on an application (210) running on the electronic device (101). According to one embodiment, the attribute data may be set based on a user input on an input interface (220).
[0061] FIG. 3 illustrates an operational flow of an electronic device (e.g., electronic device (101)) for displaying content (e.g., sentences, images, videos, audio, and / or combinations thereof) in a text input area. For example, the electronic device (101) may generate at least one sentence based on word-by-word input.
[0062] Referring to FIG. 3, in operation (301), the electronic device (101) (e.g., the processor (120)) may display a user interface including a text input area. The electronic device (101) may display an execution screen of an application (e.g., the application (210)) including a text field for inputting at least one content and a sentence related to the at least one content through a display (e.g., the display module (120)). For example, let's assume that the electronic device (101) is connected to a specific communication service (e.g., Reddit) through an Internet browser. The electronic device (101) may display a web page including content (e.g., text or photos) created by various users and a text field for writing a response (e.g., comment, reply) to the specific content. For example, the electronic device (101) may execute a food ordering application. The electronic device (101) may display a screen including food ordered through the food ordering application and a text field for writing a review of the food. For example, the electronic device (101) may be operating a shopping mall app. The electronic device (101) may display a screen including a text field for writing a review of a purchased product. For example, the electronic device (101) may display a screen including a text field for posting content (e.g., text, video, photo) on a social media application.
[0063] In operation (303), the electronic device (101) (e.g., the processor (120)) may display an input interface (e.g., a keyboard-type input interface (220) displayed on a display) including a first area corresponding to a keyboard and a second area for setting one or more properties based on at least a portion of a selection for a text input area. The electronic device (101) may obtain a selection for text input. The selection may be understood as a user's intention to write in the text input area. The electronic device (101) may display the input interface based on at least a portion of the selection. For example, the electronic device (101) may display a first keyboard interface in response to a user's input (e.g., a touch) for a text field displayed on a screen. For example, the first keyboard interface may include a keyboard (e.g., QWERTY, 3X4, Cheonjiin, Danmoeum, etc.) having character buttons arranged in a designated form for inputting characters, and visual objects (e.g., AI sentence generation, emoticons, clipboard, etc.) for executing functions associated with the keyboard. The electronic device (101) may, in response to receiving a user input for a visual object for a sentence generation function (e.g., AI sentence generation) among the visual objects, display a second keyboard interface (e.g., AI sentence generation interface) including at least one interface element for generating a sentence. The second keyboard interface may be additionally displayed while the first keyboard interface is displayed. The second keyboard interface may correspond to an input interface including the first area and the second area. The input interface may be used to set one or more properties for a text input into a text input area. The one or more properties may be related to a sentence to be generated.For example, the input interface may be a sentence generation interface, and may include at least one or more selection interfaces for selecting the format or type of the sentence to be generated, the tone of the sentence to be generated, the length of the sentence to be generated, or the language of the sentence to be generated, or at least one text field for entering simple characters / sentences for the sentence to be generated. As an example, the at least one selection interface may be displayed in a first area, and the text field may be displayed in a second area.
[0064] In one embodiment, the electronic device (101) may display an input interface (e.g., input interface (220)) with an initial value of at least one interface element set. For example, when a keyboard (e.g., a first keyboard interface) for entering a sentence into a text field included in an application is called, the electronic device (101) may check information about the application that called the keyboard (e.g., a package name of the application, an attribute of the text field, a character or sentence included in the corresponding text field) and determine one of the possible values of the attribute based on the information. The electronic device (101) may set the determined value as the initial value of the attribute. In this manner, the electronic device (101) may set an initial value of at least one interface element. The input interface may be used to set at least one of a plurality of attributes. For example, the plurality of attributes may include at least one of a format attribute, a tone attribute, a length attribute, or a language attribute.
[0065] In one embodiment, the format attribute may indicate the format of a sentence to be generated. The format attribute may indicate one selected from a plurality of formats. For example, the plurality of formats may include general, mail, social media (or Social Network Service), and / or comment or reply. The electronic device (101) may determine an initial value of the format attribute when the second keyboard interface is displayed based on at least one of attribute information of the application that called the first keyboard interface. In one embodiment, the electronic device (101) may determine the initial value of the format attribute based on the package name of the application. The package name of the application may be identified through information passed as a calling argument by the application that called the keyboard interface. Based on the package name, the format corresponding to the application may be identified. The electronic device (101) may store a format corresponding to each of a plurality of applications. The electronic device (101) may identify the format corresponding to the application through the package name of the application. The electronic device (101) may set the identified format as the default value of the format attribute. For example, the electronic device (101) may execute a mail application. When a keyboard interface is invoked by a user input to a text field displayed within the mail application, the electronic device (101) may determine whether the package name of the application is included in a specific group (e.g., Email Package Preset). If the package name of the application is included in the specific group, the electronic device (101) may set the default value of the format attribute to mail. According to one embodiment, the electronic device (101) may determine the initial value of the format attribute based on the context of the application.For example, if the application that called the keyboard interface is an Internet browser, the electronic device (101) can identify information about the address (e.g., URL (uniform resource locator)) of the currently connected web page. If the electronic device (101) determines that the address falls within a predefined group (e.g., Email URL Preset, Email service provider address), the electronic device (101) can set the default value of the format attribute to mail.
[0066] In one embodiment, the tone attribute may indicate the tone of a sentence to be generated. The tone attribute may indicate one of a plurality of different attributes. The tone attribute may be dependent on the format attribute. For example, a default value of the tone attribute may be set based on a format attribute determined based on the attributes of the application that invoked the keyboard interface. For example, the tone attribute may be one of polite, professional, and casual. For example, if the format attribute is set to "letter" based on the attributes of the application or the user's format selection, the tone attribute may be set to "professional," which corresponds to the format attribute of "letter." For example, if the format of the sentence to be generated is set to a social media format, "casual," which corresponds to the social media format, may be set as the default value of the tone attribute. In one embodiment, in the case of a communication application (e.g., IMS), the value of the tone attribute may be selected by referring to who the other party is or the content / history of a previous conversation with the other party. In one embodiment, the electronic device (101) may determine the tone attribute by checking context information. For example, the electronic device (101) may check recipient information on the screen in a currently displayed application (e.g., an email application). The electronic device (101) may determine the attribute (e.g., tone attribute) based on the recipient information. For example, if the recipient information corresponds to a first group (e.g., family, friends), the tone attribute may be set to 'casual (or friendly)'. For example, if the recipient information corresponds to a second group (e.g., coworkers), the tone attribute may be set to 'professional (or polite)'.
[0067] In one embodiment, the length attribute may indicate the length of a sentence to be generated. The length attribute may be a set value for defining the number of sentences to be generated or the total length of the sentences to be generated. For example, the electronic device (101) may determine the initial value of the length attribute as a predefined value determined according to the format of the sentence determined based on the properties of the application that called the keyboard interface. For example, the length attribute may be standard or detail. For example, if the format attribute of the sentence to be generated is set to a letter format based on the properties of the application or a user's selection, the default value of the length attribute may be set to 'detail' corresponding to the letter format. For example, if the format attribute of the sentence is set to a social media format, the default value of the length attribute may be set to 'standard' corresponding to the social media format. For example, if the length attribute is 'standard', the number of sentences to be generated may be 5 sentences. If the length attribute is 'detail', the length of the sentence to be generated may be 10 sentences. As a non-limiting example, the length value corresponding to 'standard' may be determined based on the format attribute as a sub-attribute of the length attribute. Similarly, the length value corresponding to 'detail' may also be determined based on the format attribute. In other words, the length of sentences in the standard attribute or the detailed attribute may differ depending on the format of the sentence. For example, if the format attribute is 'letter', the length attribute set to 'standard' may represent 10 sentences. If the format attribute is 'social media', the length attribute set to 'standard' may represent 5 sentences. If the format attribute is 'comment', the length attribute set to 'standard' may represent 3 sentences. As a non-limiting example, the length attribute may be set within a range from a minimum of 1 sentence to a maximum of 10 sentences.
[0068] According to one embodiment, the electronic device (101) may set the length attribute of a sentence to be generated based on the restriction information of the text input area included in the application that called the keyboard interface (e.g., information limiting the length of characters to be entered). For example, the electronic device (101) may identify the attribute of the text field that called the keyboard interface. The electronic device (101) may identify the length attribute (e.g., android: maxLength) in 'EditorInfo' passed as a calling argument. The electronic device (101) may set the length attribute of the sentence to be generated based on the length attribute.
[0069] In one embodiment, the language attribute may indicate the language of the sentence to be generated. The electronic device (101) may determine the default value of the language attribute based on the context of the application that invoked the keyboard interface. For example, suppose that a user accesses an email service through an Internet browser and attempts to compose an email. When composing a new email, the electronic device (101) may set the default language type (e.g., Korean) set in the keyboard interface as the default value of the language attribute. For example, suppose that a user accesses an email service and composes a reply email to a specific recipient. The electronic device (101) may set the body language (e.g., English) of a previous email from the recipient to whom the user wishes to respond as the default value of the language attribute.
[0070] In operation (305), the electronic device (101) (e.g., the processor (120)) may receive at least one word from a user through a first area. The first area may be used to generate text data. For example, the electronic device (101) may receive a user input for a keyboard. The electronic device (101) may obtain at least one word corresponding to the user input. As a non-limiting example, the electronic device (101) may display the at least one word in the first area. The electronic device (101) may generate text data including the at least one word. The text data may be used to generate a sentence using a language model (e.g., an artificial intelligence model (230)).
[0071] In operation (307), the electronic device (101) (e.g., the processor (120)) may obtain at least one sentence generated using at least one word and one or more attributes. The one or more attributes may indicate characteristics related to text to be input into the text input area. For example, the one or more attributes may include at least one of format, length, tone, and / or length. The electronic device (101) may determine the one or more attributes. For example, the electronic device (101) may determine an attribute having a default value set according to an application. For example, the electronic device (101) may determine an attribute having a value set according to a user's selection. Some of the attributes may be automatically set according to an application, and others may be set according to a user's selection. As a non-limiting example, among the configurable attributes, one attribute may be assigned a value while another attribute may not be assigned a value. Not all of the configurable attributes may be assigned a value, and one attribute may not be assigned a value.
[0072] The electronic device (101) may receive a user input for a sentence generation request when a character or sentence is entered in a text input area. For example, the electronic device (101) may receive a user input for a visual object indicating sentence generation while displaying at least one word received from the user in a first area of the input interface (220). In response to the user input, the electronic device (101) may identify a value (hereinafter, “attribute value”) set for each attribute in a second area of the input interface (220). The electronic device (101) may generate attribute data based on each attribute value. The electronic device (101) may generate attribute data. The attribute data may represent each attribute and a value of the corresponding attribute. The electronic device (101) may obtain at least one sentence based on text data corresponding to the at least one word and the attribute data. According to one embodiment, the electronic device (101) may generate a prompt based on the text data and the attribute data. The electronic device (101) may transmit a request signal including the prompt to the artificial intelligence model (230). For example, the electronic device (101) may transmit the request signal to a server including the artificial intelligence model (230). As another example, the electronic device (101) may transmit the request signal to a module that uses the artificial intelligence model (230) within the electronic device (101). The artificial intelligence model (230) may be configured to generate at least one sentence in response to the prompt. The electronic device (101) may obtain at least one sentence as an output of the artificial intelligence model (230). According to another embodiment, the electronic device (101) may directly provide the text data and the attribute data to the artificial intelligence model (230). For example, the artificial intelligence model (230) may be an artificial intelligence model that has been trained based on text and each attribute.Even if only text and attribute values are entered without creating a separate prompt, the artificial intelligence model (230) can be configured to generate at least one sentence based on the entered text and attribute values.
[0073] In operation (309), the electronic device (101) (e.g., the processor (120)) can display at least one sentence in a text input area. The output of the artificial intelligence model (230) can correspond to at least one sentence. The electronic device (101) can obtain the at least one sentence through the input interface (220). The electronic device (101) can display the at least one sentence in a text input area of a user interface of an application being executed. For example, the electronic device (101) can display the at least one sentence on the text input area through a user input on the input interface (220) (e.g., an input of pressing an 'Insert' button).
[0074] Although examples of displaying sentences generated by an artificial intelligence model (230) have been described in this disclosure, embodiments of the present disclosure are not limited thereto. For example, the artificial intelligence model (230) may not only generate sentences, but may also generate voice information (i.e., information about voice for reading the text) in place of the generated text. The electronic device (101) may output a voice signal based on the voice information. In addition to, or instead of, generating long sentences from short text or words input by the user, a voice signal corresponding to long sentences or multiple sentences may be output from the electronic device (101). As a non-limiting example, the voice information may include information about the user's voice. Through this, the user's voice may be reflected in the voice signal output from the electronic device (101). For example, the artificial intelligence model (230) may generate images and videos in addition to text-type sentences or voice information. The electronic device (101) may display at least one sentence, as well as display images or play videos.
[0075] FIGS. 4A to 4F illustrate examples of a user interface for generating at least one content (e.g., a sentence, an image, a video, an audio, and / or a combination thereof). The user interface may include an input interface (e.g., a keyboard-type input interface (220) displayed on a display). The input interface (220) may include one or more input interfaces (e.g., a keyboard, a button for executing a function, a text input field, a field for specifying a value of an attribute). The screens of the user interface described below through FIGS. 4A to 4F are exemplary and are not to be construed as limiting the embodiments of the present disclosure. The same reference numerals may represent the same description.
[0076] Referring to FIG. 4A, an electronic device (101) may display a screen (401). The electronic device (101) may display a user interface (410) of an application. For example, the application may be a social network service (SNS) application. The screen (401) may include a text input area (411) and a content area (413). The text input area (411) may include an area (421) where text is displayed, a visual object (422) indicating cancellation, and a visual object (423) indicating the writing of a comment.
[0077] Referring to FIG. 4B, the electronic device (101) may display a screen (402). While displaying the screen (401), the electronic device (101) may display the screen (402) in response to a user input (491). For example, the user input (491) may be an input (e.g., a touch input) for selecting a text input area (411). The screen (402) may include a first input interface (430). For example, the first input interface (430) may include a keyboard interface (431) and visual objects for a specified function (e.g., a first visual object (441), a second visual object (442), a third visual object (443), a fourth visual object (444), a fifth visual object (445), and / or a sixth visual object (446)). The keyboard interface (431) may include a keyboard on which character buttons of a specified form for inputting characters are arranged. For example, the keyboard may be a preset type of keyboard, such as a QWERTY keyboard, a Cheonjiin keyboard, or a single vowel keyboard. Although a keyboard interface is illustrated in FIG. 4B, embodiments of the present disclosure are not limited thereto. For example, not only a soft keyboard displayed on the display, but also a hardware keyboard may be used as an input means of the present disclosure. The hardware keyboard may be connected via Bluetooth or electrically connected to the electronic device (101) via a separate cable. For example, a voice input interface or a handwriting input interface may be displayed instead of the keyboard interface (431). Each of the visual objects may be used to execute a corresponding function. For example, the first visual object (441) may be used for AI-based sentence generation. The second visual object (442) may be used for icon input. The third visual object (443) may be used for translation. The fourth visual object (444) may be used for a clipboard. The fifth visual object (445) may be used for keyboard settings.The sixth visual object (446) can be used to display additional functions.
[0078] Referring to FIG. 4C, the electronic device (101) may display a screen (403). While displaying the screen (402), the electronic device (101) may display the screen (403) in response to a user input (492). For example, the user input (492) may be an input (e.g., a touch input) for selecting a first visual object (441) for AI-based sentence generation. The screen (403) may display a second input interface (440). The second input interface (440) may include a text area (440a) and an attribute area (440b). The text area (440a) may include a text input window. The text input window may be used to input a subject of a text to be written. The attribute area (440b) may include fields for specifying attributes and values of the attributes. For example, the attribute area (440b) may include an area (451) for a first attribute (e.g., format), an area (452) for a second attribute (e.g., length), an area (453) for a third attribute (e.g., tone), and an area (454) for a fourth attribute (e.g., language). The second input interface (440) may include a visual object (460) for generating a sentence. For example, the first attribute may be a format attribute. The format attribute may be set to 'Social'. For example, the second attribute may be a length attribute. The length attribute may be set to 'Standard'. For example, the third attribute may be a tone attribute. The tone attribute may be set to 'Casual'. For example, the fourth attribute may be a language attribute. The language attribute may be set to 'English'.
[0079] The screen of the user interface illustrated through FIG. 4c is exemplary and should not be construed as limiting the embodiments of the present disclosure. For example, the visual object (460) may be displayed on the area of the first input interface (430) as well as the attribute area (440b) of the second input interface (440). In addition, for example, the attribute area (440b) may be displayed as one of the visual objects for a designated function within the area of the first input interface (430) instead of being displayed separately from the area of the first input interface (430) (e.g., the first visual object (441), the second visual object (442), the third visual object (443), the fourth visual object (444), the fifth visual object (445), and / or the sixth visual object (446)). For example, instead of the fifth visual object (445) being displayed for keyboard settings as a conventional menu, a visual object for designating a specific attribute may be displayed.
[0080] Although the present disclosure describes an example in which a text input window in a text area (440a) is used to input the subject of a text to be written, embodiments of the present disclosure are not limited thereto. The text input into the text area (440a) is input text used to generate a desired sentence based on an AI model, and may be used not only as the subject of the text to be written, but also as text corresponding to a summary, conclusion, background, and / or purpose of the content the user wishes to write.
[0081] Referring to FIG. 4D, the electronic device (101) can display a screen (404). The electronic device (101) can receive a user input (493) on a first input interface (430). The electronic device (101) can generate text data corresponding to the user input (493). The text data can include at least one word (465). For example, at least one word (465) can represent 'interested'. Although a keyboard array having character buttons of a designated form for inputting characters is illustrated in FIG. 4D, embodiments of the present disclosure are not limited thereto. For example, the first input interface (430) can be provided in a form capable of inputting handwriting. The keyboard array can be switched to a form capable of inputting handwriting based on a user input, or an input interface can be provided in a form capable of inputting handwriting according to a preset setting. For example, user input (493) may include handwriting input using a part of the user's body (e.g., a finger) or a digital pen (e.g., a stylus pen). Text ('interested') and / or a drawing result based on a sketch may be received as user input (493).
[0082] Referring to FIG. 4E, the electronic device (101) can display a screen (405). The electronic device (101) can display words in a text input window not only through a user input (492) on the screen (402) and a user input (493) on the screen (404), but can also display words in the text input window through a user input on the screen (402). For example, the electronic device (101) can display the screen (402). While the screen (402) is displayed, the electronic device (101) can receive at least one word (465) through a user input (494) (e.g., an input of typing 'interested') on the first input interface (430) in a text input area (411) included in an application. The electronic device (101) can display at least one word (465) in the text input area (411). The electronic device (101) may receive a user input (495) while at least one word (465) is displayed in the text input area (411). For example, the user input (495) may be an input (e.g., a touch input) for selecting a first visual object (441) for AI-based sentence generation. The electronic device (101) may display a screen (405) in response to the user input (495). For example, the electronic device (101) may access a specific web service through an Internet browser application. A user of the electronic device (101) may select the text input area (411) to write a comment (or response) on a specific text. In response to the selection, the electronic device (101) may call an input interface (e.g., input interface (220), first input interface (430)). The user of the electronic device (101) may input at least one word (e.g., 'interested') through the input interface. At least one of the above words may be displayed in the text input area (411).When an input (e.g., user input (495)) for AI-based sentence generation is received while at least one word (465) is displayed in the text input area (411), the electronic device (101) can display a screen (405) on which at least one word (e.g., 'interested') is displayed in the text area (440a) of the second input interface (440).
[0083] According to one embodiment, a field indicating the value of each attribute on the attribute area (440b) of the second input interface (440) of the screen (405) may display a default value set for the corresponding attribute. For example, the format attribute may be set to 'Social'. The length attribute may be set to 'Standard'. The tone attribute may be set to 'Casual'. The language attribute may be set to 'English'.
[0084] Although not illustrated in FIG. 4E, the electronic device (101) may display an interface for selecting multimedia content (e.g., a music file, an image file, a video file) stored in the electronic device (101). For example, the electronic device (101) may display a visual object for inputting the multimedia content. Through a user input to the visual object, the multimedia content stored in the electronic device (101) may be input as a 'topic' for AI-based sentence generation. For example, a user of the electronic device (101) may select multimedia content stored in the electronic device (101). The selected content may be used for AI-based sentence generation.
[0085] Referring to FIG. 4F, the electronic device (101) can display a screen (404) (or screen (405)). The electronic device (101) can receive a user input (496) while at least one word (e.g., interested) is displayed on a text area (440a) of a second input interface (440). For example, the user input (496) can include a visual object (460) for generating a sentence (e.g., a 'Generate' button). In response to the user input (496) for the visual object (460), the electronic device (101) can obtain at least one sentence. The at least one sentence can be generated based on at least one word (hereinafter, 'text data') on the text area (440a) of the second input interface (440) and a value set for each attribute on the attribute area (440b) (hereinafter, 'attribute data'). The process of generating at least one sentence is described through FIGS. 5a and 5b.
[0086] As a non-limiting example, an option may be provided to set the level at which user-inputted text is reflected in AI results. In one embodiment, the electronic device (101) may display an interface for specifying the level. For example, the electronic device (101) may display the interface on or adjacent to the visual object (460). The electronic device (101) may determine the level based on user input. For example, the higher the level, the more modified sentences may be generated from the text input by the user. The lower the level, the less modified sentences may be generated from the text input by the user. The level may be referred to as a reflection level, a creative level, a modification level, an AI level, a change level, a static level, and / or equivalent terms. The determined level may be used for sentence generation, which will be described below. For example, assume a situation where three sentences are typed into a text area (e.g., text area (440a)). If the level (e.g., creative level) is set to 'upper', the generated sentences are at a level that only refers to the typed sentences, whereas if the level (e.g., creative level) is set to 'lower', the words and / or expressions written by the user can be maintained as much as possible in the generated sentences (e.g., some expressions are corrected, sentences for further explanation are added).
[0087] Although a keyboard interface is exemplified as a means for inputting text data in FIGS. 3 and 4A to 4F, embodiments of the present disclosure are not limited thereto. For example, in addition to the keyboard interface, an interface for receiving handwriting input by a user's hand or a pen may also be used as part of the input interface (220) for sentence generation. In addition, for example, an interface including a button for voice input and / or an icon indicating that voice input is being monitored may be used instead of the keyboard interface.
[0088] Although FIGS. 4A to 4F illustrate a situation in which sentences are generated through input of buttons displayed on a user interface, embodiments of the present disclosure are not limited thereto. For example, after a user inputs text into a text input area (411) via a keyboard on a screen (402), an AI assistant may be called up through a designated input (e.g., a long press of the home button, a double tap of the home button, a press of the power key, a combination of pressing the power key and another key (e.g., a volume button), a voice command). A sentence may be generated based on the context of the currently displayed screen (e.g., a screen on an application) and the entered text through the AI assistant. In other words, instead of inputting sequential visual objects of the user interface, the user may be provided with the convenience of generating sentences through designated inputs. An interface for specifying the properties of the sentence to be generated may be displayed after the AI assistant is called up, or may be displayed to re-edit the properties of the sentence after the sentence is generated.
[0089] Although examples of generating sentences in a first language in a situation where a user inputs words in a first language are described in FIGS. 4A to 4F, the embodiments of the present disclosure are not limited thereto. According to one embodiment, the electronic device (101) may determine the language of sentences to be displayed based on context. For example, if a second language is set in the electronic device (101) and / or the application, sentences in the second language may be displayed even if the language of the input words (e.g., at least one word (465)) is the first language. In addition, for example, even if the first language is set in the electronic device (101) and / or the application, if a conversation (e.g., chat) is being conducted in a second language with a party in the application (when the screen is displayed in the second language), sentences in the second language may be displayed even if the language of the input words (e.g., at least one word (465)) is the first language. The electronic device (101) may display sentences in the determined language. In one embodiment, the electronic device (101) may generate sentences in the determined language (e.g., a second language) based on AI. The electronic device (101) may display the generated sentences. In another embodiment, the electronic device (101) may translate sentences in the language of the input word (e.g., a first language) generated based on AI into sentences in the determined language (e.g., a second language). The electronic device (101) may display the translated sentences.
[0090] In the present disclosure, generating text through AI includes generating at least one sentence from a word (e.g., "interest") input by a user, but embodiments of the present disclosure are not limited thereto. In addition to expanding text in word units to provide text in sentence units, rewriting text input by a user (e.g., transforming it into text in a different style, changing its tone, correcting typos or grammar, etc.) can also be understood as an embodiment of the present disclosure.
[0091] FIGS. 5A and 5B illustrate examples of a user interface for inputting at least one content (e.g., a sentence, an image, a video, an audio, and / or a combination thereof) into a text area. The user interface may include an input interface (e.g., the input interface (220) of FIG. 2 ). The input interface (220) may include one or more input interfaces (e.g., a keyboard, a button for executing a function, a text input field, a field for specifying a value of an attribute). The screens of the user interface described below through FIGS. 5A and 5B are exemplary and are not to be construed as limiting the embodiments of the present disclosure. Like reference numerals may represent like descriptions.
[0092] Referring to FIG. 5A, the electronic device (101) can display a screen (501). The electronic device (101) can display the screen (501) in response to a user input (496) on the screen (404) (or screen (405)) of FIG. 4F. The screen (501) can include a result area (520). The result area (520) can include a result text window (521), a first visual object (531) for requesting regeneration (e.g., 'Regenerate'), and a second visual object (532) for requesting input into a text input area (e.g., 'Insert').
[0093] The electronic device (101) can obtain text data corresponding to at least one word on the text area (440a) of the second input interface (440). The electronic device (101) can obtain attribute data corresponding to values set for each attribute on the attribute area (440b) of the second input interface (440). The electronic device (101) can generate a prompt based on the text data and the attribute data. The electronic device (101) can transmit a request signal including the prompt to an artificial intelligence model (e.g., the artificial intelligence model (230) of FIG. 2). The artificial intelligence model can be located in an external electronic device (e.g., the server (108)) or stored within the electronic device (101). The output of the artificial intelligence model can be at least one sentence. The electronic device (101) can obtain the at least one sentence. According to one embodiment, the prompt may be generated based on at least one of content information obtained from the application (e.g., original text information, image information, product information), a topic (e.g., 'interested') entered in a text area (440a) of the second input interface (440), and / or a setting value of each attribute entered in an attribute area (440b) of the second input interface (440).
[0094] According to one embodiment, the format of the prompt to be generated may be determined based on a format attribute indicating the format of the sentence to be generated. The electronic device (101) may set the format of the prompt based on the value set in the format attribute. The electronic device (101) may generate the prompt based on the set prompt format. For example, when the format attribute is set to 'general', the prompt may have the following format.
[0095]
[0096] In [Table 1], "==replace==" may correspond to at least one word on the text area (440a) of the second input interface (440). In [Table 1], {TONE} may represent a setting value of a tone attribute in an attribute area (440b) of the second input interface (440). In [Table 1], {LENGTH} may represent a setting value of a length attribute in an attribute area (440b) of the second input interface (440). In [Table 1], {LANGUAGE} may represent a setting value of a language attribute in an attribute area (440b) of the second input interface (440). For example, when the format attribute is set to 'mail', the prompt may have the following format.
[0097]
[0098] In [Table 2], "==replace==" may correspond to at least one word on the text area (440a) of the second input interface (440). In [Table 2], {TONE} may represent a setting value of a tone attribute in an attribute area (440b) of the second input interface (440). In [Table 2], {LENGTH} may represent a setting value of a length attribute in an attribute area (440b) of the second input interface (440). In [Table 2], {LANGUAGE} may represent a setting value of a language attribute in an attribute area (440b) of the second input interface (440). Let us assume a situation in which a user of an electronic device (101) writes a reply to a previous email of the other party. The electronic device (101) can identify that the current user is trying to write a reply through the context of the application. When the format attribute is set to 'mail' and the text input area of the user interface (e.g., text input area (411)) is for a response, the electronic device (101) may generate a prompt different from [Table 2]. For example, the electronic device (101) may generate a prompt based on at least one of the content included in the context, the topic (e.g., 'interested') entered in the text area (440a) of the second input interface (440), and / or the setting value of each attribute entered in the attribute area (440b) of the second input interface (440). The prompt may have the following format.
[0099]
[0100] In [Table 3], [CONTEXT TEXT] may represent a context provided from a running application. In [Table 3], "==replace==" may correspond to at least one word on the text area (440a) of the second input interface (440). In [Table 3], {TONE} may represent a setting value of a tone attribute in an attribute area (440b) of the second input interface (440). In [Table 3], {LENGTH} may represent a setting value of a length attribute in an attribute area (440b) of the second input interface (440). In [Table 3], {LANGUAGE} may represent a setting value of a language attribute in an attribute area (440b) of the second input interface (440). As an example, let us assume a situation in which a reply sentence is generated for an email received from another user. The original text of the received email may be input as the context of the prompt. For example, the electronic device (101) may receive an email from the first user saying, “Hello. We would like to invite you to celebrate Children’s Day on May 5th. The location is Everland, and our staff will be there to greet you at the main gate at 9:00 AM. Please come comfortably and have a nice day. Thank you.” The user of the electronic device (101) may input the word “possible” into the text area (440a) of the second input interface (440) in response to the email. In the [Context] field of [Table 3], “Hello. We would like to invite you to celebrate Children’s Day on May 5th. The location is Everland, and our staff will be there to greet you at the main gate at 9:00 AM. Please come comfortably and have a nice day. Thank you.” may be inputted, and the word “possible” may be inputted into the [Subject] field. In [instruction], {TONE}, [LENGTH], and {LANGUAGE} can be entered as 'Professional', '10', and 'Korean', respectively.'Professional' may be the default value for the tone attribute based on the format attribute of 'Mail'. '10' may be the default value for the length attribute based on the format attribute of 'Mail'. 'Korean' may be the default value for the language attribute determined based on the keyboard settings of the electronic device (101), 'Context', or 'Subject'. For example, the following prompt may ultimately be determined or generated.
[0101]
[0102] The electronic device (101) can transmit a request signal including the prompt of [Table 4] to the artificial intelligence model (230). The prompt of [Table 4] can be input to the artificial intelligence model (230). The artificial intelligence model (230) can output at least one sentence. For example, the at least one sentence can be generated as follows.
[0103]
[0104] If "reject" instead of "possible" is entered in the text area (440a) of the second input interface (440), a prompt related to "reject" may be generated. In response to the prompt, the artificial intelligence model (230) may output the following sentences.
[0105]
[0106] The examples above describe the prompts generated when the format attribute is set to "Mail." If the format attribute is set to "Social Media," the prompts might look like this:
[0107]
[0108] In [Table 7], "==replace==" can correspond to at least one word on the text area (440a) of the second input interface (440). In [Table 7], {TONE} can represent a setting value of a tone attribute in an attribute area (440b) of the second input interface (440). In [Table 7], {LENGTH} can represent a setting value of a length attribute in an attribute area (440b) of the second input interface (440). In [Table 7], {LANGUAGE} can represent a setting value of a language attribute in an attribute area (440b) of the second input interface (440). As illustrated in [Table 7], when the format attribute is set to 'Social Media', the electronic device (101) may generate a prompt that further includes a sentence for requesting the generation of a hashtag or emoji (e.g., 'Add proper hashtags on text when needed. Use proper emoji when context is suitable for emoji.'). As an example, assume a situation where 'clear weather' is entered in a text input window of a text area (440a) of a second input interface (440). The text data may indicate 'clear weather' and the format attribute of the attribute data may indicate 'Social Media'. A prompt generated based on the text data and the attribute data may be input to the artificial intelligence model (230). The artificial intelligence model (230) may generate the following output in response to the prompt.
[0109]
[0110] The electronic device (101) can obtain at least one sentence generated according to the above-described method. The electronic device (101) can obtain the at least one sentence corresponding to the output of the artificial intelligence model (230). The electronic device (101) can display the at least one sentence in the result text window (521). For example, the at least one sentence can be 'Just like you. I'm interested in why Koreans enjoy iced drinks even when it's chilly outdoors. There seem to be various theories, but nothing definitive yet. Fascinating, isn't it?' The result area (520) can include a first visual object (531) for requesting regeneration and a second visual object (532) for requesting input to a text input area. The electronic device (101) can receive a user input (591) for the second visual object (532).
[0111] The electronic device (101) can display at least one acquired sentence in a result text window (521). The electronic device (101) can output the at least one acquired sentence through a user input (591). Meanwhile, the user of the electronic device (101) can modify at least a part of the at least one acquired sentence. The electronic device (101) can modify at least a part of the generated at least one acquired sentence through a user input to the result text window (521). The result of the modification can be displayed on the result area (520). The user of the electronic device (101) can modify at least a part of the at least one acquired sentence and then provide a user input (591).
[0112] Referring to FIG. 5B, the electronic device (101) can display a screen (502). The electronic device (101) can display the screen (502) in response to a user input (591) for a second visual object (532). The screen (502) can include a text input area (411) and a content area (413). At least one sentence of a result text window (521) of the screen (501) can be displayed in the text input area (411). For example, the electronic device (101) can display at least one sentence obtained from the artificial intelligence model (230) in the text input area (411). For example, if a user of the electronic device (101) modifies at least a portion (e.g., a word, an expression, a sentence) of at least one generated sentence, the modified result is displayed in the result text window (521), and thus the electronic device (101) can display at least one sentence corresponding to the modified result.
[0113] Although examples of specifying language properties through the second input interface (440) are described in FIGS. 4A to 5B , embodiments of the present disclosure are not limited thereto. In one embodiment, the input interface may not include a field for specifying language properties. The electronic device (101) may identify a 'language' based on a context obtained from an application (e.g., a recent post, an original email). The electronic device (101) may generate a prompt for requesting the generation of a sentence in the identified 'language'. The electronic device (101) may transmit the prompt to the artificial intelligence model (230). The electronic device (101) may obtain at least one sentence generated in response to the prompt.
[0114] Figure 6 illustrates examples of functional components for generating content (e.g., sentences, images, audio, and / or combinations thereof) using a context extraction module. Like reference numbers may represent like descriptions.
[0115] Referring to FIG. 6, an electronic device (101) (e.g., a processor (120)) can execute an application (210). The electronic device (101) can use an input interface (220) while the application (210) is executed. The electronic device (101) according to embodiments of the present disclosure can write text using an artificial intelligence model (230). In order to write text that meets a user's intention, the electronic device (101) can provide information (211) (e.g., properties of the application, a screen displayed in the application, and / or content) to the input interface (220) through the application (210). Properties that can be set through the input interface (220) can be set based on information (211) (e.g., properties of the application, a screen displayed in the application, and / or content) provided from the application (210). The electronic device (101) can transmit request information (221) to the artificial intelligence model (230) through the input interface (220). The request information (221) can be a prompt generated based on the attribute data and the text data. The prompt can be used to request the artificial intelligence model (230) to generate at least one sentence. The artificial intelligence model (230) can provide response information (231) including the at least one sentence to the input interface (220). The electronic device (101) can display the at least one sentence (241) obtained through the input interface (220). The electronic device (101) can display the at least one sentence (241) in a text input area of the user interface.
[0116] The electronic device (101) can determine the properties of a sentence to be generated by using context information (611) related to the application (210) in addition to the values (e.g., text, property setting values) input to the input interface (220). According to one embodiment, the electronic device (101) may further include a context extraction module (640). For example, the context extraction module (640) may represent a function (code or program) such as a smart clip. The smart clip may represent a module for recognizing text displayed on a screen (e.g., a screen of a web page, a document application) and analyzing the context of the text (e.g., a phone number, an address, an email date, a keyword). The electronic device (101) can obtain context information (611) including the context of the running application (210) through the context extraction module (640). For example, the context may include at least one of content, images, text displayed on the screen of the application (210), information of the application (210) (e.g., package name, category, type), user behavior information on the application (210) (e.g., recently visited pages, a series of access information, user input), information about a function being executed in the application (210), and / or information about a screen that was displayed in the application (210). The context extraction module (640) may be configured to obtain the context of the application (210) (e.g., images and / or text displayed on the current screen or a recently displayed screen). The context extraction module (640) may be configured to identify attribute information (612) related to the context of the application (210) (e.g., name, classification, content, subject of the content, information related to other users in a communication application) from the obtained context. The electronic device (101) may be configured to provide attribute information (612) related to the context of the application (210) to the input interface (220) through the context extraction module (640).
[0117] As a non-limiting example, an electronic device (101) for performing embodiments of the present disclosure may include a wearable device for providing a virtual environment (e.g., a device supporting a virtual reality (VR) / augmented reality (AR) / mixed reality (MR) environment (e.g., glasses / headset / head-mounted display (HMD))). For example, the electronic device (101) may obtain information about a surrounding environment (objects, people, locations) recognized through a sensor (e.g., an image sensor, a microphone). The information about the surrounding environment may be displayed as a 3D interface on a VR / XR / AR environment. The information about the surrounding environment may be used as context information for sentence generation.
[0118] FIG. 7 illustrates an operation flow of an electronic device (e.g., electronic device (101)) for displaying content (e.g., sentences, images, audio, and / or a combination thereof) in a text input area based on an application.
[0119] Referring to FIG. 7, in operation (701), the electronic device (101) (e.g., the processor (120)) may display a user interface (e.g., the user interface (410)) including a text input area (e.g., the text input area (411)) while an application (e.g., the application (210)) is running. For example, the electronic device (101) may run an Internet browser application. The electronic device (101) may access a specific website through the Internet browser application. A screen corresponding to the specific website may include a text input area for writing a comment on a post posted on the specific website.
[0120] In operation (703), the electronic device (101) (e.g., the processor (120)) may display an input interface (e.g., the input interface (220)) in response to a user input. For example, the user input may be a touch input for selecting the text input area. For example, the user input may be a separate command (e.g., voice, button) for calling the input interface. The electronic device (101) may display the input interface. The input interface may include an area for displaying buttons for inputting text or an image (e.g., an area corresponding to the first input interface (430)), an area for displaying text input from a user (e.g., a text area (440a)), and an area for specifying properties for a sentence to be generated (e.g., an property area (440b)). For example, a text input window may be displayed in the text area (440a). Text input into the text input window may be used as a subject of a sentence to be generated. For example, the property area (440b) may display each property and its value.
[0121] In operation (705), the electronic device (101) (e.g., the processor (120)) may generate text data through an input interface. The electronic device (101) may receive a user input through the input interface. The user input may be used to determine the subject of a sentence to be generated. For example, the user input may indicate at least one word through a keyboard (e.g., a soft keyboard, a hardware keyboard). The electronic device (101) may generate text data including the at least one word based on the user input.
[0122] In operation (707), the electronic device (101) (e.g., processor (120)) may obtain at least one sentence generated based on text data and attribute data related to the sentence to be generated.
[0123] An electronic device (101) can generate attribute data related to a sentence to be generated. The attribute data can indicate one or more attributes related to the sentence to be generated and a value of each attribute.
[0124] The electronic device (101) can display attributes related to a sentence to be generated through the input interface (220). The input interface (220) can be used to specify or change the value of the attribute. The attribute can have a default value (or initial value). According to one embodiment, the electronic device (101) can obtain context information about the application (210) (e.g., the screen being displayed, the screen that was displayed, an image within the screen, text, user input, a classification determined by the package name of the application, the type of the application). The electronic device (101) can set the default value of the attribute based on the context information. For example, if the running application (210) is an email application, the electronic device (101) can set the format attribute to 'mail'. For example, if the running application (210) is an SNS application, the electronic device (101) can set the format attribute to 'Social Media'. The value of a specific attribute among the one or more attributes may be dependent on the value of another attribute. For example, if the format attribute is set to 'mail', the length attribute may be set to 'detail'. For example, if the format attribute is set to 'Social Media', the tone attribute may be set to 'Causal'. The electronic device (101) may generate a prompt based on the text data and the attribute data. The electronic device (101) may provide the generated prompt to the artificial intelligence model (230) (e.g., LLM). The electronic device (101) may obtain at least one sentence corresponding to the output of the artificial intelligence model (230).
[0125] In operation (709), the electronic device (101) (e.g., the processor (120)) may display at least one sentence in a text input area. The output of the artificial intelligence model (230) may correspond to at least one sentence. The electronic device (101) may obtain the at least one sentence through the input interface (220). The electronic device (101) may display the at least one sentence in a text input area of a user interface of an application being executed. For example, the electronic device (101) may display the at least one sentence on the text input area through a user input (e.g., an input of pressing an 'insert' button) on the input interface (220) (e.g., a user input (591) of FIG. 5A).
[0126] The electronic device (101) may set each attribute of attribute data related to a sentence to be generated based on context information of the running application. However, embodiments of the present disclosure are not limited thereto. In addition to the context information of the currently running application, any information related to the application may be used to set attributes. For example, the electronic device (101) may determine the value of an attribute related to a sentence to be generated using another application related to the application. In addition, data associated with a user account and status information of the electronic device (101) (e.g., location, available network, battery, type of other connected devices, data received from other connected devices) may be used as additional context information. For example, the additional context information may be pre-specified based on the type of the currently used application, content provided by the application, or text (or file) entered by the user in a text input window. For another example, the additional context information may be automatically selected by the electronic device (101) depending on the type of application currently being used, the content provided by the application, or the text (or file) entered by the user in the text input window.
[0127] According to one embodiment, context information may include information obtained based on text entered by the user into an input window. The text entered by the user into the input window may include predetermined commands or keywords. For example, if the user enters the text "schedule," the electronic device (101) may obtain the user's schedule from the electronic device or another electronic device (or server) connected to the user account. The schedule may be used as context information. For example, if the user enters the text "schedule" into the input window, the electronic device (101) may identify an application related to "schedule." The electronic device (101) may obtain information about the user's schedule from the identified application. Before using the obtained schedule information, the electronic device (101) may display the obtained schedule information so that the user can confirm it. The obtained schedule information may be included in a prompt as context. For example, the obtained schedule information may be directly included in the prompt and provided to the artificial intelligence model (230). For another example, instead of the information about the acquired schedule, a designated value (e.g., “date”) may be included in the prompt, and the acquired schedule may be input at a position corresponding to the designated value among the sentences acquired from the artificial intelligence model (230). Furthermore, according to one embodiment, in addition to including a specific value in the prompt step, a method may be used in which the user directly inputs information corresponding to a specific keyword among the sentences acquired from the artificial intelligence model (230). For example, if the word “[schedule]” is included in the sentence, the electronic device (101) may, in response to receiving a user input for the word “[schedule],” display a screen of an application (e.g., a calendar application) for asking the user to select a date.For example, if the word "[location]" is included in the above sentence, the electronic device (101) may, in response to receiving a user input for the word "[location]," display a screen of an application (e.g., a map application) for asking the user for a location. As illustrated, an example of changing alternative information included in sentences generated by the artificial intelligence model (230) through user input is specifically described through FIGS. 13A to 13D .
[0128] As a non-limiting example, when an electronic device (101) is being used by a communication application, the electronic device (101) may determine that the user's current location information is required. The electronic device (101) may determine that the user's current location information is required based on conversations exchanged on the communication application and / or text entered by the user into an input field. Upon determining that the user's current location information is required, the electronic device (101) may obtain the location of the electronic device (101) via a communication circuit (e.g., GPS, Wi-Fi, BT, Cellular). The electronic device (101) may use the location information as context. The electronic device (101) may generate a prompt based on the location information.
[0129] In one embodiment, in addition to text, multimedia information (e.g., images, videos) and / or files can be used as context information. For example, a user may input the text "a picture taken with dad" into the electronic device (101). The electronic device (101) may obtain a picture corresponding to "a picture taken with dad" and use it as context information. As a non-limiting example, the multimedia information may be a plurality of objects (e.g., images, videos). A plurality of candidate objects to be used as context information may be displayed in advance to the user of the electronic device (101), and an object to be used for prompt generation may be determined based on user input. The multimedia information may be included in the prompt as context. For example, the multimedia information may be directly included in the prompt and provided to the artificial intelligence model (230). In another example, a designated value (e.g., "image file") may be included in the prompt instead of the multimedia information, and the multimedia information may be displayed in an area corresponding to the designated value among outputs from the artificial intelligence model (230).
[0130] According to one embodiment, the electronic device (101) can acquire context based on various applications, functions, input / output interfaces, and power elements, as well as other elements. For example, the electronic device (101) can intelligently connect wearable devices (e.g., smart watches, smart rings, XRs), IoT devices (e.g., home appliances, access points, speakers), home appliances, cloud servers, and / or automobiles connected to the electronic device (101) via a network. For example, the electronic device (101) can acquire sentences that take into account various situations of the user of the electronic device (101) by adding the operating status of the home appliance (e.g., during laundry), information received from the wearable device (e.g., during exercise), and / or data acquired from other applications (e.g., navigation->driving) to the prompt.
[0131] According to embodiments of the present disclosure, modules (or engines, HW, SW, interfaces) that perform operations of the electronic device (101) may be connected to a hub system, such that some of the modules or some of the sub-components of the modules are connected. The hub system may be driven by a specific processor (e.g., an always-on processor, a low-power AI processor (or engine)). The electronic device (101) may monitor various devices connected to the electronic device (101) through the processor and obtain data from each device. The processor may refer to a plurality of processors that collectively perform a plurality of operations by dividing them among the processors. Modules for performing operations of the electronic device (101) according to embodiments of the present disclosure may include software or hardware driven by an application processor or a dedicated processor.
[0132] Figure 8 illustrates examples of properties associated with content to be generated (e.g., sentences, images, videos, audio, and / or combinations thereof). Identical reference numbers may represent identical descriptions.
[0133] Referring to FIG. 8, the electronic device (101) may display an input interface (e.g., input interface (220)) for generating a sentence. The input interface may include a first input interface (430). For example, the first input interface (430) may include a keyboard interface (431) and visual objects (e.g., a first visual object (441)) for a specified function. As an example, the first visual object (441) may be used for AI-based sentence generation. The input interface may include a second input interface (440). The second input interface (440) may include a text area (440a) and an attribute area (440b). The text area (440a) may include a text input window. The text input window may be used to input a subject of a text to be written. The attribute area (440b) may include fields for specifying an attribute and a value of the attribute. For example, the attribute area (440b) may include an area (451) for a first attribute (e.g., format), an area (452) for a second attribute (e.g., length), an area (453) for a third attribute (e.g., tone), and an area (454) for a fourth attribute (e.g., language). The electronic device (101) may display candidate values for each attribute in response to a user input for the field. The electronic device (101) may change the value of the corresponding attribute in response to the user input. The second input interface (440) may include a visual object (460) for generating a sentence.
[0134] Referring to the screen (810), the format attribute can be set to one of four candidate values. For example, the format attribute can include 'General', 'Mail', 'Social Media', and 'Comment'. According to one embodiment, the candidate values can be preset according to system settings or set based on a running application or context information of the application. For example, the category of an application recently run on the electronic device (101) can be determined as one of the candidate values. As a non-limiting example, a user of the electronic device (101) can manually input a candidate value. The electronic device (101) can input text for the value of the format attribute through the first input interface (430) (e.g., a keyboard interface) (e.g., when 'article' is input, sentences corresponding to the headline and content are generated respectively).
[0135] Referring to screen (820), the tone attribute can be set to one of three candidate values. For example, the tone attribute can include 'Professional', 'Casual', and 'Polite'. In one embodiment, sub-attributes for the tone attribute can be preset according to system settings or set based on the application being executed or context information of the application. For example, sub-attributes for the tone attribute can include a tone level (e.g., corresponding to one of level 1 to level 10). When the tone attribute is set to 'Professional', the electronic device (101) can determine the level of the 'Professional' tone attribute. The level can be determined based on the target (e.g., the recipient of a mail service) to which sentences to be input in the text input area of the application are provided. For example, when writing a reply email to an email from a counterpart, the electronic device (101) can obtain context information including information about the counterpart and information indicating that the reply email is the reply email. The electronic device (101) can determine the level of the tone attribute based on the context information. For example, when the counterpart is a boss or a customer, the level of the tone attribute set to 'professional' can be determined as '10'. As a non-limiting example, the user of the electronic device (101) can manually input a candidate value. The electronic device (101) can input text for the value of the tone attribute through the first input interface (430) (e.g., when 'Strong' is input, a sentence with a strong tone is generated).
[0136] Referring to screen (830), the length attribute may be set to one of two candidate values. For example, the length attribute may include 'Standard' and 'Detail'. In one embodiment, sub-attributes for the length attribute may be preset according to system settings or set based on the application being executed or context information of the application. For example, when the length attribute is set to 'Standard', the number of sentences corresponding to the length attribute may be determined according to the screen of the application. For example, when a certain number of words or more are displayed on the screen of the application, the length attribute set to 'Standard' may be set to the maximum number of displayable sentences (e.g., 10). As a non-limiting example, a user of the electronic device (101) may manually input a candidate value. The electronic device (101) may input text for the value of the length attribute through the first input interface (430) (e.g., inputting '4' generates four sentences).
[0137] Referring to screen (840), the language attribute can be set to one of four candidate values. For example, the language attribute can include 'English', 'Korean', 'French', and 'Spanish'. According to one embodiment, the candidate values can be preset according to system settings or set based on a running application or context information of the application. For example, the languages of text displayed on the screen of the application can be determined as the candidate values. As a non-limiting example, a user of the electronic device (101) can manually input the candidate values. The electronic device (101) can input text for the value of the language attribute through the first input interface (430) (e.g., when 'Chinese' is input, a sentence in Chinese is generated).
[0138] Although not illustrated in FIG. 8, the electronic device (101) may further display options for setting the level (or degree, which may be referred to as a level) for each attribute. For example, a level field (e.g., indicated as a number 1 or greater and 5 or less, or one of upper, middle, and lower) may be further displayed next to each attribute. The user of the electronic device (101) may determine the degree of the attribute by selecting a number in the level field. In other words, when the level field is set to '1' and when the level field is set to '5', the degree to which the attribute is reflected in an actually generated sentence may differ. For example, the tone attribute may be set to 'professional'. Depending on what the level field indicates, the tone may be weakly or strongly reflected in the generated sentence.
[0139] Figure 9 illustrates examples of functional components for generating content (e.g., sentences, images, audio, and / or combinations thereof) using multiple artificial intelligence models. Like reference numbers may represent like descriptions.
[0140] Referring to FIG. 9, the electronic device (101) may utilize not only the artificial intelligence model (230) but also other artificial intelligence models. Hereinafter, the artificial intelligence model (230) may be referred to as the first artificial intelligence model (230), and the other artificial intelligence model may be referred to as the second artificial intelligence model (930). Although two artificial intelligence models are described in FIG. 9, embodiments of the present disclosure are not limited thereto. Any electronic device (101) that displays at least one sentence generated using at least one of three or more artificial intelligence models may be understood as an embodiment of the present disclosure.
[0141] An electronic device (101) (e.g., processor (120)) can execute an application (210). The electronic device (101) can use an input interface (220) while the application (210) is executed. The electronic device (101) according to embodiments of the present disclosure can write text using a first artificial intelligence model (230). In order to write text that meets a user's intention, the electronic device (101) can provide information (211) (e.g., properties of the application, a screen displayed in the application, and / or content) to the input interface (220) through the application (210). Properties that can be set through the input interface (220) can be set based on information (211) (e.g., properties of the application, a screen displayed in the application, and / or content) provided from the application (210). The electronic device (101) can identify an artificial intelligence model to be used to generate a sentence based on at least one of the parameters (e.g., text related to a topic, values of each attribute) that can be set via the information (211) and / or the input interface (220). For example, the electronic device (101) can transmit request information (221) to the first artificial intelligence model (230) via the input interface (220). The request information (221) can be a prompt generated based on the attribute data and the text data. The prompt can be used to request the first artificial intelligence model (230) to generate at least one sentence. The first artificial intelligence model (230) can provide response information (231) including the at least one sentence to the input interface (220). For example, the electronic device (101) can transmit request information (921) to the second artificial intelligence model (930) via the input interface (220). The request information (921) may be a prompt generated based on the above attribute data and the above text data.The above prompt can be used to request the second artificial intelligence model (930) to generate at least one sentence. The second artificial intelligence model (930) can provide response information (931) including the at least one sentence to the input interface (220). The electronic device (101) can obtain the at least one sentence from the first artificial intelligence model (230) or the second artificial intelligence model (930). The electronic device (101) can provide the at least one sentence (241) to the application (210) so that the result obtained through the input interface (220) can be displayed in an input area on the application (210). The electronic device (101) can display the at least one sentence (241) in the input area.
[0142] According to one embodiment, the electronic device (101) may determine an artificial intelligence model to be used for sentence generation based on the network status of the electronic device (101). For example, the first artificial intelligence model (230) may be located in an external electronic device (e.g., server (108), cloud), and the second artificial intelligence model (930) may be located within the electronic device (101). The first artificial intelligence model (230) may provide higher performance than the second artificial intelligence model (930). For example, the amount of learning of the first artificial intelligence model (230) may be greater than the amount of learning of the second artificial intelligence model (930). For example, the number of layers of the first artificial intelligence model (230) may be greater than the number of layers of the second artificial intelligence model (930). The electronic device (101) may use the first artificial intelligence model (230) when communication with the external electronic device is possible (e.g., when Internet access is possible). The electronic device (101) can use the second artificial intelligence model (930) when communication with the external electronic device is difficult (e.g., when Internet access is difficult).
[0143] According to one embodiment, the electronic device (101) can obtain sentences generated using the first artificial intelligence model (230) and correct the sentences using the second artificial intelligence model (930). For example, the first artificial intelligence model (230) can be located in an external electronic device (e.g., server (108), cloud), and the second artificial intelligence model (930) can be located inside the electronic device (101). The electronic device (101) can perform corrections (e.g., addition, deletion, correction of personal information) on sentences received from the first artificial intelligence model (230) through the second artificial intelligence model (930).
[0144] According to one embodiment, the electronic device (101) may input an image into an input area (e.g., an area where text or images are input). The input image may be used for sentence generation. For example, the first artificial intelligence model (230) may be LLM and the second artificial intelligence model (930) may be LMM. A user of the electronic device (101) may perform a posting through an SNS application. The electronic device (101) may receive a user input for a sentence generation request. In response to the user input, the electronic device (101) may acquire image data corresponding to the image on the input area as a context. The electronic device (101) may include the image data in a first prompt. For example, the electronic device (101) may convert data regarding the acquired image into string data. The electronic device (101) may input the converted string data into the context of the first prompt. For example, the electronic device (101) can encode JPEG image data into 'Base64 String' data and include the result of the encoding as the context of the first prompt. The electronic device (101) can transmit the first prompt to the second artificial intelligence model (930). The electronic device (101) can request a description of the image from the second artificial intelligence model (930). The electronic device (101) can receive description information about the image from the second artificial intelligence model (930) in the form of text. The electronic device (101) can use the received text as the context of the second prompt for sentence generation. The electronic device (101) can generate the second prompt based on the received text, text data received through the first area (440a) of the second input interface (440), and / or attribute data set through the second area (440b) of the second input interface (440). The electronic device (101) can provide the second prompt to the first artificial intelligence model (230).Through this, the electronic device (101) can receive the following sentence from the first artificial intelligence model (230).
[0145]
[0146] According to one embodiment, the electronic device (101) may not only input text or an image into an input area (e.g., an area where text or an image is input), but may also request an artificial intelligence model to modify content. The electronic device (101) may identify an artificial intelligence model that supports modifying the content. For example, the first artificial intelligence model (230) may be an LLM, and the second artificial intelligence model (930) may be an LMM. For example, a user of the electronic device (101) may modify a photo. The electronic device (101) may generate a prompt for modifying the photo based on at least one of an image corresponding to the photo, properties that can be set via the input interface (220), or information (211) provided from an application (210) (e.g., properties of the application, a screen displayed in the application, and / or content). The electronic device (101) may identify the second artificial intelligence model (930) as an artificial intelligence model that matches the function according to the prompt.
[0147] As described with reference to FIGS. 2 to 9, the electronic device (101) may generate a prompt based on contextual information related to the application, text data corresponding to the input text, and / or attribute data corresponding to the set attributes. The more information is included in the prompt, the more the artificial intelligence model can generate sentences that better match the user's intention. Meanwhile, the electronic device (101) may not include all of the acquired information in the prompt, but may include at least a portion of the acquired information in the prompt. According to one embodiment, if there is information among the information to be included in the prompt that is designated as personal information to be protected or information determined by the electronic device (101) to be personal information, the electronic device (101) may designate the information as a protection target. The electronic device (101) may change the information as a protection target to substitute information. For example, the electronic device (101) may include classification information for the protection target or keywords, symbols, and / or text in a pre-arranged format in the prompt as the substitute information. The electronic device (101) may provide the prompt to the artificial intelligence model (230). The artificial intelligence model (230) may generate at least one sentence in response to the prompt. The at least one sentence may include a portion corresponding to the substitute information (hereinafter, referred to as a protected portion). The artificial intelligence model (230) may input the substitute information (e.g., classification information or keywords, symbols, and / or text in a predetermined format) into the protected portion in the at least one generated sentence. The electronic device (101) may replace the substitute information with actual information in the sentence received from the artificial intelligence model (230). For example, the electronic device (101) may display the protected portion and the non-protected portion separately. As an example, the electronic device (101) may further display an emphasis effect (e.g., highlight) on the protected portion.When a user of the electronic device (101) selects the above-described protected portion, the electronic device (101) may provide a user interface that allows text input or selection from among suggested candidate texts. For example, the prompt may include '[account number]' as substitute information instead of the actual account number. The sentence generated from the artificial intelligence model (230) may include '[account number]'. In response to the user input for '[account number]', the electronic device (101) may replace '[account number]' in the sentence with the actual account number pre-stored in the electronic device (101). As a non-limiting example, instead of replacing '[account number]' in the sentence with the actual account number pre-stored, a user interface for entering a separate account number may be displayed.
[0148] Figures 10a through 10e illustrate examples of user interfaces for generating content (e.g., sentences, images, videos, audio, and / or combinations thereof) based on images. Like reference numerals may represent like descriptions.
[0149] Referring to FIG. 10A, an electronic device (101) can display a screen (1001). The electronic device (101) can display a user interface (1010) of an application. For example, the application may be a social network service (SNS) application. A user of the electronic device (101) can input text for a new posting.
[0150] The screen (1001) may include a text input area (1011) and an image area (1013). The text input area (1011) may be an area where text is displayed. The image area (1013) may be an area where an image to be uploaded is displayed. An image input into the image area (1013) may be used to generate a prompt. The electronic device (101) may use image data of the image as a context. The screen (1001) may include a first input interface (430). For example, the first input interface (430) may include a keyboard interface (431) and visual objects for a specified function (e.g., a first visual object (441), a second visual object (442), a third visual object (443), a fourth visual object (444), a fifth visual object (445), and / or a sixth visual object (446)). For example, the first visual object (441) can be used for AI-based sentence generation. The second visual object (442) can be used for icon input. The third visual object (443) can be used for translation. The fourth visual object (444) can be used for a clipboard. The fifth visual object (445) can be used for keyboard settings. The sixth visual object (446) can be used to display additional functions.
[0151] Referring to FIG. 10b, an electronic device (101) may display a screen (1002). The electronic device (101) may receive a user input for a first visual object (441). In response to the user input, the electronic device (101) may display a screen (1002). The screen (1002) may display a second input interface (440). The second input interface (440) may include a text area (440a) and an attribute area (440b). The text area (440a) may include a text input window. The text input window may be used to input a subject of a text to be written. The attribute area (440b) may include fields for specifying attributes and values of the attributes. For example, the attribute area (440b) may include an area (451) for a first attribute (e.g., format), an area (452) for a second attribute (e.g., length), an area (453) for a third attribute (e.g., tone), and an area (454) for a fourth attribute (e.g., language). The second input interface (440) may include a visual object (460) for generating sentences. The electronic device (101) may receive a user input (1091) on the first input interface (430).
[0152] Referring to FIG. 10c, the electronic device (101) can display a screen (1003). The electronic device (101) can generate text data corresponding to a user input (1091). The electronic device (101) can display text (1065) (e.g., 'So cute') represented by the text data on the first area (440a). The electronic device (101) can receive a user input (1092) for a visual object (460) on the second input interface (440).
[0153] The electronic device (101) can generate a prompt in response to a user input (1092). The electronic device (101) can obtain text data corresponding to at least one word (e.g., 'So cute') on a text area (440a) of the second input interface (440). The electronic device (101) can obtain attribute data corresponding to values set for each attribute (e.g., Format: Social, Length: Standard, Tone: Casual, Language: English) on an attribute area (440b) of the second input interface (440). The electronic device (101) can use image data as context, as described in FIG. 10A. The electronic device (101) can transmit the image data to a separate artificial intelligence model (e.g., the second artificial intelligence model (930), LMM). The electronic device (101) can receive an expression text representing a description of the image received from the artificial intelligence model. The electronic device (101) can input the expression text as context data of the prompt. The electronic device (101) can generate a prompt based on the context data, the text data, and the attribute data. For example, the electronic device (101) can generate a prompt as follows.
[0154]
[0155] The device (101) may transmit the generated prompt to an artificial intelligence model (e.g., artificial intelligence model (230)). Referring to FIG. 10d, the electronic device (101) may display a screen (1004). The screen (1004) may include a result area (520). The result area (520) may include a result text window (521), a first visual object (531) for requesting regeneration (e.g., 'Regenerate'), and a second visual object (532) for requesting input into a text input area (e.g., 'Insert'). The result area (520) may include a first visual object (531) for requesting regeneration and a second visual object (532) for requesting input into a text input area.
[0156] According to one embodiment, the electronic device (101) may obtain voice information as an output of the artificial intelligence model (230). The artificial intelligence model (230) may generate the voice information. The voice information may correspond to a result on a prompt input through the electronic device (101). For example, the artificial intelligence model (230) may generate voice information corresponding to text data and attribute data requested by the user. As an example, voice data in the form of a user reading sentences generated by the artificial intelligence model (230) in a specific voice may be generated as the voice information. As another example, background music associated with sentences generated by the artificial intelligence model (230) may be generated as the voice information. As an example, when a cat image is input, a cat's meowing sound may be generated as the voice information. Such voice information may be provided as a replacement for sentences generated by the artificial intelligence model (230) or may be provided additionally in addition to the sentences generated by the artificial intelligence model (230).
[0157]
[0158] Although examples in which the LLM and LMM are configured separately are described in FIGS. 10A to 10E, embodiments of the present disclosure are not limited thereto. In one embodiment, the LLM AI engine and the LMM AI engine may be implemented as a single artificial intelligence model (e.g., artificial intelligence model (230)).
[0159] Figure 11 illustrates an example user interface for setting the length of content to be generated (e.g., sentences, images, videos, audio, and / or combinations thereof). Like reference numbers may represent like descriptions.
[0160] Referring to FIG. 11, the electronic device (101) may display a screen (1101). The screen (1101) may include a first input interface (430). For example, the first input interface (430) may include a keyboard interface (431) and visual objects for designated functions (e.g., a first visual object (441), a second visual object (442), a third visual object (443), a fourth visual object (444), a fifth visual object (445), and / or a sixth visual object (446)). The keyboard interface (431) may include a keyboard on which character buttons of designated shapes for entering characters are arranged. The screen (1101) may include a result area (520). The result area (520) may include a result text window (521), a first visual object (531) for requesting regeneration (e.g., 'Regenerate'), and a second visual object (532) for requesting input into a text input area (e.g., 'Insert'). The result area (520) may include a first visual object (531) for requesting regeneration and a second visual object (532) for requesting input into a text input area.
[0161] An electronic device (101) may provide a user with an option to adjust the length of a sentence. The electronic device (101) may display a first option (1151) for setting the length attribute to 'Standard' and a second option (1152) for setting the length attribute to 'Detail'. When the first option (1151) is selected on the first screen (1101), the electronic device (101) may display at least one sentence according to the length attribute set to 'Standard'. The length attribute set to 'Standard' may indicate that the maximum number of sentences that can be displayed is two. For example, the at least one sentence displayed on the first screen (1101) may be 'Just like you. I'm interested about why Koreans enjoy iced drinks even when it's chilly outdoors'. When a user input (1191) for the second option (1152) is received, the electronic device (101) may display a second screen (1102). The second screen (1102) may display at least one sentence based on a length attribute according to the second option (1152). A length attribute set to "detail" may indicate that the maximum number of displayable sentences is five. For example, the at least one sentence displayed on the second screen (1102) may be "I share your interest in why Koreans opt for iced beverages even during colder months. From what I've gathered so far, there isn't one concrete reason—just speculations. One theory suggests it may have something to do with our tradition of consuming cold water consistently throughout the year. But again, this...'may be. When a user input (1192) for the first option (1151) is received on the second screen (1102), the electronic device (101) may display the first screen (1101).
[0162] According to one embodiment, the electronic device (101) may receive a plurality of sentences generated with different lengths. For example, the electronic device (101) may obtain a first set of sentences corresponding to a length attribute set to a 'standard' value and a second set of sentences corresponding to a length attribute set to a 'detail' value from the artificial intelligence model (230). Thereafter, the electronic device (101) may display the sentences corresponding to the set length attributes in a result text window (521) of a result area (520) according to a user input (e.g., user input (1191), user input (1192)).
[0163] A user of an electronic device (101) may select a first visual object (531) to request regeneration if the generated sentences do not match the user's intention. The electronic device (101) may receive a user input for the first visual object (531). The electronic device (101) may generate a prompt in response to the user input. The electronic device (101) may generate the prompt based on preset parameters (e.g., context, text data, attribute data). The electronic device (101) may transmit the generated prompt to an artificial intelligence model (230). The electronic device (101) may receive at least one regenerated sentence from the artificial intelligence model (230). The electronic device (101) may display the at least one regenerated sentence in a result text window (521) of a result area (520). As a non-limiting example, the electronic device (101) may change an argument whenever a user input for the first visual object (531) is received. The argument may be a parameter used for generating a prompt that is not displayed to the user. The electronic device (101) may change the value of the argument whenever a user input for regenerating a sentence (e.g., a user input for the first visual object (531)) is received. The argument may be an internal parameter that is not set on the input interface (230) (e.g., the first input interface (430), the second input interface (440)). The internal parameter may be included when generating the prompt. As an example, the argument may represent a level for an attribute. As another example, the argument may represent a number of re-requests. As yet another example, the argument may be new context information acquired at the time of requesting regeneration.
[0164] According to one embodiment, a user of an electronic device (101) may select a sentence that does not match the intent from among generated sentences. The user of the electronic device (101) may select a first visual object (531) for requesting regeneration along with the selection of the sentence. The electronic device (101) may receive a user input for the first visual object (531). The electronic device (101) may obtain information about the selected sentence from among the displayed sentences. The selected sentence may represent a sentence that the user determines does not match the user's intent. The electronic device (101) may generate a prompt in response to the user input. The electronic device (101) may include information in the prompt indicating that the user has not reflected the information about the selected sentence. For example, the electronic device (101) may regenerate the prompt by changing the degree of the length property and / or the degree of the tone property of the sentence related to the sentence.
[0165] Figures 12a to 12c illustrate examples of user interfaces for generating content (e.g., sentences, images, videos, audio, and / or combinations thereof) in a shopping service. Figures 12a to 12c illustrate a situation in which an electronic device (101) accesses a website providing a shopping service via an Internet browser application. The same reference numbers may be used throughout the description.
[0166] Referring to FIG. 12A, the electronic device (101) may display a screen (1201). The screen (1201) may include information regarding the address of a website accessed via an Internet browser application. The website may be a website providing a shopping service. For example, the screen (1201) may be a review writing screen for a specific purchased product on the website. The screen (1201) may include content (e.g., an image, text (1210) (satisfaction rating: 5 points), and text (1211) (e.g., "Please leave a review of your satisfaction level after using the device for a month (at least 10 characters)").
[0167] The electronic device (101) may display a screen (1202) in response to a user input (491) while displaying a screen (1201). For example, the user input (491) may be an input (e.g., a touch input) for selecting a text input area (411). The user input (491) may be used to write a product review. The above content (e.g., image, text (1210) (satisfaction level 5 points), text (1211) (e.g., 'Please leave a review about the change in satisfaction after using it for a month (at least 10 characters)') can be input as the context of a prompt that is input (or transmitted) to an artificial intelligence model for outputting at least one sentence to be generated. The electronic device (101) can display a screen (1202). The screen (1202) can include a keyboard and visual objects for executing a specific function. For example, the screen (1202) can include a first visual object (441) used for AI-based sentence generation. The electronic device (101) can receive a user input (492) while displaying the screen (1202). The electronic device (101) can receive a user input (492) for selecting the first visual object (441) for AI-based sentence generation and / or editing.
[0168] Referring to FIG. 12B, the electronic device (101) can display a screen (1203). While displaying the screen (1202), the electronic device (101) can display the screen (1203) in response to a user input (492). The electronic device (101) can display the screen (1203) in response to a user input (492) for selecting a first visual object (441) for AI-based sentence generation and / or editing. The screen (1203) can include at least one visual object for sentence generation and editing (e.g., a seventh visual object (447), an eighth visual object (448), and a ninth visual object (449). For example, the seventh visual object (447) can be used for AI-based sentence generation. The eighth visual object (448) can be used for style conversion of text data input into the text input area (411). The ninth visual object (449) can be used for spelling and grammar correction of text data entered into the text input area (411).
[0169] The electronic device (101) may display a screen (1204) in response to a user input (492-1) for selecting a seventh visual object (447) for AI-based sentence generation. The screen (1204) may include a second input interface (440) including a text area (440a) and an attribute area (440b). For example, the text area (440a) may be used to input a subject to be written. The attribute area (440b) may include fields for specifying attributes and values of the attributes. For example, the attribute area (440b) may include an area (451) for a first attribute (e.g., format) and an area (453) for a third attribute (e.g., tone). In one embodiment, the second input interface (440) may not include a field for specifying a language attribute corresponding to the second attribute. The above language attribute may be identified based on context information (e.g., purchase product information) obtained from the application. The second input interface (440) may include a visual object (460) for generating a sentence. For example, the first attribute may be a format attribute, and the format attribute may be set to 'standard'. The third attribute may be a tone attribute, and the tone attribute may be set to 'polite'. As a non-limiting example, the tone attribute may be a default value of the tone attribute according to the format attribute. The electronic device (101) may generate text data corresponding to a user input on a keyboard interface. The text data may include at least one word (465). For example, at least one word (465) may represent '10 out of 10'. In response to user input (496) for a visual object (460) for sentence generation, a prompt can be generated while at least one word (465) is displayed on a text area (440a).The electronic device (101) can generate a prompt based on at least one of the content included in the context (e.g., content on the review writing screen (1201) (e.g., text (1210), text (1211)), the topic entered in the text area (440a) (e.g., '10 out of 10'), and / or the setting value of each attribute entered in the attribute area (440b). For example, when the format attribute is set to 'standard', the prompt can further include a sentence for requesting the generation of at least one sentence and a sentence for requesting the generation of an emoji. The generated prompt is transmitted to an artificial intelligence model (e.g., artificial intelligence model (230)), and the artificial intelligence model can generate at least one sentence based on the prompt and output the generated sentence.
[0170] Referring to FIG. 12C, the electronic device (101) may display a screen (1205). The electronic device (101) may display the screen (1205) in response to a user input (496) regarding a visual object (460) for sentence generation. Text and visual objects (461) indicating that sentence generation (or writing) is in progress by AI may be provided as an animation effect. When an AI-based generated sentence is received or the AI-based sentence generation is completed, the electronic device (101) may display the screen (1206).
[0171]
[0172] According to one embodiment, the electronic device (101) may receive a user input for selecting a first visual object (531) for requesting regeneration after a user's editing operation is performed on at least one generated sentence in a result text input window (521). In response to the user input, the electronic device (101) may regenerate at least one sentence as (or based on) the edited at least one sentence as a 'subject'. According to one embodiment, the result area (520) may further include a first visual object (531) (e.g., Refresh 1) for requesting regeneration for at least one word (465) input by the user as a 'subject' and a visual object (e.g., Refresh 2) for requesting regeneration for at least one edited sentence. Based on the user input for the visual objects (e.g., Refresh 1, Refresh 2) for the regeneration request, the electronic device (101) may regenerate at least one sentence.
[0173] Figures 13a through 13c illustrate examples of user interfaces for generating content (e.g., sentences, images, videos, audio, and / or combinations thereof) based on alternative information. Figures 13a through 13c assume a situation where an email application displays a mail composition window. Figures 13a through 13c illustrate an embodiment where additional information is input via a link to another app linked to a keyword. The same reference numbers may be used throughout the description.
[0174] Referring to FIG. 13A, the electronic device (101) may display a screen (1301). The screen (1301) may be an email writing screen of an email application. In response to a user input (491) for selecting a text input window (411) for writing an email body, the electronic device (101) may display a first visual object (441) and a keyboard interface for AI-based sentence generation and / or editing, overlapping a portion of the screen (1301). In response to a user input (492) for selecting the first visual object (441), the electronic device (101) may display a screen (1302).
[0175] The screen (1302) may include at least one visual object (e.g., a seventh visual object (447), an eighth visual object (448), and a ninth visual object (449)) for sentence generation and editing. For example, the seventh visual object (447) may be used for AI-based sentence generation. The eighth visual object (448) may be used for style conversion of text data input into the text input area (411). The ninth visual object (449) may be used for spelling and grammar correction of text data input into the text input area (411). The electronic device (101) may receive a user input (492-1) for selecting the seventh visual object (447) for AI-based sentence generation.
[0176] Referring to FIG. 13B, the electronic device (101) can display a screen (1303). The electronic device (101) can display the screen (1303) in response to a user input (492-1). The format attribute of the email application can be set to 'email' and the tone attribute can be set to the default value 'professional' according to the format attribute. The electronic device (101) can generate text data corresponding to the user input. The text data can include at least one word (465). For example, the at least one word (465) can represent 'moving visit estimate'. The electronic device (101) can display the at least one word (465) on the text area (440a). While the at least one word (465) is displayed, the electronic device (101) can receive a user input (496) for a visual object (460) for sentence generation. The electronic device (101) may acquire at least one sentence in response to a user input (496). Although not illustrated in FIG. 13B, in response to a user input (496) for a visual object (460) for sentence generation, text and visual objects may be provided as animation effects to indicate that sentence generation (or writing) is in progress by AI. When the AI-based generated sentence is received or the AI-based sentence generation is completed, the electronic device (101) may display a screen (1304).
[0177] The screen (1304) may include a result text window (521). At least one AI-generated sentence may be displayed in the result text window (521). The at least one generated sentence may include at least one alternative information (e.g., first alternative information (1311), second alternative information (1312), and third alternative information (1313)) including keywords, symbols, and / or text in a predetermined format. As an example, the first alternative information (1311) may be 'planned moving date'. As an example, the second alternative information (1312) may be 'current address'. As an example, the third alternative information (1313) may be 'new address'. For example, the predetermined symbol may be '[ ]'. In the at least one sentence, alternative information and non-alternative information may be displayed separately. For example, the alternative information may be displayed using an emphasis effect (e.g., green highlighting).
[0178] Referring to FIG. 13C, the electronic device (101) may display a screen (1305). The screen (1305) may include a result text window (521). At least one sentence of the result text window (521) may include at least one alternative information (e.g., first alternative information (1311), second alternative information (1312), and third alternative information (1313)) including keywords, symbols, and / or text in a predetermined format. The electronic device (101) may receive a user input (1391) for selecting one of the at least one piece of alternative information (e.g., first alternative information (1311)).
[0179] The electronic device (101) can display a screen (1306) based on a user input (1391). The electronic device (101) can identify an application (e.g., a calendar application) linked to the selected alternative information (e.g., the first alternative information (1311)) in response to the user input (1391). The electronic device (101) can display an app execution screen (1321) through link information of the identified application. For example, the app execution screen (1321) can be displayed in a floating form. Unlike as illustrated in FIG. 13b, if the pre-arranged keyword is not 'date' but 'address' (e.g., the second alternative information (1312), the third alternative information (1313)), the electronic device (101) can identify a map application and display an app execution screen corresponding to the map application.
[0180] The electronic device (101) may display a screen (1307) based on a user input (e.g., a user input for selecting a specific date) on an app execution screen (1321). The date specified by the user input may be displayed in the result text window (521) in place of the second alternative information (1312).
[0181] Figure 14 is a schematic diagram of an exemplary AI system.
[0182] Referring to FIG. 14, the AI system (1400) may include an input / output interface (1410), an AI framework (1420), a generative AI model (1430) (e.g., an artificial intelligence model (230)), and / or a knowledge repository (1490).
[0183] The input / output interface (1410) can receive input. The input can include user input and / or data acquired or generated by an electronic device (e.g., the electronic device (101) described above). The data can include images, videos, and / or sensor data generated by at least one processor (e.g., at least one processor (210) or processor (1720)) of the electronic device (e.g., illuminance data around the electronic device acquired from a sensor or sensor hub (e.g., a coprocessor (1723), posture data (or orientation data) of the electronic device, temperature inside the electronic device (e.g., temperature of the display (220) or temperature of the at least one processor (210)), size information of a display area of the display (220), and / or images acquired through an image sensor (e.g., included in a camera module (1780)) of the electronic device). The user input may include natural language, touch data obtained through touch circuitry included within the display panel (160) (e.g., used to identify input from a finger and / or a stylus), images displayed (and / or to be displayed) on the display panel (160), and / or video. As a non-limiting example, the user input may be received by the input / output interface (1410) together with context information. The context information may be described as additional information obtained in relation to the user input. The context information may relate to a state when the user input is received (e.g., including a state of the electronic device and / or a state surrounding the electronic device (e.g., a user state)). For example, the context information may include information about one or more software applications running within the electronic device when the user input is received.For example, the contextual information may include information about the location of the electronic device (or the location of the user of the electronic device) at the time the user input is received. For example, the user input may be integrated with the contextual information. For example, the user input integrated with the contextual information may be received by the input / output interface (1410).
[0184] The input / output interface (1410) can transmit (or provide) output. The output may include a result (or result information) generated or obtained by the AI system (1400) based at least in part on the input. The format of the output may vary. For example, the output may include natural language. For example, the output may include content (e.g., including media content and / or multimedia content). For example, the output may include an action related to a user of the electronic device. For example, the output may have a format according to a user setting of the electronic device.
[0185] The input / output interface (1410) can be described as a user query / response interface (1410).
[0186] The AI framework (1420) can be used to obtain information (or data) about the input from the input / output interface (1410) and control one or more components related to the AI system (1400) using the obtained information.
[0187] For example, the prompt design component (1421) within the AI framework (1420) can use the acquired information to generate or obtain a prompt for a generative AI model (1430) (e.g., including an LLM (large language model) or an LMM (large multimodal model)). For example, the prompt design component (1421) can be described as an AI component that uses a learning algorithm and / or a neural network to provide enhanced prompts over time. For example, the prompt design component (1421) can use the acquired information to access a knowledge component (e.g., a knowledge repository (1490)) that includes user preference data, a prompt library, and / or prompt examples to generate or obtain a prompt. The generated prompt can be provided to the generative AI model (1430) (e.g., including an LLM or an LMM).
[0188] For example, the API / plugin management component (1422) within the AI framework (1420) may be utilized to facilitate communication for additional information requested (or induced) in connection with the prompt provided (or to be provided) to the generative AI model (1430) (e.g., the artificial intelligence model (230)). For example, the API / plugin management component (1422) may be utilized to create or establish channels for communication with various data sources (e.g., a knowledge repository (1490)). For example, the API / plugin management component (1422) may facilitate access to at least some of the data sources. For example, the API / plugin management component (1422) may be utilized to request another component (e.g., an application / service component (1480)) to perform feedback (or response) in response to the prompt. As a non-limiting example, information obtained (or generated) through the API / plugin management component (1422) may be provided to the prompt design component (1421) for generating a prompt. As a non-limiting example, information obtained (or generated) through the API / plugin management component (1422) may be provided to a generative AI model (1430) (e.g., an artificial intelligence model (230)).
[0189] For example, the improvement component (1423) within the AI framework (1420) can at least partially tune (or adjust) (or change) the result (e.g., content) obtained (or output) from the generative AI model (1430) (e.g., the artificial intelligence model (230)). For example, the improvement component (1423) can determine or verify whether the content obtained from the generative AI model (1430) is related to the input. For example, the improvement component (1423) can determine or verify whether the content obtained from the generative AI model (1430) includes biased content. For example, the improvement component (1423) can determine or verify whether the content obtained from the generative AI model (1430) includes harmful content. For example, the improvement component (1423) may support or assist in performing additional processing to improve content obtained from the generative AI model (1430). For example, the improvement component (1423) may support providing hints to the user to improve the content.
[0190] A generative AI model (1430) (e.g., artificial intelligence model (230)) can be described as an artificial intelligence neural network that generates feedback in response to a prompt. For example, the feedback may include additional data and / or information related to the prompt, but relative to the prompt. For example, the feedback may include new content related to the prompt. For example, the generative AI model (1430) may include a model that generates images and / or a model that generates language. For example, the model that generates images may include a generative adversarial network (GAN) and / or a variational autoencoder (VAE). For example, the model that generates images may include a diffusion-based generative model (e.g., a transformer VAE). For example, the model that generates language may include CHAT-GPT 3 and / or CHAT-GPT 4. For example, a generative AI model (1430) may include an LMM that generates the feedback by recognizing text, images, and / or speech.
[0191] As a non-limiting example, the AI framework (1420) and / or the generative AI model (1430) (e.g., the artificial intelligence model (230)) may be included within an AI module (e.g., including a processing circuit) within the electronic device. For example, the AI module may be operatively coupled with at least one processor of the electronic device (e.g., at least one processor (210) or processor (1720)). For example, the AI module may be operatively coupled with a display driving circuit of the electronic device (e.g., the display driving circuit (221) or DDI (1830)). For example, the AI module may be operatively coupled with a sensor hub of the electronic device for one or more sensors within the electronic device.
[0192] An electronic device (101) according to embodiments of the present disclosure can display sentences that match the user's intention to a user of the electronic device (101) by generating a sentence based on one or more attributes related to the sentence to be generated. By generating a prompt based on contextual information related to the application as well as the running application, the electronic device (101) can make the result of sentence generation using an AI model more match the user's intention. In other words, the electronic device (101) according to embodiments of the present disclosure can provide sentences that match the user's intention by determining the value of an attribute related to the sentence to be generated based on a user's input and / or the application in use, and obtaining a sentence generated according to the attribute of the determined value.
[0193] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.
[0194] An electronic device (101) according to embodiments of the present disclosure can perform a sentence generation function.
[0195] Although the above-described examples describe an example in which an additional input interface (e.g., a second input interface (440)) is displayed in addition to a keyboard interface (e.g., a first input interface (430)) to set the properties of a sentence to be generated, embodiments of the present disclosure are not limited thereto. According to one embodiment, the electronic device (101) may generate text data through the keyboard interface. Thereafter, when an input to a visual object (e.g., a first visual object (441)) for AI-based sentence generation is detected, the electronic device (101) may generate a prompt based on property data related to the application and the text data. Each property of the property data may be set based on the context of the application. The electronic device (101) may generate property data without user intervention to arbitrarily change the properties.
[0196] Although the examples described above describe an example in which a result text window (521) is displayed to display generated sentences, embodiments of the present disclosure are not limited thereto. Instead of displaying the result text window (521), a pop-up UI may be displayed that prompts the user to input sentences received from the artificial intelligence model (230). Thereafter, when a user's acceptance input is received, the electronic device (101) may display the sentences in a text input area (e.g., text input area (411)). As a non-limiting example, sentences received from the artificial intelligence model (230) may be displayed directly in a text field without the pop-up UI.
[0197] Although the above-described examples describe an example in which a separate visual object (e.g., a first visual object (441)) for executing the AI-based sentence generation function is displayed on the keyboard interface, embodiments of the present disclosure are not limited thereto. Instead of always displaying the visual object on the input interface, the electronic device (101) may display a visual object (e.g., an icon, an indicator) for executing the AI-based sentence generation function on the user interface when a user's text input is detected. The electronic device (101) may execute the AI-based sentence generation function in response to a user input for the visual object. For example, the electronic device (101) may display a second input interface (440) for specifying an attribute in response to the user input or may generate a prompt based on currently set information. As a non-limiting example, the visual object may be a floating icon.
[0198] According to one embodiment, the electronic device (101) may execute an AI-based sentence generation function while a first application is running. For example, the electronic device (101) may receive an input of 'funny' as text data. The values of attributes related to a sentence to be generated as attribute data may be set to default values that are automatically set according to the first application. The electronic device (101) may obtain at least one sentence generated based on the set values and the input text while the first application is running. Thereafter, the electronic device (101) may execute an AI-based sentence generation function while a second application is running. The electronic device (101) may obtain at least one sentence generated based on the same text data and attribute data while the second application is running. The implementation of the present disclosure can be confirmed based on the difference in characteristics (e.g., number of sentences, tone, sentence length, language, format) of sentences generated through the first application and the characteristics (e.g., number of sentences, tone, sentence length, language, format) of sentences generated through the second application. For example, the first application may be an email application and the second application may be an SNS application. The implementation of the present disclosure can be confirmed based on the fact that the number of sentences generated for the first application is greater than the number of sentences generated for the second application. As another example, assume a case where a first set of sentences is generated in a first application that has no character limit and a case where a second set of sentences is generated in a second application that has a character limit. The implementation of the present disclosure can be confirmed based on the fact that the total length of the sentences in the first set is longer than the total length of the sentences in the second set.
[0199] In one embodiment, the electronic device (101) can execute an AI-based sentence generation function while a browser application is running. The electronic device (101) can access a post through the browser application. For example, when accessing a post written in English, the electronic device (101) can obtain a first set of sentences through the AI-based sentence generation function. For example, when accessing a post written in Korean, the electronic device (101) can obtain a second set of sentences through the AI-based sentence generation function. The implementation of the present disclosure can be confirmed by the fact that the language used in the first set of sentences differs from the language used in the second set of sentences.
[0200] In embodiments, an electronic device (101) is provided. The electronic device (101) includes a display; at least one processor; and a memory storing instructions, which, when executed by the at least one processor, collectively or individually cause the electronic device (101) to display a user interface including a text input area through the display while an application is being executed, display an input interface through the display in response to a user input while the user interface is being displayed, generate text data through the input interface, obtain at least one sentence generated based on the text data and attribute data related to the sentence to be generated, and display the at least one sentence in the text input area through the display. The attribute data may be set based on the application.
[0201] For example, the instructions, when executed by the at least one processor, collectively or individually, may cause the electronic device (101) to generate a prompt based on the text data and the attribute data to obtain the at least one sentence, and to obtain the at least one sentence generated by the generative artificial intelligence model (230) according to the prompt.
[0202] For example, the instructions, when executed by the at least one processor, collectively or individually, may cause the electronic device (101) to transmit a request message including the prompt to a server including the generative artificial intelligence model (230) to obtain the at least one sentence, and to receive a response message including the at least one sentence from the server.
[0203] For example, the user input may include a touch input to the text input area. The input interface may include a text area for generating the text data and an attribute area for generating the attribute data.
[0204] For example, the attribute data may be generated based on context information included in a screen displayed through the application or a screen currently being displayed through the application. The context information may include at least one of text, an image, a graphical user interface (GUI) attribute, or a user setting for the application.
[0205] For example, the attribute data may include a language attribute. If the application is used to compose a new email, the language attribute may be set to the default language of the input interface. If the application is used to compose a reply to a received email, the language attribute may be set to the language used in the received email.
[0206] For example, a first property among the properties of the above property data may be set based on the application. A second property among the properties of the above property data may be set based on user input.
[0207] For example, the instructions, when executed by the at least one processor, may collectively or individually cause the electronic device (101) to receive a user input for refreshing while displaying the at least one sentence in the text input area, and, in response to the user input for refreshing, obtain at least one other sentence based on the text data and the attribute data, and display the at least one other sentence in the text input area.
[0208] In embodiments, the electronic device (101) may include at least one processor; and a memory storing instructions. The instructions, when executed by the at least one processor, may collectively or individually cause the electronic device (101) to display a user interface including a text input area through a touch screen, and, based on at least a portion of a user input related to the text input area, display an input interface including a first area corresponding to a keyboard and a second area for setting one or more properties for text input into the text input area, receive at least one word from a user through the first area, obtain at least one sentence generated using the at least one word and the one or more properties, and display the at least one sentence in the text input area.
[0209] For example, the user interface may be displayed in response to the execution of a first application. At least one of the one or more properties may be automatically set based on contextual information including the first application. The second area may include a settings interface for the user to change the at least one property.
[0210] For example, the instructions, when executed by the at least one processor, may collectively or individually cause the electronic device (101) to determine whether application classification information pre-specified for the first application is identified, and based on a determination that the application classification information is identified (e.g., based on a determination that the application classification information is identified), to use the application classification information as the context information, and based on a determination that the application classification information is not identified (e.g., based on a determination that the application classification information is not identified), to use content displayed on the user interface as the context information.
[0211] For example, the context information may include a uniform resource locator (ULR) connected via the first application.
[0212] For example, the context information may include a uniform resource locator (URL) for providing the user interface.
[0213] For example, the context information may include information obtained from a first application of the user interface or a second application associated with the at least one word.
[0214] For example, the at least one sentence may be generated by a generative artificial intelligence model (230) that receives a prompt including the at least one word and the one or more attributes.
[0215] For example, the one or more properties may include a first property and a second property. A sub-property of the second property may change depending on a change in the first property.
[0216] For example, the first attribute may correspond to format, and the second attribute may correspond to tone.
[0217] For example, the format may correspond to a first format attribute or a second format attribute. If the format is set to the first format attribute, the generated at least one sentence may include sentences corresponding to each of the title, content, and conclusion. If the format is set to the second format attribute, the generated at least one sentence may include a hashtag.
[0218] For example, the one or more properties may further include at least one of a third property corresponding to length or a fourth property corresponding to language.
[0219] For example, the instructions, when executed by the at least one processor, may collectively or individually cause the electronic device (101) to display the first area in response to the selection of the text input area, and to display the second area and another text input area in response to a designated user input for the first area. While the other text input area is displayed, at least a portion of the text input area may not be displayed on the screen.
[0220] In embodiments, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium may include a memory storing instructions. The instructions, when executed by at least one processor, may collectively or individually cause an electronic device (101) to display a user interface including a text input area while an application is running on the electronic device (101), display the input interface in response to a user input while the user interface is displayed, generate text data through the input interface, obtain at least one sentence generated based on the text data and attribute data related to the sentence to be generated, and display the at least one sentence in the text input area. The attribute data may be set based on the application.
[0221] In embodiments, a method performed by an electronic device (101) is provided. The method may include: displaying a user interface including a text input area while an application is running on the electronic device (101); displaying the input interface in response to a user input while the user interface is displayed; generating text data through the input interface; obtaining at least one sentence generated based on the text data and attribute data related to the sentence to be generated; and displaying the at least one sentence in the text input area. The attribute data may be set based on the application.
[0222] For one or more embodiments, at least one of the components described in one or more of the preceding drawings may be configured to perform one or more operations, techniques, processes, and / or methods as described herein. For example, a processor (e.g., a baseband processor) described herein with respect to one or more of the preceding drawings may be configured to operate according to one or more examples described herein. For another example, circuitry associated with a user equipment (UE), a base station, a network element, and the like, as described above with respect to one or more of the preceding drawings, may be configured to operate according to one or more examples described herein.
[0223] Any of the embodiments described above may be combined with any other embodiment (or combination of embodiments) unless explicitly stated otherwise. The foregoing description of one or more implementations provides examples and descriptions, but is not intended to be exhaustive or limit the scope of the embodiments to the precise forms disclosed. Modifications and variations are possible in light of the above teachings or may be learned from practicing various embodiments.
[0224] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, electronic devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.
[0225] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.
[0226] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0227] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.
[0228] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0229] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.
Claims
1. In electronic devices, display; at least one processor; and A memory for storing instructions, wherein the instructions, when executed by the at least one processor, collectively or individually cause the electronic device to: While the application is running, a user interface including a text input area is displayed through the display, Displaying an input interface through the display in response to user input while the user interface is displayed; Generate text data through the above input interface, Obtain at least one sentence generated based on the above text data and attribute data related to the sentence to be generated, Causing said at least one sentence to be displayed through said display in said text input area, The above attribute data is set based on the above application, Electronic devices.
2. In claim 1, The instructions, when executed by the at least one processor, collectively or individually cause the electronic device to obtain the at least one sentence: Generate a prompt based on the above text data and the above attribute data, causing the acquisition of at least one sentence generated by a generative artificial intelligence model according to the above prompt; Electronic devices.
3. In claim 2, The instructions, when executed by the at least one processor, collectively or individually cause the electronic device to obtain the at least one sentence: Sending a request message including the prompt to a server including the generative artificial intelligence model, causing a response message containing at least one sentence to be received from the server; Electronic devices.
4. In claim 2, The above user input includes a touch input to the text input area, The above input interface includes a text area for generating the text data and an attribute area for generating the attribute data. Electronic devices.
5. In claim 1, The above attribute data is generated based on context information included in the screen displayed through the application or the screen being displayed through the application, The context information includes at least one of text, an image, a graphical user interface (GUI) attribute, or a user setting for the application. Electronic devices.
6. In claim 1, The above attribute data includes language attributes, When the above application is used to compose a new email, the language attribute is set to the language set as the default in the input interface, If the above application is used to write a reply to a received email, the language attribute is set to the language used in the received email. Electronic devices.
7. In claim 1, Among the properties of the above property data, the first property is set based on the application, Among the properties of the above property data, the second property is set based on user input. Electronic devices.
8. In claim 1, The above instructions, when executed collectively or individually by the at least one processor, cause the electronic device to: While displaying at least one sentence in the text input area, receiving user input for refreshing, In response to a user input for the above refresh, obtaining at least one other sentence based on the text data and the attribute data, causing at least one other sentence to be displayed in the text input area; Electronic devices.
9. In electronic devices, at least one processor; and A memory for storing instructions, wherein the instructions, when executed by the at least one processor, collectively or individually cause the electronic device to: Display a user interface including a text input area via a touch screen, Displaying an input interface including a first area corresponding to a keyboard and a second area for setting one or more properties for text entered in the text input area, based on at least a portion of user input related to the text input area; Through the first area, at least one word is received from the user, Obtaining at least one sentence generated using the at least one word and the one or more properties, causing at least one sentence to be displayed in the text input area; Electronic devices.
10. In claim 9, The above user interface is displayed in response to the execution of the first application, At least one of the above one or more properties is automatically set based on context information including the first application, The second area includes a settings interface for the user to change at least one property. Electronic devices.
11. In claim 9, The above instructions, when executed collectively or individually by the at least one processor, cause the electronic device to: Determine whether the application classification information pre-specified for the first application of the above user interface is confirmed, Based on the determination that the above application classification information is confirmed, the above application classification information is used as context information, Based on a determination that the above application classification information is not verified, causing the content displayed on the user interface to be used as at least part of the above context information. Electronic devices.
12. In claim 9, The above context information includes a uniform resource locator (URL) for providing the user interface. Electronic devices.
13. In claim 9, The context information includes information obtained from a first application of the user interface or a second application associated with the at least one word. Electronic devices.
14. In claim 9, wherein said at least one sentence is generated by a generative artificial intelligence model that receives a prompt including said at least one word and said one or more attributes. Electronic devices.
15. In a non-transitory computer-readable recording medium, Contains memory that stores instructions, The above instructions, when executed collectively or individually by at least one processor, cause the electronic device to: Displaying a user interface including a text input area while the application is running on the electronic device; Displaying an input interface in response to user input while the above user interface is displayed, Generate text data through the above input interface, Obtain at least one sentence generated based on the above text data and attribute data related to the sentence to be generated, causing at least one sentence to be displayed in the text input area; The above attribute data is set based on the above application, Non-transitory computer-readable recording medium.
Citation Information
Patent Citations
Terminal device and control method thereof
KR1020150000273A
Electronic apparatus method for controlling thereof and non-transitory computer readable recording medium
KR1020180102987A
Secondary battery
KR1020250030648A
Bamboo toothbrush manufacturing apparatus, bamboo toothbrush manufacturing method, and bamboo toothbrush manufactured thereof
KR102508482B1
Leveraging inferred context to improve suggested messages
US11962546B1