Electronic apparatus and control method thereof

Abstract

This electronic apparatus comprises at least one processor and a memory storing at least one instruction, wherein the instructions, when executed by the at least one processor, instruct the electronic device to: identify content preference information related to a user on the basis of natural language processing (NLP); determine content architecture information corresponding to a UI screen and visual element information corresponding to the UI screen on the basis of the content preference information and context information related to the user; and generate the UI screen on the basis of the content architecture information and the visual element information.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device providing a UI screen and a method for controlling the same.

[0002] Advances in electronic technology have led to the development and proliferation of various types of electronic devices. In particular, display devices such as TVs and laptops are being developed in various forms.

[0003] Typically, display devices offer a variety of UI screens, such as the home UI screen. However, users often face the inconvenience of having to go through numerous manipulations to customize the UI screens.

[0004] According to one or more embodiments, an electronic device includes: at least one processor; and a memory storing at least one instruction; wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to identify content preference information related to a user based on natural language processing (NLP), determine content architecture information corresponding to a user interface (UI) screen and visual element information corresponding to the UI screen based on the content preference information and context information related to the user, and generate the UI screen based on the content architecture information and the visual element information.

[0005] The at least one instruction may cause the electronic device to identify visual preference information related to the user and digital action preference information related to the user based on the natural language processing, determine integrated preference information related to the user based on the content preference information, the visual preference information, and the digital action preference information, and determine the context architecture information and the visual element information based on the integrated preference information and the context information.

[0006] The at least one instruction causes the electronic device to filter preferred content among a plurality of contents based on the content preference information and the context information, and input information about the preferred content into a first neural network model to determine the content architecture information, wherein the content architecture information may include category information and priority information about the filtered content.

[0007] The at least one instruction causes the electronic device to determine information related to a visual layout based on the content architecture information, and input the information related to the visual layout into a second neural network model to determine the visual element information, wherein the visual element information may include UI layout information and UI asset information for the UI screen.

[0008] The at least one instruction may cause the electronic device to filter preferred content among a plurality of contents based on the content preference information and the context information, and input information related to the preferred content and information related to the visual layout into a third neural network model to determine the content architecture information and the visual element information.

[0009] The at least one instruction may cause the electronic device to determine the content preference information by applying the natural language processing to user information obtained using at least one of an onboarding process of the electronic device, a chatbot service, a voice recognition secretary service, or an external device.

[0010] The at least one instruction may cause the electronic device to identify an intimacy level between the user and the electronic device based on at least one of a level of the user information, a frequency of use of the user in relation to the electronic device, or a usage behavior of the user in relation to the electronic device, select a question for obtaining additional information related to the user based on the intimacy level, obtain additional information of the user using at least one of the chatbot service and the voice recognition secretary service based on the selected question, and update the UI screen based on the additional information.

[0011] The at least one instruction may cause the electronic device to determine the context information based on an internal context of the electronic device, an external context of the electronic device, a real-time context of the user, and a usage context based on the user's usage history.

[0012] The at least one instruction may enable the electronic device to obtain the content preference information based on the natural language processing using a Large Language Model (LLM).

[0013] The at least one instruction may cause the electronic device to generate a personalized home UI screen including a personalized home background image, a content category, a representative image for each content category, and a UI font for the user based on the content architecture information and the visual element information.

[0014] A method for controlling an electronic device according to one embodiment includes: identifying content preference information of a user based on natural language processing (NLP); determining content architecture information corresponding to a user interface (UI) screen and visual element information corresponding to the UI screen based on the content preference information and context information related to the user; and obtaining the UI screen based on the content architecture information and the visual element information.

[0015] The control method further includes a step of identifying visual preference information related to the user and digital action preference information related to the user based on the natural language processing; and a step of determining integrated preference information related to the user based on the content preference information, the visual preference information, and the digital action preference information; wherein the context architecture information and the visual element information can be determined based on the integrated preference information and the context information.

[0016] The control method further includes a step of filtering preferred content among a plurality of contents based on the content preference information and the context information; and a step of inputting information about the preferred content into a first neural network model to determine the content architecture information; wherein the content architecture information may include category information and priority information about the filtered content.

[0017] The control method further includes a step of obtaining information related to a visual layout based on the content architecture information; and a step of inputting the information related to the visual layout into a second neural network model to obtain the visual element information; wherein the visual element information may include UI layout information and UI asset information for the UI screen.

[0018] In one embodiment, a computer instruction is stored that, when executed by a processor of an electronic device, causes the electronic device to identify content preference information related to a user based on natural language processing (NLP), determine content architecture information corresponding to a user interface (UI) screen and visual element information corresponding to the UI screen based on the content preference information and context information related to the user, and obtain the UI screen based on the content architecture information and the visual element information.

[0019] FIG. 1 is a diagram illustrating the operation of an electronic system according to one or more embodiments.

[0020] FIG. 2A is a block diagram showing the configuration of an electronic device according to one embodiment.

[0021] FIG. 2b is a block diagram specifically illustrating a configuration of an electronic device according to one or more embodiments.

[0022] FIG. 3 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0023] FIG. 4 is a diagram for explaining natural language processing according to one or more embodiments.

[0024] FIG. 5 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0025] FIG. 6a is a diagram for explaining an information acquisition method using a neural network model according to one or more embodiments.

[0026] FIG. 6b is a diagram for explaining an information acquisition method using a neural network model according to one or more embodiments.

[0027] FIG. 7 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0028] FIG. 8 is a diagram illustrating an information acquisition method using a neural network model according to one or more embodiments.

[0029] FIG. 9 is a diagram illustrating the operation of an electronic device according to one or more embodiments.

[0030] FIG. 10 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0031] FIG. 11a is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0032] FIG. 11b is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0033] FIG. 11c is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0034] FIG. 11d is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0035] FIG. 11e is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0036] FIG. 11f is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0037] FIG. 11g is a drawing for explaining a method for providing a personalized UI screen according to one or more embodiments.

[0038] FIG. 11h is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0039] FIG. 11i is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0040] FIG. 11J is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0041] FIG. 11k is a diagram illustrating a method for providing a personalized UI screen according to one or more embodiments.

[0042] FIG. 12A is a diagram illustrating an example of a personalized UI screen according to one or more embodiments.

[0043] FIG. 12b is a diagram illustrating an example of a personalized UI screen according to one or more embodiments.

[0044] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0045] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0046] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0047] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0048] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0049] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0050] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0051] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0052] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0053] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module, or may be implemented as a "module" or "part" that requires implementation with at least one processor or specific hardware.

[0054] According to one or more embodiments, various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing drawn in the attached drawings.

[0055] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0056] FIG. 1 is a diagram illustrating the operation of an electronic system according to one or more embodiments.

[0057] According to FIG. 1, the plurality of devices (1000) may include an electronic device (100), an electronic device (10), an electronic device (20), and an electronic device (30). The plurality of devices (1000) may communicate through a server (40). According to one or more examples, the plurality of devices (1000) may be various Internet of Things (IoT) devices. For example, the plurality of devices (1000) may be IoT (Internet of Things) devices that support a WiFi module. For example, the plurality of devices (1000) may be implemented as various types of electronic devices such as a TV, a mobile phone, a smart phone, a tablet, a notebook, a laptop, an electronic picture frame, a PMP (Portable Media Layer), an MP3 player, a game console, a LFD (Large Format Display), a monitor, a projector system, and the like.

[0058] According to one or more examples, a first electronic device (100) among a plurality of devices (1000) may be implemented as a TV. For example, a user may log in to a server (200) through a user account on the first electronic device (100). The first electronic device (100) may communicate with the server (200) based on the logged-in user account. The server (200) may manage a plurality of devices (1000) registered to the user account. Here, the server (200) may be implemented as a cloud server, but is not limited thereto.

[0059] When multiple devices (1000) are registered to a user account according to one or more examples, the server (200) may transmit data received from the multiple devices (1000) to a first electronic device (100) that communicates with the server (200) based on the user account in which the multiple devices (1000) are registered. According to one or more examples, the server (200) may transmit data stored in the server (200) or data received from the multiple devices (1000) to the electronic device (100) at the request of the electronic device (100).

[0060] Meanwhile, the electronic device (100) may provide a UI screen, for example, a home UI screen, based on a logged-in user account. Below, various embodiments of providing a personalized UI screen based on user preference information and context information will be described.

[0061] FIG. 2A is a block diagram showing the configuration of an electronic device according to one embodiment.

[0062] According to FIG. 2a, the electronic device (100) includes a memory (110) and at least one processor (120).

[0063] The electronic device (100) may be implemented as various types of display devices such as a TV, monitor, personal computer (PC), kiosk, tablet PC, electronic picture frame, mobile phone, HMD (Head mounted Display), NED (Near Eye Display), LFD (Large format display), Digital Signage (Digital signage), DID (Digital Information Display), video wall, projector display, etc., or as an image processing device (e.g., set-top box, one connected box) that provides images to the display device.

[0064] The memory (110) can store data used in various embodiments. The memory (110) may be implemented as at least one of a memory type embedded in the electronic device (100') and a memory type detachable from the electronic device (100), depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100'), and data for expanding the functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100). In one example, the memory embedded in the electronic device (100) may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, and solid state drive (SSD). In addition, the memory that can be detachably attached to the electronic device (100') may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), and external memory that can be connected to a USB port (e.g., USB memory). It can be implemented in one or more of the following forms.

[0065] According to one or more examples, the memory (110) may store a computer program including at least one instruction or instructions for controlling the electronic device (100).

[0066] At least one processor (120) controls the overall operation of the electronic device (100). For example, at least one processor (120) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100). For example, at least one processor (120) may be electrically connected to the display (110) and the memory (110) to control the overall operation of the electronic device (100). At least one processor (120) may be composed of one processor or multiple processors.

[0067] At least one processor (120) can perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (110).

[0068] According to one or more examples, functions related to artificial intelligence according to the present disclosure may be operated via a processor and memory of an electronic device.

[0069] At least one processor (120) may be composed of one or more processors. In this case, the one or more processors may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an NPU (Neural Processing Unit), but are not limited to the examples of the processors described above.

[0070] CPUs are general-purpose processors capable of performing not only general calculations but also artificial intelligence calculations. Their multi-layered cache structure allows for the efficient execution of complex programs. CPUs are advantageous for serial processing, enabling organic linking of previous and subsequent calculation results through sequential calculations. General-purpose processors are not limited to the examples described above, except where specifically identified as CPUs.

[0071] A GPU is a processor designed for large-scale computations, such as floating-point operations used in graphics processing. It integrates a large number of cores to perform large-scale computations in parallel. In particular, GPUs may be advantageous over CPUs in parallel processing methods, such as convolution operations. Furthermore, GPUs can be used as coprocessors to supplement the functions of CPUs. Processors for large-scale computations are not limited to the examples described above, except in cases where they are specifically referred to as GPUs.

[0072] An NPU is a processor specialized in artificial intelligence computation using artificial neural networks, and each layer of the artificial neural network can be implemented in hardware (e.g., silicon). Since NPUs are designed specifically according to the company's specifications, they have less freedom than CPUs or GPUs, but can efficiently process the AI ​​computations requested by the company. Meanwhile, as a processor specialized in AI computation, an NPU can be implemented in various forms, such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), or a Vision Processing Unit (VPU). Except as specifically designated as an NPU, an AI processor is not limited to the examples described above.

[0073] Additionally, at least one processor (120) may be implemented as a SoC (System on Chip). In this case, the SoC may further include, in addition to at least one processor (120), a memory (110), and a network interface such as a bus for data communication between the processor (120) and the memory (110).

[0074] When a plurality of processors are included in a SoC (System on Chip) included in an electronic device (100), the electronic device (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of a neural network model) by using some of the plurality of processors. For example, the electronic device may perform operations related to artificial intelligence by using at least one of a GPU, NPU, VPU, TPU, or hardware accelerator specialized in artificial intelligence operations such as convolution operations or matrix multiplication operations among the plurality of processors. However, this is only one or more embodiments, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor.

[0075] Additionally, the electronic device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in a single processor. In particular, the electronic device can perform artificial intelligence operations, such as convolution operations and matrix multiplication operations, in parallel by utilizing multiple cores included in the processor.

[0076] At least one processor (120) is controlled to process input data according to predefined operating rules or neural network models (or artificial intelligence models) stored in memory (110). The predefined operating rules or neural network models are characterized by being created through learning.

[0077] Here, "created through learning" means that a predefined set of behavioral rules or a neural network model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the artificial intelligence according to the present disclosure is implemented, or through a separate server / system.

[0078] A neural network model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a recurrent neural network (RNN), a deep neural network (DNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0079] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, thereby enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0080] According to one or more embodiments, the processor (120) may identify content preference information related to a user based on Natural Language Processing (NLP). For example, the processor (120) may identify the user's content preference information based on natural language processing using a Large Language Model (LLM). For example, the processor (120) may identify the content preference information by applying natural language processing to at least one of user information acquired during the onboarding process of the electronic device (100), user information acquired through a chatbot service, user information acquired through a voice recognition assistant service, or user information acquired from an external device.

[0081] According to one or more embodiments, the processor (120) may obtain context information based on an internal context of the electronic device (100), an external context of the electronic device (100), a real-time context of the user, and a context based on the user's usage history.

[0082] According to one or more embodiments, the processor (120) may identify or determine content architecture information corresponding to a user interface (UI) screen and visual element information corresponding to the UI screen based on content preference information and context information.

[0083] According to one or more embodiments, the processor (120) may generate or obtain a UI screen based on content architecture information and visual element information. For example, the processor (120) may obtain a personalized home UI screen including a personalized home background image, content categories, representative images for each content category, and UI fonts based on the content architecture information and visual element information.

[0084] According to one or more embodiments, the processor (120) may identify or determine visual preference information associated with the user and digital action preference information associated with the user based on natural language processing.

[0085] According to one or more embodiments, the processor (120) may identify integrated preference information associated with a user based on content preference information, visual preference information, and digital action preference information, and may identify or determine context architecture information and visual element information based on the integrated preference information and context information associated with the user.

[0086] According to one or more embodiments, the processor (120) may filter preferred content among a plurality of contents based on content preference information and user context information, and input information about preferred content into a first neural network model to obtain or determine content architecture information.

[0087] According to one or more embodiments, the processor (120) may obtain information related to a visual layout based on content architecture information, and input the information related to the visual layout into a second neural network model to obtain or determine visual element information.

[0088] According to one or more embodiments, the processor (120) may filter preferred content among a plurality of contents based on content preference information and context information, and input information related to preferred content and information related to visual layout into a third neural network model to obtain or determine content architecture information and visual element information.

[0089] According to one or more examples, at least one of the first neural network model, the second neural network model, and the third neural network model may be included in at least one of the electronic device (100) and at least one server. According to one or more examples, at least some of the first neural network model, the second neural network model, or the third neural network model may be implemented in the electronic device (100), and the remaining some may be implemented in at least one server.

[0090] FIG. 2b is a block diagram specifically illustrating a configuration of an electronic device according to one or more embodiments.

[0091] According to FIG. 2b, the electronic device (100') may include a memory (110), at least one processor (120), a display (130), a camera (140), a user interface (150), a communication interface (160), and a speaker (170). The configuration illustrated in FIG. 2b may be similar to the configuration illustrated in FIG. 2a, and redundant or duplicative descriptions thereof will be omitted.

[0092] The display (130) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (130) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. According to one or more examples, a touch sensor that detects a touch operation in the form of a touch film, a touch sheet, a touch pad, etc. may be disposed on the front of the display (130) so as to be implemented so as to detect various types of touch inputs. For example, the display (130) can detect various types of touch inputs, such as a touch input by a user's hand, a touch input by an input device such as a stylus pen, and a touch input by a specific electrostatic material. Here, the input device can be implemented as a pen-type input device that can be referred to by various terms such as an electronic pen, a stylus pen, an S-pen, etc. According to one or more examples, the display (130) can be implemented as a flat display, a curved display, a flexible display that can be folded or / and rolled, etc.

[0093] According to one or more examples, the display (130) may provide an output image comprised of a binocular image including at least one new viewpoint image.

[0094] The camera (140) can be turned on or activated to perform shooting according to a preset event. The camera (140) can convert a captured image into an electrical signal and generate image data based on the converted signal. For example, a subject can be converted into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), and the converted image signal can be amplified and converted into a digital signal and then signal processed. For example, the camera (120) can include at least one of a general (or basic) camera and an ultra-wide-angle camera.

[0095] According to one or more examples, the camera (140) may acquire a user-captured image and provide it to the processor (120). The processor (120) may detect the user's face location from the user-captured image, identify the user's expression by identifying the user's eye and mouth locations on the user's face, and identify the user's preference for content or the user's mood based on the user's expression. Various methods, such as direct recognition methods and statistical methods, may be used as methods for detecting the face region. The direct recognition method creates rules using physical features of the face image, such as the contour, skin color, and the size or distance between components, and compares, inspects, and measures according to the rules. The statistical method may detect the face region according to a pre-learned algorithm. For example, this is a method of digitizing unique features included in an input face and comparing and analyzing them with a relatively large database. In particular, the face region may be detected according to a pre-learned algorithm, and methods such as MLP (Multi-Layer Perceptron) and SVM (Support Vector Machine) may be used.

[0096] The user interface (150) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen that can also perform the display function and operation input function described above.

[0097] The communication interface (160) may be implemented as various interfaces depending on the implementation example of the electronic device (100'). For example, the communication interface (140) may communicate with an external device, an external storage medium (e.g., a USB memory), an external server (e.g., a web hard drive), etc. through a communication method such as Bluetooth, AP-based Wi-Fi (Wireless Local Area Network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc. According to one or more examples, the communication interface (160) may communicate with another electronic device, an external server, and / or a remote control device.

[0098] The speaker (170) may be configured to output various audio data as well as various notification sounds or voice messages. The processor (120) may control the speaker (170) to output feedback or various notifications in audio format according to various embodiments of the present disclosure.

[0099] In addition, the electronic device (100') may include sensors and microphones, etc., depending on the implementation example.

[0100] The sensors may include various types of sensors, such as touch sensors, proximity sensors, acceleration sensors (or gravity sensors), geomagnetic sensors, gyro sensors, pressure sensors, position sensors, distance sensors, light sensors, etc.

[0101] A microphone is a device configured to receive a user's voice or other sounds and convert them into audio data. However, according to another embodiment, the electronic device (100') may receive a user's voice input via an external device through a communication interface (160).

[0102] FIG. 3 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0103] According to FIG. 3, in operation 310, the electronic device (100) can identify user preference information based on natural language processing (NLP).

[0104] According to one or more examples, the user preference information may include at least one of content preference, visual preference, and digital action preference.

[0105] Content preference is a concept that represents a user's preference for a specific type of content and can vary depending on the user's tastes and interests. For example, content preference may include at least one of the following: genre preference, media type preference, language and style preference, depth of information preference, character preference, or content length preference.

[0106] According to one or more examples, the electronic device (100) may identify content preference information based on at least one of user information acquired during the onboarding process, user information acquired through a chatbot service, user information acquired through a voice recognition assistant service, or user information acquired from an external device. For example, the electronic device (100) may identify content preference based on the user's content usage history, reaction information regarding content, etc. acquired through various methods. For example, the reaction regarding content may include at least one of a voice reaction, a chat reaction, a facial expression reaction, or a gesture reaction.

[0107] Onboarding can refer to the process by which a user using an electronic device (100) for the first time becomes familiar with the functions and usage of the service. For example, the electronic device (100) can obtain user information through questions about service preferences, content preferences, and device familiarity during the onboarding process.

[0108] A chatbot service may refer to a service that interacts with an electronic device (100) using chatbot technology to improve the user's viewing experience.

[0109] A voice recognition assistant service may refer to a service that interacts with an electronic device (100) using voice recognition technology to improve a user's viewing experience.

[0110] For example, the electronic device (100) can obtain user information based on various user requests / questions, such as requests and / or questions through a chatbot service and / or a voice recognition secretary service, for example, user questions about program schedules and broadcast information, user questions about real-time TV trends and issues, requests for reminder settings and notifications, and requests for event guidance.

[0111] Visual preferences can be a user's preferred visual features or aspects. For example, visual preferences can include a person's preferences for visual content, such as design style, color, layout, image style, font style, and font size.

[0112] Digital action preferences can indicate how a user prefers certain actions or activities in a digital environment. For example, digital actions can include behaviors across various digital platforms, such as application usage habits, online activities (e.g., online shopping, online gaming), digital media consumption, social media activity, and digital service use. For example, digital action preferences can be identified based on information such as whether a user frequently uses a specific type of app, actively engages in a specific type of social media activity, actively consumes media through streaming services or online video platforms, or actively plays online games.

[0113] According to one or more examples, the electronic device (100) may acquire user preference information based on natural language processing. Natural language processing (NLP) is a technology that enables machines to understand and process human language. It may be a technology that enables computers to understand and interpret human language to extract information or understand meaning. For example, the electronic device (100) may perform natural language processing on text corresponding to various user information using a Large Language Model (LLM).

[0114] In operation 320, the electronic device (100) may identify content architecture information for a UI screen based on the user's preference information and the user's context information. According to one or more examples, the electronic device (100) may identify integrated preference information related to the user based on the content preference information, the visual preference information, and the digital action preference information. According to one or more examples, the electronic device (100) may identify or determine context architecture information based on the integrated preference information and the context information related to the user.

[0115] Meanwhile, the electronic device (100) may update user preference information based on updated context information, as well as chatbot services and / or voice recognition assistant services. For example, the electronic device (100) may obtain updated context information based on the updated internal context of the electronic device (100), the updated external context of the electronic device (100), the updated real-time context of the user, and the updated context based on the user's usage history.

[0116] Contextual information can include various types of information, such as device information, time information, environmental information, location information, social information, and physiological status information. For example, device information may include information about the device used by the user, such as the device type, screen size, and operating system. For example, time information may include time-related information, such as the current time, day of the week, and season. Environmental information may include information about the surrounding environment, such as weather information, temperature information, and lighting information. Social information may include social network-related information, such as the user's social media activity, friends list, and contact information. Physiological status information may include health-related data or physiological information, such as heart rate, sleep patterns, and activity level, obtained through wearable devices.

[0117] Content architecture information may include information about the composition of content for composing a UI screen. For example, content architecture information may include category information, content type, and / or priority information for the content. For example, a content category may include category information for each preferred content. For example, a content type may include type information such as live broadcast, OTT (Over The Top) content, or VOD (Video On Demand) content. For example, content priority information may include priority information for each category and content within a category.

[0118] In operation 330, the electronic device (100) may identify visual element information for a user interface (UI) screen based on content architecture information. For example, the visual element information may include UI layout information and UI asset information for the UI screen.

[0119] UI layout can refer to the way design elements of a user interface are arranged and structured. For example, UI layout can include information about the structure of a page or screen, the position, size, and spacing of UI elements, and more.

[0120] UI asset information can refer to resources used to design and build user interfaces. For example, UI assets can include UI component elements such as background images, theme images, skin images, fonts, buttons, icons, color palettes, and tooltips.

[0121] In operation 340, the electronic device (100) may obtain a UI screen based on content architecture information and visual element information. For example, the electronic device (100) may obtain a UI screen based on content category information, UI layout information, and UI asset information. The UI screen may represent a visual area in the user interface (UI) where a specific task is performed or information is displayed. For example, the UI screen may be, but is not limited to, a home UI screen.

[0122] As illustrated in FIG. 3, the order of all operations is mapped for convenience of explanation, but the embodiment is not limited thereto, and one or more operations may be performed in any order, for example, in parallel. FIG. 4 is a diagram for explaining natural language processing according to one or more embodiments.

[0123] According to FIG. 4, the electronic device (100) can perform natural language processing using a preprocessing module (410), a voice recognition module (410), a natural language understanding module (430), and a user preference identification module (440). In an embodiment, the natural language processing module 430 may also be referred to as a natural language understanding module. For example, each module may be implemented with at least one software, at least one hardware, and / or a combination thereof. For example, at least one of the modules may be implemented to use a predefined algorithm, a predefined formula, and / or a neural network model. Each module and its operation may be included within the electronic device (100), but may be distributed to at least one external device (e.g., a server) according to one or more examples.

[0124] The preprocessing module (410) can perform preprocessing on an audio signal received through a microphone. Specifically, the preprocessing module (410) can receive an analog audio signal containing the user's voice through the microphone and convert the analog signal into a digital signal. Furthermore, the preprocessing module (410) can calculate the energy of the converted digital signal to extract the user's voice segment.

[0125] The speech recognition module (420) can convert the user's speech in the form of audio data received from the preprocessing module (410) into text data. At this time, the speech recognition module (420) can include an acoustic model and a language model. The acoustic model can include information related to pronunciation, and the language model can include information about unit phoneme information and a combination of unit phoneme information. The speech recognition module (420) can convert the user's speech into text data using the information about the pronunciation and the unit phoneme information. Information about the acoustic model and the language model can be stored, for example, in an automatic speech recognition database (ASR DB).

[0126] The natural language processing module (430) can perform tokenization, syntactic analysis, or semantic analysis based on text data of the user's voice acquired through voice recognition to identify the domain and user intent of the user's voice. At this time, grammatical analysis can divide the user input into grammatical units (e.g., words, phrases, morphemes, etc.) and identify what grammatical elements the divided units have. Semantic analysis can be performed using semantic matching, rule matching, formula matching, etc. In particular, the natural language processing module (430) can obtain a natural language understanding result, a category of the user's voice, an intent of the user's voice, and a slot (or entity, parameter, etc.) for executing the intent of the user's voice.

[0127] The user preference identification module (440) can obtain user preference information based on the output data of the natural language processing module (430).

[0128] FIG. 5 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0129] According to FIG. 5, in operation 510, the electronic device (100) can obtain user preference information and user context information.

[0130] In operation 520, preferred content may be filtered from among a plurality of contents based on the user's preference information and the user's context information. According to one or more examples, the electronic device (100) may filter preferred content from among a plurality of contents including content stored in the electronic device (100) and content provided from an external device.

[0131] In operation 530, the electronic device (100) may obtain information about filtered preferred content. For example, the information about preferred content may include content attribute information. For example, content attributes may include various attributes such as the content itself, content episode, sub-genre (e.g., baseball, reality variety), broadcast time, characters, items, emotions, images, background images, sounds, and scenes.

[0132] In operation 540, the electronic device (100) can obtain content architecture information based on information about preferred content.

[0133] According to one or more examples, the electronic device (100) may obtain first data for input into the first neural network model based on information about preferred content. For example, the electronic device (100) may obtain the first data by processing the information about preferred content into a series of data sequences. For example, the series of data sequences may be, but is not limited to, a text sequence.

[0134] According to one or more examples, the electronic device (100) may acquire content architecture information by inputting first data into a first neural network model, as illustrated in FIG. 6A. For example, the first neural network model may be implemented as a generative AI model. A generative AI model is an artificial intelligence model that generates new content based on a given input, and may generate various types of data, such as text, images, and voice. A generative AI model may be pre-trained on a large amount of data (e.g., text data) and may perform fine-tuning for various tasks based on the pre-trained data. A generative AI model may learn patterns of input data and generate new data based on a deep learning architecture, such as a Recurrent Neural Network (RNN), a Long Short-Term Memory (LSTM), a Gated Recurrent Unit (GRU), or a Transformer. For example, the first neural network model can be implemented with various AI models such as the Generative Adversarial Network (GAN) Model, Style Transfer Models, AI Content Generators, Transformer-Based Models, and Genetic Algorithms for Layout Optimization.

[0135] In operation 550, the electronic device (100) may obtain information related to the visual layout based on content architecture information. The information related to the visual layout may include information such as design style, color, layout, image style, font style, font size, etc.

[0136] In operation 560, the electronic device (100) can obtain visual element information based on information related to the visual layout.

[0137] For example, the electronic device (100) may process information about a visual layout into a series of data sequences to obtain second data. For example, the series of data sequences may be, but is not limited to, a text sequence.

[0138] According to one or more examples, the electronic device (100) may input second data into a second neural network model to acquire or determine visual element information, as illustrated in FIG. 6B . For example, the second neural network model may be implemented as a generative AI model. For example, the second neural network model may be implemented as various AI models, such as a Generative Adversarial Network (GAN) model, Style Transfer Models, AI Content Generators, Transformer-Based Models, Genetic Algorithms for Layout Optimization, and the like.

[0139] When the first neural network model and the second neural network model are implemented in one server according to one or more examples, the electronic device (100) can transmit information about preferred content and information about visual layout to the server and receive visual element information from the server.

[0140] As another example, if the first neural network model and the second neural network model are implemented on different servers, the electronic device (100) may transmit information about preferred content to the first server and receive information about the visual layout from the first server. Subsequently, the electronic device (100) may transmit information about the visual layout to the second server and receive information about visual elements from the second server.

[0141] As another example, if the first neural network model and the second neural network model are implemented as on-device models included in the electronic device (100), the electronic device (100) can input information about preferred content into the first neural network model to obtain information about the visual layout. Subsequently, the electronic device (100) can input information about the visual layout into the second neural network model to obtain information about visual elements.

[0142] In operation 570, the electronic device (100) can obtain a UI screen based on content architecture information and visual element information.

[0143] According to one or more examples, the electronic device (100) may generate or obtain a UI screen by combining content architecture information and visual element information. For example, the electronic device (100) may generate or obtain a UI screen by combining content category information, UI layout information, and UI asset information. For example, the UI screen may be, but is not limited to, a home UI screen.

[0144] According to one or more examples, the electronic device (100) may generate or obtain a UI layout screen based on UI layout information. According to one or more examples, the electronic device (100) may identify representative images and / or titles of contents included in content architecture information, and may obtain a UI screen by arranging the representative images and / or titles of the contents based on category information and priority information of the contents. For example, the electronic device (100) may obtain a final UI screen by alpha blending a first display layer including a UI layout screen and a second display layer including a representative image and / or title. Here, alpha blending refers to a method of mixing and displaying a background RGB value and an RGB value above it by assigning a new value called A (Alpha) to the color value RGB in order to produce a transparent effect when overlaying another image on top of an image.

[0145] For example, the Alpha value can be divided into 0~255 or 0.0~1.0, where 0 can mean completely transparent, and the opposite, 255 (or the highest value such as 1.0), can mean fully opaque. However, it is not limited to this. For example, 0 can mean completely opaque, and the opposite, 255 (or the highest value such as 1.0) can mean fully transparent. For example, if 8 bits are allocated to the Alpha value and can express values ​​from 0 to 255, the larger the value, the higher the ratio of the corresponding pixel, and the lower the value, the lower the ratio. For example, the transparency of the area corresponding to the representative image and / or title of the contents can be increased so that the representative image and / or title of the contents can be provided in the area.

[0146] As illustrated in FIG. 5, the order of all operations is mapped for convenience of explanation, but the embodiment is not limited thereto, and one or more operations may be performed in any order, for example, in parallel. FIG. 7 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0147] According to FIG. 7, in operation 710, the electronic device (100) can obtain user preference information and user context information.

[0148] In operation 720, the electronic device (100) may obtain first data related to preferred content and second data related to visual layout based on the user's preference information and the user's context information.

[0149] In operation 730, the electronic device (100) can obtain content architecture information and visual element information based on the first data and the second data.

[0150] According to one or more examples, the electronic device (100) can input the first data and the second data into a third neural network model to obtain content architecture information and visual element information.

[0151] For example, the electronic device (100) may process information about preferred content into a series of data sequences to obtain first data, and process information about a visual layout into a series of data sequences to obtain second data. For example, the series of data sequences may be, but is not limited to, a text sequence.

[0152] According to one or more examples, the electronic device (100) may input first data and second data into a third neural network model to obtain visual element information, as illustrated in FIG. 8. For example, the third neural network model may be implemented as a generative AI model.

[0153] When the third neural network model is implemented on the server according to one or more examples, the electronic device (100) can transmit information about preferred content and information about visual layout to the server, and receive visual element information from the server.

[0154] In another example, when the third neural network model is implemented as an on-device model included in the electronic device (100), the electronic device (100) can obtain visual element information by inputting information about preferred content and information about visual layout into the third neural network model.

[0155] In operation 740, the electronic device (100) can obtain a UI screen based on content architecture information and visual element information.

[0156] As illustrated in FIG. 7, the order of all operations is mapped for convenience of explanation, but the embodiment is not limited thereto, and one or more operations may be performed in any order, for example, in parallel. FIG. 9 is a drawing for explaining the operation of an electronic device according to one or more embodiments.

[0157] According to FIG. 9, the electronic device (100) may include a User Style Understanding Module (910), a Context and Content Module (920), a Content Architecture Generation Module (930), a user experience / user interface (UX / UI) Layout Generation Module (940), a Visual Asset Generation Module (950), an Aggregation Module (960), and a Render Scheduling Module (970). For example, each module may be implemented with at least one software, at least one hardware, and / or a combination thereof. For example, at least one of the modules may be implemented to utilize a predefined algorithm, a predefined formula, and / or a neural network model. Each module and its operation may be included within the electronic device (100), but may be distributed to at least one external device (e.g., a server) according to one or more examples.

[0158] The User Style Understanding Module (910) can obtain user preferences based on at least one of user data, usage history, and context data.

[0159] The Context and Content Aggregation Module (920) can obtain information related to UI / UX customization based on user preferences, content data, and context data.

[0160] The Content Architecture Generation Module (930) can obtain content architecture information and content priority information.

[0161] The UX / UI Layout Generation Module (940) can obtain visual layout information based on the content architecture.

[0162] The Visual Asset Generation Module (950) can obtain UI asset information for the UX / UI layout.

[0163] The Aggregation Module (960) can compile content architecture information, visual layout information, and UI asset information into final UX / UI data.

[0164] The Render Scheduling Module (970) can render the UI screen based on the final UX / UI data.

[0165] FIG. 10 is a flowchart illustrating a method of controlling an electronic device according to one or more embodiments.

[0166] For example, in natural language conversations between a user and an electronic device (100), the user may feel uncomfortable revealing their preferences. Accordingly, the user's preferences can be naturally updated during the initial use phase and as the user's familiarity with the system increases. For example, the user's level of familiarity with the system can be identified and categorized into multiple levels, and questions designed to determine the user's preferences can be differentiated based on the level of familiarity.

[0167] According to FIG. 10, in operation 1010, the electronic device (100) can identify an intimacy level between the user and the electronic device (100) based on at least one of the level of user information, the user's usage frequency, and the user's usage behavior.

[0168] Intimacy between a user and an electronic device (100) may refer to the strength of the relationship between a specific user and that user's device.

[0169] The level of user information may include the depth level of previously acquired user information. For example, the electronic device (100) may identify or determine that the level of intimacy between the user and the electronic device (100) increases as the depth level of previously acquired user information increases.

[0170] The user's usage frequency may include usage history information, such as the viewing frequency of content provided by the electronic device (100). For example, the electronic device (100) may identify or determine that the level of intimacy between the user and the electronic device (100) increases as the user uses the electronic device (100) more frequently.

[0171] The user's usage behavior may include information such as the user's attitude toward content, questions, etc. provided by the electronic device (100). For example, the electronic device (100) may identify that the more positive responses the user makes, the higher the level of intimacy between the user and the electronic device (100).

[0172] In operation 1020, the electronic device (100) may identify a question for obtaining additional information related to the user based on the intimacy level. For example, the electronic device (100) may determine that a lower intimacy level corresponds to a lower depth level of the question, and a higher intimacy level corresponds to a higher depth level of the question. For example, the electronic device (100) may classify intimacy levels into levels 0, 1, and 2, and adjust the depth level of the question differently based on each level.

[0173] According to one or more examples, the electronic device (100) may obtain non-user-related information, such as external information, Assistant settings, etc., at the first level. For example, the electronic device (100) may identify a question at the level such as "What language do you prefer?"

[0174] According to one or more examples, the electronic device (100) may obtain information superficially related to the user, such as family composition, at a second level. For example, the electronic device (100) may identify a question at a level such as "What is your family composition?"

[0175] According to one or more examples, the electronic device (100) can obtain personal and internal information, such as the user's personal preferences, at a third level. For example, the electronic device (100) can identify questions at a relatively low depth level, such as "Do you have any topics or categories that interest you?", "Do you have any services or apps that you have used before?", "What color do you prefer?", and "What font and font size do you prefer?"

[0176] In operation 1030, the electronic device (100) may obtain additional information about the user through at least one of a chatbot service and a voice recognition assistant service based on the identified question. For example, the electronic device (100) may obtain additional information about the user by providing the user with the identified question based on a preset event. For example, the preset event may include an event of conversing with the user through a chatbot service and / or a voice recognition assistant service.

[0177] According to one embodiment, the electronic device (100) may update user preference information based on the user's additional information, and obtain updated content architecture information and / or updated visual element information based on the updated preference information. According to one or more examples, the electronic device (100) may obtain an updated UI screen based on the updated content architecture information and / or updated visual element information. For example, the updated UI screen may be a UI screen in which the content included in the UI screen has been changed or the visual elements have been changed.

[0178] Meanwhile, the electronic device (100) can update user preference information based on updated context information, as well as chatbot services and / or voice recognition assistant services. For example, the electronic device (100) can obtain updated context information based on the updated internal context of the electronic device (100), the updated external context of the electronic device (100), the updated real-time context of the user, and the updated context based on the user's usage history.

[0179] FIGS. 11A to 11K are drawings for explaining a method of providing a personalized UI screen according to one or more embodiments.

[0180] According to one or more embodiments, the electronic device (100) may provide a personalized UI screen, for example, a home UI screen, based on at least one of user information acquired through an onboarding process, user information acquired through a chatbot service, user information acquired through a voice recognition secretary service, or user information acquired from an external device.

[0181] For example, according to FIG. 11A, when the onboarding process is initiated through a user account, the electronic device (100) may provide a welcome screen (1111) and inquire whether to connect an external device (e.g., a mobile phone) connected to the user account. For example, the electronic device (100) may inquire the user through voice through a voice recognition assistant service. However, the present invention is not limited thereto, and the electronic device (100) may of course inquire the user by displaying text through a chatbot service.

[0182] According to FIG. 11b, the electronic device (100) can perform a connection with an external device based on a user response to connect to the external device and provide the connection process through a screen (1112). For example, the electronic device (100) can perform a connection with the external device based on a user's voice response, text response, button response using a remote control, etc.

[0183] According to FIGS. 11c and 11d, the electronic device (100) can obtain a user photo from an external device and create and provide a user avatar (1113, 1114). For example, the electronic device (100) can provide guidance such as "I created a personal avatar using photos from the gallery" and "Choose the one you like" in voice or text.

[0184] According to FIG. 11e, the electronic device (100) may provide a UI screen (1115) that includes a question for obtaining user information. For example, the electronic device (100) may provide a UI screen (1115) that includes a question such as "Tell me about your interests and hobbies." In this case, the electronic device (100) may also provide the question via voice.

[0185] According to FIG. 11f, the electronic device (100) may provide a UI screen (1116) including a user response to a question. For example, if the user provides responses such as “travel,” “surfing,” and “baseball,” the electronic device (100) may provide a UI screen (1116) including the corresponding response.

[0186] According to FIG. 11g, the electronic device (100) may provide a UI screen (1117) that includes a question for setting the service characteristics of a voice assistant service or chatbot service. For example, the electronic device (100) may provide a UI screen (1117) that includes a question such as, "What kind of personality would you like me to have?" In this case, the electronic device (100) may also provide the question via voice.

[0187] According to FIGS. 11h and 11i , the electronic device (100) may provide a UI screen (1118) including a user response to a question. For example, if the user provides a response such as, "I wish I could be cheerful when I'm in a good mood, but I'd prefer a calm atmosphere when I'm tired or late at night," the electronic device (100) may provide a UI screen (1119) including the response or a keyword of the response.

[0188] According to FIG. 11j, the electronic device (100) may provide a UI screen (1120) including a phrase that guides or indicates that a UI screen is provided based on collected user information. For example, the electronic device (100) may provide a UI screen (1120) including a guide such as "It is personalized to match the photo you have."

[0189] According to FIG. 11k, the electronic device (100) may provide a personalized UI screen (1121) based on collected user information. For example, the electronic device (100) may provide a UI screen (1120) in which the user's preferred content is arranged based on the user's preferred UI layout and UI assets.

[0190] FIGS. 12A and 12B are drawings illustrating examples of personalized UI screens according to one or more embodiments.

[0191] According to one or more embodiments, the electronic device (100) may provide a personalized UI screen, for example, a personalized home UI screen, based on user preference information and user context information.

[0192] According to one or more examples, the first home UI screen (1210) illustrated in FIG. 12a and the second home UI screen (1220) illustrated in FIG. 12b may be home UI screens generated based on user preference information and user context information.

[0193] According to one or more examples, the electronic device (100) may selectively provide either the first home screen (1210) or the second home UI screen (1220) depending on the real-time context. For example, the electronic device (100) may provide different home UI screens depending on the time, weather, mood, etc.

[0194] For example, a first home UI screen (1210) generated based on content that the user mainly views in the morning may be provided, and a second home UI screen (1220) generated based on content that the user mainly views in the evening may be provided.

[0195] For example, a first home UI screen (1210) generated based on content that the user mainly views on weekday evenings may be provided, and a second home UI screen (1220) generated based on content that the user mainly views on weekend evenings may be provided.

[0196] According to the various embodiments described above, the electronic device (100) can utilize natural language processing and generative AI technologies to naturally communicate with the user and generate and provide a personalized UI screen in real time based on the user's preferences and experiences identified through the communication. This can enhance user convenience.

[0197] Meanwhile, the methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device and / or server.

[0198] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0199] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0200] Furthermore, according to one or more embodiments of the present disclosure, the methods according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a commodity between sellers and buyers. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0201] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0202] Although the present disclosure has been illustrated and described above with respect to one embodiment, the embodiments are not limited to the specific embodiments described above, and various modifications may be implemented by a person having ordinary skill in the art pertaining to the present disclosure without departing from the gist of the present disclosure claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, at least one processor; and a memory storing at least one instruction; The at least one instruction, when executed by the at least one processor, causes the electronic device to: Identify user-related content preference information based on Natural Language Processing (NLP), Based on the content preference information and the context information related to the user, content architecture information corresponding to the UI (user interface) screen and visual element information corresponding to the UI screen are determined, An electronic device that generates the UI screen based on the content architecture information and the visual element information.

2. In paragraph 1, The at least one instruction causes the electronic device to identify visual preference information related to the user and digital action preference information related to the user based on the natural language processing, Determine integrated preference information related to the user based on the content preference information, the visual preference information, and the digital action preference information, An electronic device that determines the context architecture information and the visual element information based on the integrated preference information and the context information.

3. In paragraph 1, The at least one instruction causes the electronic device to filter preferred content among a plurality of contents based on the content preference information and the context information, Inputting information about the above preferred content into the first neural network model to determine the above content architecture information, The above content architecture information is: An electronic device comprising category information and priority information for the filtered content.

4. In paragraph 1, The at least one instruction causes the electronic device to determine information related to a visual layout based on the content architecture information, Inputting information related to the above visual layout into a second neural network model to determine the above visual element information, The above visual element information is, An electronic device including UI layout information and UI asset information for the above UI screen.

5. In paragraph 1, At least one instruction of the electronic device, Filtering preferred content among multiple contents based on the above content preference information and the above context information, An electronic device that inputs information related to the preferred content and information related to the visual layout into a third neural network model to determine the content architecture information and the visual element information.

6. In paragraph 1, At least one instruction of the electronic device, An electronic device that determines the content preference information by applying the natural language processing to user information obtained using at least one of the onboarding process of the electronic device, a chatbot service, a voice recognition secretary service, or an external device.

7. In paragraph 6, The at least one instruction causes the electronic device to identify an intimacy level between the user and the electronic device based on at least one of a level of the user information, a frequency of use of the user in relation to the electronic device, or a usage behavior of the user in relation to the electronic device. Select questions to obtain additional information about the user based on the intimacy level; Obtaining additional information of the user using at least one of the chatbot service and the voice recognition secretary service based on the selected question, An electronic device that updates the UI screen based on the above additional information.

8. In paragraph 1, At least one instruction of the electronic device, An electronic device that determines the context information based on the internal context of the electronic device, the external context of the electronic device, the real-time context of the user, and the usage context based on the usage history of the user.

9. In paragraph 1, At least one instruction of the electronic device, An electronic device that obtains content preference information based on the natural language processing using LLM (Large Language Model).

10. In paragraph 1, An electronic device wherein the at least one instruction causes the electronic device to generate a personalized home UI screen including a personalized home background image, a content category, a representative image for each content category, and a UI font based on the content architecture information and the visual element information.

11. In a method for controlling an electronic device, A step of identifying user content preference information based on natural language processing (NLP); A step of determining content architecture information corresponding to a UI (user interface) screen and visual element information corresponding to the UI screen based on the content preference information and context information related to the user; and A control method, comprising: a step of obtaining the UI screen based on the content architecture information and the visual element information.

12. In paragraph 11, The above control method is, A step of identifying visual preference information related to the user and digital action preference information related to the user based on the natural language processing; and Further comprising a step of determining integrated preference information related to the user based on the content preference information, the visual preference information, and the digital action preference information; The above context architecture information and the above visual element information, A control method determined based on the above integrated preference information and the above context information.

13. In paragraph 11, The above control method is, A step of filtering preferred content among a plurality of contents based on the content preference information and the context information; and It further includes a step of inputting information about the preferred content into a first neural network model to determine the content architecture information; The above content architecture information is: A control method comprising category information and priority information for the above filtered content.

14. In paragraph 11, The above control method is, A step of obtaining information related to a visual layout based on the above content architecture information; and It further includes a step of obtaining the visual element information by inputting information related to the visual layout into a second neural network model; The above visual element information is, A control method including UI layout information and UI asset information for the above UI screen.

15. When executed by a processor of an electronic device, the electronic device identifies content preference information related to a user based on natural language processing (NLP), and determines content architecture information corresponding to a user interface (UI) screen and visual element information corresponding to the UI screen based on the content preference information and context information related to the user. A non-transitory computer-readable medium storing a computer command for obtaining the UI screen based on the content architecture information and the visual element information.

Images