Method and electronic device for providing content
The electronic device and method provide an intuitive interface for users to customize content using generative artificial intelligence, addressing the challenge of user difficulty in editing and personalizing content, resulting in enhanced user experiences through personalized content generation.
Patent Information
- Application Number
- PCT/KR2024/003773
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-08
- Filing Date
- 2024-03-26
- Publication Date
- 2024-11-14
AI Technical Summary
Users find it difficult to edit and customize content using generative artificial intelligence technology due to a lack of an intuitive and efficient user interface, limiting their ability to personalize content experiences.
An electronic device and method that generate a user interface with editable and creatable content elements based on the played content, allowing users to select and modify content elements using generative artificial intelligence to create personalized content, including images, audio, and text.
Enables users to easily customize and personalize content by providing an intuitive interface for editing and creating content elements, enhancing the user experience through personalized content generation.
Smart Images

Figure KR2024003773_14112024_PF_FP_ABST
Abstract
Description
Methods and electronic devices for providing content
[0001] The present disclosure relates to an electronic device for generating personalized content and a method of operating the same.
[0002] Recently, generative AI, a technology that utilizes artificial intelligence to generate new data based on input data, has emerged, and the amount of content produced using this generative AI is increasing. However, editing content using generative AI technology can be challenging for general users, requiring an intuitive and efficient user interface.
[0003] In order to provide various customization options for the appearance, voice, text, etc. of objects within the content, it is required to provide a new content experience to users by implementing an environment for utilizing generative artificial intelligence through an optimized user interface.
[0004] According to one aspect of the present disclosure, a method for providing content by an electronic device may be provided. The method may include obtaining information related to first content being played. The method may include generating a user interface including at least one of editable content elements and creatable content elements from the first content based on the information related to the first content. The method may include receiving a user input for selecting a content element through the user interface. The method may include generating second content corresponding to the first content based on the content element corresponding to the user input. The method may include generating personalized content including the first content and the second content.
[0005] According to one aspect of the present disclosure, an electronic device for providing content may be provided. The electronic device may include a communication interface; a display; a memory storing one or more instructions; and at least one processor for executing the one or more instructions. The at least one processor may, by executing the one or more instructions, obtain information related to first content being played. The at least one processor may, by executing the one or more instructions, generate a user interface including at least one of editable content elements and creatable content elements in the first content based on the information related to the first content. The at least one processor may, by executing the one or more instructions, receive a user input for selecting a content element through the user interface. The at least one processor may, by executing the one or more instructions, generate second content corresponding to the first content based on a content element corresponding to the user input. The at least one processor can display personalized content including the first content and the second content on the screen of the display by executing the one or more instructions.
[0006] According to one aspect of the present disclosure, a computer-readable recording medium having recorded thereon a program for causing an electronic device to execute any one of the aforementioned and hereinafter described methods for providing content may be provided.
[0007] FIG. 1 is a schematic diagram illustrating an electronic device providing a content service according to one embodiment of the present disclosure.
[0008] FIG. 2 is a flowchart illustrating operations of an electronic device generating content according to one embodiment of the present disclosure.
[0009] FIG. 3 is a diagram illustrating the overall operation of an electronic device according to one embodiment of the present disclosure.
[0010] FIG. 4A is a diagram for explaining an operation of executing a second content creation mode in an electronic device according to one embodiment of the present disclosure.
[0011] FIG. 4b is a diagram for explaining an operation of executing a second content creation mode in an electronic device according to one embodiment of the present disclosure.
[0012] FIG. 5 is a diagram illustrating predefined settings used by an electronic device when generating a user interface according to one embodiment of the present disclosure.
[0013] FIG. 6 is a diagram illustrating an example of an electronic device generating second content according to an embodiment of the present disclosure.
[0014] FIG. 7A is a diagram illustrating a user interface for generating second content generated by an electronic device according to one embodiment of the present disclosure.
[0015] FIG. 7b is a diagram illustrating an operation of an electronic device according to one embodiment of the present disclosure to generate second content in a background process.
[0016] FIG. 8A is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0017] FIG. 8b is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0018] FIG. 9 is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0019] FIG. 10A is a diagram for explaining an operation of a user of an electronic device generating second content through user input according to one embodiment of the present disclosure.
[0020] FIG. 10b is a diagram illustrating a result of an electronic device generating second content by user input according to an embodiment of the present disclosure.
[0021] FIG. 10c is a diagram for explaining an operation of a user of an electronic device editing second content using another electronic device according to one embodiment of the present disclosure.
[0022] FIG. 11A is a diagram for explaining an operation of an electronic device according to one embodiment of the present disclosure to store second content information in association with user information.
[0023] FIG. 11b is a diagram for explaining an operation of an electronic device according to one embodiment of the present disclosure to store second content information in association with user information.
[0024] FIG. 12 is a block diagram illustrating a configuration of an electronic device according to one embodiment of the present disclosure.
[0025] Hereinafter, terms used in this specification will be briefly described, and the present disclosure will be described in detail. In this disclosure, the expression “at least one of a, b, or c” can refer to “a,” “b,” “c,” “a and b,” “a and c,” “b and c,” “all of a, b, and c,” or variations thereof.
[0026] The terms used in this disclosure are selected from widely used, common terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of the disclosure.
[0027] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art described herein. Furthermore, terms containing ordinal numbers, such as "first" or "second," used herein may be used to describe various components, but such components should not be limited by such terms. Such terms are used solely to distinguish one component from another.
[0028] When a part of the specification is said to "include" a component, unless otherwise specifically stated, this does not exclude other components but rather implies the inclusion of other components. Furthermore, terms such as "part" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.
[0029] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily practice the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, portions irrelevant to the description have been omitted for clarity of explanation, and similar reference numerals have been used throughout the specification to designate similar parts.
[0030] The present disclosure will be described in detail with reference to the attached drawings below.
[0031] FIG. 1 is a schematic diagram illustrating an electronic device providing a content service according to one embodiment of the present disclosure.
[0032] An electronic device (2000) according to one embodiment may be a device that outputs images and / or moving images, including a display. For example, the electronic device (2000) may include, but is not limited to, a smart TV, a smartphone, a tablet PC, a laptop PC, a picture frame display, etc., and may be implemented as various types and forms of electronic devices that include an electronic device (2000) display. In addition, the electronic device (2000) may include a speaker to output audio.
[0033] In the present disclosure, the original content played on the electronic device (2000) is referred to as first content. According to one embodiment, the electronic device (2000) can customize the first content played on the electronic device. For example, the electronic device (2000) can generate second content (120), which is new content, based on content elements (110) included in the first content. The second content (120) can include at least one of an image, audio, and text.
[0034] The electronic device (2000) can generate personalized content by combining the generated second content (120) with the first content. For example, the electronic device (2000) can replace at least one of the content elements (110) included in the first content, such as an image, audio, or text, with the second content (120).
[0035] In one embodiment, when the electronic device (2000) generates second content, a generative artificial intelligence model may be utilized. The generative artificial intelligence model refers to an artificial intelligence model for generating content having similar characteristics to the content elements included in the first content or having new characteristics. The electronic device (2000) may generate the second content using the generative artificial intelligence model and provide the user with personalized content comprised of the first content and the second content.
[0036] Specific operations for generating personalized content by having the electronic device (2000) generate secondary content and apply it to the original content will be described in more detail in the drawings and descriptions thereof described below.
[0037] FIG. 2 is a flowchart illustrating operations of an electronic device generating content according to one embodiment of the present disclosure.
[0038] In step S210, the electronic device (2000) obtains information related to the first content being played.
[0039] In the present disclosure, the first content refers to various media contents including movies, drama series, documentaries, entertainment, TV programs, other video contents, etc., and may also be referred to as multimedia contents.
[0040] In one embodiment, the first content may be a digital file format created in a standardized manner for packaging multimedia data. Examples of media container formats that may be used include, but are not limited to, MP4, AVI, MKV, MOV, and WMV.
[0041] The first content may include various types of media data. For example, the first content may include video, audio, and text. The electronic device (2000) may identify the first content being played on the electronic device (2000) and obtain information related to the first content.
[0042] The first content may include various information related to the content. For example, the first content may include metadata such as the title, producer, duration, genre, performer information, platform information, and channel information. However, the information related to the content is not limited to the examples described above.
[0043] In step S220, the electronic device (2000) generates a user interface including at least one of editable content elements and creatable content elements in the first content based on information related to the first content. The user interface refers to an intuitive interface that allows a user of the electronic device (2000) to customize the first content.
[0044] In one embodiment, the electronic device (2000) may generate a user interface that includes editable content elements in the first content. The editable content elements may be at least one of an image, audio, or text included in the first content. For example, the electronic device (2000) may generate a user interface that allows a user to edit the appearance of a person in the first content.
[0045] In one embodiment, the electronic device (2000) can generate a user interface that includes content elements that can be generated from the first content. The content elements that can be generated may be at least one of an image, audio, or text that can be added to the first content. For example, the electronic device (2000) can generate a user interface that allows a user to add a new virtual object within the first content.
[0046] In one embodiment, a user interface may be generated based on information related to the first content. For example, the electronic device (2000) may generate a user interface based on the genre of the first content, including predefined content elements for each genre. The electronic device (2000) generating a user interface based on predefined settings is further described with reference to FIG. 5 .
[0047] In step S230, the electronic device (2000) receives a user input for selecting a content element through a user interface.
[0048] In one embodiment, a user of the electronic device (2000) may change a content element within the first content or create a new content element to be added to the first content through a user interface.
[0049] In one embodiment, a user of the electronic device (2000) may select at least one of an image, audio, and text included in the first content through a user interface. For example, a user of the electronic device (2000) may select a person within the first content through the user interface to edit the appearance of the person within the first content.
[0050] In one embodiment, a user of the electronic device (2000) may select at least one of an image, audio, and text to be added to the first content via a user interface. For example, a user of the electronic device (2000) may select a virtual object via the user interface to add the virtual object to the first content.
[0051] In one embodiment, the electronic device (2000) may modify a content element within the first content or generate a new content element based on user input. In the present disclosure, the content element modified and / or generated by the electronic device (2000) is referred to as "second content."
[0052] In step S240, the electronic device (2000) according to one embodiment generates second content corresponding to the first content based on a content element corresponding to the user input.
[0053] In one embodiment, the electronic device (2000) can generate second content by modifying content elements within the first content that correspond to user input. For example, the second content may be an image of a new person whose appearance has been modified from that of the person within the first content.
[0054] In one embodiment, the electronic device (2000) can generate second content by generating a content element corresponding to a user input. For example, the second content may be an image of a virtual object that can be added to the first content.
[0055] In one embodiment, the electronic device (2000) can generate second content using a generative artificial intelligence model. The electronic device (2000) can generate second content including at least one of an image, audio, and text.
[0056] In step S250, an electronic device (2000) according to one embodiment generates personalized content including first content and second content.
[0057] In one embodiment, the first content is original content, and the second content is content generated by the user on the electronic device (2000). Therefore, when the second content is combined with the first content, personalized content unique to the user that is not provided to other users can be generated.
[0058] FIG. 3 is a diagram illustrating the overall operation of an electronic device according to one embodiment of the present disclosure.
[0059] In step S310, the electronic device (2000) executes a second content generation mode. The second content refers to content corresponding to the first content, which is original content that is being played or can be played on the electronic device (2000). For example, the first content refers to content that is composed of content elements such as images, audio, and text, and can be additionally combined with the first content. The operations of the electronic device described above in FIG. 2 for generating the second content and generating personalized content can be performed by executing the second content generation mode.
[0060] In one embodiment, the home menu of the electronic device (2000) may include a second content creation mode (e.g., a generative artificial intelligence mode). The electronic device (2000) may identify that the user is executing the second content creation mode from the home menu.
[0061] In one embodiment, while the electronic device (2000) is operating, a user of the electronic device (2000) can execute a second content creation mode (e.g., a creation-type artificial intelligence mode) through a shortcut menu. In this case, the electronic device (2000) can identify the execution of the second content creation mode while playing the first content and directly execute the second content creation mode.
[0062] The electronic device (2000) can acquire information related to the first content to generate the second content. Specifically, the electronic device (2000) can identify the genre of the first content and execute an optimization function corresponding to the identified genre. The genre of the first content can be categorized into, for example, current affairs, sports, movies, entertainment, music, debates, economics, documentaries, etc., but is not limited thereto.
[0063] In step S320, the electronic device (2000) may determine whether to generate a user interface for generating second content based on information related to the first content. For example, the electronic device (2000) may determine whether to generate a user interface based on the genre of the first content.
[0064] The electronic device (2000) may generate a user interface for generating second content, which is additional content corresponding to the first content, based on the genre of the first content. The user interface may include at least one of editable content elements and creatable content elements in the first content.
[0065] A user of the electronic device (2000) can generate second content through user input via a user interface. In this case, the user interface provided to the user may have different formats depending on the genre of the first content, and may be optimized for each genre of the content. If the genre of the first content is set to not require the creation of a user interface, the electronic device (2000) can automatically perform optimization to generate second content related to the first content.
[0066] In step S330, the electronic device (2000) may generate second content. For example, the electronic device (2000) may generate second content based on user input via a user interface. Alternatively, the electronic device (2000) may generate second content based on elements within the first content. The generated second content may be updated on a server in conjunction with user information. For example, if a user logs in to the electronic device (2000) and account information exists, the electronic device (2000) may accumulate, store, update, and manage second content generated for the user. The history of second content stored for the user may be used by the user to generate another second content for another first content at a future point in time.
[0067] FIG. 4A is a diagram for explaining an operation of executing a second content creation mode in an electronic device according to one embodiment of the present disclosure.
[0068] In one embodiment, the electronic device (2000) may display a menu (410) for a second content creation mode (e.g., a generative artificial intelligence mode) on the home menu. Furthermore, the menu (410) for the second content creation mode may display a content genre. For example, content genres such as discussion (420), music broadcast (430), and sports (440) may be displayed.
[0069] When a content genre is selected by a user, the electronic device (2000) can generate a user interface (400) optimized for the corresponding genre and display it on the screen of the electronic device (2000). For example, when the discussion (420) genre is selected on the electronic device (2000), multiple contents included in the discussion (420) genre are displayed. When the user selects one of the multiple contents (first contents), the first contents may be played. In this case, a viewing screen of the first contents is displayed, and a user interface (400) for generating second contents corresponding to the first contents may be displayed. In the example of FIG. 4A, since the content genre selected by the user is discussion (420), a user interface (400) optimized for the discussion (420) genre may be displayed. However, this is merely an example, and a user interface optimized for each genre may be displayed differently depending on the genre of the first contents. This is further described with reference to Figure 5.
[0070] FIG. 4b is a diagram for explaining an operation of executing a second content creation mode in an electronic device according to one embodiment of the present disclosure.
[0071] In one embodiment, while the electronic device (2000) is operating, the user of the electronic device (2000) can execute a second content creation mode (e.g., a creation-type artificial intelligence mode) through a shortcut menu. In this case, while the viewing screen (450) of the first content is displayed, the electronic device (2000) can execute the second content creation mode and display a genre-optimized user interface (400) together with the viewing screen (450) of the first content.
[0072] Meanwhile, in FIG. 4A, the user selects the genre of the first content, whereas in FIG. 4B, the second content creation mode is executed by the user while the first content is being played without selecting a genre. In this case, the electronic device (2000) can perform a series of operations to identify the genre of the first content.
[0073] In one embodiment, the electronic device (2000) may utilize metadata of the first content, which is information related to the first content. The metadata may include, but is not limited to, the title, producer, playback time, genre, performer information, platform information, and channel information of the first content. If the metadata does not include genre information of the first content, the electronic device (2000) may infer the genre of the first content using other information (e.g., title, etc.) included in the metadata. If the metadata includes genre information of the first content, the electronic device (2000) may identify the genre of the first content.
[0074] In one embodiment, if the genre of the first content is not inferred from metadata or if metadata is not obtained, the electronic device (2000) may analyze the first content to identify the genre of the first content. For example, the electronic device (2000) may perform scene analysis to obtain scene context data. The electronic device (2000) may utilize one or more artificial intelligence models for scene analysis.
[0075] The electronic device (2000) can select one or more video frames of video data included in the first content and detect at least one object within the video frame. The electronic device (2000) can use an object detection model, which is an artificial intelligence model, to detect the object. The object detection model may be a deep neural network model that receives an image as input and outputs information representing the detected objects. For example, the object detection model may receive an image as input and output bounding boxes representing the detected objects. The object detection model can be implemented using various known deep neural network architectures and algorithms, or through modifications of various known deep neural network architectures and algorithms.
[0076] The electronic device (2000) can recognize the category of at least one object detected within a video frame. The electronic device (2000) can utilize an object recognition model, which is an artificial intelligence model, for object recognition. The object recognition model may be a deep neural network model that receives an image as input and outputs information representing object class label(s). For example, the object recognition model may receive an image from which an object has been cropped and output one or more object class labels (e.g., "car," "person," etc.) and a confidence score. The object recognition model may be implemented using various known deep neural network architectures and algorithms, or through modifications of various known deep neural network architectures and algorithms.
[0077] The electronic device (2000) can detect relationships between recognized objects. The electronic device (2000) can use an object relationship detection model, which is an artificial intelligence model, to detect object relationships. The object relationship detection model may be a deep neural network model that receives information about detected objects and outputs information indicating relationships between the objects. For example, the object relationship detection model may be a model that receives information about detected objects "person a" and "person b" and outputs a relationship "next to" between the two objects, indicating that the person is on the roof. The object relationship detection model may be implemented using various known deep neural network architectures and algorithms, or through modifications of various known deep neural network architectures and algorithms.
[0078] The electronic device (2000) can generate a scene graph based on the result data of object detection, object recognition, and object relationship detection. Here, one or more nodes of the scene graph represent one or more objects, and one or more edges represent relationships between one or more objects. In addition, the electronic device (2000) can generate a scene context based on the scene graph. The electronic device (2000) can generate a scene context for each of one or more video frames and identify the genre of the first content based on the scene context.
[0079] In the example of FIG. 4b, if the first content on the viewing screen (450) is "TV Debate," the genre of the first content may be identified as "Debate" through scene analysis, and a user interface (400) optimized for the debate genre may be displayed. However, this is merely an example, and the optimized user interface for each genre may be displayed differently depending on the genre of the first content. This will be further described with reference to FIG. 5.
[0080] FIG. 5 is a diagram illustrating predefined settings used by an electronic device when generating a user interface according to one embodiment of the present disclosure.
[0081] In one embodiment, the electronic device (2000) may utilize predefined settings (500) when generating a user interface for generating second content corresponding to first content.
[0082] The predefined settings (500) may define content elements (520) to be included in the user interface according to the genre (510) of the first content, and may define second content (530) to be generated through the user interface.
[0083] For example, if the genre (510) of the first content is "News, Discussion," the predefined content element corresponding to the "News, Discussion" genre may be "People, Conversation." In this case, the user interface for generating the second content (530) may include the content element "People, Conversation."
[0084] A user of an electronic device (2000) can create second content (530) through a user interface. For example, for first content in the "News, Discussion" genre (510), the user can edit the content element "Person." Specifically, the user can create a 3D character as the second content (530). Using the second content (530), the user can change the appearance of a person appearing in the news or discussion into a 3D character. For example, the user can create a content element "Conversation Content." Specifically, the user can create keyword extraction and analysis results, conversation summaries, etc. as the second content (530). Using the second content (530), the user can display the keyword analysis results, conversation summaries, etc. overlaid on the screen where the news or discussion is in progress. Meanwhile, the second content, such as keyword analysis results and conversation summary, may be provided to the user after the first content has finished playing on the electronic device (2000), and may also be provided to the user through an electronic device other than the electronic device (2000).
[0085] An electronic device (2000) according to one embodiment may generate a user interface optimized for each genre of the first content based on predefined settings (500). For another example, if the genre (510) of the first content is "sports," a content element (520) included in a user interface for generating the second content may be "person, background," and a user may generate the second content (530) by editing or generating the "person, background" through the user interface.
[0086] FIG. 6 is a diagram illustrating an example of an electronic device generating second content according to an embodiment of the present disclosure.
[0087] Referring to FIG. 6, an electronic device (2000) according to one embodiment can identify a first content being played (e.g., a live channel broadcast (600)).
[0088] The electronic device (2000) can obtain information related to the first content being played. For example, the electronic device (2000) can identify the genre (610) of the first content based on metadata including the title, producer, playing time, genre, performer information, platform information, and channel information of the first content. Alternatively, the electronic device (2000) can obtain scene context data for the first content through scene analysis and recognize the genre of the first content. Since the electronic device (2000) recognizes the genre of the first content as described above with respect to FIG. 4B , a repeated description thereof will be omitted.
[0089] In one embodiment, the electronic device (2000) may generate a user interface for generating second content (630) based on the genre (610) of the first content. The user interface may include at least one of editable content elements and generable content elements in the first content, and may include predefined content elements optimized for each genre.
[0090] For example, if the genre (610) of the first content is “discussion (612),” the user interface may include content elements such as “person, conversation content,” which are predefined to correspond to “discussion (612).” The user of the electronic device (2000) may create second content (630) according to the user’s preference, such as editing the person appearing in the discussion or creating keywords, summaries, etc. related to the conversation content through the user interface.
[0091] In one embodiment, the electronic device (2000) can generate second content (630) using a generative artificial intelligence model (620). The generative artificial intelligence model (620) can be a deep neural network model that performs at least one task among image generation (or video generation, 3D object generation), audio generation (or voice generation, audio generation), and text generation. The generative artificial intelligence model (620) can be implemented using various known deep neural network architectures and algorithms, or through modifications of various known deep neural network architectures and algorithms.
[0092] In one embodiment, the electronic device (2000) may utilize multiple generative artificial intelligence models (620). For example, the generative artificial intelligence models may include a first generative artificial intelligence model that generates images, a second generative artificial intelligence model that generates audio, and a third generative artificial intelligence model that generates text.
[0093] The first generative AI model may be an AI model that performs an image generation task. The first generative AI model may be implemented using various known deep neural network architectures and algorithms for generating images, videos, 3D objects, etc., or through modifications of various known deep neural network architectures and algorithms. The first generative AI model may be implemented using, but is not limited to, generative adversarial networks (GANs), variational autoencoders (VAEs), etc.
[0094] In one embodiment, the second content (630) may be an image (632). The electronic device (2000) may generate the image (632) using the first generative artificial intelligence model. For example, the electronic device (2000) may generate an image having similar characteristics to the people, objects, and backgrounds of the first content, or an image having new characteristics, using the first generative artificial intelligence model. Specifically, for a person, a new image may be generated with changed hair color, facial expression, clothing type, etc.
[0095] The second generative AI model may be an AI model that performs an audio generation task. The second generative AI model may be implemented using various known deep neural network architectures and algorithms for generating audio, such as speech, music, and other sounds, or through variations of various known deep neural network architectures and algorithms. The second generative AI model may be implemented using, but is not limited to, recurrent neural networks (RNNs), variational autoencoders (VAEs), WaveNet, Tacotron, and the like.
[0096] In one embodiment, the second content (630) may be audio (634). The electronic device (2000) may generate the audio (634) using the second generative artificial intelligence model. For example, the electronic device (2000) may generate audio having similar characteristics to the audio associated with people, musical instruments, etc. of the first content, or audio having new characteristics, using the second generative artificial intelligence model. Specifically, for a human voice, a new voice may be generated by changing the tone, voice, gender, language, conversation content, etc.
[0097] The third generative AI model may be an AI model that performs a text generation task. The third generative AI model may be implemented using various known deep neural network architectures and algorithms for text generation, or through modifications of various known deep neural network architectures and algorithms. The third generative AI model may be implemented using, but is not limited to, recurrent neural networks (RNNs) and Transformers.
[0098] In one embodiment, the second content (630) may be text (636). The electronic device (2000) may generate the text (636) using a third generative artificial intelligence model. For example, the electronic device (2000) may generate text having similar characteristics to the text related to the human conversation of the first content, or text having new characteristics, using the third generative artificial intelligence model. Specifically, text representing the human conversation content, such as the tendencies, emotions, keywords, and topics of the conversation content, may be generated.
[0099] Meanwhile, the first generative artificial intelligence model, the second generative artificial intelligence model, and the third generative artificial intelligence model described above may be implemented as a single integrated generative artificial intelligence model.
[0100] The electronic device (2000) can generate personalized content by generating second content (630) including at least one of an image (632), audio (634), and text (636), and combining the generated second content (630) with first content (e.g., a live channel broadcast (600)). Then, the electronic device (2000) can provide the personalized content to the user.
[0101] FIG. 7A is a diagram illustrating a user interface for generating second content generated by an electronic device according to one embodiment of the present disclosure.
[0102] In one embodiment, the electronic device (2000) can generate a user interface (700) including at least one of editable content elements and creatable content elements in the first content based on information related to the first content.
[0103] In one embodiment, the user interface (700) may be optimized for the genre of the first content. For example, if the genre of the first content is "Discussion," the user interface (700) for using the second content may be configured to include content elements optimized for the "Discussion" genre. Specifically, characters and conversations corresponding to key content elements in the "Discussion" genre may be included in the user interface (700).
[0104] A user of the electronic device (2000) can create second content by selecting a content element displayed on the user interface (700) and editing or creating the content element.
[0105] For example, a user may select a first person (710), which is a content element displayed on a user interface (700). The user may create a new image (second content) by changing the hair color, facial expression, clothing type, etc. of the first person (710). Alternatively, the user may create a new voice (second content) by changing the tone, voice, gender, language, conversation content, etc. of the voice of the first person (710). In this case, when a discussion program between people, which is the first content, is played on the electronic device (2000), the appearance, voice, etc. of the first person (710) may be replaced with the second content created by the user. In the same manner, the user of the electronic device (2000) may change the appearance, voice, etc. of the second person (712) and the third person (714) by applying the second content created by the user to the appearance, voice, etc. of the second person (712) and the third person (714) through the user interface (700).
[0106] For example, a user may select a conversation content (720) displayed on a user interface (700). The user may generate text representing the tendencies, emotions, keywords, topics, etc. of the conversation content of the first person (712) and the conversation content of the second person (714). In this case, when the first content, the discussion program between the people, is played, a summary of the conversation content of the first person (712) and a summary of the conversation content of the second person (714) may be displayed together.
[0107] FIG. 7b is a diagram illustrating an operation of an electronic device according to one embodiment of the present disclosure to generate second content in a background process.
[0108] Referring to the first screen (730) illustrated in FIG. 7B, an electronic device (2000) according to one embodiment can display personalized content including first content and second content. In this case, the second content may be automatically generated or generated at the user's request.
[0109] In one embodiment, the first content being played on the electronic device (2000) may be changed to other content. For example, the channel of the electronic device (2000) may be changed by the user's manipulation of the electronic device (2000), so that other content, such as the second screen (740), may be played.
[0110] If the electronic device (2000) determines that the first content currently being played on the electronic device (2000) has changed to another content, the electronic device (2000) may continue to generate the second content in a background process without terminating the generation of the second content. For example, the second content generated by the electronic device (2000) may be a summary of the first content. The electronic device (2000) may summarize the conversations of the discussion participants until the discussion program ends, and when the first content being broadcast on another channel ends, the electronic device (2000) may provide the user with a second content generation notification (742).
[0111] A user of the electronic device (2000) can check the second content through the screen of the electronic device (2000) or check the second content using an electronic device other than the electronic device (2000).
[0112] FIG. 8A is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0113] In one embodiment, the electronic device (2000) can generate personalized content including first content and second content.
[0114] The electronic device (2000) may generate a user interface including at least one of editable content elements and creatable content elements in the first content based on information related to the first content. For example, if the genre of the first content is sports (short-distance running), the electronic device (2000) may display predefined editable content elements for the sports genre. The predefined content elements for the sports genre may include, but are not limited to, people, backgrounds, etc.
[0115] The electronic device (2000) can receive user input for selecting a content element through a user interface and generate second content based on the content element corresponding to the user input.
[0116] For example, if the electronic device (2000) selects the first athlete (812) participating in a short-distance run, the electronic device (2000) can generate the second content (820) by changing the appearance (e.g., hair color, facial expression, type of clothing, etc.) of the first athlete (812) through a generative artificial intelligence model. Accordingly, when the short-distance run content is played, the second content, which is the changed appearance, can be applied and displayed instead of the actual appearance of the first athlete (812). The electronic device (2000) can also change the appearances of the second athlete (814), the third athlete (816), etc. based on user input. In this case, the generative artificial intelligence model can be used.
[0117] Alternatively, the electronic device (2000) may generate new content as the second content in addition to modifying content elements within the first content. For example, the electronic device (2000) may generate a virtual character (810) as the second content using a generative artificial intelligence model. The electronic device (2000) may generate graphics to make it appear as if the virtual character (810) is racing together with the first athlete (812), the second athlete (814), and the third athlete (816) during a running race.
[0118] FIG. 8b is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0119] In one embodiment, if the genre of the first content is sports (soccer), the electronic device (2000) may display predefined content elements that can be edited for the sports genre, as described above in FIG. 8A . Predefined content elements for the sports genre may include, but are not limited to, people, backgrounds, etc.
[0120] In one embodiment, the electronic device (2000) may generate second content based on information related to the content and / or user input. For example, if the first content is a soccer match, the information related to the first content may include player information. In this case, the electronic device (2000) may identify a first player (822), a soccer player preferred by the user, within the first content, or may identify the first player (822) based on user input. In addition, the electronic device (2000) may generate second content (820) by adding graphic effects to the first player (822). For example, the electronic device (2000) may generate second content (820) by applying processing such as enlarging, highlighting, or changing the uniform of the first player (812) using a generative artificial intelligence model. Accordingly, when a soccer match content is played, the user's preferred first player (822) may be displayed with a changed appearance, that is, the second content (820), applied instead of the actual appearance. In the same manner, the electronic device (2000) may generate second content corresponding to each of the second player (824) and the third player (816), which are content elements existing within the first content.
[0121] Meanwhile, in describing FIGS. 8A and 8B , in the case of the sports genre, it was explained as an example that sports players are content elements that can be changed and / or generated, but this is not limited thereto. For example, the electronic device (2000) can generate secondary content based on other content elements such as background, stadium, audience, weather, etc., and combine the generated secondary content with the primary content, thereby generating personalized content.
[0122] FIG. 9 is a diagram illustrating an example of an electronic device providing personalized content according to one embodiment of the present disclosure.
[0123] In one embodiment, the electronic device (2000) may determine whether to generate a user interface for generating second content based on information related to the first content. For example, the electronic device (2000) may determine whether to generate a user interface based on the genre of the first content.
[0124] In one embodiment, a user interface for generating second content may be predefined. For example, if the genre of the first content is "music broadcast," the creation of a user interface may be set to be unnecessary. In this case, the electronic device (2000) may automatically generate the second content. Specifically, if the first content is an orchestral performance broadcast, the electronic device (2000) may use a generative artificial intelligence model to generate a 3D object (910) visualizing the orchestral performance as the second content.
[0125] Meanwhile, the generation of the second content is based on the assumption that the second content generation mode is executed in the manner described above in FIGS. 4A and 4B. That is, the electronic device (2000) can only play the first content when the second content generation mode is not executed, and can play personalized content including the first content and the second content when the second content generation mode is executed.
[0126] FIG. 10A is a diagram for explaining an operation of a user of an electronic device generating second content through user input according to one embodiment of the present disclosure.
[0127] In one embodiment, the electronic device (2000) may analyze the genre of the first content based on information related to the first content and provide an intuitive user interface (1010) that allows the user to personalize the first content according to the genre of the first content. In this case, the user of the electronic device (2000) may select a content element to be personalized within the user interface (1010) and create second content by editing and / or changing the selected element.
[0128] In one embodiment, the electronic device (2000) may generate second content using a generative artificial intelligence model. For example, when a user selects a content element, person A (1020), the electronic device (2000) may provide the user with options (1030) that can modify the appearance, voice, etc. of person A (1020). For example, when the user selects appearance from the options (1030), second content may be generated with changed face, hair color, eye shape, mouth corners, clothing color, clothing type, etc. Alternatively, when the user selects voice from the options (1030), second content may be generated with changed voice, tone, gender, language, etc.
[0129] In one embodiment, the electronic device (2000) may provide at least one of the appearance, voice, and genre of the first content of person A (1020) as input data to the generative artificial intelligence model. Based on the input data, the generative artificial intelligence model may generate and output data having similar characteristics to the input data or having new characteristics.
[0130] In one embodiment, the electronic device (2000) may display a plurality of style images (1040) applicable to first content. The electronic device (2000) may receive a user input for selecting one of the plurality of style images (1040). In this case, the selected style image may be applied to the first content, thereby generating second content. The electronic device (2000) may transfer the style using a generative artificial intelligence model to apply the style selected by the user to the first content.
[0131] FIG. 10b is a diagram illustrating a result of an electronic device generating second content by user input according to an embodiment of the present disclosure.
[0132] In one embodiment, the electronic device (2000) may display multiple results of second content generated using a generative artificial intelligence model. For example, the electronic device (2000) may display second contents (1060) generated by changing the appearance of a first user (1050) within the first content. When the user of the electronic device (2000) selects one of the second contents (1060), a facial image (1062) of the selected second content may be applied to the first content, thereby generating personalized content.
[0133] An electronic device (2000) according to one embodiment can generate multiple types of second content. For example, personalized content may include a facial image (1062), which is second content of the image type, and a speech bubble (1070), which is second content of the text type converted from the voice of the first user (1050), may be displayed.
[0134] In one embodiment, the electronic device (2000) may provide a variety of options for a user to create secondary content. For example, the electronic device (2000) may change the face of a person in the primary content, change the hair color, clothing, etc. of a person in the primary content, change the voice of a person in the primary content to a specific voice (e.g., an agitated voice, a calm voice, etc.), visualize the voice and display it as subtitles, or mute the voice.
[0135] FIG. 10c is a diagram for explaining an operation of a user of an electronic device editing second content using another electronic device according to one embodiment of the present disclosure.
[0136] In one embodiment, when the electronic device (2000) executes the second content creation function, the electronic device (2000) may enable the user of the electronic device (2000) to use another electronic device (e.g., a smartphone).
[0137] As described above in FIG. 4B, the electronic device (2000) according to one embodiment may perform metadata or scene analysis when the first content is played, and may add and display the second content (1080) to the first content. In this case, in order to perform a detailed operation for modifying the second content, the electronic device (2000) may identify another electronic device of the user. For example, the electronic device (2000) may identify another electronic device connected to the same network (e.g., Wi-Fi) as the electronic device (2000). Alternatively, the electronic device (2000) may identify another electronic device using a user account logged into the electronic device (2000) through a server.
[0138] A user of the electronic device (2000) can edit the second content (1080) using another electronic device (e.g., a smartphone). For example, the user can use the other electronic device to change the clothing worn by the virtual object of the second content (1080) or change its gender. Alternatively, the user can use the other electronic device to change the second content (1080) to another virtual object or add a new virtual object.
[0139] In one embodiment, the electronic device (2000) may identify another electronic device (2000) of the user and provide an interface that allows the user to edit and / or create second content on the other electronic device (2000). Specifically, the electronic device (2000) may be a TV, and the other electronic device of the user may be a smartphone. Since interaction via a smartphone is more convenient, the user may perform the task of editing and / or creating second content using the smartphone in a more detailed and easier manner.
[0140] FIG. 11A is a diagram for explaining an operation of an electronic device according to one embodiment of the present disclosure to store second content information in association with user information.
[0141] In one embodiment, the electronic device (2000) may store information about second content created by a user on a server in association with the user information. The information about the second content stored in association with the user information may be utilized when the user of the electronic device (2000) wishes to create second content at a different time in the future. For example, if first content is played on Channel 1 and the user has a history of creating second content based on the first content, information about the second content may be stored on the server. After some time has passed, if the first content is played (rebroadcast) again on Channel 1 and the same user is identified, the electronic device (2000) may receive information about the second content in real time from the server and apply the second content to create personalized content. Additionally, the electronic device (2000) may provide the user with a second content creation function that allows the user to create and edit the second content while providing the second content based on information about the second content received from the server.
[0142] FIG. 11b is a diagram for explaining an operation of an electronic device according to one embodiment of the present disclosure to store second content information in association with user information.
[0143] In one embodiment, the electronic device (2000) may store information regarding second content created by a user on a server in association with the user information. The information regarding the second content stored in association with the user information may be utilized when the user of the electronic device (2000) wishes to create second content on another channel. For example, if first content is played on Channel 1 and the user has a history of creating second content based on the first content, information regarding the second content may be stored on the server.
[0144] In one embodiment, the broadcast channel of the electronic device (2000) may be changed from Channel 1 to Channel 2, and the same person as the first content played on Channel 1 may appear in the content displayed on Channel 2. In this case, the electronic device (2000) may receive information about the second content in real time from the server and apply the second content to create personalized content. In addition, the electronic device (2000) may provide the second content based on the information about the second content received from the server, and may additionally provide the user with a second content creation function that allows the user to create and edit the second content.
[0145] FIG. 12 is a block diagram illustrating a configuration of an electronic device according to one embodiment of the present disclosure.
[0146] In one embodiment, the electronic device (2000) may include a communication interface (2100), a display (2200), a memory (2300), and a processor (2400).
[0147] The communication interface (2100) can perform data communication with other electronic devices under the control of the processor (2300).
[0148] The communication interface (2100) may include a communication circuit that can perform data communication between the electronic device (2000) and other devices using at least one of data communication methods including, for example, wired LAN, wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliances (WiGig), and RF communication.
[0149] According to one embodiment, a communication interface (2100) can transmit data for generating second content to an external device. For example, the communication interface (2100) can receive first content, or receive multiple artificial intelligence models including a generative artificial intelligence model and training data for training the artificial intelligence models.
[0150] The display (2200) can output a video signal to the screen of the electronic device (2000) under the control of the processor (2400). The display (2200) can display first content, second content, and personalized content under the control of the processor (2400).
[0151] The memory (2300) may store instructions, data structures, and program codes that can be read by the processor (2400). In the disclosed embodiments, operations performed by the processor (2400) may be implemented by executing instructions or codes of a program stored in the memory (2300).
[0152] The memory (2300) may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or a SRAM (Static Random Access Memory).
[0153] According to one embodiment, the memory (2300) may store one or more instructions and / or programs that cause the electronic device (2000) to operate to generate second content. For example, the memory (2300) may store a content analysis module (2310), a user interface generation module (2320), and a content generation module (2330).
[0154] The processor (2400) can control the overall operations of the electronic device (2000). For example, the processor (2400) can control the overall operations of the electronic device (2000) to generate second content by executing one or more instructions of a program stored in the memory (2300).
[0155] The processor (2400) may be configured with at least one of, but is not limited to, a central processing unit, a microprocessor, a graphic processing unit, an application specific integrated circuits (ASICs), a digital signal processor (DSPs), a digital signal processing device (DSPDs), a programmable logic device (PLDs), a field programmable gate array (FPGAs), an application processor, a neural processing unit, or an artificial intelligence processor designed with a hardware structure specialized for processing an artificial intelligence model.
[0156] In one embodiment, the processor (2400) may execute a content analysis module (2310) to analyze the first content. The processor (2400) may analyze scenes of the first content using one or more artificial intelligence models and obtain information related to the first content (e.g., the genre of the first content). Since the operations of the content analysis module (2310) have been described above in the previous drawings, a repetitive description will be omitted.
[0157] In one embodiment, the processor (2400) may execute the user interface generation module (2320) to generate a user interface for generating second content. The user interface may include at least one of editable content elements and generable content elements in the first content. A user of the electronic device (2000) may select a content element to be personalized within the user interface and edit and / or change the selected element to generate the second content. The user may edit the content using the electronic device (2000). The electronic device (2000) may display a user interface for editing the content and receive user input for editing the media content from the user. Since the operations of the user interface generation module (2320) have been described above in the previous drawings, a repeated description thereof will be omitted.
[0158] In one embodiment, the processor (2400) may execute the content generation module (2330) to generate second content. The processor (2400) may provide characteristics of the first content (e.g., content elements within the first content) as input to a generative artificial intelligence model to generate second content with changed characteristics of the content elements. The second content may be combined with the first content and provided to the user as personalized content. The second content may include at least one of an image, audio, and text. In one embodiment, the second content may be a summary of the first content. The user may be provided with the summary of the first content through the electronic device (2000). Since the operations of the content generation module (2330) have been described above in the previous drawings, a repeated description thereof will be omitted.
[0159] The present disclosure provides a user interface that utilizes generative artificial intelligence to enable users viewing content to generate personalized content, and provides the generated personalized content to the user. The technical challenges addressed by the present disclosure are not limited to those described above, and other technical challenges not mentioned herein will be readily apparent to those skilled in the art, based on the description herein.
[0160] According to one aspect of the present disclosure, a method for an electronic device to provide content may be provided.
[0161] The method may include a step of obtaining information related to the first content being played.
[0162] The method may include generating a user interface including at least one of editable content elements and creatable content elements in the first content based on information related to the first content.
[0163] The method may include receiving user input for selecting a content element via the user interface.
[0164] The method may include a step of generating second content corresponding to the first content based on a content element corresponding to the user input.
[0165] The method may include a step of generating personalized content including the first content and the second content.
[0166] Information related to the first content may include at least one of a title, producer, playback time, genre, performer information, platform information, and channel information.
[0167] The method may include a step of generating scene context data using a video frame included in the first content.
[0168] The method may include a step of identifying a genre of the first content based on at least one of information related to the first content and the scene context data.
[0169] The step of generating the user interface may be generating the user interface based on the genre of the first content.
[0170] The step of generating the user interface may be generating the user interface so as to include predefined content elements corresponding to the genre of the first content.
[0171] The step of generating the second content may include a step of obtaining the second content by inputting a content element corresponding to the user input into a generative artificial intelligence model.
[0172] The above generative artificial intelligence model may include at least one of a first generative artificial intelligence model that generates an image, a second generative artificial intelligence model that generates audio, and a third generative artificial intelligence model that generates text.
[0173] The second content may include at least one of an image, audio, and text.
[0174] The method may include a step of storing the second content in association with user information.
[0175] The method may include a step of providing a summary of the first content to the user based on the termination of playback of the first content.
[0176] The step of providing a summary of the first content to the user may include a step of generating a summary of the first content in a background process when the first content being played on the electronic device is changed to other content.
[0177] According to one aspect of the present disclosure, an electronic device providing content may be provided.
[0178] The electronic device may include a communication interface; a display; a memory storing one or more instructions; and at least one processor executing the one or more instructions.
[0179] The at least one processor can obtain information related to the first content being played by executing the one or more instructions.
[0180] The at least one processor may, by executing the one or more instructions, generate a user interface including at least one of editable content elements and createable content elements in the first content based on information related to the first content.
[0181] The at least one processor may receive user input for selecting a content element through the user interface by executing the one or more instructions.
[0182] The at least one processor may generate second content corresponding to the first content based on a content element corresponding to the user input by executing the one or more instructions.
[0183] The at least one processor can display personalized content including the first content and the second content on the screen of the display by executing the one or more instructions.
[0184] Information related to the first content may include at least one of a title, producer, playback time, genre, performer information, platform information, and channel information.
[0185] The at least one processor may generate scene context data using a video frame included in the first content by executing the one or more instructions.
[0186] The at least one processor can identify a genre of the first content based on at least one of information related to the first content and the scene context data by executing the one or more instructions.
[0187] The at least one processor may generate the user interface based on the genre of the first content by executing the one or more instructions.
[0188] The at least one processor may generate the user interface to include a predefined content element corresponding to a genre of the first content by executing the one or more instructions.
[0189] The at least one processor can obtain the second content by inputting a content element corresponding to the user input into a generative artificial intelligence model by executing the one or more instructions.
[0190] The above generative artificial intelligence model may include at least one of a first generative artificial intelligence model that generates an image, a second generative artificial intelligence model that generates audio, and a third generative artificial intelligence model that generates text.
[0191] The at least one processor can store the second content in association with user information by executing the one or more instructions.
[0192] The at least one processor may provide a summary of the first content to the user based on the termination of playback of the first content by executing the one or more instructions.
[0193] The at least one processor may generate a summary of the first content in a background process when the first content being played on the electronic device is changed to other content by executing the one or more instructions.
[0194] Meanwhile, embodiments of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and include both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include computer storage media and communication media. Computer storage media include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media may typically include computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules.
[0195] Additionally, a computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.
[0196] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.
[0197] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.
[0198] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.
Claims
1. In a method for an electronic device to provide content, A step of obtaining information related to the first content being played; A step of generating a user interface including at least one of editable content elements and creatable content elements in the first content based on information related to the first content; A step of receiving user input for selecting a content element via the user interface; A step of generating second content corresponding to the first content based on a content element corresponding to the user input; and A method comprising the step of generating personalized content including the first content and the second content.
2. In paragraph 1, The above method, A step of generating scene context data using a video frame included in the first content; and Further comprising a step of identifying a genre of the first content based on at least one of information related to the first content and the scene context data, The steps for creating the above user interface are: A method for generating the user interface based on the genre of the first content.
3. In paragraph 2, The steps for creating the above user interface are: A method for generating the user interface so as to include predefined content elements corresponding to the genre of the first content.
4. In any one of paragraphs 1 to 3, The step of generating the second content is as follows: A method comprising the step of obtaining the second content by inputting a content element corresponding to the user input into a generative artificial intelligence model.
5. In paragraph 4, A method, wherein the generative artificial intelligence model comprises at least one of a first generative artificial intelligence model that generates an image, a second generative artificial intelligence model that generates audio, and a third generative artificial intelligence model that generates text.
6. In paragraph 1, The above method, A method further comprising the step of providing a summary of the first content to the user based on the termination of playback of the first content.
7. In paragraph 6, The step of providing a summary of the above first content to the user is: A method comprising: generating a summary of the first content in a background process when the first content being played on the electronic device is changed to other content.
8. For electronic devices providing content, communication interface; display; Memory for storing one or more instructions; and comprising at least one processor executing one or more of the above instructions; The at least one processor, by executing the one or more instructions, Obtain information related to the first content being played, Based on information related to the first content, generating a user interface including at least one of editable content elements and createable content elements in the first content, Receiving user input for selecting a content element via the above user interface, Generate second content corresponding to the first content based on the content element corresponding to the user input, An electronic device that displays personalized content including the first content and the second content on a screen of the display.
9. In paragraph 8, The at least one processor, by executing the one or more instructions, Generate scene context data using the video frame included in the above first content, Identifying the genre of the first content based on at least one of the information related to the first content and the scene context data, An electronic device that generates the user interface based on the genre of the first content.
10. In paragraph 9, An electronic device wherein said at least one processor generates the user interface by executing said one or more instructions to include a predefined content element corresponding to a genre of said first content.
11. In any one of paragraphs 8 to 10, An electronic device wherein said at least one processor obtains said second content by inputting a content element corresponding to said user input into a generative artificial intelligence model by executing said one or more instructions.
12. In paragraph 11, An electronic device, wherein the generative artificial intelligence model includes at least one of a first generative artificial intelligence model that generates an image, a second generative artificial intelligence model that generates audio, and a third generative artificial intelligence model that generates text.
13. In paragraph 8, An electronic device wherein said at least one processor provides a summary of the first content to a user based on the termination of playback of the first content by executing said one or more instructions.
14. In paragraph 13, An electronic device wherein said at least one processor generates a summary of the first content in a background process when the first content being played on the electronic device is changed to other content by executing said one or more instructions.
15. A computer-readable recording medium having recorded thereon a program for executing the method of any one of clauses 1 to 7 on a computer.
Citation Information
Patent Citations
Haptic effect signal processing
KR1020200081211A
Mask with strap separation and connection structure
KR1020220114938A
Driving chip and display device including the same
KR1020240124479A
buoy and ship route guide system using the same
KR102490585B1
KR20220106005A
