Electronic device and control method therefor

Abstract

This electronic device comprises: a communication unit; a memory for storing at least one instruction; and one or more processors for executing the at least one instruction, wherein the one or more processors: generate a first video corresponding to content acquired via the communication unit; acquire person information included in the content on the basis of at least one of video information or audio information corresponding to the content; identify a main person of the content on the basis of data including the acquired person information and additional information; and generate a second video including the person information when a preset event occurs.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device capable of recognizing a key person within content and a method for controlling the same.

[0002] An electronic device can perform operations such as generating images corresponding to content or displaying images. Such an electronic device can provide the user not only with images corresponding to the content selected by the user, but also with various information related to the content.

[0003] Embodiments of the present disclosure may solve at least one of the previously described problems and / or disadvantages and provide the advantages described below. Accordingly, the embodiments of the present disclosure provide an electronic device capable of recognizing a key person within content and a method for controlling the same.

[0004] Additional embodiments will be presented in the detailed description below, some of which are obvious from the detailed description, and other embodiments can also be presented through learning from the presented embodiments.

[0005] An electronic device according to an embodiment of the present disclosure is disclosed. The electronic device comprises a communication unit, a memory for storing at least one instruction, and one or more processors for executing said at least one instruction.

[0006] The above one or more processors generate a first image corresponding to content obtained through the communication unit, obtain person information included in the content based on at least one of image information and audio information corresponding to the content, identify a key person of the content based on data including the obtained person information and additional information, and generate a second image including person information when a preset event occurs.

[0007] The above one or more processors can generate data including person information for each person included in the content and time information within the content where the person appears.

[0008] The above one or more processors can identify key figures among multiple figures based on at least one of appearance time per person, appearance proportion, mention frequency calculated based on audio information, and web search results.

[0009] The second image containing the above person information may include information corresponding to the identified key person.

[0010] When one or more processors receive user input corresponding to a person search, they can identify a person who appeared within a preset time and at the time the input was received, and generate a second image containing information about the identified person.

[0011] The above one or more processors acquire an image corresponding to the content at a preset period and can identify whether a person included in the acquired image is a recognized person.

[0012] The above one or more processors can identify a person in the acquired image based on at least one of image comparison of a face region included in the acquired image and text recognition results within the acquired image.

[0013] The above one or more processors can perform a function corresponding to speech recognition using the audio information and obtain information related to a name based on the result of the performed function.

[0014] One or more processors can obtain name information included in the content and weight information for each of the name information based on a title corresponding to the content.

[0015] The above one or more processors can generate a second video including at least one of content related to a person and advertising content corresponding to the acquired person information.

[0016] A control method for an electronic device according to one embodiment of the present disclosure comprises: a step of acquiring content; a step of generating a first image corresponding to the acquired content; a step of acquiring person information included in the content based on at least one of image information and audio information corresponding to the content; a step of identifying a key person of the content based on data including the acquired person information and additional information; and a step of generating a second image including the person information when a preset event occurs.

[0017] The control method may further include the step of generating data including person information for each person included in the content and time information within the content where the person appears.

[0018] The step of determining the main character above can identify the main character among a plurality of characters based on at least one of the appearance time, appearance proportion, mention frequency calculated based on audio information, and web search results per character.

[0019] The second image containing the above person information may include information corresponding to the identified key person.

[0020] The step of generating the second image above can identify a person who appeared within a preset time and at the time when the input was received, when a user input corresponding to a person search is received, and generate a second image including information about the identified person.

[0021] The control method may further include the step of acquiring an image corresponding to the content at a preset period and identifying whether a person included in the acquired image is a recognized person.

[0022] The step of confirming whether the person is the previously recognized person can identify the person in the acquired image based on at least one of image comparison of the face region included in the acquired image and text recognition results within the acquired image.

[0023] The control method may further include the step of performing a function corresponding to speech recognition using the audio information and obtaining information related to a name based on the result of the performed function.

[0024] The control method may further include the step of obtaining name information included in the content and weight information for each of the name information based on a title corresponding to the content.

[0025] A non-transient computer-readable recording medium storing a program for executing a control method for an electronic device according to one embodiment of the present disclosure, wherein the control method comprises the steps of: acquiring content; generating a first image corresponding to the acquired content; acquiring person information included in the content based on at least one of image information and audio information corresponding to the content; identifying a key person of the content based on data including the acquired person information and additional information; and generating a second image including the person information when a preset event occurs.

[0026] The above-described or other aspects, features, and benefits of embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings. In the accompanying drawings:

[0027] FIG. 1 is a drawing for explaining a person information provision operation according to one embodiment of the present disclosure,

[0028] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0029] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0030] FIG. 4 is a diagram illustrating the operation of creating a person information DB according to one embodiment of the present disclosure,

[0031] FIG. 5 is a diagram illustrating the operation of acquiring person information according to one embodiment of the present disclosure,

[0032] FIG. 6 is a drawing illustrating an example of a person database created according to one embodiment of the present disclosure,

[0033] FIG. 7 is a drawing for explaining an example of a screen according to one embodiment of the present disclosure,

[0034] FIG. 8 is a drawing for explaining an example of a screen according to one embodiment of the present disclosure,

[0035] FIG. 9 is a flowchart illustrating the control operation of an electronic device according to one embodiment of the present disclosure.

[0036] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0037] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0038] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0039] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0040] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components, etc.) and do not exclude the presence of additional features.

[0041] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0042] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0043] Where it is stated that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., a third component).

[0044] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0045] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0046] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0047] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0048] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0049] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0050] Meanwhile, an electronic device according to various embodiments of the present disclosure may include, for example, at least one of a terminal device, a tablet PC, a desktop PC, a laptop PC, a server, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit.

[0051] In some embodiments, the electronic device is, for example, a television, a DVD (digital video disk) player, audio, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync). TM , Apple TV TM , or Google TV TM ), game console (e.g., Xbox) TM PlayStation TM It may include at least one of an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame. Meanwhile, among the electronic devices described above, a device equipped with a display may be referred to as a display device. Meanwhile, even if the electronic device of the present disclosure does not have a display, it may be a set-top box or a PC that provides images to a display device.

[0052] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0053] FIG. 1 is a drawing for explaining a person information provision operation according to one embodiment of the present disclosure.

[0054] Referring to FIG. 1, the electronic device (100) can display a second image (220) containing person information (210) in response to a user's request while displaying a first image corresponding to the content. Meanwhile, in the illustrated example, for ease of explanation, it is assumed that the electronic device directly displays the image, but in implementation, the electronic device may be a device that outputs the image without having a display.

[0055] Here, the first image is an image corresponding to the content, and the second image is an image containing information about a person selected by the user. In the illustrated example, it is assumed that the image corresponding to the content and the person information are displayed together; however, in implementation, only information about the person selected by the user may be displayed, or a list of main characters appearing in the content may be displayed instead of information about a specific person. Such examples are described later in FIGS. 7 and 8.

[0056] Here, content refers to providing movies, music, plays, photographs, comics, animations, computer games, text, shapes, colors, sounds, movements, or images, or a combination of the aforementioned, through an electronic device. Among such content, regarding content such as movies, plays, and dramas, users may be interested in the characters appearing within the content, or in the actors who played those characters.

[0057] For example, while watching content, users may be curious about other works by the actor currently playing the role, or about the costumes worn by the character.

[0058] Here, "characters" refer to individuals appearing in the relevant content, such as novels, plays, movies, and novels. Here, the term "character" does not refer specifically to human beings but can include anthropomorphic animals, mythical beings, and even inanimate objects within the content. In other words, in games, animations, etc., characters, animals, mascots, etc., that are not human can also be considered characters. Such characters may be referred to as protagonists, players, narrators, personas, or characters.

[0059] Previously, if a user was curious about a character, they had to search for the character's name on search sites or look up information related to the content on content provider servers. Alternatively, since some content provided information about key figures using metadata, users could utilize that metadata.

[0060] However, this method presented the inconvenience of requiring users to manually perform the search process and locate desired information one by one from a vast amount of data. Furthermore, while searching required knowing the name of the actor or character, performing the search itself was difficult if this information was unknown. Additionally, existing methods failed to provide information when users wanted to view other scenes featuring a specific actor.

[0061] To solve these problems, the present disclosure may create a character database for the relevant content and provide a service using the created database. Such a character database may include character information and time information on when the character appears.

[0062] Here, character information within the content may include role names, actual cast member names, or arbitrary identifiers, and may additionally include captured face images for image comparison. Such character information may include character information, actor information, etc.

[0063] In addition, time information may include the time or frame number at which the person appears. Whether the person appears on the screen can be determined in various ways. For example, whether certain criteria are met, such as whether they are maintained for a certain amount of time (e.g., 0.5 seconds) or whether they are included in a pre-set area on the screen, may be considered. Through such actions, it is possible to prevent people who appear very briefly within the content (e.g., extras) from being included in the person database.

[0064] In this manner, the electronic device (100) can create a database containing the characters appearing in the content. Although it has been described above that the database contains only actor information and time information regarding the actor information, in implementation, the database may additionally include background information (e.g., filming location, etc.) or additional information (e.g., score information in the case of a sports match).

[0065] Additionally, the electronic device (100) can select key figures using the database. For example, it is possible to determine a figure who has a significant presence in the content as a key figure and provide the aforementioned key figure to the user first when displaying figure information. There may be various methods for selecting whether a figure is a key figure, and examples thereof will be described later in FIG. 5.

[0066] The specific configuration and operation of the electronic device (100) of the present disclosure will be described later with reference to FIG. 2.

[0067] Meanwhile, the above describes and illustrates an example in which an electronic device independently generates a database and provides person information to a user using the generated database. However, in implementation, it is also possible for a separate server to generate a database corresponding to the content and for a separate device to receive and use the database generated by the server.

[0068] For example, regarding content such as VOD, an electronic device may acquire a database created in advance for the content from another device and perform the operation described above. In such a case, the database may contain a wider variety of information. For example, it may include information such as clothes or shoes worn by a specific actor in a specific situation (or sponsorship information, advertising information), and the database may include not only person information but also information about the background (or location) at that time.

[0069] In addition, although the illustrated example shows the electronic device directly displaying the image, in implementation, the electronic device may be a device that performs only the image generation operation as described above and provides the generated image to a separate display.

[0070] As described above, the electronic device according to the present disclosure creates a database having person information and time information and utilizes it, making it possible to provide various information related to a person to a user.

[0071] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0072] Referring to FIG. 2, the electronic device (100) may include a communication unit (110), a memory (120), and a processor (130).

[0073] The communication unit (110) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication unit (110) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module, etc. Here, each communication module may include at least one hardware chip or hardware circuit.

[0074] Wi-Fi modules and Bluetooth modules can perform communication via Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various types of information can be transmitted and received.

[0075] The infrared communication module performs communication according to infrared communication (IrDA, Infrared Data Association) technology, which uses infrared rays located between visible light and millimeter waves to wirelessly transmit data over short distances.

[0076] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0077] In addition, the communication unit (110) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or a UWB (Ultra Wide-Band) module.

[0078] According to one example, the communication unit (110) may use the same communication module (e.g., Wi-Fi module) to communicate with external devices such as a remote control and an external server.

[0079] According to other examples, the communication unit (110) may use different communication modules (e.g., Wi-Fi modules) to communicate with external devices such as a remote control and external servers. For example, the communication unit (110) may use at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may use a BT module to communicate with an external device such as a remote control. However, this is merely one embodiment, and the communication unit (110) may use at least one of various communication modules when communicating with multiple external devices or external servers.

[0080] The communication unit (110) can receive content. This content can be diverse, such as movies, music videos, dramas, short videos, etc. And while the content is assumed to be video, it may also be an image or referred to as video.

[0081] The communication unit (110) can receive information from a website or social media. For example, the communication unit (110) can provide a query (or search term, query, etc.) to a specific site under the control of the processor (130) and receive response information corresponding to the query. Here, the query may be a combination of keywords necessary to obtain search results for a person displayed in the content, such as "title and person" or "title and performer" corresponding to the specific content.

[0082] And the communication unit (110) can receive not only content but also information necessary for providing various applications and services of the electronic device (100) from an external device. For example, in cases where a database of people is not created directly, the communication unit (110) can obtain a database stored on an external server.

[0083] And when the communication unit (110) uses an external DB or external module for person search, it can transmit an image for the search and receive corresponding result information.

[0084] And the communication unit (110) can transmit the database generated in the process described later to an external server. In this way, by registering the database generated by one device to the server, other devices can use the database registered to the server.

[0085] The memory (120) may be implemented as internal memory such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM included in the processor (130), or as memory separate from the processor (130). In this case, the memory (120) may be implemented in the form of memory embedded in the electronic device (100) or in the form of memory that can be attached to and detached from the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that can be attached to and detached from the electronic device (100).

[0086] The memory (120) can store a database created in the process described below. Although the above description assumes that the electronic device (100) directly creates and uses a database, it is also possible to receive and use a database created by an external device.

[0087] And the memory (120) can store various contents (e.g., broadcast content, applications, etc.) received through the communication unit (110) described above.

[0088] Meanwhile, the memory embedded in the electronic device (100) is implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD), and the memory that can be attached to and detached from the electronic device (100) can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0089] Meanwhile, although the illustrated example shows the electronic device (100) being composed of a single memory, when distinguishing between volatile memory and non-volatile memory, the electronic device (100) may be described as including multiple memories.

[0090] The processor (130) can perform overall control operations of the electronic device (100). Specifically, the processor (130) functions to control the overall operation of the electronic device (100).

[0091] The processor (130) may be implemented as a digital signal processor (DSP), microprocessor, or time controller (TCON) that processes digital signals. However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), microcontroller unit (MCU), microprocessing unit (MPU), controller, application processor (AP), graphics-processing unit (GPU), communication processor (CP), or ARM processor. Additionally, the processor (130) may be implemented as a System on Chip (SoC) or large-scale integration (LSI) with built-in processing algorithms, or may be implemented in the form of a Field Programmable Gate Array (FPGA). Furthermore, the processor (130) can perform various functions by executing computer executable instructions stored in memory. Meanwhile, although FIG. 2 illustrates that the electronic device (100) includes only one processor, in implementation, multiple processors may be included. It may also include processors (e.g., CPU + GPU, CPU + DSP).

[0092] The processor (130) acquires content by using the communication unit (110). For example, when the processor (130) receives a playback command for specific content from a user, it can acquire the content by controlling the communication unit (110) to receive the content. Such content may be a video such as VOD, but may also be a real-time streaming video provided by a specific server, or broadcast content transmitted by a broadcasting station.

[0093] The processor (130) generates a first image corresponding to the acquired content. For example, if the content is real-time streaming, it can construct a screen using video data from the received streaming data and generate a first image using the screen.

[0094] At this time, the processor (130) can determine whether a person database for the content exists. For example, if the current content is content such as VOD, the communication unit (110) can be controlled to search for a person database for the content on a content provider or an external server to obtain the corresponding person database.

[0095] If there is no search result person database or if the current content is real-time content, the processor (130) can create a person database for the content.

[0096] For example, the processor (130) can obtain person information included in the content by using at least one of the video information and audio information corresponding to the content. Such a person database may include person information and additional information for each person included in the content. Here, the additional information may be time information, but may also be various other information such as information on the clothing worn by the person, advertising information related to the person, information on other works of the person, etc.

[0097] Such character databases may be referred to as character information, data, actor lists, etc.

[0098] The processor (130) captures video corresponding to the content at a preset interval and can check whether a person included in the captured image is a recognized person. Although it is mentioned here that video capture is performed periodically, it may also be performed non-periodically. For example, capture may be performed when a change in the scene of the video is detected, and the capture interval may be used differently depending on the genre of the scene or content.

[0099] If the person included in the image is a previously recognized person, time information corresponding to the currently captured video can be added to the previously recognized person in the database. If the person is not a previously recognized person, the processor (130) can add a new person to the database.

[0100] Confirmation of whether the person is such can be performed by comparing the face image of a previously registered person with the face image of the currently captured foreign object, or by performing OCR on the subtitles in the video. Meanwhile, if the name of the person is obtained through the process, the processor (130) can update the name of the person in the database.

[0101] Meanwhile, if multiple individuals are identified in a single image, the processor (130) may perform the aforementioned recognition operation for individuals larger than a preset size based on the size of the individuals' faces. At this time, the processor (130) may check the display time of the individuals within the image and perform the aforementioned recognition operation only when they are displayed for a certain period of time or longer.

[0102] And the processor (130) can perform a function corresponding to voice recognition using audio information, and can obtain name-related information (e.g., role name, actor name, role title, role nickname, etc.) based on the result of the performed voice recognition function. And by combining the voice recognition result and the video analysis result, it can obtain the person displayed at the time of current capture and the name of the person, etc.

[0103] By carrying out this process in a chronological order, the database for the content can be continuously updated. Meanwhile, although the analysis of video and audio has been described above as being performed individually, the analysis described above can be performed in conjunction. For example, in the case where multiple people are included in the video, although it was stated that person recognition is performed by considering the size of the people in the above case, when implementing, even if the proportion is small, if a person is conducting a conversation, it is highly likely to be a main person. Therefore, the processor (130) can identify a person by considering audio information and video information together, such as the person conducting the conversation and the conversation state.

[0104] The processor (130) identifies key figures based on the generated character database. For example, the processor (130) can identify key figures among multiple figures based on at least one of the appearance time per person, appearance proportion, mention frequency calculated based on audio information, and web search results.

[0105] The determination of such key figures can be made at the time when video output for the relevant content is completed, by evaluating currently recognized figures when figure information is needed, or by identifying key figures at regular time intervals.

[0106] The processor (130) can obtain name information, etc. included in the content based on a title corresponding to the content. For example, the processor (130) can obtain information about a person displayed in the content (e.g., person's name, etc.) through web search or social search. At this time, weight information for each person can also be obtained by considering the search frequency of each person.

[0107] The processor (130) can perform clustering of people. For example, depending on the situation, a person may be recognized as a different person depending on whether they are wearing different clothes, glasses, etc. Therefore, the processor (130) can perform a process of clustering people within the database when the database is completed or at preset time intervals to check whether one person is recognized as multiple people, and merging them into one person.

[0108] The processor (130) generates a second image containing person information when a preset event occurs. For example, the processor (130) can generate a second image containing information corresponding to a key person obtained when a preset event occurs. Here, the preset event may be a user command requesting a list of key people within the content, or a user command requesting information about a person within the currently displayed image.

[0109] Alternatively, when the processor (130) receives user input corresponding to a person search, it may search for a person that appeared within the time at which the input was received and a preset time, and generate a second image containing information about the searched person. An example of such an implementation is described later in FIG. 7.

[0110] Additionally, the processor (130) can generate content corresponding to a person selected by the user. For example, if the user selects performer A, the processor (130) can generate content about the person by using time information corresponding to that person to extract and merge videos within that time information within the content. For example, if the user requests past scenes of the current batter while watching a baseball game, the processor (130) can check all segments where the batter appeared and generate a video by combining those segments.

[0111] As described above, the electronic device according to the present disclosure generates a database containing information about a person included in the content and can provide the person information to a user using the same, so that the user can know the person information about the content more accurately.

[0112] Meanwhile, although only a simple configuration constituting the electronic device (100) has been illustrated and described above, various additional configurations may be provided during implementation. This will be explained below with reference to FIG. 3.

[0113] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0114] Referring to FIG. 3, the electronic device (100') may include a communication unit (110), memory (120), processor (130), input / output interface (140), microphone (150), display (160), and speaker (170).

[0115] The configuration of the communication unit (110), memory (120), and processor (130) was previously described in FIG. 2, and only the operation different from FIG. 2 will be described below.

[0116] The input / output interface (140) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface).

[0117] The input / output interface (140) can input and output at least one of audio and video signals. Depending on the implementation example, the input / output interface (260) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0118] And the input / output interface (140) can provide a video signal corresponding to a screen generated by the electronic device (100') or an audio signal together with the video signal to an external device (e.g., a display device, an STB, etc.).

[0119] The microphone (150) can receive the user's voice when active. For example, the microphone (150) may be formed integrally on the upper side, front side, or side side of the electronic device (100'). The microphone (150) may include various configurations such as a microphone for collecting analog user voice, an amplifier circuit for amplifying the collected user voice, an A / D conversion circuit for sampling the amplified user voice and converting it into a digital signal, and a filter circuit for removing noise components from the converted digital signal.

[0120] When a user's voice is input through such a microphone (150), the processor (130) can check the content of the user's voice and perform an action corresponding to the content of the voice. For example, the content of the voice may be a request for information about a person output in the content or a request for information about a person currently appearing on the screen.

[0121] Meanwhile, although it has been described above that user voice is input through the microphone (150), the microphone may be provided in a remote control for controlling the electronic device (100'), and user voice input through the microphone provided in the remote control may be input to the electronic device (100') and processed through the communication unit (110) described above.

[0122] The electronic device (100') can operate not only based on the configuration or remote control provided in the electronic device (100'), but also according to the control command of the terminal device. For example, if the electronic device is a TV or a set-top box, recently, manufacturers provide applications for controlling the TV or set-top box. Such applications can provide a function that allows the terminal device to be used as a remote control for the electronic device.

[0123] Accordingly, when a user executes an application to control a TV or set-top box using a terminal device and inputs a voice command through the terminal device, the electronic device (100') can perform a voice recognition operation and a corresponding voice recognition result using the voice signal input through the terminal device.

[0124] The display (160) can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, and a PDP (Plasma Display Panel). The display (160) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, and an OTFT (organic TFT). Meanwhile, the display (160) can be implemented as a touchscreen combined with a touch sensor, a flexible display, a 3D display, etc.

[0125] The display (160) can display various images. For example, the display (160) can display a first image or a second image generated by the processor (130).

[0126] The speaker (170) can output sound. Specifically, the speaker (170) may be a component that outputs various audio data processed at the input / output interface, as well as various notification sounds or voice messages. Additionally, the speaker (170) may output result information (e.g., person information) corresponding to the voice recognition operation described later.

[0127] Meanwhile, although the electronic device (100) in FIG. 3 is illustrated and described as including a display (160), if the electronic device (100') is a device such as a set-top box that does not include a display, the display configuration may be omitted. Also, depending on the implementation form, the speaker and microphone described above may also be omitted. Additionally, although not illustrated in FIG. 3, other components (e.g., camera, human body detection sensor) may be further included.

[0128] FIG. 4 is a diagram illustrating the operation of creating a person information DB according to one embodiment of the present disclosure.

[0129] Referring to FIG. 4, the DB creation module (400) may include a content analysis module (410), a metadata analysis module (420), and a search module (430). The operation of each of these modules may be performed by a single processor or by multiple processors individually. For example, the metadata analysis module (420) and the search module (430) may be performed by a CPU, and the content analysis module (410) may be performed by a GPU or a DSP.

[0130] The content analysis module (410) can acquire a person using video information and / or audio information within the content. To perform this operation, the content analysis module (410) can capture the video (or audio) within the content at a preset periodic interval. This periodic interval may vary depending on the scene or content genre, and may also vary depending on the scene attributes within the video. The capture periodic interval may be slow in scenes with little change in the person, and short in dialogue scenes with a lot of change in the person. Additionally, a method may be used in which the capture proceeds in response to the occurrence of audio, rather than performing periodic capture.

[0131] The content analysis module (410) may include a person analyzer (411), an OCR detector (413), and an ASR (415).

[0132] The person analyzer (411) can recognize a person within an image captured at preset intervals within the content. For example, the person analyzer (411) can obtain a face region within the captured image using an object detection method.

[0133] If, in the process, multiple face regions are acquired, the person analyzer (411) may perform an individual person recognition process for each face region. Meanwhile, if many face regions are detected, the person analyzer (411) may determine only a few face regions for which person recognition will be performed by considering at least one of the size and location among the detected face regions, and may perform a person recognition operation only on the determined face regions.

[0134] And the person analyzer (411) can perform face recognition on the acquired face region. Such face recognition may utilize an object recognition model. This object recognition model may be a module that uses only internal data, or a module that uses information from an external server. This object recognition model may be implemented as a neural network model, etc.

[0135] These neural network models can be referred to as neural models, AI models, or learning models. These neural network models are computer systems that implement human-level intelligence, where the machine learns and makes judgments on its own, and the recognition rate improves with use.

[0136] The OCR (Optical Character Recognition) detector (413) can recognize video subtitles within a captured image and output text (e.g., a person's name) as a result. The OCR detector can also use a neural network model that outputs text when it receives an image as input.

[0137] And the OCR detector (413) receives subtitle data for content, not a captured image, and can obtain person information within the subtitle data.

[0138] Automatic Speech Recognition (ASR) (415) is a configuration that recognizes speech using audio information. For example, the ASR can convert audio information input in segments into text and obtain information related to a person (e.g., name) within the converted text. A neural network model that outputs a name corresponding to the input text can be used for such an ASR (415).

[0139] The metadata analysis module (420) is configured such that the Content Provider obtains metadata provided by the Service Provider.

[0140] Here, metadata may include the title, cast, description information, etc.

[0141] The metadata analysis module (420) can obtain the title of the content using the title among the obtained metadata, and can obtain performer information using the description information. The performer information obtained here may be some of the people included in the content.

[0142] The search module (430) obtains person information related to the content on the internet. For example, the search module (430) can perform a search based on the title of the content obtained from metadata and vectorize the search results. Here, vectorization involves extracting words from the search results and storing them in a word dictionary. When the same word is extracted from the search results, the vectorization of the person's name can be performed using a method (Counter Vectorization) that additionally stores the frequency.

[0143] These search modules (430) may include a web search module (431) and an SNS search module (433).

[0144] The web search module (431) can search for and obtain person information, etc. on the internet using the content title (or title + actor) obtained from metadata as a keyword.

[0145] The SNS search module (433) can obtain person information, etc. by using an SNS service application with the content title obtained from metadata as a keyword.

[0146] The search module (430) can assign TF (Term Frequency) to the search results. For example, weights can be assigned to each person based on the number of occurrences or repetitions among the information obtained from web search and SNS search. Here, TF is the number of times a specific word appears in a single search result; the higher the frequency of the word, the higher the importance of the word. Additionally, additional weights can be assigned to actors who do not appear in other content search results but appear frequently only in that content. For example, a method of assigning a high weight when the IDF is small can be applied. Here, IDF (Inverse Document Frequency) is the number of times a specific word appears in the entire parameter of search results.

[0147] The DB creation module (400) can create a candidate database (441) that selects important character candidates having information about characters appearing in the content, including the configuration described above. Such a database may have character information and time information. This will be described later in FIG. 6.

[0148] Then, key figures can be selected using the generated candidate database (441), and a final key figure database (451) can be generated accordingly. Specific selection methods, etc., will be described later in FIG. 5.

[0149] Meanwhile, although the creation of a database using multiple modules has been illustrated and described above, some of the aforementioned configurations may be omitted during implementation.

[0150] FIG. 5 is a drawing for explaining the operation of acquiring person information according to one embodiment of the present disclosure.

[0151] First, when content is received, metadata corresponding to the content can be obtained (505). Then, search keywords can be set based on the content of the metadata. For example, a title within the metadata can be checked, and a search keyword can be created using keyword 1 corresponding to the title and keyword 2 corresponding to the actor. Or, if actor information is included in the metadata, a search keyword can be created using keyword 1 corresponding to the title and keyword 2 corresponding to the actor's name. When implementing, the electronic device can create multiple queries by combining the keywords described above.

[0152] By generating multiple queries in this way, you can perform a search based on the generated queries.

[0153] Meanwhile, the actor information listed in the metadata is not information about all actors or characters displayed in the content, so separately, information about the output actor (or character) can be collected using the title in the metadata (510).

[0154] During this process, vectorization of the characters can be performed, and character-specific vectorization can be carried out based on the frequency of identical words within the search results. In this vectorization process, high weights can be assigned to characters listed in the metadata or those recognized as lead actors. Additionally, as previously explained, such weights can be set by considering TF and IDF.

[0155] A list of candidate people can be generated by combining the results of the search through this process (520).

[0156] Apart from the search using the aforementioned metadata, person search can be performed using content. For example, video and / or audio corresponding to the content can be periodically captured, and person identification can be performed using the captured data. For example, when using video, face recognition within the captured image can be recognized using vision recognition, and person recognition can be performed through object recognition of the recognized face.

[0157] Meanwhile, when only captured video is used, the aforementioned face recognition may utilize an algorithm that compares whether a previously recognized person and a newly captured person are the same person, and when an external DB (or external server) is used, the aforementioned face recognition may obtain a result that outputs the actor's name. Meanwhile, in movies or dramas, names corresponding to the roles may be used instead of the actor's name in audio or subtitles, and thus, by using the audio search results and subtitle results described below, a correspondence result (or mapping result) between the actor's name and the role name may be obtained, or the correspondence result between the actor's name and the role name may be obtained and used by utilizing the search result data described above.

[0158] OCR recognition is a method that utilizes subtitle data within a video. For example, in entertainment shows or sports, information about actors or characters is often displayed as video subtitles. Therefore, if text such as subtitles is detected within the video, the name of the person in the video can be obtained by recognizing that text.

[0159] Dialogue recognition is a method of using audio data to create text by speech recognition of the audio data and obtaining words such as names within the created text (545).

[0160] In this way, by combining information obtained through video and audio information, person information and time information included in the content can be generated.

[0161] For example, time information corresponding to current video and audio information and person information at that time can be organized chronologically (550). This operation can be performed periodically, and the importance or priority of each person can be determined based on the accumulated count, time ranking, etc. At this time, each list can store the name of the person and weight information (or time information).

[0162] Meanwhile, since the above-described face recognition result is performed using actual face images, there is a possibility that it will be searched for by the name of the actual performer rather than the name of the character within the content. Therefore, additional correspondence information between the name of the actual performer and the name of the character can be generated through searching, etc. (555). This operation can be performed through web search or generated through mapping between video and audio information.

[0163] For example, through image analysis, a result named Name 1 can be obtained for a face image named A. Meanwhile, if audio analysis yields a result that the person's name at that point in time is Name 3, association information between Name 1 (actor's name) and Name 3 (character's name) can be obtained. Alternatively, through a web search, association information that Name 1 is acting as Name 3 in a specific movie can be obtained.

[0164] Through this process, when candidate figures are identified, a database containing information included in the aforementioned process and metadata can be created (560).

[0165] Then, by considering time information within the person, etc., it is determined whether the person is a key figure, and finally, a database of key figures can be created (570). For example, by obtaining actor information using metadata, video information, and audio information, the weight or importance of each person can be distinguished by considering whether the person is searched in each of the three passes, the weights described above, and the time when the person appears (580). Alternatively, a high priority can be assigned to the results analyzed in the video during the process described above.

[0166] Such results can be calculated as specific numerical values, and may also be distinguished as whether the actor is a lead actor or a supporting actor. An example of a database generated through this process is shown in Fig. 6.

[0167] Meanwhile, although the above description explains the creation of a separate database for key figures, a method of creating a single database and managing only the indexes corresponding to key figures separately may also be used.

[0168] Meanwhile, although Figures 4 and 5 illustrate and explain the use of web search using metadata, if the content does not include metadata during implementation, person analysis may be performed using only video and audio information.

[0169] FIG. 6 is a drawing illustrating an example of a person database created according to one embodiment of the present disclosure.

[0170] Referring to Fig. 6, the database includes name information, time information, and key person information.

[0171] Name information may include actors identified using metadata, image information, and voice information as described in FIG. 5. In the illustrated example, only four people are included, but in implementation, fewer than three people may be included, or more people may be included.

[0172] Additionally, all characters appearing in the content may be included in the list, and in implementation, only users who have spent a certain amount of time (or more than a certain weight value) may be included in the list instead of all performers. In this case, the name information may be represented as the actual username, or it may be sorted by the character name within the content.

[0173] Time information may be the time information where the person appears. In implementation, this time information refers to the time information within the video where the person appears, and may be a numerical value displayed as a time value based on the start time, or a numerical value displayed as a frame number based on the start frame.

[0174] Lead status is information indicating whether the person is a main character. In the illustrated example, it simply indicates whether the person is a main character or not, but in implementation, it may be expressed as a specific numerical value rather than O or X.

[0175] FIG. 7 is a drawing for explaining an example of a screen according to an embodiment of the present disclosure. For example, FIG. 7 is a drawing illustrating an example of a screen that can be displayed on an electronic device when a command requesting person information is input while watching a video.

[0176] Referring to FIG. 7, the user interface window (700) may include a content area (710) and person information (720).

[0177] For example, while watching a video of content, a user may become curious about the people appearing in the content. Accordingly, if the user inputs a command requesting information about the performers (e.g., voice or specific button input), the electronic device may display a second video containing person information (720) about the main person of the current content as illustrated.

[0178] In the illustrated example, an example is shown in which the content area (710) and person information (720) are displayed together, but in implementation, it is possible to display only the list. Also, in the illustrated example, the actors for the content are displayed in a list format, but in implementation, it is possible to display the faces together or just the faces, or instead of a list, information about the person that the user is expected to be most curious about may be displayed immediately. In this case, a user interface window such as that shown in FIG. 8, which will be described later, may be displayed.

[0179] Meanwhile, in the illustrated example, a list of several people is displayed and then the user selects a specific person, but in the implementation, information about a specific person can be displayed immediately without displaying a list.

[0180] For example, as described, if a user command requesting person information is entered during display of performer A in the first video corresponding to the content, the electronic device may immediately display information about performer A.

[0181] FIG. 8 is a drawing for explaining an example of a screen according to one embodiment of the present disclosure.

[0182] Referring to FIG. 8, the user interface window (800) may include a content area (810), person information (720), and time information (830).

[0183] The content area (810) is an area where a video corresponding to the content currently being played is displayed.

[0184] The character information (720) is an area that displays information about a character selected by the user from the preceding list or a character displayed on the current screen. For example, as illustrated, information about other works in which the actor who played the character appeared may be displayed. Alternatively, other interview videos, etc., may be displayed in addition to the aforementioned other works.

[0185] If the user selects another displayed work or content, the electronic device may acquire and display a website or content corresponding to the work or content selected by the user.

[0186] Time information (830) can display the time period in which the character appears. Users can select from the item to view other areas in the content where the actor appears. Meanwhile, in the illustrated example, the time information was displayed in a form that visually displays the area corresponding to the location where the character appears in the area (or timeline) corresponding to the content, but in implementation, it is possible to display it simply in a list form, or to display highlights (or key scenes) by section in a thumbnail form.

[0187] In addition, when implementing, it is also possible for the electronic device (100) to use the corresponding time information to combine only the area output by the actor to create content corresponding to the actor.

[0188] Meanwhile, although FIGS. 7 and 8 illustrate and describe providing only text information such as appearance information for a specific person, when implementing, it is also possible to use the preceding time information to extract only the video segments where the person selected by the user or the currently displayed person appears, and to create new content for that person and provide it to the user.

[0189] FIG. 9 is a flowchart illustrating the control operation of an electronic device according to one embodiment of the present disclosure.

[0190] Referring to FIG. 9, the electronic device (100) acquires content (S910). For example, if a playback command for specific content is received from a user, the content can be acquired.

[0191] A first image corresponding to the acquired content is generated (S920). Meanwhile, if the electronic device is equipped with a display capable of displaying an image, the first image can be displayed.

[0192] For example, if the content is real-time streaming, a screen can be constructed using video data from the received streaming data, and a first video can be generated using the screen.

[0193] At this time, data having foreign object information corresponding to the content can be generated (S930). For example, person information included in the content can be obtained by using at least one of video information and audio information corresponding to the content. Such data may include person information for each person included in the content and time information within the content where the foreign object appears.

[0194] Video corresponding to the content is captured at a preset interval, and it is possible to check whether a person included in the captured image is a previously recognized person. If the person included in the image is a previously recognized person, time information corresponding to the currently captured video can be added to the previously recognized person. If the person is not a previously recognized person, a new person can be added to the corresponding database.

[0195] Verification of whether the individual is the same person can be performed by comparing the face image of a previously registered person with the face image of the currently captured foreign object, or through OCR of the subtitles within the video. Meanwhile, if the name of the person is obtained through this process, the name of the person in the corresponding database can be updated.

[0196] Meanwhile, if multiple individuals are identified in a single image, the recognition operation described above can be performed for individuals larger than a preset size based on the size of the individuals' faces. At this time, the electronic device (100) can check the display time of the individuals in the image and perform the recognition operation described above only when they are displayed for a certain period of time or longer.

[0197] Furthermore, audio information is used to perform functions corresponding to speech recognition, and name-related information (e.g., role name, actor name, role title, role nickname, etc.) can be obtained based on the results of the performed functions. Additionally, by combining these speech recognition results and video analysis results, the displayed person at the time of current capture and the name of that person can be obtained.

[0198] By carrying out this process chronologically, the database for the content can be continuously updated.

[0199] Then, based on the previously generated data, the main characters of the content are identified (S930). For example, the electronic device (100) can identify the main characters among a plurality of characters based on at least one of the appearance time per character, appearance proportion, mention frequency calculated based on audio information, and web search results.

[0200] Identification of such key figures can be performed when video output for the relevant content is completed, by evaluating currently recognized figures at the time when figure information is needed, or by determining key figures at regular intervals.

[0201] Name information, etc., included in the content can be obtained based on the title corresponding to the content. For example, information about a person displayed in the content (e.g., person's name, etc.) can be obtained through web search or social search. At this time, weight information for each person can also be obtained by considering the search frequency of each person.

[0202] When a preset event occurs, the electronic device (100) generates a second image containing person information (S940). For example, when a preset event occurs, the electronic device (100) can generate a second image containing information corresponding to the acquired main person.

[0203] Meanwhile, although the above description illustrates that a second video containing the aforementioned person information is displayed when a user requests actor information, it is also possible to display information about the currently recognized person along with the video even without a user request, when a person is recognized on the current screen.

[0204] When a person search command is received, the electronic device searches for a person who appeared within a preset time range relative to the time when the command was entered, and generates a second image containing information about the searched person.

[0205] Meanwhile, although the illustrated example depicts and explains an instance where an electronic device directly creates and uses a database, in implementation, when a user selects content, it is also possible to search for whether a person database corresponding to that content exists, and if such a database is found, to acquire and use the retrieved database.

[0206] As described above, the control method according to the present disclosure creates a database containing information about a person included in the content and can provide the person information to a user using the database, so that the user can know the person information about the content more accurately.

[0207] Meanwhile, methods according to at least some of the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0208] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0209] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0210] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium. For example, a 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TMIt can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., terminal devices). For online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0211] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0212] When the above-described instruction is executed by a processor, the processor may perform a function corresponding to the instruction directly or by using other components under the control of said processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0213] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Communications Department; Memory for storing at least one instruction; and It includes one or more processors that execute at least one of the above instructions; The above one or more processors, A first image corresponding to the content obtained through the above communication unit is generated, and Based on at least one of the video information and audio information corresponding to the above content, person information included in the content is obtained, and Identify the main characters of the content based on data including the acquired character information and additional information, and An electronic device that generates a second image containing person information when a preset event occurs.

2. In Paragraph 1, The above one or more processors, An electronic device that generates data including person information for each person included in the above content and time information within the above content in which the person appears.

3. In Paragraph 1, The above one or more processors, An electronic device that identifies a key figure among multiple figures based on at least one of the following: appearance time per figure, appearance proportion, frequency of mention calculated based on audio information, and web search results.

4. In Paragraph 1, The second image containing the above-mentioned person information is, An electronic device containing information corresponding to the above-identified key person.

5. In Paragraph 1, The above one or more processors, An electronic device that, upon receiving user input corresponding to a person search, identifies a person who appeared within a preset time and at the time the input was received, and generates a second image containing information about the identified person.

6. In Paragraph 1, The above one or more processors, An electronic device that acquires an image corresponding to the above content at a preset interval and identifies whether a person included in the acquired image is a previously recognized person.

7. In Paragraph 6, The above one or more processors, An electronic device that identifies a person within an acquired image based on at least one of image comparison of a face region included in the acquired image and text recognition results within the acquired image.

8. In Paragraph 1, The above one or more processors, An electronic device that performs a function corresponding to voice recognition using the above audio information and obtains information related to a name based on the result of the above-described function.

9. In Paragraph 1, The above one or more processors, An electronic device that obtains name information included in the content and weight information for each of the name information based on a title corresponding to the content.

10. In Paragraph 1, The above one or more processors, An electronic device that generates a second image including at least one of content related to a person and advertising content corresponding to the above-mentioned acquired person information.

11. In a method for controlling an electronic device, The stage of acquiring content; A step of generating a first image corresponding to the above-mentioned acquired content; A step of obtaining person information included in the content based on at least one of video information and audio information corresponding to the content; A step of identifying key figures of the content based on data including the acquired figure information and additional information; and A control method comprising the step of generating a second image containing the person information when a preset event occurs.

12. In Paragraph 11, A control method further comprising the step of generating data including person information for each person included in the above content and time information within the above content in which the person appears.

13. In Paragraph 11, The step of determining the above key figures is, A control method for identifying a key person among a plurality of people based on at least one of appearance time per person, appearance proportion, mention frequency calculated based on audio information, and web search results.

14. In Paragraph 11, The second image containing the above-mentioned person information is, A control method comprising information corresponding to the identified key person.

15. A non-transient computer-readable recording medium storing a program for executing a method of controlling an electronic device, The above control method is, The stage of acquiring content; A step of generating a first image corresponding to the above-mentioned acquired content; A step of obtaining person information included in the content based on at least one of video information and audio information corresponding to the content; A step of identifying key figures of the content based on data including the acquired figure information and additional information; and A computer-readable recording medium comprising the step of generating a second image containing the person information when a preset event occurs.

Images