Electronic device and control method therefor

Abstract

This electronic device comprises: a memory storing at least one instruction; a communication device for communicating with an external electronic device; and at least one processor connected to the memory and the communication device and controlling the electronic device, wherein the at least one processor: obtains at least one image of interest from among a plurality of images included in content played by the electronic device on the basis of a first user manipulation input received from the external electronic device; obtains at least one search result corresponding to an object to be searched for from among at least one object included in the at least one image of interest; and obtains a multi-view image including the at least one image of interest, the at least one search result, and the played content.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more specifically, to an electronic device and a method for controlling the same that obtain a search result from content played on the electronic device and obtain a multi-view image including the search result.

[0002] With the advancement of electronic technology, the use of electronic products capable of playing content on a screen while simultaneously providing related information is increasing.

[0003] In particular, recently, electronic products are being developed that can automatically search for objects included in specific scenes of videos, such as movies or dramas, while playing them, and provide the search results.

[0004] An electronic device according to one embodiment of the present disclosure comprises a memory storing at least one instruction, a communication device for communicating with an external electronic device, and at least one processor connected to the memory and the communication device to control the electronic device, wherein the at least one processor acquires at least one image of interest among a plurality of images included in content played by the electronic device based on a first user operation input received from the external electronic device, acquires at least one search result corresponding to an object that is a search target among at least one object included in the at least one image of interest, and acquires a multi-view image including the at least one image of interest, the at least one search result, and the content being played.

[0005] The first user operation input is an input for capturing an image included in the content, and the at least one processor can capture the at least one image of interest among the plurality of images to obtain the at least one image of interest.

[0006] The first user operation input above includes information about at least one object of interest, and the at least one processor can acquire at least one image of interest including the at least one object of interest among the plurality of images based on the information.

[0007] The above memory stores a neural network model trained to recognize objects included in an input image, and the at least one processor inputs the at least one image of interest into the neural network model to obtain at least one object included in the at least one image of interest, and can obtain at least one search result corresponding to one of the at least one objects that is the search target among the at least one obtained object.

[0008] The above at least one processor acquires at least one UI (User interface) including each of the at least one acquired object, and upon receiving a second user operation input for selecting the search target among the at least one object, acquires at least one search result corresponding to the object that is the search target, and can acquire the multi-view image including at least one interest image in which the at least one UI is displayed, the at least one search result, and the content being played.

[0009] When the above at least one processor receives a third user operation input from the external electronic device for selecting one of at least one object included in the at least one image of interest as a search target, it can obtain at least one search result corresponding to the object that is the search target.

[0010] The third user operation input corresponds to a touch input input to the external electronic device to select one of the at least one object as a search target, and the at least one processor, upon receiving the third user operation input, obtains a trajectory corresponding to the touch input and can obtain at least one search result corresponding to at least one object included in the area formed by the trajectory among the at least one image of interest.

[0011] The above at least one processor obtains the above at least one search result classified according to a plurality of information types, and the plurality of information types may include at least one of a shopping list, reviews, and video content.

[0012] The memory stores a neural network model trained to output batch information of a plurality of search results based on personal information regarding a user of the electronic device, and the at least one processor can obtain the at least one search result based on the batch information obtained by inputting the at least one search result and the personal information into the neural network model.

[0013] The electronic device further includes a display, and the at least one processor controls the display to display the image of interest, the at least one search result, and the content being played in each of a plurality of partial areas of the multi-view image during a search mode for searching for at least one object included in the at least one image of interest, and controls the display to display the content being played in the entire area when a fourth user operation input for switching from the search mode to the playback mode is received, and controls the communication device so that at least one of the at least one search result and the at least one image of interest is provided to the external electronic device.

[0014] A control method for an electronic device according to another embodiment of the present disclosure may include the steps of: acquiring at least one image of interest among a plurality of images included in content played by the electronic device based on a first user operation input received from an external electronic device; acquiring at least one search result corresponding to an object that is a search target among at least one object included in the at least one image of interest; and acquiring a multi-view image including the at least one image of interest, the at least one search result, and the content being played.

[0015] The first user operation input is an input for capturing an image included in the content, and the step of acquiring at least one image of interest may include the step of acquiring at least one image of interest among the plurality of images by capturing the at least one image of interest based on the first user operation input for capturing an image included in the content.

[0016] The first user operation input includes information about at least one object of interest, and the step of acquiring the at least one image of interest may include the step of acquiring at least one image of interest among the plurality of images that includes the at least one object of interest, based on the information about at least one object of interest included in the first user operation input.

[0017] The step of obtaining at least one search result includes the step of inputting at least one image of interest into a neural network model to obtain at least one object included in the at least one image of interest, and the step of obtaining at least one search result corresponding to one of the at least one objects obtained that is the search target, and the neural network model may be a model trained to recognize an object included in the input image.

[0018] The step of obtaining at least one search result includes the step of obtaining at least one UI (User interface) including each of the at least one obtained object, and the step of obtaining at least one search result corresponding to the object that is the search target when a second user operation input for selecting the search target among the at least one obtained object is received, and the step of obtaining the multi-view image may include the step of obtaining the multi-view image including at least one interest image in which the at least one UI is displayed, the at least one search result, and the content being played.

[0019] The step of obtaining at least one search result may include receiving a third user operation input from the external electronic device for selecting one of at least one object included in the at least one image of interest as a search target, and obtaining at least one search result corresponding to the object that is the search target.

[0020] The third user operation input corresponds to a touch input input to the external electronic device to select one of the at least one object as a search target, and the step of obtaining the at least one search result may include the step of obtaining a trajectory corresponding to the touch input upon receiving the third user operation input, and the step of obtaining at least one search result corresponding to at least one object included in an area determined by the trajectory among the at least one image of interest.

[0021] The step of obtaining at least one search result includes the step of obtaining at least one search result classified according to a plurality of information types, and the plurality of information types may include at least one of a shopping list, reviews, and video content.

[0022] The step of obtaining at least one search result includes the step of obtaining at least one search result based on the batch information obtained by inputting the at least one search result and personal information into a neural network model, and the neural network model may be a model trained to output batch information of a plurality of search results based on the personal information regarding a user of the electronic device.

[0023] A non-transient computer-readable recording medium storing computer instructions that cause an electronic device to perform an operation when executed by at least one processor of an electronic device according to another embodiment of the present disclosure, wherein the operation comprises: a step of acquiring at least one image of interest among a plurality of images included in content played by the electronic device based on a first user operation input received from an external electronic device; a step of acquiring at least one search result corresponding to one of at least one object included in the at least one image of interest that is a search target; and a step of acquiring a multi-view image including the at least one image of interest, the at least one search result, and the content played.

[0024] The above and other aspects and features of the present disclosure will become more apparent by describing embodiments in detail with reference to the accompanying drawings.

[0025] FIG. 1 is a drawing for explaining the operation of an electronic device and an external electronic device according to one or more embodiments of the present disclosure.

[0026] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one or more embodiments of the present disclosure.

[0027] FIG. 3 is a block diagram illustrating the configuration of an external electronic device according to one or more embodiments of the present disclosure.

[0028] FIG. 4 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure.

[0029] FIG. 5 is a drawing for explaining a multi-view according to one or more embodiments of the present disclosure.

[0030] FIG. 6 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0031] FIG. 7 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0032] FIG. 8 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0033] FIG. 9 is a drawing for illustrating an image of interest including an object of interest according to one or more embodiments of the present disclosure.

[0034] FIG. 10 is a drawing for illustrating selectable objects according to one or more embodiments of the present disclosure.

[0035] FIG. 11 is a drawing for explaining a trajectory according to one or more embodiments of the present disclosure.

[0036] FIG. 12 is a drawing for explaining related content according to one or more embodiments of the present disclosure.

[0037] FIG. 13 is a drawing for explaining related content according to one or more embodiments of the present disclosure.

[0038] FIG. 14 is a drawing for illustrating search results according to one or more embodiments of the present disclosure.

[0039] FIG. 15 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments of the present disclosure.

[0040] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0041] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0042] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0043] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0044] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0045] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0046] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0047] When it is stated that a certain component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the said certain component may be directly connected to the said other component or connected through another component (e.g., a third component).

[0048] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0049] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0050] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” in conjunction with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may refer to a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0051] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0052] Meanwhile, various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present disclosure is not limited by the relative sizes or spacing depicted in the attached drawings.

[0053] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0054] FIG. 1 is a drawing for explaining the operation of an electronic device and an external electronic device according to one or more embodiments of the present disclosure.

[0055] According to FIG. 1, an electronic device (100) is shown as an external electronic device. The electronic device (100) can be connected to an external electronic device (200-1, 200-2, ...) to display a screen.

[0056] Here, the electronic device (100) is a smartphone, tablet PC (Personal computer), desktop PC, laptop PC, PC, set-top box, OTT service (Over-the-top media service) server, console (video game console), Blu-ray player, DVD (Digital Video Disc or Digital Versatile Disc) player, home automation control panel, security control panel, media box (e.g., Samsung HomeSync TM , Apple TV TM , or Google TV TM ), game console (e.g., Xbox) TM PlayStation TM It can be implemented as at least one of ). However, it is not limited to this.

[0057] For example, if the electronic device (100) is implemented as a set-top box, the electronic device (100) can receive content from an external source. The electronic device (100) can provide the received content to an external display device equipped with a display. Here, the electronic device (100) can output a screen of the content received from the external source.

[0058] Here, the electronic device (100) can not only play the content of the electronic device (100), but also obtain information related to the content. For example, information related to the content may include information related to the content itself, such as the content creator, the year of production, and the genre. However, it is not limited to this, and the content information may include linked content, online search results related to the content, etc.

[0059] Here, the search result may correspond to information provided when a search is performed on an object included in a specific scene while using the electronic device (100). For example, the search result may include detailed information about the object, related images, recommendations for similar products, etc.

[0060] Here, the detailed information may include a link to a website for purchasing the object, and the detailed information may include various information obtainable through the website, such as the size and weight of the object.

[0061] Here, similar products may refer to products that are similar in appearance and / or function to the object in question. Alternatively, similar products may refer to items manufactured by the same manufacturer as the object in question. Alternatively, similar products may include items of a size similar to that of the object in question.

[0062] Accordingly, the electronic device (100) can provide the acquired relevant information to an external display device, etc. The external display device can display the information received from the electronic device (100) on a screen.

[0063] When the electronic device (100) is implemented as a display device (e.g., smartphone, tablet, TV, etc.) equipped with a display, it can output an image included in the content through the display. Here, the content may include at least one of text, images, and videos received from an external content source or stored in the electronic device (100). At this time, the electronic device (100) can display the content and related information together through the screen.

[0064] External electronic devices (200-1, 200-2, ...) can communicate with the electronic device (100). External electronic devices (200-1, 200-2, ...) can provide data (e.g., media content) used in the operation of the electronic device (100) or provide signals that can control the operation of the electronic device (100). Here, the external electronic devices (200-1, 200-2, ...) can be implemented as a smartphone, PC, etc., just like the electronic device (100) described above.

[0065] For example, an external electronic device (200-1, 200-2, ...) may provide various control signals to the electronic device (100). For example, the external electronic device (200-1, 200-2, ...) may provide a control signal for capturing a screen of content being played by the electronic device (100). Also and / or alternatively, the external electronic device (200-1, 200-2, ...) may provide a control signal for searching for a captured screen. However, it is not limited thereto, and the external electronic device (200-1, 200-2, ...) may provide a signal for controlling the playback of the content being played (e.g., play, pause, fast forward, etc.) and may provide a control signal for receiving search results obtained by the electronic device (100).

[0066] When an external electronic device (200-1, 200-2, ...) is implemented as a user terminal device (200-1) such as a smartphone, the user terminal device (200-1) can receive a user's touch input (e.g., an input for defining an area (1001) on an image displayed on the screen of the electronic device (100)) through a touch display, etc. The user terminal device (200-1) can provide a control signal corresponding to the user's touch input to the electronic device (100).

[0067] Also and / or alternatively, in addition to the examples described above, external electronic devices (200-1, 200-2, ...) may be implemented as a remote control (200-2), etc., capable of controlling the operation of the electronic device (100). Here, the remote control (200-2) may be a wireless device that enables remote control of the electronic device (100), such as a TV or an audio device. In this case, the remote control (200-2) can operate various functions of the electronic device (100) by transmitting a signal received through a button on the remote control (200-2) to the electronic device (100).

[0068] Here, the remote control (200-2) may include a remote control (200-2) equipped with a gyro sensor. Here, the gyro sensor may also be referred to as a gyroscope sensor. A gyroscope sensor may correspond to a device that detects and measures the rotational motion of an object. When the remote control (200-2) is equipped with a gyro sensor, the remote control (200-2) may detect the rotational movement of the remote control (200-2) and transmit a signal corresponding to the rotational movement to the electronic device (100).

[0069] Even if the external electronic device (200-1, 200-2) is implemented as a user terminal device (200-1) such as a smartphone, it can perform the function of the remote control (200-2) as described above. For example, the operation of the electronic device (100) can be controlled through an application installed on the user terminal device (200-1). Here, the application may include a command to transmit a control signal to the electronic device (100) through the user terminal device (200-1).

[0070] FIG. 1 illustrates that external electronic devices (200-1, 200-2) are implemented in the form of a user terminal device (200-1) such as a smartphone and a remote control (200-2), but are not limited thereto, and external electronic devices (200-1, 200-2, ...) can be implemented in various forms of devices capable of controlling the operation of the electronic device (100).

[0071] As described above, the electronic device (100) can perform various functions by receiving control signals, etc. provided by external electronic devices (200-1, 200-2, ...). The electronic device (100) can acquire information related to the content being played while playing the content. The electronic device (100) can acquire an image to provide the acquired information together while continuously playing the content.

[0072] Here, the video may refer to a multi-view screen where the content playback screen and content-related information are displayed together. Multi-view video refers to video that displays multiple visual contents and / or videos simultaneously on a single screen. Multi-view video can efficiently convey information by providing users with diverse information and / or visual experiences at the same time.

[0073] For example, the electronic device (100) can acquire a multi-view video that can provide not only the content being played but also search results for items appearing in the content. The acquired multi-view video can be provided through a screen equipped in the electronic device (100) and can be provided to an external display device (TV, smartphone, tablet, etc.).

[0074] The specific operation steps for the electronic device (100) to acquire a multi-view image will be explained in detail through FIGS. 2 and FIGS. 4, which will be described later.

[0075] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one or more embodiments of the present disclosure.

[0076] According to FIG. 2, the electronic device (100) may include a memory (110), a communication device (120), and at least one processor (130).

[0077] The memory (110) is electrically connected to at least one processor (130) and can store data necessary for various embodiments of the present disclosure. For example, the memory (110) may be implemented as an internal memory such as ROM (Read-Only Memory) (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM (Random Access Memory) included in the processor (130), or it may be implemented as a memory separate from at least one processor (130).

[0078] Depending on the purpose of data storage, the memory (110) may be implemented in the form of a memory embedded in the electronic device (100) or in the form of a memory that can be attached to and detached from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in a memory that can be attached to and detached from the electronic device (100). When implemented as memory embedded in an electronic device (100), the memory (110) may be at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).

[0079] In the illustrated example, the electronic device (100) is illustrated as including a single memory, but the present disclosure is not limited thereto. When distinguishing between volatile memory and non-volatile memory, the electronic device (100) may be described as including a plurality of memories.

[0080] A memory (110) according to one or more embodiments may store at least one instruction. Here, the at least one instruction may correspond to at least one command for the electronic device (100) to acquire a multi-view image. In addition, the memory (110) may store information related to the operation of the electronic device (100).

[0081] A memory (110) according to one or more embodiments may store at least one neural network model. Here, the neural network model may be a computer system and / or software module for implementing human-level intelligence, and the neural network model enables the machine to learn and make judgments on its own. The recognition rate of the neural network model is characterized by improving with use.

[0082] Neural network models are based on machine learning (e.g., deep learning) technology that uses algorithms to classify / learn features of input data on their own, and / or on elemental technologies that utilize machine learning algorithms to mimic functions such as cognition and judgment of the human brain.

[0083] Here, neural network models may also be referred to as learning models, AI (Artificial intelligence) models, deep learning models, etc.

[0084] The elemental technologies may include, for example, at least one of linguistic understanding technology that recognizes human language / characters, visual understanding technology that recognizes objects like human vision, reasoning / prediction technology that judges information to logically reason and predict, and knowledge representation technology that processes human experience information into knowledge data.

[0085] Neural network models can correspond to Generative AI (Artificial Intelligence) models. Generative AI is an AI system focused on generating new data; it learns patterns from given data to create data that is similar or entirely new. Generative AI can be distinguished from general AI, which aims at data interpretation and pattern recognition.

[0086] For example, a neural network model may correspond to a model trained to take videos included in content as input and output videos of interest suitable for the user. Here, videos of interest suitable for the user may refer to videos that the user is presumed to prefer based on the user's profile information (e.g., age, gender, viewing history, etc.) and viewing characteristics (e.g., preferred genre, main viewing time, etc.).

[0087] For example, the neural network model may correspond to an object recognition model that recognizes objects included in an image. For example, a user may set a specific type of object as an object of interest, and the neural network model may correspond to a model trained to identify and output an image containing such an object of interest. Alternatively, the object recognition model may correspond to a model trained to recognize at least one object included in an image of interest. Here, examples of objects of interest will be described in FIG. 9.

[0088] The neural network model is not limited to the example described above, and the neural network model can be implemented with various models trained to perform the operations necessary for the electronic device (100) to acquire multi-view images.

[0089] The communication device (120) is configured to communicate with various types of external devices according to various types of communication methods. The communication device (120) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0090] Wi-Fi modules and Bluetooth modules can perform communication using Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various information can be transmitted and received.

[0091] The infrared communication module can perform communication according to infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared rays located between visible light and millimeter waves.

[0092] The wireless communication module may include at least one communication chip characterized by performing communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation), in addition to the communication method described above.

[0093] In addition, the communication device (120) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or an UWB (Ultra Wide-Band) module, etc. Such a communication device (120) may also be referred to as a transceiver.

[0094] According to one or more embodiments, the electronic device (100) may communicate with an external electronic device (user terminal device, remote control, etc.) through a communication device (120). For example, the electronic device (100) may receive various types of user operation inputs from an external electronic device. For example, the user operation input may correspond to a signal for acquiring an image of interest among a plurality of images included in the content. Alternatively, the user operation input may correspond to a signal for acquiring at least one search target.

[0095] For example, here the user operation input may correspond to a remote control signal received through the communication device (120). Alternatively, the user operation input may correspond to an operation signal corresponding to a touch input input to an external electronic device.

[0096] The electronic device (100) can provide acquired multi-view images, search results, etc. to an external device through a communication device (120). Here, the external device may be a display device (TV, smartphone, tablet PC, etc.) for displaying multi-view images or search results. However, it is not limited thereto.

[0097] At least one processor (130) can perform overall control operations of the electronic device (100).

[0098] At least one processor (130) may be implemented, for example, as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a time controller (TCON). However, at least one processor (130) is not limited thereto and may include or be defined by one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an ARM processor. Additionally, at least one processor (130) may be implemented as a System on Chip (SoC) or a large-scale integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). Furthermore, at least one processor (130) may perform various functions by executing computer executable instructions stored in memory (110). Although 2 is illustrated as including only one processor (130) in the electronic device (100), the at least one processor (130) may include multiple processors (e.g., CPU + GPU, CPU + DSP).

[0099] According to one or more embodiments, at least one processor (130) can acquire at least one image of interest among a plurality of images included in the content played by the electronic device based on a first user operation input received from an external electronic device (200-1, 200-2, etc.).

[0100] For example, at least one processor (130) can acquire at least one image of interest by capturing at least one image of interest among a plurality of images based on a first user operation input. Here, the first user operation input may correspond to an operation for capturing an image included in the content. This will be described in detail later in FIGS. 6, 7, and 8.

[0101] For example, the first user operation input may correspond to an input for capturing and saving the screen currently being played among the videos being played by the electronic device (100). In this case, the user operation input may correspond to a signal corresponding to a remote control signal provided by an external electronic device or a touch signal input to an external electronic device.

[0102] For example, at least one processor (130) can acquire at least one image of interest containing at least one object of interest among a plurality of images based on information about at least one object of interest included in the first user operation input. This will be explained in detail in FIG. 9.

[0103] According to one or more embodiments, at least one processor (130) can obtain at least one search result corresponding to one of the search targets among at least one object included in at least one image of interest.

[0104] For example, at least one processor (130) can input an image of interest into a neural network model to obtain at least one object included in at least one image of interest. Here, the neural network model may correspond to a neural network model trained to recognize an object included in an input image.

[0105] Afterwards, at least one processor (130) can obtain at least one search result corresponding to one of the at least one objects that are the search target.

[0106] For example, at least one processor (130) may acquire at least one UI (User interface) including each of at least one acquired object, and upon receiving a second user operation input, may acquire at least one search result corresponding to the object to be searched. This will be described in detail later in FIG. 10.

[0107] Here, the second user operation input may correspond to an input for selecting a search target among at least one object. Here, the second user operation input may correspond to a signal provided by an external electronic device, similar to the first user operation input. For example, the second user operation input may correspond to an input received by an external electronic device (e.g., a directional key input or a touch input). Here, the directional key input or the touch input may correspond to each of multiple directions, such as up, down, left, and right. A detailed explanation thereof will be provided later in FIG. 10.

[0108] At least one processor (130) can obtain at least one search result corresponding to an object to be searched when it receives a third user operation input from an external electronic device.

[0109] Here, the third user operation input may correspond to an input for selecting one of at least one object included in at least one image of interest as a search target. For example, it may correspond to an operation input corresponding to a touch input to an external electronic device to select one of the objects as a search target.

[0110] For example, a third user operation input may correspond to a signal corresponding to a touch input for forming a trajectory surrounding an object included in the image of interest. Here, the touch input may correspond to a touch input input by a user to an external electronic device.

[0111] For example, a third user operation input may correspond to a remote control signal for forming a trajectory surrounding an object. That is, if the external electronic device is implemented as a remote control equipped with a gyroscope sensor, a trajectory may be formed based on the tilt and direction of the remote control.

[0112] Subsequently, when at least one processor (130) receives a third user operation input, it can acquire a trajectory corresponding to the touch input and acquire at least one search result corresponding to at least one object included in the area formed by the trajectory among at least one image of interest. This will be described in detail later in FIG. 11.

[0113] According to one or more embodiments, a multi-view image including at least one image of interest, at least one search result, and content being played can be obtained. Here, the multi-view image will be described in detail in FIGS. 4 and FIGS. 5, etc.

[0114] For example, at least one search result can be obtained that is classified according to multiple information types. Here, the multiple information types may include at least one of shopping lists, reviews, and video content. A more specific description of the multiple information types will be provided later in FIG. 14.

[0115] For example, at least one search result can be obtained based on batch information obtained by inputting at least one search result and personal information into a neural network model. Here, the neural network model may correspond to a neural network model trained to output batch information of multiple search results based on personal information about a user.

[0116] Although FIG. 2 illustrates that the electronic device (100) includes only basic components (e.g., memory, processor, communication device), the electronic device (100) may include various additional components in addition to the components described above. For example, the electronic device (100) may include a display.

[0117] The display is configured to display a multi-view image acquired by the electronic device (100), the operating status of the electronic device (100), a notification message, a UI screen, etc. The display can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, a PDP (Plasma Display Panel), etc. The display may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0118] The display can be implemented as a flexible display, a 3D display, or a three-dimensional display. Alternatively, the display may be implemented using only one or more light-emitting elements.

[0119] According to one or more embodiments, at least one processor (130) can display a multi-view image. For example, during a search mode, at least one processor (130) can control the display to display an image of interest, at least one search result, and content being played in each of a plurality of partial regions of the multi-view image.

[0120] Here, the search mode may correspond to an operation mode for searching for at least one object included in at least one image of interest. For example, the electronic device (100) may operate in a search mode according to a user operation input for using the electronic device (100) in a search mode. For example, the user operation input here may correspond to a user input for capturing an image of interest. However, the present disclosure is not limited thereto.

[0121] Afterward, at least one processor (130) can control the display to display the content being played over the entire area when a fourth user operation input is received. Here, the fourth user operation input may correspond to an input for switching from search mode to playback mode.

[0122] For example, the fourth user operation input may correspond to a signal requesting 'Go Back' or 'Cancel'. Here, 'Go Back' or 'Cancel' may be displayed as a UI on an external electronic device, etc. However, it is not limited thereto.

[0123] Subsequently, the communication device (120) can be controlled so that at least one of at least one search result and at least one image of interest is provided to an external electronic device. That is, when the search mode is terminated according to the fourth user operation input, the acquired search result, etc., can be provided to an external electronic device.

[0124] Accordingly, the electronic device (100) can enlarge the previously played content and display it over the entire area of ​​the display. Additionally, the external electronic device can display the search results provided by the electronic device (100) so that the user can view them through the external electronic device. Thus, the user can not only continue the existing viewing experience through the full screen of the electronic device (100), but also have a search experience using the electronic device (100) on an external electronic device (e.g., a user terminal device, etc.).

[0125] FIG. 3 is a block diagram illustrating the configuration of an external electronic device according to one or more embodiments of the present disclosure.

[0126] According to FIG. 3, the external electronic device (200) may include a memory (210), a communication device (220), a display (230), and at least one processor (240).

[0127] The memory (210) can store at least one instruction related to the operation of the external electronic device (200).

[0128] The memory (210), like the memory (110) of the electronic device (100), can be electrically connected to at least one processor (240) and can store data related to various embodiments of the present disclosure. Here, the memory (210) can be implemented in various forms such as ROM, RAM, etc.

[0129] The memory (210) can store at least one instruction for generating a control signal corresponding to a user operation input to an external electronic device (200). The memory (210) can store at least one instruction for providing the generated control signal to the electronic device (100), etc.

[0130] The memory (210) may store at least one instruction for receiving a search result from an electronic device (100) and displaying it on an external electronic device (200). However, the present disclosure is not limited thereto.

[0131] Meanwhile, the communication device (220) can communicate with the electronic device (100) to transmit and receive data.

[0132] The communication device (220) can transmit and receive the aforementioned control signal and search results by communicating with the electronic device (100). Here, the communication device (220) can communicate with the communication device (120) of the electronic device (100) via wired or wireless means according to various methods. For example, the communication device (220) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module.

[0133] The external electronic device (200) can display video data through the display (230). Here, the display (230) may be implemented similarly to the display of the electronic device (100), but the display (230) may be implemented as a touch screen combined with a touch sensor.

[0134] Here, the touch screen can perform the function of receiving touch input from a user along with the function of displaying video data of an external electronic device (200). For example, the external electronic device (200) can display a UI for selecting an object included in the image of interest through the touch screen. At this time, the user can select one object included in the object of interest by touching the UI displayed on the touch screen of the external electronic device (200) with a finger.

[0135] Accordingly, the external electronic device (200) can convert the touch signal into an operation signal for operating the electronic device (100) and provide it to the electronic device (100) through the communication device (220). However, the function of the touch screen is not limited to the example described above, and the external electronic device (200) can display a UI for selecting one of a plurality of directions (up, down, left, right, etc.) through the touch screen, and can also receive a touch operation for selecting one direction.

[0136] The display (230) can display information regarding the operating status of the electronic device (100) and / or the operating status of the external electronic device (200) through the display (230). For example, information regarding the operating status of the external electronic device (200) may correspond to an on / off state, current channel, remaining battery level, etc. Information regarding the operating status of the electronic device (100) may correspond to information regarding the channel currently being broadcast by the electronic device (100), volume, content title and search mode, and / or whether the electronic device (100) is in playback mode, etc.

[0137] For example, if the external electronic device (200) is implemented as a remote control, the display (230) may display information regarding the on / off status of the remote control, the current channel, the remaining battery level, the volume of the electronic device (100), the title of the content, and / or the operating mode of the electronic device (100).

[0138] At least one processor (240) can perform overall control operations of the external electronic device (200).

[0139] At least one processor (240) can be implemented as various types of processors (e.g., digital signal processor, microprocessor, CPU, MCU, MPU, etc.), just like at least one processor (130) provided in the electronic device (100). Also, although FIG. 3 is illustrated as including only one processor in the external electronic device (200), it may include multiple processors in implementation.

[0140] For example, at least one processor (240) can convert a user operation signal input through a display (230), etc., into a control signal for controlling an electronic device (100).

[0141] For example, at least one processor (240) can detect changes in current, pressure, etc., according to a user's touch input and convert them into numerical data. At least one processor (240) can generate a control signal based on the converted data and provide it to an electronic device (100) through a communication device (220).

[0142] The operation of at least one processor (240) is not necessarily limited to this, and the electronic device (100) can perform various control operations to acquire multi-view images.

[0143] In addition to the configuration described above, the external electronic device (200) may further include at least one button for receiving user operation input. For example, the external electronic device (200) may receive various types of user operation input described above through at least one button.

[0144] Additionally, the external electronic device (200) may further include a microphone for receiving user voice input. The microphone receives the user voice and can transmit the received user voice to at least one processor (240). Subsequently, the at least one processor (240) can input the received user voice into a speech recognition model to perform speech recognition. For example, the at least one processor (240) can perform speech recognition on the user voice by performing STT (Speech to Text) on the user voice.

[0145] For example, an external electronic device (200) can receive input through a microphone, such as an operation input for a user to select at least one object or an operation input for acquiring an image of interest. That is, the user can utter a voice such as "I am curious about the second item from the left" or "Take a screenshot now," and the microphone can receive the user's uttered voice and provide it to at least one processor (240).

[0146] Subsequently, at least one processor (240) can perform voice recognition on the provided voice. At least one processor (240) can perform a function suitable for the intent based on the result of the voice recognition (e.g., text, etc.). For example, at least one processor (240) can provide a control signal to the electronic device (100) through the communication device (220). Here, the control signal may correspond to a signal that obtains a search result for the second item from the left among the items displayed on the screen.

[0147] Although an external electronic device (200) including various configurations is illustrated in FIG. 3, some of the illustrated configurations may be omitted. Additionally, other configurations that are not illustrated may also be included.

[0148] Hereinafter, the operation of the electronic device (100) and the external electronic device (200) acquiring a multi-view image by the various configurations described above will be explained in detail.

[0149] FIG. 4 is a drawing for explaining the operation of an electronic device according to one or more embodiments of the present disclosure.

[0150] Referring to FIG. 4, the operation (400) of an electronic device (100) acquiring a multi-view image (431) is shown in each step. The operation of each step may be performed in one processor, and each step may be performed in a different processor (or another device).

[0151] According to one embodiment, the electronic device (100) can acquire an image of interest (411) from playback content (410), select one object (420) included in the image of interest, acquire a search result (430) corresponding to the selected object, and acquire a multi-view image (431-3) including a plurality of partial regions (431-1, 431-2, 431-3).

[0152] Here, the playback content (410) may correspond to video content currently being played by the electronic device (100). For example, the electronic device (100) may play content provided through a TV channel, such as live content, or the electronic device (100) may play content provided through an OTT service. However, the present disclosure is not limited thereto.

[0153] The electronic device (100) can acquire at least one image of interest (411) from the content (410) being played. The content (410) being played may include multiple images. Here, multiple images may refer to multiple frames included in a video. Here, multiple frames or multiple images may each be referred to as scenes, scenes, or screens, but are not limited thereto.

[0154] The electronic device (100) can acquire one of a plurality of images as an image of interest (411). The image of interest (411) may correspond to a scene captured by the user or an image presumed to be of interest to the user. The image presumed here may correspond to an image presumed by a neural network model based on the user's profile information, etc., as described above.

[0155] Here, the operation of capturing a scene of interest to the user may be performed through an external electronic device (200) or through an operation interface (e.g., a button, a microphone, etc.) separately provided in the electronic device (100). However, the present disclosure is not limited thereto. An example of the operation of acquiring the image of interest (411) will be described in detail below in FIG. 6.

[0156] The electronic device (100) can select one object (420) among at least one object included in the acquired image of interest (411). Here, the object (or object) may mean an object included in a specific image. However, the object here may mean not only a specific object included in a specific image, but also the color, pattern, text, etc. of a specific object.

[0157] Here, the electronic device (100) may select one object based on the received user operation input. Alternatively, the electronic device (100) may select an object that is presumed to be of interest to the user even if the user does not perform an operation to select an object through an external electronic device (200), etc. The object presumed to be here may be obtained by a neural network model. Here, the neural network model may correspond to a model trained to obtain an object that is expected to be of interest to the user based on the user's profile information, etc.

[0158] The electronic device (100) can provide a search function, and the selected object (420) may be a search target. That is, the electronic device (100) can make a selected object among various objects included in the image of interest (411) a search target.

[0159] The electronic device (100) can obtain search results for an object to be searched. Here, the search results may correspond to search results containing one or more contents. Here, the search results may include product information, shopping information, reviews, product advertisements, and similar products for the selected object (420).

[0160] Here, the electronic device (100) can obtain information about a search target from an application, website, etc. that provides video or shopping services. The electronic device (100) can classify the obtained information by type of information and obtain search results according to the classification. This will be described in detail later in FIG. 14.

[0161] Information regarding the search target obtained by the electronic device (100) may include product advertisements. The electronic device (100) may obtain search results by receiving advertising content provided by an external advertising platform. Here, the advertising platform may be an online system or server device, etc., in which an advertiser can create and manage advertisements, and distribute and analyze advertisements. An advertising platform may be used to create and distribute various forms of digital advertisements.

[0162] The electronic device (100) can acquire a multi-view image (431) including playback content (410), an image of interest (411), and a search result (430). The entire area of ​​the multi-view image (431) may include a plurality of partial areas (431-1, 431-2, 431-3).

[0163] Here, the electronic device (100) can obtain a multi-view image by placing playback content (410) in a first partial area (431-1), placing an image of interest (411) in a second partial area (431-2), and placing a search result in a third partial area (431-3).

[0164] At this time, the first partial area (431-1) where the playback content (410) is placed may correspond to the area where the video, which is the playback content (410), is displayed. Here, "video is displayed" may mean that multiple images included in the video are displayed sequentially over time.

[0165] In FIG. 4, the position where each partial region (431-1, 431-2, 431-3) is displayed may not represent the display position of each partial region (431-1, 431-2, 431-3) in the actual multi-view image (431). That is, each partial region (431-1, 431-2, 431-3) may be placed at any position within the entire area of ​​the multi-view image (431).

[0166] FIG. 5 is a drawing for explaining a multi-view according to one or more embodiments of the present disclosure.

[0167] According to FIG. 5, an electronic device (100) can acquire a multi-view image (500) including an image of interest (510), a content playback image (520), and a search result (530). Here, the image of interest (510) may include an object (511). Here, the image of interest (510) is depicted as including one object (511), but it may also include various other objects, such as objects, colors, and patterns included in the image of interest (510).

[0168] Here, the content playback video (520) may refer to a screen in which content is being played. For example, the electronic device (100) may sequentially display multiple videos included in the content over time. At this time, the multi-view video (500) may include the video currently displayed by the electronic device (100) as the content playback video (520). Accordingly, in the area of ​​the multi-view video (500) where the content playback video (520) is located, a video may be played and displayed as time passes.

[0169] The multi-view video (500) may include an interest video (510) located at the top left, a search result (530) located at the right, and a content playback video (520) located at the bottom left. However, the present disclosure is not limited thereto, and each video (e.g., interest video, search result, and content playback video) may be placed in various other locations.

[0170] The multi-view video (500) may include various related content related to the currently playing content in addition to the content playback video and search results described above. For example, information about background music (BGM) included in the playing content and related videos related to the playing content may be included in the multi-view video (500). This will be described in detail below in FIG. 12.

[0171] Accordingly, the electronic device (100) can ensure that the viewer's viewing experience is not interrupted by providing a screen through a multi-view video including a screen where content is being played, even while the user is using the search function. Through this, the search experience of the electronic device (100) and a continuous content viewing experience can be integrated and provided.

[0172] From now on, specific operations for the electronic device (100) to acquire an image of interest in order to provide a search function to the user will be described.

[0173] FIG. 6 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0174] Referring to FIG. 6, the user (10) can capture an image of interest (600) through an external electronic device (200).

[0175] Here, the user (10) can input a capture operation to capture the screen of the content, and accordingly, the external electronic device (200) can provide a capture signal to the electronic device (100).

[0176] Here, "capture" may refer to the process of saving a specific screen of content currently being played as an image. In this context, the capture operation may correspond to an action where a user selects a frame of a video currently playing using a specific button or UI and saves it as an image.

[0177] For example, the user (10) can press a screen capture button during video playback to select a desired scene, and the scene can be saved to an electronic device as an image file.

[0178] The electronic device (100) can play content and output a first video (610) and a second video (620). Here, the first video (610) may correspond to the video (610) displayed by the electronic device (100) at the time when the user (10) inputs a capture operation through the external electronic device (200). Here, the second video (620) may correspond to the video displayed by the electronic device (100) at the time when it receives a capture signal through the external electronic device (200).

[0179] At this time, the image that the user (10) intends to capture may correspond to the first image. However, the image acquired by the electronic device (100) according to the user's input may correspond to the second image (620). In this case, the second image (620) may correspond to a different image from the first image (610).

[0180] However, the plurality of objects (refrigerator, person, oven, sofa, etc.) included in the first image (610) and the plurality of objects (refrigerator, person, oven, sofa, etc.) included in the second image (620) may be the same. In this case, even if the second image (620) is different from the first image (610), the electronic device (100) can acquire the second image (620) as an image of interest that corresponds to the user's (10) intention to input a capture operation.

[0181] However, if the multiple objects included in the second image (620) and the multiple objects included in the first image are different, the second image (620) may not be the image that the user (10) intends to capture. This will be described in detail below.

[0182] FIG. 7 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0183] According to FIG. 7, the electronic device (100) can play content and output each of a plurality of images sequentially over time. Accordingly, the electronic device (100) can output a first image (710) and then output a second image (720) according to a change in scene. That is, the first image (710) and the second image (720) may correspond to different images.

[0184] Specifically, the plurality of objects (refrigerator, one person, oven, sofa, etc.) included in the first image (710) and the plurality of objects (refrigerator, dining table, chair, lighting, etc.) included in the second image (720) may be different. In this case, the second image (620) may correspond to an image that does not correspond to the user's (10) intention when inputting the capture operation.

[0185] For example, a user (10) may input a capture operation to search for information about the refrigerator (711) included in the first video (710) of the content being played. An external electronic device (200) that receives the capture operation may provide a corresponding capture signal to the electronic device (100).

[0186] At this time, the electronic device (100) can output a second video (720), which is a changed scene, while playing the content. The electronic device (100) can capture and acquire the second video (720) at the time when a capture signal is input by the electronic device (100). Here, the second video (720) may correspond to a video that does not include the refrigerator (711) that the user wanted to search for.

[0187] Here, the second image (720) may correspond to an image that does not correspond to the user's intention when inputting the capture operation. The operation of the electronic device (100) to resolve this will be explained in detail in the following section.

[0188] FIG. 8 is a drawing for explaining the operation of capturing an image of interest according to one or more embodiments of the present disclosure.

[0189] According to FIG. 8, an electronic device (100) can acquire a multi-view image (800). Here, the multi-view image (800) may include a plurality of captured images (810, 820-1, 820-2). The plurality of captured images (810, 820-1, 820-2) may correspond to a first image (820-1), a second image (810), and a third image (820-2), respectively.

[0190] The electronic device (100) can play content and acquire the aforementioned multiple captured images (810, 820-1, 820-2) based on object changes. Here, the electronic device (100) can recognize objects included in each image of the content through a neural network model. Here, the neural network model may correspond to a model trained to recognize at least one object included in the input image.

[0191] For example, the electronic device (100) can sequentially play multiple videos included in the content over time and acquire a captured video based on changes in objects included in each video. That is, the electronic device (100) can acquire the current video as a captured video when the object included in the current video among the multiple videos is different from the object included in the previous video.

[0192] According to one embodiment, if the current image contains a new object that was not included in the previous image, the current image can be acquired as a captured image. Alternatively, if at least one of the objects included in the previous image is not included in the current image, the current image can be acquired as a captured image.

[0193] The electronic device (100) can obtain a plurality of captured images (810, 820-1, 820-2) by repeating the process described above. Accordingly, the plurality of captured images may correspond to images obtained whenever the scene changes. Here, a change in the scene may mean that the composition of the object included in each image changes as described above.

[0194] The electronic device (100) can acquire a multi-view image (800) including a plurality of captured images (810, 820-1, 820-2) and can select one of the plurality of captured images (810, 820-1, 820-2).

[0195] For example, the user (10) may input an operation to an external electronic device (200) to select one of the multiple captured images (810, 820-1, 820-2). Alternatively, the user (10) may input an operation to sequentially switch and display each of the multiple captured images (810, 820-1, 820-2). Here, the operation may correspond to an operation to select the left, right, up, or down direction.

[0196] For example, a user (10) may input an operation to select a second image (810) containing a refrigerator among a plurality of captured images (810, 820-1, 820-2). An external electronic device (200) provides a selection signal corresponding to the operation input to an electronic device (100), and the electronic device (100) may select and acquire the second image (810) as an image of interest based on the selection signal.

[0197] Accordingly, problems that may arise due to the difference between the time when the user (10) inputs the capture operation and the time when the electronic device (100) acquires the captured image can be resolved. That is, the electronic device (100) acquires and stores multiple captured images (810, 820-1, 820-2) in advance based on scene changes, and allows the user to select one of the multiple images (810, 820-1, 820-2), thereby enabling the user (10) to effectively acquire the image of interest they want.

[0198] FIG. 9 is a drawing for illustrating an image of interest including an object of interest according to one or more embodiments of the present disclosure.

[0199] According to FIG. 9, the electronic device (100) can acquire a plurality of images of interest (910-1, 910-2, 910-3) among a plurality of images (910) included in the content (900). Here, the plurality of images (910) may refer to a plurality of frames included in the video.

[0200] Here, the electronic device (100) can acquire a plurality of images of interest (910-1, 910-2, 910-3) based on information about an object of interest. Here, the information about the object of interest may correspond to information included in user operation input. Here, the user operation input may correspond to input for the user to set the object of interest.

[0201] Here, an object of interest may refer to an object pre-configured by the user. The user can configure the system to automatically recognize and capture the object of interest when it is included in a specific scene. In this context, the object of interest may refer to an object or person appearing in a specific scene. Objects of interest can be configured for the purpose of detecting subjects preferred or of interest to the user in real-time and recording the scene.

[0202] For example, a user can set the refrigerator shown in FIG. 9 as an object of interest (913). The user can input an operation to set the object of interest (913) into an external electronic device (200), and the external electronic device (200) can provide the user operation input to the electronic device (100) to set the refrigerator as an object of interest (913).

[0203] Subsequently, the electronic device (100) can sequentially input a plurality of images (910) included in the content (900) into a neural network model to recognize at least one object included in each of the plurality of images (910). Here, the neural network model may correspond to a neural network model for object recognition as described in FIG. 8.

[0204] Meanwhile, the user can further subdivide the types of items, such as 'two-door refrigerator'. In this case, the electronic device (100) can search for 'refrigerator with two doors' in each scene through a neural network model and capture the current scene if the refrigerator is present in the current scene.

[0205] The electronic device (100) can identify a plurality of images (frames) containing a refrigerator, which is an object of interest (913), among a plurality of images (910) and acquire a plurality of images of interest (910-1, 910-2, 910-3). The user can select one of the plurality of images of interest (910-1, 910-2, 910-3) and select the object of interest included therein as a search target. However, it is not limited thereto, and the electronic device (100) can acquire a multi-view image containing search results for the object of interest (913) without the user inputting a selection operation to select one of the plurality of images of interest.

[0206] Accordingly, the user inputs information about the object of interest into the electronic device (100) in advance, so that the electronic device (100) can automatically capture a scene containing the set object of interest without the user needing to manually input an operation to capture a scene in which the object of interest appears while watching content. Therefore, the user's content viewing experience may not be interrupted, and the process for obtaining search results may be simplified, thereby increasing convenience of use.

[0207] The electronic device (100) can acquire an image of interest in various ways as described above, and can acquire a search result by searching for an object included in the image of interest. This will be explained in detail in the following section.

[0208] FIG. 10 is a drawing for illustrating selectable objects according to one or more embodiments of the present disclosure.

[0209] According to FIG. 10, the electronic device (100) can acquire a multi-view image (1000) including an image of interest (1010), a content playback image (1020), and a search result (1030).

[0210] Here, the image of interest (1010) may include a plurality of UIs (1010-1, 1010-2, 1010-3, 1010-4) each containing an object included in the image of interest (1010). Here, each of the plurality of UIs may correspond to a rectangular shape as shown in FIG. 10, but is not limited thereto and may correspond to various shapes such as a circle or an ellipse.

[0211] Here, a plurality of UIs (1010-1, 1010-2, 1010-3, 1010-4) may each include a first UI (1010-1), a second UI (1010-2), a third UI (1010-3), and a fourth UI (1010-4). Here, each of the first to fourth UIs (1010-1, 1010-2, 1010-3, 1010-4) may include different objects included in the image of interest (1010).

[0212] For example, the electronic device (100) can recognize multiple objects included in an image of interest (1010) through a neural network model for object recognition described above. The electronic device (100) can acquire multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) each containing one of the recognized multiple objects. The multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) acquired here may differ in position and size depending on the position of each recognized object.

[0213] The user can input an operation to select a search target through an external electronic device (200). The external electronic device (200) can provide a second user operation input to the electronic device (100) to select a search target.

[0214] For example, the user can select the sofa located furthest to the right among the objects recognized by the electronic device (100) through a neural network model in the image of interest (1010) as a search target. The user can input an operation to select the sofa through an external electronic device (200), and the external electronic device (200) can provide a second user operation input to the electronic device (100) to select the sofa as a search target.

[0215] Subsequently, the electronic device (100) can select a fourth UI (1010-4). At this time, the selected fourth UI (1010-4) can be displayed separately from the first UI (1010-1), the second UI (1010-2), and the third UI (1010-3). Accordingly, a search result for an object (sofa) corresponding to the fourth UI (1010-4) can be obtained and displayed in a multi-view image.

[0216] In this way, an image of interest (1010) with multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) displayed can be displayed through the display of the electronic device (100). However, it is not limited thereto, and an image of interest (1010) with multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) displayed can be displayed through the display of an external electronic device (200). At this time, the user can select one of the multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) displayed on the external electronic device (200).

[0217] When a user changes and selects another UI among the multiple UIs (1010-1, 1010-2, 1010-3, 1010-4), a search result for an object included in the other selected UI can be obtained. Accordingly, the user can select one of the multiple UIs (1010-1, 1010-2, 1010-3, 1010-4) as a search target and receive the desired search result through the multi-view image (1000).

[0218] The user may select a search target through the UI obtained in advance as described above, or select a search target by selecting an area included in the video of interest. This will be explained in detail in FIG. 11.

[0219] FIG. 11 is a drawing for explaining a trajectory according to one or more embodiments of the present disclosure.

[0220] According to FIG. 11, a first image of interest (1110) and a second image of interest (1120) are shown. Here, the first image of interest (1110) shows a trajectory (1112) containing an object (1112), and the second image of interest (1120) shows a UI (1121) containing a search target (1122).

[0221] The user can input an operation to select a search target among a plurality of objects included in the image of interest (1110) through an external electronic device (200). Here, the operation may correspond to touch input using the user's finger or touch operation using a touch pen. However, it is not limited thereto, and for example, the operation may correspond to an operation using a remote control button.

[0222] Accordingly, the external electronic device can provide a third user operation input corresponding to the user operation to the electronic device (100).

[0223] When an external electronic device is implemented as a remote control equipped with a gyro sensor, a third user operation input may correspond to an input based on the angle, tilt, and / or direction of the remote control.

[0224] The electronic device (100) can acquire a trajectory corresponding to a third user operation input and acquire a first image of interest (1110) containing the trajectory. Here, the trajectory corresponding to the third user operation input may correspond to a shape having the same shape as the trajectory formed on the external electronic device (200) according to the touch input. Here, the shape having the same shape may correspond to a shape that has the same shape as the trajectory formed on the external electronic device but has a different size (or ratio).

[0225] For example, a user can input an operation to select an oven (1112) through an external electronic device (200). Accordingly, the electronic device (100) can acquire a trajectory (1111) from a first image of interest (1110). The electronic device (100) can recognize an oven (1112) included in the trajectory (1111). Here, the electronic device (100) can recognize the oven (1112) through a neural network model for object recognition.

[0226] The trajectory (1112) may be of various forms used to recognize the search target. For example, as illustrated in FIG. 11, the trajectory (1111) may be a closed curve in the first image of interest (1110). In this case, the electronic device (100) can recognize at least one object, such as an oven (1112), within any closed curve.

[0227] The trajectory (1111) may be in the form of a diagonal line, grid line, or curve (a curve that is not a closed curve) marking a specific area of ​​the first image of interest (1110). In this case, the electronic device (100) can recognize at least one object included in the area marked by the diagonal line, grid, or curve.

[0228] When the electronic device (100) identifies that at least one object is included in the trajectory (1111) or in the area occupied by the trajectory (1111), it can obtain a UI (1121) containing said object. Here, the electronic device (100) can obtain a second image of interest (1120) containing the UI (1121). Here, unlike the trajectory (1111) in the first image of interest (1110), the UI (1121) can be implemented as a regular shape.

[0229] For example, the electronic device (100) may acquire a rectangular UI (1121) containing an oven, which is a recognized object. However, it is not limited thereto, and the UI (1121) may be implemented as various regular shapes containing the recognized object. Here, a regular shape may refer to a shape with a mathematically defined regular form, such as a triangle, a square, or a circle.

[0230] Through this, the user can easily recognize that there is a searchable object in the trajectory (1111) formed by user operation through the second interest image (1120).

[0231] The electronic device (100) can identify an object corresponding to a trajectory (1111) of various forms as a search target. The electronic device (100) can obtain a search result corresponding to the search target.

[0232] Afterwards, when the user inputs an operation to form a different trajectory through an external electronic device, the electronic device (100) may obtain a search result corresponding to an object included in the different trajectory.

[0233] The search results obtained in this way may include one or more categories, which will be explained in detail in FIG. 14.

[0234] FIG. 12 is a drawing for explaining related content according to one or more embodiments of the present disclosure.

[0235] Referring to FIG. 12, the multi-view video (1200) may include an image of interest (1210), a content playback video (1220), and a related content video (1230).

[0236] Here, the related content video (1230) may correspond to a video related to the content currently being played.

[0237] For example, when the electronic device (100) plays a refrigerator advertisement video, it can display the advertisement video as a content playback video (1220). At this time, the electronic device (100) can extract the background music of the advertisement video and obtain information about the background music. At this time, the electronic device (100) can extract audio data of the advertisement video being played. The electronic device (100) can obtain background music information by comparing the extracted audio data with audio data stored in a database stored in memory, or by comparing the extracted audio data with a database stored on a server or cloud.

[0238] The electronic device (100) can display information about the acquired music as a related content video (1230). Here, the information about the background music may include the title of the extracted background music, album video, information about the artist, etc. However, the present disclosure is not limited thereto.

[0239] Accordingly, the electronic device (100) can provide the user with related content, such as background music information, from content such as an advertisement video being played. Accordingly, the electronic device (100) can receive various additional content related to the content being played itself, in addition to search results for objects in the current scene. Furthermore, the electronic device (100) can increase user convenience by providing these functions without requiring operation of a separate user terminal device.

[0240] For the above, an example embodiment has been described in which the electronic device (100) acquires related content from the content itself that is being played and acquires a multi-view image (1200) containing information about it. However, the electronic device (100) may acquire content related to an object corresponding to a region based on user operation, such as selecting a portion of the image of interest, and acquire a multi-view image containing the acquired content related to the object.

[0241] FIG. 13 is a drawing for explaining related content according to one or more embodiments of the present disclosure.

[0242] According to FIG. 13, the multi-view video (1300) may include an image of interest (1310), a content playback video (1320), and related content videos (1330, 1340). Here, the image of interest (1310) may include a curved trajectory (1311).

[0243] For example, while the user is watching content displayed in a content playback video (1320), the user may input an operation to form the above trajectory on the video of interest (1310) through an external electronic device (100). Here, the operation may correspond to the touch operation described above. However, the present disclosure is not limited thereto.

[0244] An external electronic device can convert a touch signal into an operation signal for operating the electronic device (100) and provide it to the electronic device (100). Here, the operation signal may correspond to a signal of the same type as the third user operation input described above.

[0245] The electronic device (100) can acquire an image of interest (1310) including a trajectory (1311) based on a control signal. At this time, the electronic device (100) can acquire related content according to the area formed by the trajectory (1311).

[0246] For example, as illustrated in FIG. 11, the image of interest (1310) may include an object such as a robot. When the electronic device (100) forms a trajectory (1311) according to user operation input, the electronic device (100) may recognize that an object such as a robot is included in the area formed by the trajectory (1311). Here, the electronic device (100) may recognize the robot according to the neural network model for object recognition described above.

[0247] Accordingly, the electronic device (100) can identify content related to robots among the content that the electronic device (100) can provide. For example, the electronic device (100) can identify content related to robots among the content that the electronic device (100) can provide. TM PlayStation TM When connected to an external device that provides game content such as ), the game console can identify a game related to robots among the game content that can be provided. The electronic device (100) can acquire a representative icon (Icon) of the recognized game as a related content image (1330).

[0248] Here, the representative icon may correspond to a UI for running a game. The user can select the UI to input operations for running the game through the electronic device (100). Accordingly, the electronic device (100) can run the game through a game console connected to the electronic device (100).

[0249] However, the related content provided by the electronic device (100) is not limited to games, and the electronic device (100) can identify various related content, such as video content and audio content related to the robot recognized using a neural network model.

[0250] Meanwhile, the electronic device (100) can extract background music from the content being played, as in the example described in FIG. 12, and provide information about the background music as a related content video (1340).

[0251] Accordingly, the electronic device (100) can recognize at least one object included in the image of interest (1310), identify content related to the recognized object, and obtain a related content image (1330) related thereto. That is, by selecting one of the objects included in the image of interest (1310), the user can receive not only search results for it but also various other content, such as games playable through the electronic device (100).

[0252] FIG. 14 is a drawing for illustrating search results according to one or more embodiments of the present disclosure.

[0253] According to FIG. 14, the multi-view video (1400) may include an interest video (1410), a content playback video (1420), and a search result video (1430).

[0254] The interest image (1410) may include a search target object (1412) and a UI (1411) including the search target object (1412).

[0255] The search result image (1430) includes a plurality of information types (1431, 1432, 1433), and among the plurality of information types, the shopping list (1431) may include at least one search result (1431-1 to 1431-4).

[0256] Here, the plurality of information types may include at least one of a shopping list (1431), reviews (1432), and video content (1433). Here, the shopping list (1431) may include a plurality of search results (1431-1 to 1431-4) corresponding to each of the plurality of websites.

[0257] For example, the user may input an operation to select an oven, which is one of the objects (1412) included in the image of interest (1410). This operation may be input through an external electronic device. Since the operation of the electronic device (100) selecting one object as a search target has been specifically described in FIG. 11, a redundant description will be omitted.

[0258] The electronic device (100) can obtain search results for a search target. The electronic device (100) can obtain information about the search target as search results from an application, website, etc. that provides video or shopping services. The electronic device (100) can obtain search results by classifying the obtained information by type. Here, the type may correspond to an information type corresponding to each search result.

[0259] According to one embodiment, the electronic device (100) may obtain a shopping list (1431) corresponding to a search target from a website or application. Each of the multiple search results included in the shopping list (1431) may correspond to a link to a website where the product of the search target can be purchased. The user may move to the execution screen of the website or application by performing an operation (such as clicking) to select a link. At this time, each of the multiple search results may include not only a website link but also an advertisement image of the product of the search target.

[0260] The electronic device (100) can obtain reviews (1432) corresponding to the search target from a website or application. Here, each of the multiple search results included in the reviews (1432) may correspond to a link to a website where the reviews can be viewed. Each of the multiple search results included in the reviews (1432) may include experiences or opinions written by consumers after using the product.

[0261] The electronic device (100) can obtain video content (1433) corresponding to the search target from a website or application. Here, each of the multiple search results corresponding to the type of video content (1433) may include a video of the product that is the search result. Here, each of the multiple search results may correspond not only to the video itself but also to a website link for playing the video or a thumbnail image of each video.

[0262] According to FIG. 14, a plurality of information types (1431, 1432, 1433) may include a shopping list (1431), reviews (1432), and video content (1433). However, this is merely an example, and at least one of these information types may be omitted depending on the search results.

[0263] The information types (1431, 1432, 1433) of the search results obtained by the electronic device (100) have been described above as examples, but this is merely an example, and the electronic device (100) may obtain search results that can be classified into various other information types in addition to or alternatively to the aforementioned multiple information types (1431, 1432, 1433).

[0264] In the search result image (1430), each search result can be displayed according to a pre-set order. That is, in the embodiment of FIG. 14, a plurality of information types (1431, 1432, 1433) can be arranged in the order of a shopping list (1431), reviews (1432), and video content (1433). At this time, a plurality of search results (1431-1 to 1431-4) included in the shopping list (1431) can be arranged in one of various orders, such as newest, accurate, or popular.

[0265] Here, the order in which multiple information types (1431, 1432, 1433) are displayed and the order in which each search result (1431-1 to 1431-4) is displayed can be set by the user. However, the present disclosure is not limited thereto, and the electronic device (100) can automatically arrange the order of the search results to obtain a search result image (1430).

[0266] For example, the electronic device (100) can obtain a search result image (1430) based on the user's personal information, etc. At this time, the search result image (1430) may include a plurality of information types and a plurality of search results arranged by the electronic device (100).

[0267] According to one embodiment, the electronic device (100) can acquire a search result image (1430) using a neural network model. Here, the neural network model may correspond to a model trained to output a batch information of multiple search results based on personal information about a user. The electronic device (100) can acquire the batch information by inputting the acquired search results and personal information into the neural network model.

[0268] Here, the user's personal information may include at least one of the user's profile information (age, gender, viewing history, etc.) and viewing characteristics (preferred genre, main viewing time, etc.). That is, the electronic device (100) can input the personal information into a neural network model to obtain placement information that is presumed to be preferred by the user.

[0269] For example, if the user mainly watches shopping content, the electronic device (100) can obtain a search result image (1430) by placing the shopping list (1431) first among a plurality of information types (1431, 1432, 1433), for example, by placing the shopping list (1431) on the far left in the multi-view image (1400) as shown in FIG. 14.

[0270] At this time, if the user has frequently purchased products from a specific website, the shopping content of that website can be placed at the very top of the multi-view video (1400) based on the user's purchase history to obtain a search result video (1430). Alternatively, the electronic device (100) can obtain a search result video (1430) by placing a website with a high usage rate among the user's age group and gender at the very top of the multi-view video (14000).

[0271] Accordingly, even if the user does not set the order of the search results in advance, the electronic device (100) can provide a search result screen that displays search results presumed to be preferred by the user first (or highlights them), thereby providing a search experience that is suitable for the user's personal characteristics and highly satisfying to the user.

[0272] FIG. 15 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments of the present disclosure.

[0273] Based on a first user operation input received from an external electronic device, at least one image of interest among a plurality of images included in the content played by the electronic device can be obtained (S1510).

[0274] According to one or more embodiments, the first user operation input is an input for capturing an image included in the content, and the electronic device (100) can acquire at least one image of interest by capturing at least one image of interest among a plurality of images based on the first user operation input for capturing an image included in the content.

[0275] According to one or more embodiments, the first user operation input includes information about at least one object of interest, and the electronic device (100) can acquire at least one image of interest including at least one object of interest among a plurality of images based on the information about at least one object of interest included in the first user operation input.

[0276] Next, the electronic device (100) can obtain at least one search result corresponding to the object to be searched among at least one object included in at least one image of interest (S1520).

[0277] According to one or more embodiments, the electronic device (100) can input at least one image of interest into a neural network model to obtain at least one object included in at least one image of interest.

[0278] The electronic device (100) can obtain at least one search result corresponding to one of the search targets among at least one acquired object. The neural network model may be a model trained to recognize objects included in the input image.

[0279] Next, the electronic device (100) can acquire a multi-view image including at least one image of interest, at least one search result, and content being played (S1530).

[0280] According to one or more embodiments, the electronic device (100) can obtain at least one search result classified according to a plurality of information types. Here, the plurality of information types may include at least one of a shopping list, reviews, and video content.

[0281] According to one or more embodiments, the electronic device (100) can display an image of interest, at least one search result, and content being played in each of a plurality of partial regions of a multi-view image during a search mode.

[0282] Subsequently, when a fourth user operation input is received, the electronic device (100) can control the display to display the content being played over the entire area. Additionally, the electronic device (100) can provide at least one of at least one search result and at least one image of interest to an external electronic device.

[0283] Through this, the electronic device (100) can ensure that the existing viewing experience is not interrupted by providing a screen in which content is being played while the search function is being provided to the user. Additionally, the electronic device (100) can provide a search experience suitable for the user's personal characteristics by utilizing various neural network models, such as a neural network model for object recognition, and can ensure that the existing search experience is not interrupted by providing search results to the user terminal device even after the search operation is terminated. Accordingly, the electronic device (100) can not only provide a continuous viewing experience and a search experience, but also harmoniously integrate the viewing experience and the search experience.

[0284] In Fig. 15, the order of all steps has been mapped for convenience of explanation, but it goes without saying that the order of steps that are not related to the order or can be performed in parallel is not necessarily limited to that order.

[0285] Methods according to at least some of the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0286] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented solely by software upgrades and / or hardware upgrades for existing electronic devices.

[0287] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0288] According to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium. For example, A 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TMIt can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0289] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100-1)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0290] When the above-described instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0291] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Memory where at least one instruction is stored; A communication device for performing communication with an external electronic device; and It includes at least one processor connected to the memory and the communication device to control the electronic device; The above-mentioned at least one processor is, Based on a first user operation input received from the above external electronic device, at least one image of interest is obtained among a plurality of images included in the content played by the electronic device, and Obtaining at least one search result corresponding to the object to be searched among at least one object included in the above at least one interest image, and An electronic device for acquiring a multi-view image including at least one image of interest, at least one search result, and content being played.

2. In Paragraph 1, The above first user operation input is an input for capturing a video included in the content, and The above-mentioned at least one processor is, An electronic device that captures at least one image of interest among the plurality of images and acquires at least one image of interest.

3. In Paragraph 1, The first user operation input above includes information about at least one object of interest, and The above-mentioned at least one processor is, An electronic device that acquires at least one image of interest including at least one object of interest among the plurality of images based on the above information.

4. In Paragraph 1, The above memory stores a neural network model trained to recognize objects included in an input image, and The above-mentioned at least one processor is, Input the above at least one image of interest into the neural network model to obtain at least one object included in the above at least one image of interest, and An electronic device that obtains at least one search result corresponding to one of the search targets among at least one object obtained above.

5. In Paragraph 4, The above-mentioned at least one processor is, At least one UI (User interface) including each of the at least one object obtained above is obtained, and When a second user operation input for selecting a search target among at least one object is received, at least one search result corresponding to the object that is the search target is obtained, and An electronic device for acquiring a multi-view image including at least one interest image with at least one UI displayed, at least one search result, and content being played.

6. In Paragraph 1, The above-mentioned at least one processor is, An electronic device that, upon receiving a third user operation input for selecting one of at least one object included in at least one image of interest from the above external electronic device as a search target, obtains at least one search result corresponding to the object as a search target.

7. In Paragraph 6, The above third user operation input is, Corresponding to a touch input input to the external electronic device to select one of the above at least one object as a search target, The above-mentioned at least one processor is, When the above third user operation input is received, a trajectory corresponding to the above touch input is obtained, and An electronic device that obtains at least one search result corresponding to at least one object included in the region formed by the trajectory among at least one interest image.

8. In Paragraph 1, The above-mentioned at least one processor is, Obtaining at least one search result classified according to multiple information types, and An electronic device comprising at least one of the above plurality of information types, shopping list, reviews, and video content.

9. In Paragraph 8, The above memory is, A neural network model trained to output arrangement information of multiple search results based on personal information of the user of the electronic device is stored, and The above-mentioned at least one processor is, An electronic device that obtains at least one search result based on batch information obtained by inputting at least one search result and personal information into the above neural network model.

10. In Paragraph 1, It further includes a display; The above-mentioned at least one processor is, During a search mode for searching for at least one object included in at least one interest image, the display is controlled to display the interest image, the at least one search result, and the content being played in each of a plurality of partial regions of the multi-view image. When a fourth user operation input for switching from the above search mode to the playback mode is received, the display is controlled to display the content being played in the entire area, and An electronic device that controls the communication device so that at least one of the above at least one search result and the above at least one image of interest is provided to the external electronic device.

11. In a method for controlling an electronic device, A step of acquiring at least one image of interest among a plurality of images included in the content played by the electronic device based on a first user operation input received from an external electronic device; A step of obtaining at least one search result corresponding to a search target object among at least one object included in the at least one interest image; and A control method comprising the step of acquiring a multi-view image including at least one image of interest, at least one search result, and content being played.

12. In Paragraph 11, The above first user operation input is an input for capturing a video included in the content, and The step of acquiring at least one image of interest above is, A control method comprising: a step of capturing at least one image of interest among the plurality of images based on a first user operation input for capturing an image included in the above content and acquiring the at least one image of interest.

13. In Paragraph 11, The first user operation input above includes information about at least one object of interest, and The step of acquiring at least one image of interest above is, A control method comprising: a step of acquiring at least one image of interest that includes at least one object of interest among the plurality of images, based on information about at least one object of interest included in the first user operation input.

14. In Paragraph 11, The step of obtaining at least one search result mentioned above is, A step of inputting at least one image of interest into a neural network model to obtain at least one object included in the at least one image of interest; and The method includes the step of obtaining at least one search result corresponding to one of the search targets among the at least one object obtained above; and A control method in which the above neural network model is a model trained to recognize objects included in an input image.

15. A non-transient computer-readable recording medium storing computer instructions that cause said electronic device to perform an operation when executed by at least one processor of said electronic device, wherein said operation is, A step of acquiring at least one image of interest among a plurality of images included in the content played by the electronic device based on a first user operation input received from an external electronic device; A step of obtaining at least one search result corresponding to one of the search targets among at least one object included in the at least one interest image; and A non-transient computer-readable recording medium comprising the step of acquiring a multi-view image including at least one image of interest, at least one search result, and content being played.

Images