Electronic device and control method therefor

Abstract

This electronic device comprises: a communication unit; a memory for storing at least one instruction; and one or more processors for executing the at least one instruction. The one or more processors: when an object is identified in a first image acquired through the communication unit, acquire object information and a background image on the basis of the first image; and generate, on the basis of the acquired object information and background image, a second image in which at least one of the position, size, or shape of the object on the background image is changed.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device capable of displaying objects within content by changing them, and a method for controlling the same.

[0002] An electronic device can perform operations such as generating images corresponding to content or displaying images. Such an electronic device can provide the user not only with images corresponding to the content selected by the user, but also with various information related to the content.

[0003] The embodiments of the disclosure may solve at least one of the previously described problems and / or disadvantages and provide the advantages described below. Accordingly, the embodiments of the present disclosure provide an electronic device capable of displaying an object within content by changing it, and a method for controlling the same.

[0004] Additional embodiments will be presented in the detailed description below, some of which are obvious from the detailed description, and other embodiments can also be presented through learning from the presented embodiments.

[0005] An electronic device according to an embodiment of the present disclosure is disclosed. The electronic device comprises a communication unit, a memory for storing at least one instruction, and one or more processors for executing said at least one instruction. When an object is identified in a first image obtained through the communication unit, the one or more processors may obtain object information and a background image based on said first image, and, based on said obtained object information and background image, generate a second image in which at least one of the position, size, and shape of said object is changed on said background image.

[0006] One or more processors may generate a background image corresponding to a second aspect ratio different from the first aspect ratio using the first image corresponding to the first aspect ratio, and generate a second image by compositing the generated background image with an object corresponding to the first aspect ratio.

[0007] The above one or more processors can obtain a background image by upscaling the first aspect ratio of the first image to correspond to a second aspect ratio, obtain an object image of the first aspect ratio within the first image by upscaling it while maintaining the first aspect ratio, and generate the second image by synthesizing the upscaled background image and the object image.

[0008] One or more processors above can acquire a new object image based on the object information and synthesize the background image and the acquired new object image to generate a second image.

[0009] One or more processors above can acquire a new object image based on the country information of the electronic device.

[0010] The above one or more processors can adjust at least one of the saturation and luminance of the new object image based on at least one of the saturation information and luminance information of the background image, and can generate a second image by synthesizing the adjusted new object image and the background image.

[0011] One or more processors can identify the size of an object within the second image and, based on the identified object size, synthesize the object and the background image to generate the second image.

[0012] The above one or more processors can identify the size of the object based on the screen size of the display showing the second image, distance information between the display and the user, and age information of the user.

[0013] One or more processors above can inpaint an object area within the first image to obtain a background image.

[0014] The above object is a subtitle within the video, and the above one or more processors can generate a second video in which at least one of the position, size, and font of the subtitle on the background image is changed.

[0015] A control method for an electronic device according to one embodiment of the present disclosure comprises the steps of: acquiring a first image; when an object is identified within the first image, acquiring object information and a background image based on the first image; and generating a second image in which at least one of the position, size, and shape of the object on the background image is changed based on the acquired object information and background image.

[0016] The step of acquiring the object information and background image may generate a background image corresponding to a second aspect ratio different from the first aspect ratio using the first image corresponding to the first aspect ratio, and the step of generating the second image may generate a second image by compositing the generated background image with an object corresponding to the first aspect ratio.

[0017] The step of acquiring the object information and background image may involve upscaling the first aspect ratio of the first image to correspond to the second aspect ratio to acquire the background image, and upscaling the object image corresponding to the first aspect ratio within the first image while maintaining the first aspect ratio to acquire the object image, and the step of generating the second image may involve synthesizing the upscaled background image and object image to generate the second image.

[0018] The step of acquiring the object information and background image may acquire a new object image based on the object information, and the step of generating the second image may generate the second image by synthesizing the background image and the acquired new object image.

[0019] The step of generating the second image may involve adjusting at least one of the saturation and luminance of the new object image based on at least one of the saturation information and luminance information of the background image, and compositing the adjusted new object image with the background image to generate the second image.

[0020] The step of generating the second image may determine the size of an object within the second image and generate the second image by compositing the object and the background image based on the determined object size.

[0021] The step of generating the second image can identify the size of the object based on the screen size of the display showing the second image, distance information between the display and the user, and age information of the user.

[0022] The step of acquiring the object information and background image can be performed by inpainting the object area within the first image to acquire the background image.

[0023] The above object is a subtitle within the video, and the step of generating the second video may generate a second video in which at least one of the position, size, and font of the subtitle on the background image is changed.

[0024] A non-transient computer-readable recording medium storing a program for executing a control method for an electronic device according to one embodiment of the present disclosure, wherein the control method comprises the steps of: acquiring a first image; acquiring object information and a background image based on the first image when an object is identified within the first image; and generating a second image in which at least one of the position, size, and shape of the object on the background image is changed based on the acquired object information and background image.

[0025] The above-described or other aspects, features, and benefits of embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings. In the accompanying drawings:

[0026] FIG. 1 is a drawing for explaining an object change operation according to one embodiment of the present disclosure,

[0027] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0028] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0029] FIG. 4 is a drawing for explaining the operation of changing the aspect ratio of content according to one embodiment of the present disclosure,

[0030] FIG. 5 is a drawing for explaining an object change operation according to an embodiment of the present disclosure,

[0031] FIG. 6 is a drawing for explaining decomposition and synthesis operations according to one embodiment of the present disclosure,

[0032] FIG. 7 is a drawing for explaining an operation to determine whether a logo change is necessary according to an embodiment of the present disclosure,

[0033] FIG. 8 is a drawing for explaining an operation to determine the size of an object according to one embodiment of the present disclosure,

[0034] FIG. 9 is a drawing for illustrating an example of changing the size of subtitles according to one embodiment of the present disclosure, and,

[0035] FIG. 10 is a flowchart illustrating the control operation of an electronic device according to one embodiment of the present disclosure.

[0036] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0037] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0038] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0039] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0040] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components, etc.) and do not exclude the presence of additional features.

[0041] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0042] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0043] Where it is stated that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., a third component).

[0044] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0045] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0046] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing the said operation (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing the said operation by executing one or more software programs stored in a memory device.

[0047] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0048] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0049] Meanwhile, various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0050] Meanwhile, an electronic device according to various embodiments of the present disclosure may include, for example, at least one of a terminal device, a tablet PC, a desktop PC, a laptop PC, a server, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit.

[0051] In some embodiments, the electronic device is, for example, a television, a DVD (digital video disk) player, audio, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync). TM , Apple TV TM, or Google TV TM ), game console (e.g., Xbox) TM PlayStation TM It may include at least one of an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame. Meanwhile, among the electronic devices described above, a device equipped with a display may be referred to as a display device. Meanwhile, even if the electronic device of the present disclosure does not have a display, it may be a set-top box or a PC that provides images to a display device.

[0052] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0053] FIG. 1 is a drawing for explaining an object change operation according to one embodiment of the present disclosure.

[0054] Referring to FIG. 1, when an electronic device (100) receives a first image (10), it can generate and display a second image (101) in which at least one of the position, size, and shape of an object (11) in the received first image is changed. Meanwhile, in the illustrated example, for ease of explanation, it is assumed that the electronic device (100) directly displays the image, but in implementation, the electronic device may not have a display and may be a device that outputs the image to another device.

[0055] Here, as illustrated, the first image (10) is an image corresponding to the content, and an object (11) may be positioned on one side of the image. Here, the object may be a logo, subtitle, etc., inserted into the captured image through image processing such as editing. However, when implementing, the operation described below may also be applied to a logo within the image (i.e., a logo existing in the actual environment, rather than an image added through editing).

[0056] And the logo may be a manufacturer logo, a broadcaster logo, a content creator logo, an emblem such as a national flag, a sports team emblem, an association logo, etc., and the subtitle may be provided as subtitle data and displayed, or it may be an image-based subtitle added during the broadcaster's video editing process.

[0057] Furthermore, an object can be anything added to a video for a specific purpose, such as a logo, subtitle, or watermark. Such objects may be referred to by various names, including video products, image objects, keyword images, company images, and manufacturer images.

[0058] Among such objects, logos are generally placed within the video in a size that does not interfere with the user's viewing. However, if the image is small or has poor resolution, its proportion within the video may be relatively large to ensure the logo's identification.

[0059] Conversely, as video resolution has recently increased and display screens have grown larger, the relative proportion of objects such as logos within the video is decreasing.

[0060] However, when displaying previously created video on a modern display, the video is displayed enlarged as is, which can reduce the user's immersion due to the enlarged logo. Additionally, when displaying past content with a 4:3 aspect ratio on a display with a 16:9 aspect ratio, if the screen changes to match the aforementioned ratio, the logo within the video also changes to match that ratio, causing the shape of the displayed logo to become distorted.

[0061] Furthermore, while the logo may change into a different form over time, when displaying an image featuring a past logo, there was no choice but to display the existing logo shape as is.

[0062] To solve such problems, the electronic device (100) according to the present disclosure can identify objects such as logos and subtitles within an image, and when an object is identified, generate a second image (101) in which the position, size, and shape of the object are changed. The specific configuration and operation of the electronic device (100) will be described later with reference to FIG. 2.

[0063] As described above, the electronic device according to the present disclosure can change objects within an image, thereby preventing user immersion from being disrupted by the objects. Additionally, if there is a change to a logo, it is possible to display a new logo.

[0064]

[0065] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0066] Referring to FIG. 2, the electronic device (100) may include a communication unit (110), a memory (120), and a processor (130).

[0067] The communication unit (110) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication unit (110) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module, etc. Here, each communication module may include at least one hardware chip or hardware circuit.

[0068] Wi-Fi modules and Bluetooth modules can perform communication via Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various information can be transmitted and received.

[0069] The infrared communication module performs communication according to infrared communication (IrDA, Infrared Data Association) technology, which uses infrared rays located between visible light and millimeter waves to wirelessly transmit data over short distances.

[0070] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0071] In addition, the communication unit (110) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or a UWB (Ultra Wide-Band) module.

[0072] According to one example, the communication unit (110) may use the same communication module (e.g., Wi-Fi module) to communicate with external devices such as a remote control and an external server.

[0073] According to other examples, the communication unit (110) may use different communication modules (e.g., Wi-Fi modules) to communicate with external devices such as a remote control and external servers. For example, the communication unit (110) may use at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may use a BT module to communicate with an external device such as a remote control. However, this is merely one embodiment, and the communication unit (110) may use at least one of various communication modules when communicating with multiple external devices or external servers.

[0074] The communication unit (110) can receive content. This content can be diverse, such as movies, music videos, dramas, short videos, etc. And while the content is assumed to be video, it may also be an image or referred to as video.

[0075] The communication unit (110) can obtain object information. For example, it can obtain a new object image and image information corresponding to the new object from an external device.

[0076] And the communication unit (110) can receive not only content but also information necessary for providing various applications and services of the electronic device (100) from an external device.

[0077] The memory (120) may be implemented as internal memory such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM included in the processor (130), or as memory separate from the processor (130). In this case, the memory (120) may be implemented in the form of memory embedded in the electronic device (100) or in the form of memory that can be attached to and detached from the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in memory that can be attached to and detached from the electronic device (100).

[0078] The memory (120) can store object images, background images, etc. generated during the process described below, and can also store new object images obtained from an external device.

[0079] And the memory (120) can store various contents (e.g., broadcast content, applications, etc.) received through the communication unit (110) described above.

[0080] Meanwhile, the memory embedded in the electronic device (100) is implemented as at least one of volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD), and the memory that can be attached to and detached from the electronic device (100) can be implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory that can be connected to a USB port (e.g., USB memory).

[0081] Meanwhile, although the illustrated example shows the electronic device (100) being composed of a single memory, when distinguishing between volatile memory and non-volatile memory, the electronic device (100) may be described as including multiple memories.

[0082] The processor (130) can perform overall control operations of the electronic device (100). Specifically, the processor (130) functions to control the overall operation of the electronic device (100).

[0083] The processor (130) may be implemented as a digital signal processor (DSP), microprocessor, or time controller (TCON) that processes digital signals. However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), microcontroller unit (MCU), microprocessing unit (MPU), controller, application processor (AP), graphics-processing unit (GPU), communication processor (CP), or ARM processor. Additionally, the processor (130) may be implemented as a System on Chip (SoC) or large-scale integration (LSI) with built-in processing algorithms, or may be implemented in the form of a Field Programmable Gate Array (FPGA). Furthermore, the processor (130) can perform various functions by executing computer executable instructions stored in memory. Meanwhile, although FIG. 2 illustrates that the electronic device (100) includes only one processor, in implementation, multiple processors may be included. It may also include processors (e.g., CPU + GPU, CPU + DSP).

[0084] The processor (130) acquires content using the communication unit (110). For example, when the processor (130) receives a playback command for specific content from a user, it can acquire the content by controlling the communication unit (110) to receive the content. Such content may be a video such as VOD, but may also be a real-time streaming video provided by a specific server, or broadcast content transmitted by a broadcasting station.

[0085] The processor (130) generates a first image corresponding to the acquired content. For example, if the content is real-time streaming, it can construct a screen using video data from the received streaming data and generate a first image using the screen. Such content may be a video, game content, a photo, etc.

[0086] The processor (130) can determine whether there is an object in the image. Specifically, the processor (130) can perform image analysis on the image to determine whether an object, such as a logo or subtitle, is located in the image.

[0087] At this time, the processor (130) may use a pre-trained model that identifies objects. This learning model may be a model capable of receiving an image as input and outputting coordinate information, size information, and class of the object. Here, the coordinate information is the coordinates of the object's center point, and the size information includes information regarding the width (or number of horizontal pixels) and the height (or number of vertical pixels). The class is an index indicating what kind of logo, etc., the object is.

[0088] And the memory (120) can store index information for the new logo and index information for the past logo. In this case, if the processor (130) confirms that the confirmed index is a past logo or that there is a new index corresponding to the logo, it can control the communication unit (110) to acquire a new logo image from the outside.

[0089] When an object is identified within the first image, the processor (130) can determine whether a change is necessary for the object. For example, the processor (130) can determine whether the object in the image is a past object, whether the size of the object is excessive or, conversely, whether the size of the object is too small, and whether the brightness, saturation, etc. of the object are too different when compared to the background image. Specific details of the checks will be described later with reference to FIG. 7.

[0090] When an object is identified within the first image, the processor (130) can obtain object information and a background image based on the first image. For example, the processor (130) can generate a background image corresponding to a second aspect ratio different from the first aspect ratio using the first image corresponding to the first aspect ratio.

[0091] At this time, if it is necessary to perform upscaling processing on the first image described above, the processor (130) can obtain a background image by upscaling the first aspect ratio of the first image to correspond to the second aspect ratio. At this time, the processor (130) can obtain a background image by inpainting an object area within the first image when generating the background image.

[0092] Here, upscaling is a technology that increases the resolution of photos or videos by inserting new pixels between existing ones. For example, this could involve converting SD quality video into HD quality video. This upscaling can be performed by taking into account the aspect ratio of the input video and the aspect ratio of the display.

[0093] And the processor (130) can perform upscaling on the object as well, while maintaining the first aspect ratio.

[0094] If a change in the shape of an object is required, that is, if the image in the first image is a past logo or if the processor (130) is a logo that is not currently used in the country, the processor (130) may obtain a logo image currently used in the country from an external device. In such cases where a change in the logo is required, the processor (130) may have previously obtained and stored a new logo image, etc., from an external device.

[0095] The processor (130) can generate a second image in which at least one of the position, size, and shape of an object on a background image is changed based on the acquired object information and background image. For example, the processor (130) can generate a second image by compositing the generated background image with an object corresponding to a first aspect ratio.

[0096] At this time, the processor (130) can, in the process of synthesizing the background image and the object, adjust at least one of the saturation and luminance of the new object image based on at least one of the saturation information and luminance information of the background image, and synthesize the adjusted new object image and the background image to generate a second image.

[0097] Additionally, the processor (130) may determine the size of an object within the second image and generate the second image by compositing the object and the background image based on the determined object size. For example, the processor (130) may determine the object size within the image based on the screen size of the display showing the second image, distance information between the display and the user, and age information of the user.

[0098] Meanwhile, although the above-described operation was explained assuming it is a logo, it may be performed on subtitles during implementation. In this case, the processor (130) can change the size or position of the subtitles, as well as change the font. Additionally, the processor (130) can perform OCR on the subtitles and filter the content of the subtitles. For example, regarding subtitles related to racial slurs, sexism, profanity, etc., the processor (130) may perform processing such as deleting problematic text or blurring the relevant parts of the subtitles.

[0099] As described above, the electronic device according to the present disclosure displays objects such as logos and subtitles within an image by adjusting them to a size suitable for the current screen, thereby allowing for more immersive viewing of the content.

[0100] Meanwhile, although only a simple configuration constituting the electronic device (100) has been illustrated and described above, various additional configurations may be provided during implementation. This will be explained below with reference to FIG. 3.

[0101]

[0102] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0103] Referring to FIG. 3, the electronic device (100') may include a communication unit (110), memory (120), processor (130), input / output interface (140), microphone (150), display (160), and speaker (170).

[0104] The configuration of the communication unit (110), memory (120), and processor (130) was previously described in FIG. 2, and only the operation different from FIG. 2 will be described below.

[0105] The input / output interface (140) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface).

[0106] The input / output interface (140) can input and output at least one of audio and video signals. Depending on the implementation example, the input / output interface (260) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0107] And the input / output interface (140) can provide a video signal corresponding to a screen generated by the electronic device (100') or an audio signal together with the video signal to an external device (e.g., a display device, an STB, etc.). The screen generated here may be the second image described above.

[0108] The microphone (150) can receive the user's voice when active. For example, the microphone (150) may be formed integrally on the upper side, front side, or side side of the electronic device (100').

[0109] The microphone (150) may include various configurations such as a microphone that collects user voice in an analog form, an amplifier circuit that amplifies the collected user voice, an A / D conversion circuit that samples the amplified user voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0110] When a user's voice is input through such a microphone (150), the processor (130) can check the content of the user's voice and perform an action corresponding to the content of the voice. For example, the content of the voice may be a playback command for specific content.

[0111] Additionally, during implementation, a command to move an object (or change its size) may be received from the user, and the movement command (or change its size) selected by the user may be executed. For example, a user who finds it inconvenient to have the logo on the right side may give a voice command to change the position of the logo, and the processor (130) may change the logo from the top right to the top left. Alternatively, subtitles may be displayed on the video, but if the subtitles are small, the user may give a voice command to enlarge the size of the subtitles. In response to such voice commands, the processor (130) may enlarge and display the subtitles within the video.

[0112] Meanwhile, although it has been described above that user voice is input through the microphone (150), the microphone may be provided in a remote control for controlling the electronic device (100'), and user voice input through the microphone provided in the remote control may be input to the electronic device (100') and processed through the communication unit (110) described above.

[0113] The electronic device (100') can operate not only based on the configuration or remote control provided in the electronic device (100'), but also according to the control command of the terminal device. For example, if the electronic device is a TV or a set-top box, recently, manufacturers provide applications for controlling the TV or set-top box. Such applications can provide a function that allows the terminal device to be used as a remote control for the electronic device.

[0114] Accordingly, when a user executes an application to control a TV or set-top box using a terminal device and inputs a voice command through the terminal device, the electronic device (100') can perform a voice recognition operation and a corresponding voice recognition result using the voice signal input through the terminal device.

[0115] The display (160) can be implemented as various types of displays such as LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diodes) display, PDP (Plasma Display Panel), Micro LED, etc. The display (160) may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as a-si TFT, LTPS (low temperature poly silicon) TFT, OTFT (organic TFT), etc. Meanwhile, the display (160) can be implemented as a touchscreen combined with a touch sensor, a flexible display, a 3D display, etc.

[0116] The display (160) can display various images. For example, the display (160) can display a first image or a second image generated by the processor (130).

[0117] The speaker (170) can output sound. Specifically, the speaker (170) may be a component that outputs various audio data processed at the input / output interface, as well as various notification sounds or voice messages. The speaker (170) may also output result information corresponding to the voice recognition operation described later.

[0118] Meanwhile, although the electronic device (100) in FIG. 3 is illustrated and described as including a display (160), if the electronic device (100') is a device such as a set-top box that does not include a display, the display configuration may be omitted. Also, depending on the implementation form, the speaker and microphone described above may also be omitted. Additionally, although not illustrated in FIG. 3, other components (e.g., a camera) may be included.

[0119]

[0120] FIG. 4 is a drawing for explaining the operation of changing the aspect ratio of content according to one embodiment of the present disclosure.

[0121] Referring to FIG. 4, first to third images (10, 20, 410) are shown. Here, the first image (10) is an original image and has a 4:3 aspect ratio. This first image may be referred to as an input image, input content, etc.

[0122] And the second image (20) and the third image (410) are images that have been changed to a 16:9 ratio using the first image (10).

[0123] Specifically, when a user plays content (10) created in a 4:3 ratio on a display having a 16:9 ratio, the electronic device (100) can display it according to the content ratio or change it to match the ratio of the display.

[0124] For example, when changing to match the aspect ratio of the display, the electronic device (100) can change the resolution to correspond to the aspect ratio of the display and display it.

[0125] If a logo (11) is positioned at the top right of a video (10) having a 4:3 aspect ratio, then a logo (21) is also positioned at the top right of a second image (20) having a 16:9 aspect ratio. However, this change in aspect ratio applies to the logo as well as the image. Consequently, the shape (or aesthetics) of the logo within the video may be compromised, so it is preferable not to change the logo to a 16:9 aspect ratio.

[0126] Additionally, the logo image is inserted relatively large for logo identification on small screens, but on large screens or high resolutions, there is no need to insert the logo image as large as before. However, when the image is enlarged, the logo is also enlarged at the same ratio, so an unnecessarily large logo (21) is displayed.

[0127] Accordingly, the electronic device of the present disclosure distinguishes between a background image and a logo, performs scaling or scaling up on the background image as in the conventional manner, and performs image processing to distinguish the logo so that its ratio, etc., is not changed and it does not have an excessive size within the image. Accordingly, when the logo object (411) in the new image (410) is compared with the object logo (11) in the first image (10), the ratio and size, etc., are different. A specific method of change is described below with reference to FIGS. 4 and 5.

[0128] Meanwhile, although FIG. 4 illustrates an example in which only the aspect ratio (and / or size) of the existing logo is varied, the shape of the logo may be changed during implementation, and the display position of the logo (specifically, the center position of the logo) may be changed.

[0129] In addition, although the company logo was used as the subject in Fig. 4, it can be applied not only to the logo but also to subtitles, flags, emblems, national flags, etc. during implementation.

[0130]

[0131] FIG. 5 is a drawing for explaining an object change operation according to one embodiment of the present disclosure.

[0132] Referring to FIG. 5, it may include a video encoder (530), a data encoder (540), a detection unit (560), and a video decoder (570).

[0133] First, the video encoder (530) receives an input image (510) in frame units and a mask (520) for detecting objects within the image. Accordingly, the video encoder (530) receives the input image (510) and the mask (520) and can output an image block corresponding to a specific area within the image.

[0134] In the illustration, the input image (510) is shown as being input to the video encoder (530) with the mask (520) merged, but in the implementation, it may be input to the video encoder (530) individually and used.

[0135] The data encoder (540) can receive information such as logo information, device size, viewing time, logo and subtitle content, viewing environment, and viewer characteristics, and output corresponding vector information. That is, the data encoder (540) can receive various information related to object changes and output a latent vector corresponding to the information. Such latent vectors are pairs of independent latent variables and can influence the determination of the size (and / or location) of an object within the video to be displayed.

[0136] In this case, the aforementioned information can be used by simply tokenizing it or by converting it into latent vectors. For example, viewing situations that can be considered may include logo information, device size, viewing time, content of the logo and subtitles, viewing environment, and viewer characteristics.

[0137] The image block of the video encoder and the vector information of the data encoder (540) are combined and input to the detection unit (560). In the illustrated example, the two data described above are not combined and input to the detection unit (560), but can be input to the detection unit (560) individually.

[0138] The detection unit (560) can detect objects within the image. Such a detection unit (560) can operate using a learning model. Here, the learning model can detect objects within the image or output the size, location, and class of the detected objects. If no object is detected, the video decoder (570) described later may not perform a separate merging operation.

[0139] Such a learning model may be a model that considers detected and modified logo information, device size, viewing time, content of logos and subtitles, viewing environment, and viewer characteristics during the training of the inpainting and generative models.

[0140] Meanwhile, such object detection operations can be performed on a frame-by-frame basis or in cycles of several frames, and common information can be utilized within those cycles. That is, for example, if a logo is identified in the first frame, operations to display a new logo at the same shape and location or to display a logo at a changed size can be performed in frames 1 through 10.

[0141] Meanwhile, such behavior may vary depending on the type of object. For example, if the object is a logo, the presence or change of the logo can be checked at relatively long intervals. Conversely, if it is a subtitle, the presence or other details of the subtitle may be checked more frequently.

[0142] The video decoder (570) can change at least one of the shape, position, and size of an object in the image based on the output information of the detection unit (560). Meanwhile, although the video decoder (570) has been depicted above as performing the change and synthesis simultaneously, the object change and synthesis operations may be performed separately in different configurations.

[0143]

[0144] FIG. 6 is a drawing for explaining decomposition and synthesis operations according to one embodiment of the present disclosure.

[0145] Referring to FIG. 6, the first to third images (610, 620, 630, 640) are shown.

[0146] First, the first image (610) is an image corresponding to the input content, and a logo of a specific manufacturer may be located on the right side of the image.

[0147] When such a first image is input, an operation to check whether an object is located within the image can be performed. For example, operations such as segmentation and detection may be used for this operation.

[0148] These input images may be a single frame image from the video, a single image from a sequence of consecutive frames, or a single image randomly selected from a sequence of frames. Additionally, the images within the video may be square or various geometric shapes (lateral, circular, segmentation map, etc.).

[0149] In this way, when an object exists, the electronic device (100) can identify that the object is located within the image (610). Specifically, the electronic device (100) is equipped with a network for object identification and can input an input image into the network. This network is a deep learning model that performs object identification. This deep learning model may be referred to as a learning model, an AI model, a network model, etc.

[0150] When inputting more than the input to the network in this way, the electronic device (100) can check the center position of the object, its size information, and class information indicating what kind of object it is.

[0151] When information related to such an object is identified, the electronic device (100) can acquire a background image (630) and an object image (620) based on the identified information.

[0152] First, the electronic device (100) can acquire an object image within a first image based on identified location information and size information. Prior to this, the electronic device (100) can determine whether a logo within the image is a past object based on acquired class information, and if it is a past object, it can acquire a new object. For example, in the illustrated example, an image with text inside an ellipse is a past object, and an image containing only text is a new object. In this case, if an elliptical logo is acquired, the electronic device (100) can acquire a logo image corresponding to a new object without an ellipse.

[0153] Meanwhile, in the illustrated example, there is a certain correlation, but at the time of implementation, there may be no correlation between the two logos depending on the company's policy and the country's policy. For example, if the content is displayed in Japan, the "SAMSUNG" logo may be changed to a logo such as "Galaxy".

[0154] And the electronic device (100) can obtain a background image in which an object has been deleted using the first image. For example, the electronic device (100) can delete pixel information corresponding to an object within the first image (610) and fill the area where the pixel information has been deleted using surrounding pixel information. At this time, the electronic device (100) may use not only the image but also images before and after, such as information from the previous frame.

[0155] In this process, the electronic device (100) can generate a background image using an inpainting method or a generative AI. Here, the inpainting method is a method of partially restoring an image that is damaged to the extent that it is difficult for a person to notice. The generative AI is a deep learning method and can generate a background image by utilizing a more pre-trained model to consider various objects and elements such as the background within the image.

[0156] When an object and a background image are acquired in this manner, the electronic device (100) can determine the position and size of the object within the second image. This operation is described in detail in FIG. 8.

[0157] And the electronic device (100) can generate a second image by positioning an object on a background image based on a determined position and size.

[0158] Although FIG. 6 illustrates an example of changing to a logo different from the logo in the first image, it is also possible to maintain the shape of the logo during implementation, as shown in FIG. 5. In addition, the above-described operation is also possible for subtitles other than logos. This will be described later in FIG. 9.

[0159] Meanwhile, in FIG. 6, the operation of generating object information and a background image and the operation of synthesizing the two separately generated pieces of information can be performed in a single device (or configuration), or can be performed in two separate devices (or three devices).

[0160]

[0161] FIG. 7 is a drawing for explaining the operation of checking whether a logo change is necessary according to one embodiment of the present disclosure.

[0162] Referring to FIG. 7, it may first be confirmed that an object exists in the input image. In this case, as illustrated, a process of determining whether a change in the shape of the object is necessary and a process of determining whether a change in the size of the object is necessary may be performed.

[0163] First, it can be determined whether the aspect ratio of the input image is the same as the aspect ratio to be displayed on the display (710). For example, the aspect ratio here may be the ratio of the width and height of the image. For example, if the size ratio of the image is W:H and the size ratio of the device is W_d:H_d, the ratios of the two images can be checked, and if they are the same, the first flag (Scale_flag_1) is not set, and if they are different, the first flag is set.

[0164] Here, the first flag indicates whether to use the first scale value for changing the size of the detected object according to the size ratio of the image.

[0165] And it can be verified whether the size of the object is appropriate relative to the viewing distance (720). For example, it is verified whether the size of the object is suitable for the distance between the viewer and the device (viewing distance). At this time, the screen size of the display device and the distance between the display device and the user (or viewer) can be verified first. For example, a camera equipped in the electronic device can be used, or it can be obtained through the analysis of video captured by an external device.

[0166] If the distance between the device and the viewer is D, and the appropriate viewing distance based on the device size is denoted as D_T, then examples of those values ​​are as follows.

[0167] TV Size Viewing Distance 138 cm 1.7 m 163 cm 2.0 m 189 cm 2.3 m 214 cm 2.6 m 247 cm 3.0 m

[0168] Referring to the table above, if the viewing distance corresponding to the size of the display (hereinafter referred to as the optimal viewing distance) is the same as the actual viewing distance, the second flag (Scale_flag_2) may not be set. On the other hand, if there is a difference between the optimal viewing distance and the actual viewing distance—specifically, if the difference between the two viewing distances is greater than a certain ratio—the second flag may be set. In this process, the second scale may be calculated in advance or may be calculated during the application process described later. Here, the second flag indicates whether to use the second scale for changing the size of the detected object according to the viewer's viewing distance.

[0169] And, it can be checked whether the size of the object is appropriate for the viewer's age (730). For example, first, user age recognition can be performed using a video of the user, or the user's age can be obtained based on the user's login information. And based on the obtained user age, if the user is an adult (e.g., 19 to 65 years old), the third flag may not be set. On the other hand, if the user is 65 years old or older, or 19 years old or younger, the third flag may be set.

[0170] Here, the third flag indicates whether to use a third scale value to change the size of the object according to the viewer's age group.

[0171] Meanwhile, although the illustrated example shows checking three items, some items may be omitted during implementation, and new items not illustrated may be added. Additionally, although the illustrated example explains that both the flag and the scaling value are used, during implementation, the flag may not be used separately, and the scaling value may be changed in situations where the aforementioned flag value needs to be set. Conversely, a method may be applied where the scaling value is maintained at 1 when the flag value is not set.

[0172] The following describes the operation for determining whether a shape change of an object is necessary.

[0173] First, it can be checked whether the shape of the object is new or not (740). For example, a correlation coefficient between a previously stored new object and a detected object can be calculated, and based on the calculated correlation coefficient, it can be checked whether it is new or not. Alternatively, when distinguishing the shape of the object using a learning model, it can be checked whether there is another new class corresponding to the class determined by the learning model. For example, regarding the logo for Samsung, 10 and 11 are registered, and the electronic device knows that 11 is a newer logo than 10, and when the learning model outputs class 10 for the input image, the electronic device can confirm the existence of a new object called 11.

[0174] And it is possible to verify whether the detected object is an object suitable for the current country. For example, a logo can be universally applicable worldwide or may differ by country. Therefore, it is possible to verify whether the currently detected object is a logo suitable for the current country. For example, as illustrated, if the logo "Samsung" is used in Korea but the logo "Galaxy" is used in Japan, the region of the current electronic device is identified, and a new object shape corresponding to the region can be used.

[0175] And the noise of the object can be checked (S760). Specifically, it can be calculated whether the object impairs the immersion of the video. Such calculations can determine the amount of noise or blur. For example, if the PSNR (Peak-Signal to Noise Ratio) is measured with the original logo and is 30dB or less, the object is considered to be severely damaged and a change flag can be set. Conversely, if the PSNR measurement is not 30dB or higher, the object is considered to be lightly damaged and a change flag may not be set.

[0176] And it can be checked whether the immersion is compromised by the difference in luminance between the object and the background (770). For example, the electronic device (100) can check whether the difference in luminance between the object and the original object does not compromise immersion by checking the difference in luminance between the detected object and the original object and checking whether the degree of damage has exceeded a certain level. For example, the electronic device (100) can determine that the damage to the luminance component of the object is severe if it measures the original logo and the SSIM (Structural Similarity Index Measure) and the result is 0.8 or less. Conversely, if the SSIM measurement is 0.8 or higher, the damage to the luminance component of the object can be considered to be minor.

[0177] And it can be checked whether the difference in saturation between the object and the background does not impair immersion (780). For example, the electronic device (100) can analyze the difference in saturation between the detected object and the original object and change to the latest logo when the degree of damage exceeds a certain level. For example, if the original logo and SSIM are measured and the result is 0.8 or less, the damage to the object's chrominance component can be considered severe. Conversely, if the SSIM measurement is 0.8 or higher, the damage to the object's chrominance component can be considered minor.

[0178] Meanwhile, in the illustrated example, it was described and explained that a change flag is not set when there are no problems with the five items, and a change flag is set when it is confirmed that one of the five is present. However, in implementation, some of the items described above may be omitted. Additionally, other items not illustrated may be added and used.

[0179] Next, the operation of changing the logo using the flag confirmed by the above-described process is explained.

[0180] Meanwhile, in the illustrated example, setting the flag was referred to as 1 and not setting the flag was used as 0, but in implementation, setting the flag can be 0 and the opposite case can be 1. In addition, values ​​other than 0 or 1 may be used.

[0181]

[0182] FIG. 8 is a drawing for explaining an operation for determining the size of an object according to an embodiment of the present disclosure. Specifically, it is a drawing for explaining a method for determining the final size of an object according to first to third scale flags.

[0183] Referring to FIG. 8, the size of an object can be changed according to a scale flag, and if a first scale flag is set, a first scale can be multiplied to the size of the object (810). For example, if a first scale flag is set, the first scale can be set so that the ratio of the object to the image before resizing ((w×h) / (W×H)) and the ratio of the object to the entire device after final resizing ((w_T^'×h_T^') / (W_T×H_T )) are equal. Here, the horizontal scale_1_w and scale_1_h can be expressed as follows.

[0184] And if the second scale flag is set, the second scale can be multiplied by the size of the object (820). For example, if the second scale flag is set, the second scale can be selected as the ratio of the actual viewing distance to the recommended viewing distance by measuring, through a camera or sensor, how much the actual viewing distance D_T of the viewer is compared to the recommended viewing distance according to the device size. Conversely, if the second scale flag is not set, the second scale can be set to a value of 1 so that it does not change according to the viewing distance.

[0185] Likewise, if the third scale flag is set, the third scale can be multiplied by the size of the object (830). If neither flag is set, the scaling for each step can be a value of 1. For example, if the third scale flag is set, the third scale can be 1 to 2 depending on the age of the viewer. It can be 2 for children (0-12 years), 1.5 for adolescents (13-18 years), and 1.5 for the elderly (65 years or older). Conversely, if the third scale flag is not set, the third scale value becomes 1, so no change based on age can be applied.

[0186] Meanwhile, although the illustrated example shows the first to third scale flags being checked in order, the order of application during implementation may differ from the illustrated example. Additionally, the scaling value may be used directly without separately checking whether the flags are set, that is, without using the flags.

[0187] Through this operation, once each scaling value is calculated, the calculated scaling values ​​can be combined to determine the size of the final object to be applied.

[0188] And you can check if the change flag is set (840).

[0189] If a logo change is set (840-Y), a new object stored in the buffer (or memory) (850) can be obtained. Accordingly, an object to be applied to the second image can be determined. For example, as illustrated, if a change flag is set, an object different from the shape of the existing object can be used, and if the flag is not set, an object identical to the shape of the existing object can be used.

[0190] Meanwhile, although the object was used as 0 or 1 in the above description, a specific integer and 0 may be used during implementation. That is, a value of 0 means that the shape change of the object is not used, and an integer value other than 0 may use an image corresponding to a specific integer value. That is, for example, if the change flag has a value of 33, an image corresponding to 33 may be used.

[0191] As described above, the electronic device according to the present embodiment can maintain an immersive environment for viewers by readjusting the size of the logo and subtitles according to the size of the device. Additionally, by readjusting the size of the object according to the size of the logo, the brand logo can be maintained robustly in any environment. Furthermore, by changing the shape of the logo to the latest logo at the time of viewing, the logo can always be maintained in its current form. Moreover, by changing the brightness, saturation, and size of the logo and subtitles according to the viewing environment and the viewer, more effective information delivery is possible.

[0192]

[0193] FIG. 9 is a drawing for illustrating an example of changing the size of subtitles according to one embodiment of the present disclosure.

[0194] Referring to FIG. 9, first and second images (910, 920) are shown. Here, the first image (910) is an original image and includes subtitles (911) in a specific area within the image.

[0195] And the second image is a second image (920) in which the size of an object in the first image has been changed according to the present disclosure. For example, if a subtitle is located in the first image, but the user viewing the image is old or the size of the subtitle in the existing image is very small, the electronic device can expand the subtitle object in the first image.

[0196] Specifically, the electronic device (100) can check whether a subtitle object exists within the video.

[0197] If a subtitle object is identified as a result of the verification, the size of the subtitle object can be identified, and whether a size change is required for the subtitle object can be determined.

[0198] If it is identified that a change in the size of a subtitle object is required, as explained above, the size to be changed can be calculated, subtitle information having the calculated size can be verified, and a corresponding background image can be generated.

[0199] And a second image (920) can be obtained by combining the background image and the resized subtitle.

[0200] Meanwhile, although the implementation simply illustrates and explains the change in subtitle size, the content of the subtitle may be reviewed during implementation. To this end, the electronic device (100) may perform OCR on the detected subtitle and verify the content based on the OCR results. If there are matters requiring filtering, such as profanity or racial discrimination, in the identified text, deletion or blurring of the content may be performed. Such actions may be performed depending on the user watching the video. That is, if it is confirmed that the viewer watching the video is a minor, phrases corresponding to profanity, racial discrimination, or gender discrimination may be deleted or blurred.

[0201]

[0202] FIG. 10 is a flowchart illustrating the control operation of an electronic device according to one embodiment of the present disclosure.

[0203] Referring to FIG. 10, first, a first image is acquired (S1010). For example, when a playback command for specific content is received from a user, an image of the content can be acquired. Such content may be a video, game content, a photo, etc.

[0204] When an object is identified within the first image, object information and a background image can be obtained based on the first image (S1020). For example, a background image corresponding to a second aspect ratio different from the first aspect ratio can be generated using a first image corresponding to a first aspect ratio. At this time, if it is necessary to perform upscaling processing on the first image described above, the first aspect ratio of the first image is upscaled to correspond to the second aspect ratio to obtain a background image, and upscaling is also performed on the object while maintaining the first aspect ratio.

[0205] If a change in the object shape is required, that is, if the image in the first image is a past logo or a logo not currently used in the relevant country, a logo image currently used in the relevant country can be obtained from an external device. In cases where such a logo change is required, a new logo image, etc., may be obtained from an external device in advance and stored.

[0206] Based on the acquired object information and background image, a second image can be generated in which at least one of the position, size, and shape of an object on the background image is changed (S1030). For example, the second image can be generated by compositing the generated background image with an object corresponding to a first aspect ratio. At this time, the background image may be upscaled. Furthermore, since the present disclosure assumes that the object ratio in the second image is smaller than the object ratio in the first image, the object area in the first image can be inpainted when generating the background image to obtain the background image.

[0207] And the composite object may be a new object acquired from an external source, rather than an image within the first image.

[0208] In addition, during the process of compositing the background image and the object, at least one of the saturation and luminance of the new object image may be adjusted based on at least one of the saturation information and luminance information of the background image, and a second image may be generated by compositing the adjusted new object image and the background image. Alternatively, the object size within the second image may be identified, and the object and the background image may be composited based on the identified object size to generate the second image.

[0209] For example, the size of an object in an image can be determined based on the screen size of the display showing the second image, distance information between the display and the user, and age information of the user.

[0210] And if the electronic device is equipped with a display, it can display the second image. If the electronic device is not equipped with a display, for example, if it is a set-top box, the second image can be transmitted to an external device.

[0211] Meanwhile, although the above-described operation was explained assuming it is a logo, it can also be performed on subtitles during implementation. In this case, not only can the size or position of the subtitles be changed, but the font can also be changed. Additionally, OCR can be performed on the subtitles, and filtering of the subtitle content can be performed. For example, regarding subtitles containing racial slurs, sexism, profanity, etc., processing such as deleting the problematic text or blurring the relevant parts of the subtitle can be performed.

[0212] As described above, the control method according to the present disclosure displays objects such as logos and subtitles within the video by adjusting them to a size suitable for the current screen, thereby allowing for more immersive viewing of the content.

[0213] Meanwhile, methods according to at least some of the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0214] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0215] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0216] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily on the storage medium. For example, a 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TMIt can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., terminal devices). For online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0217] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0218] When the above-described instruction is executed by a processor, the processor may perform the function corresponding to the above-described instruction directly or by using other components under the control of the above-described processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0219] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Communications Department; Memory for storing at least one instruction; and It includes one or more processors that execute at least one of the above instructions; The above one or more processors, When an object is identified within a first image obtained through the communication unit, object information and a background image are obtained based on the first image, and An electronic device that generates a second image in which at least one of the position, size, and shape of the object is changed on the background image, based on the object information and background image obtained above.

2. In Paragraph 1, The above one or more processors, An electronic device that generates a background image corresponding to a second aspect ratio different from the first aspect ratio using the first image corresponding to the first aspect ratio, and generates a second image by compositing the generated background image with an object corresponding to the first aspect ratio.

3. In Paragraph 1, The above one or more processors, A background image is obtained by upscaling the first aspect ratio of the above first image to correspond to the second aspect ratio, and An object image corresponding to the first aspect ratio in the first image is upscaled while maintaining the first aspect ratio to obtain an object image, and An electronic device that generates the second image by synthesizing the upscaled background image and the object image.

4. In Paragraph 1, The above one or more processors, Based on the above object information, a new object image is obtained, and An electronic device that generates a second image by synthesizing the above background image and the above acquired new object image.

5. In Paragraph 4, The above one or more processors, An electronic device that acquires a new object image based on the country information of the above electronic device.

6. In Paragraph 4, The above one or more processors, An electronic device that adjusts at least one of the saturation and luminance of the new object image based on at least one of the saturation information and luminance information of the background image, and synthesizes the adjusted new object image and the background image to generate a second image.

7. In Paragraph 1, The above one or more processors, An electronic device that identifies the size of an object within the second image and synthesizes the object and the background image based on the identified object size to generate the second image.

8. In Paragraph 7, The above one or more processors, An electronic device that identifies the size of an object based on the screen size of a display showing the second image, distance information between the display and the user, and age information of the user.

9. In Paragraph 1, The above one or more processors, An electronic device that inpaints an object area within the first image to obtain a background image.

10. In Paragraph 1, The above object is a subtitle within the video, and The above one or more processors, An electronic device that generates a second image in which at least one of the position, size, and font of the subtitle on the background image is changed.

11. In a method for controlling an electronic device, Step of acquiring a first image; When an object is identified within the first image, a step of obtaining object information and a background image based on the first image; and A control method comprising the step of generating a second image in which at least one of the position, size, and shape of the object is changed on the background image based on the object information and background image obtained above.

12. In Paragraph 11, The step of acquiring the above object information and background image is, Using the first image corresponding to the first aspect ratio, a background image corresponding to a second aspect ratio different from the first aspect ratio is generated, and The step of generating the second image above is, A control method for generating a second image by compositing the above-generated background image with an object corresponding to the above-generated first aspect ratio.

13. In Paragraph 11, The step of acquiring the above object information and background image is, A background image is obtained by upscaling the first aspect ratio of the above first image to correspond to the second aspect ratio, and An object image corresponding to the first aspect ratio in the first image is upscaled while maintaining the first aspect ratio to obtain an object image, and The step of generating the second image above is, A control method for generating the second image by synthesizing the upscaled background image and the object image.

14. In Paragraph 11, The step of acquiring the above object information and background image is, Based on the above object information, a new object image is obtained, and The step of generating the second image above is, A control method for generating a second image by synthesizing the above background image and the above acquired new object image.

15. A non-transient computer-readable recording medium storing a program for executing a method of controlling an electronic device, The above control method is, Step of acquiring a first image; When an object is identified within the first image, a step of obtaining object information and a background image based on the first image; and A computer-readable recording medium comprising: a step of generating a second image in which at least one of the position, size, and shape of the object is changed on the background image based on the object information and background image obtained above.

Images