Electronic device and control method therefor

Abstract

This electronic device comprises a communication interface, at least one processor, and a memory for storing at least one instruction, wherein the at least one instruction, when executed by the at least one processor, causes the electronic device to: acquire an image by using the communication interface; provide the image as an input to a neural network model to acquire probability information about a plurality of classifications including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres; determine current genre information corresponding to a current image on the basis of the acquired probability information and previous genre information corresponding to a previous frame; and perform, on the current image, image quality processing corresponding to the genre information.

Description

Electronic device and control method thereof

[0001] The present disclosure relates to an electronic device capable of adaptively adjusting image quality according to video genre and a method for controlling the same.

[0002] Electronic devices can perform operations such as generating images corresponding to content or displaying images. Recent electronic devices can perform various image quality processing to display acquired images more realistically.

[0003] Such image quality processing can be performed by adjusting control factors such as image sharpness, contrast ratio, and saturation, and these control factors could be applied differently depending on the type of image.

[0004] The embodiments of the present disclosure may solve at least one of the previously described problems and / or disadvantages and provide the advantages described below. Accordingly, the embodiments of the present disclosure provide an electronic device capable of adaptively adjusting image quality for a video genre and a method for controlling the electronic device.

[0005] Additional aspects will be partially explained in the following detailed description, and will also be partially apparent from the description or can be understood by practicing the presented embodiments.

[0006] An electronic device according to an embodiment of the present disclosure is disclosed. The electronic device comprises a communication interface, at least one processor, and a memory for storing at least one instruction. When the at least one instruction is executed by the at least one processor, the electronic device acquires an image using the communication interface and provides the image as input to a neural network model to acquire probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres. Based on the acquired probability information and previous genre information corresponding to a previous frame, the electronic device determines current genre information corresponding to a current image and performs image quality processing corresponding to the genre information on the current image.

[0007] The above neural network model is trained to output probability values ​​for each of the above first classification, the above second classification, and the above third classification, wherein the first classification is associated with a first feature corresponding to the above first genre, the above second classification is associated with the above first feature and is not associated with the above second feature, and the above third classification may not be associated with the above first feature and the above second feature.

[0008] The first genre mentioned above includes the sports genre, and the second classification may be associated with at least one of a close-up video of a specific person, a crowd video of multiple people clustered together, and a data graphic video.

[0009] The above neural network model is trained to classify sports events, and the second classification can be associated with different image conditions corresponding to multiple sports types.

[0010] When the above at least one instruction is executed by the above at least one processor, the electronic device may acquire additional graphic information from the image and determine genre information corresponding to the current image by additionally considering a determination of whether the additional graphic information was acquired from a previous frame and a determination of whether the additional graphic information was acquired from the current image.

[0011] The above additional graphic information may include at least one of a broadcast station logo, a sports federation logo, and score status information at a preset location in the above image.

[0012] When the above at least one instruction is executed by the above at least one processor, the electronic device may acquire audio information corresponding to the image, acquire audio characteristic information based on the audio information, and determine genre information corresponding to the current image by additionally considering the previous audio characteristic information corresponding to the previous frame and the current audio characteristic information corresponding to the current image.

[0013] When the above at least one instruction is executed by the above at least one processor, the electronic device determines the genre of the current image as the first genre based on probability information indicating that the first probability value corresponding to the first classification is greater than or equal to a predetermined value, and maintains the genre of the current image as the previous genre corresponding to the previous frame based on probability information indicating that the first probability value is smaller than the predetermined value and the second probability value corresponding to the second classification is the highest probability value included in the probability information.

[0014] When the above at least one instruction is executed by the above at least one processor, the electronic device can determine the current genre information based on the mode among the genre corresponding to a preset number of frames and the genre corresponding to the highest exchange rate value among the acquired probability information.

[0015] When the above at least one instruction is executed by the above at least one processor, the electronic device can control the display to display the processed image.

[0016] A control method for an electronic device according to one embodiment of the present disclosure comprises the steps of: providing an image as input to a neural network model to obtain probability information for a plurality of classifications including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres; determining current genre information corresponding to a current image based on the obtained probability information and previous genre information corresponding to a previous frame; and performing image quality processing corresponding to the genre information on the current image.

[0017] The above neural network model is trained to output probability values ​​for each of the above first classification, the above second classification, and the above third classification, wherein the first classification is associated with a first feature corresponding to the above first genre, the above second classification is associated with the above first feature and is not associated with the above second feature, and the above third classification may not be associated with the above first feature and the above second feature.

[0018] The first genre mentioned above includes the sports genre, and the second classification may be associated with at least one of a close-up video of a specific person, a crowd video of multiple people clustered together, and a data graphic video.

[0019] The control method further includes the step of acquiring additional graphic information from the image, and the genre information can be determined by additionally considering a judgment on whether the additional graphic information was acquired from a previous frame and a judgment on whether the additional graphic information was acquired from the current image.

[0020] In a non-transient computer-readable recording medium storing a program for executing a control method for an electronic device according to one embodiment of the present disclosure, the control method comprises: providing an image as input to a neural network model to obtain probability information for a plurality of classifications including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres; determining current genre information corresponding to a current image based on the obtained probability information and previous genre information corresponding to a previous frame; and performing image quality processing corresponding to the genre information on the current image.

[0021] The above-described or other aspects, features, and benefits of embodiments of the present disclosure will become more apparent from the following description with reference to the accompanying drawings. In the accompanying drawings:

[0022] FIG. 1 is a drawing for explaining a content recommendation operation according to one embodiment of the present disclosure,

[0023] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0024] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure,

[0025] FIG. 4 is a drawing for explaining a classification method according to one embodiment of the present disclosure,

[0026] FIG. 5 is a drawing illustrating classification items according to one embodiment of the present disclosure,

[0027] FIG. 6 is a drawing for explaining a classification method according to one embodiment of the present disclosure,

[0028] FIG. 7 is a drawing for explaining another classification example of one embodiment of the present disclosure,

[0029] FIG. 8 is a drawing for illustrating a different classification method in one embodiment of the present disclosure, and,

[0030] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0031] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0032] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description may be omitted.

[0033] Additionally, the following specific embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these specific embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0034] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0035] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components, etc.) and do not exclude the presence of additional features.

[0036] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” and “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0037] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0038] Where it is stated that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component may be directly connected to the other component or connected through the other component (e.g., a third component).

[0039] However, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0040] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0041] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0042] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0043] Operations performed by a module, program, or other component according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0044] The various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0045] An electronic device according to various embodiments of the present disclosure may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a server, or a wearable device. The wearable device may include at least one of an accessory type (e.g., a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD)), a fabric or clothing integrated type (e.g., electronic clothing), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit.

[0046] In some embodiments, the electronic device is, for example, a television, a DVD (digital video disk) player, audio, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync). TM , Apple TV TM , or Google TVTM ), game console (e.g., Xbox) TM PlayStation TM It may include at least one of an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame. Among the electronic devices according to the present disclosure, a device having a display may be referred to as a display device. Additionally, even if the electronic device of the present disclosure does not have a display, it may be a set-top box or a PC that provides images to a display device.

[0047] Hereinafter, embodiments according to the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them.

[0048] FIG. 1 is a drawing for explaining a content recommendation operation according to one embodiment of the present disclosure.

[0049] Referring to FIG. 1, an electronic device (100) can display content selected by a user. Here, the content is provided through the electronic device as a movie, music, play, photo, cartoon, animation, computer game, text, figure, color, sound, motion or picture, or a combination of the above.

[0050] The electronic device (100) can correct the video to match the sports genre and display the corrected video if the content selected by the user is a sports genre.

[0051] Here, "genre" refers to a category in which various content, such as literature, art, and film, is classified according to specific criteria. Accordingly, content genres can be classified into sports, movies, dramas, animations, news, etc. According to an embodiment of the present disclosure, the genre of the content can be used to perform image processing (or image quality processing) or sound processing suitable for that genre. Therefore, rather than classifying content genres individually, there is greater interest in whether they are classified by specific genre (e.g., sports genre).

[0052] The electronic device (100) may use features related to a sports genre to verify or determine whether the current video is a sports genre. For example, the first screen (10) may have a composition consisting of a batter, a catcher, and an umpire, which are commonly seen in baseball games. Therefore, the electronic device (100) may determine that the current video is related to a sports genre if such a composition is identified in the video. As another example, since sports such as soccer (or football) can be played on green grass, the electronic device may determine that the video is related to a sports genre based on identifying or detecting multiple people handling a ball on the green grass in the video. However, this is merely an example, and features corresponding to a sports genre are not limited to this. For example, other features may be obtained by using learning based on various sports videos.

[0053] However, the same video may contain video features that are difficult to use to determine the sports genre. Therefore, it may be difficult to identify the sports genre based solely on the features of scenes that frequently appear in the sports genre or related scenes. For example, the video may include a second screen (20) showing a close-up of a specific player, a screen showing the audience, a screen showing graphic information, a screen showing an interstitial advertisement, etc.

[0054] Some shots that frequently appear in sports videos may contain compositions and / or characteristics commonly found not only in the sports genre but also in other genres such as news, movies, and dramas. Therefore, if a close-up shot of a specific person is identified and classified as belonging to the sports genre, videos containing such close-ups, such as news, movies, and dramas, may be mistakenly perceived as belonging to the sports genre.

[0055] Based on the above, close-up shots or shots showing crowds could be classified as a genre other than sports.

[0056] While this classification method based on this approach has the advantage that close-up shots of specific individuals in genres such as news and movies may not be classified as sports, it also has the disadvantage that scenes containing close-ups within sports sequences may not be recognized as sports genres.

[0057] For example, an electronic device may process the screen brightness of video corresponding to the sports genre at a relatively high (e.g., relatively bright) first brightness level. The electronic device may determine that the video corresponds to the sports genre if a close-up shot of a specific match or a spectator shot is maintained for a certain period of time. Accordingly, the electronic device may process the screen brightness at a relatively lower (e.g., relatively dark) second brightness level compared to the previous screen. Subsequently, if the screen is again determined to be of the sports genre based on a subsequent screen, the screen brightness may be processed back to the first brightness level. Therefore, as described above, if the image quality processing method changes frequently even when continuously watching the same sports content, it may interfere with video viewing.

[0058] Therefore, even when displaying the aforementioned situations related to sports matches, such as close-up shots of specific players or shots of the crowd, it is necessary to consistently recognize the video as belonging to the sports genre. Furthermore, when the aforementioned close-up shots are displayed or when a screen containing a large number of spectators is identified in content such as news, it is necessary not to identify the video as belonging to the sports genre.

[0059] Accordingly, in consideration of the above-described points, the embodiments of the present disclosure identify close-up screens, audience screens, graphic images, etc., as separate items, and when such items are identified, analyze or determine the final genre of the video by considering the previous identification results.

[0060] For example, if the classification in the previous screen was Sports and the current is recognized as Close-up, it can be determined that the current genre of the video maintains the previous classification, Sports. As another example, if the previous genre was News and the current video is recognized as Close-up, it can be determined that the current genre ultimately maintains the previous classification, News.

[0061] The specific configuration and operation of the electronic device (100) according to one embodiment of the present disclosure will be described later with reference to FIG. 2.

[0062] As such, the electronic device according to the embodiment of the present disclosure classifies close-ups, crowd images, and graphic images into separate items, and analyzes the final genre of these items by referring to previous classification results (e.g., previous analysis results or previous decision results), thereby enabling more accurate and consistent genre analysis or determination. Furthermore, the electronic device can perform accurate and consistent image quality processing for the images based on accurate genre analysis.

[0063] Although the description of FIG. 1 above describes an example in which an electronic device directly displays an image, the present embodiment is not limited thereto. For example, in some embodiments, the electronic device may output an image to another device without including a display.

[0064] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0065] Referring to FIG. 2, the electronic device (100) may include a communication interface (110), a memory (120), and a processor (130). Such an electronic device (100) may be a server, at least one of a set-top box and a TV, or include at least one of these. Below, cases where information such as viewing history of another device can be collected and used are described in relation to the device of FIG. 2, and cases where viewing history is directly stored and managed are described in relation to the device of FIG. 3.

[0066] The communication interface (110) is a configuration that performs communication with various types of external devices according to various types of communication methods. The communication interface (110) may include a wired communication module, a Wi-Fi module, a Bluetooth module, an infrared communication module, and a wireless communication module, etc. Here, each communication module may include at least one hardware chip or hardware circuit.

[0067] Wi-Fi modules and Bluetooth modules can perform communication via Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as SSID and session key, is transmitted and received first; after establishing a communication connection using this information, various information can be transmitted and received.

[0068] The infrared communication module performs communication according to infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared rays located between visible light and millimeter waves.

[0069] In addition to the communication method described above, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0070] In addition, the communication interface (110) may include at least one wired communication module that performs communication using a LAN (Local Area Network) module, an Ethernet module, a pair cable, a coaxial cable, a fiber optic cable, or a UWB (Ultra Wide-Band) module.

[0071] The communication interface (110) can receive content. For example, the content may include at least one of a movie, a music video, a drama, or a short video. The content may include at least one of a video, game content, etc.

[0072] The communication interface (110) can obtain content information corresponding to the content. For example, the content information may include field information and metadata. For example, the example of video analysis described below may not simply perform screen analysis but may also utilize the content information described above.

[0073] According to the present embodiment, a screen is an image displayed on the display of an electronic device, or may include such an image. An image may also be referred to as a frame. Various types of objects, such as icons, text, photos, videos, widgets, etc., may be displayed on the screen.

[0074] The communication interface (110) can receive information used in various applications of the electronic device 100 in addition to content, and can also receive information for providing services from an external device.

[0075] The memory (120) may be implemented as at least one of the internal memories included in the processor (130), such as ROM (e.g., EEPROM (electrically erasable programmable read-only memory)) or RAM, or it may be implemented as a memory separate from the processor (130). For example, the memory (120) may be implemented as a memory embedded in the electronic device (100) or as at least one of the memory that can be attached to and detached from the electronic device (100), depending on the purpose of data storage. For example, data for operating the electronic device (100) may be stored in the memory embedded in the electronic device (100), and data for the expansion function of the electronic device (100) may be stored in the memory that can be attached to and detached from the electronic device (100).

[0076] The memory (120) can store probability information, recognition results, etc. generated during the process described below. Accordingly, the memory (120) can store probability information or information about the final genre within a certain time range (e.g., "10" seconds).

[0077] And the memory (120) can store content, metadata, etc. received using the communication interface (110) described above. In addition, the memory (120) can temporarily store a video with improved image quality.

[0078] According to the present embodiment, the memory embedded in the electronic device (100) is implemented as at least one of volatile memory (e.g., at least one of DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM)), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash, etc.), hard drive, and solid state drive (SSD), and the memory detachable from the electronic device (100) is implemented in the form of a memory card (e.g., CF (compact flash), SD (secure digital), Micro-SD (micro secure digital), Mini-SD (mini secure digital), xD (extreme digital), MMC (multi-media card), etc.), external memory connectable to a USB port (e.g., USB memory), etc. It is possible.

[0079] Although the electronic device (100) is exemplified as being composed of a single memory, the present embodiment is not limited thereto. For example, when distinguishing between volatile memory and non-volatile memory, the electronic device (100) may be described as including multiple memories.

[0080] The processor (130) can perform overall control operations of the electronic device (100). For example, the processor (130) functions to control the overall operation of the electronic device (100).

[0081] The processor (130) may be implemented as a digital signal processor (DSP) that processes digital signals, a microprocessor, or a time controller (TCON). However, it is not limited thereto, and may include or be defined by one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an ARM processor. Additionally, the processor (130) may be implemented as a System on Chip (SoC) or a large-scale integration (LSI) with a built-in processing algorithm, or may be implemented in the form of a Field Programmable Gate Array (FPGA). The example illustrated in FIG. 2 includes only one processor, but the embodiment is not limited thereto and may include multiple processors (e.g., a CPU and a GPU, or a CPU and a DSP) when implemented.

[0082] The processor (130) can obtain content using the communication interface (110). For example, the processor (130) can obtain information about the content that the electronic device can provide.

[0083] The content that the aforementioned electronic device can provide may include content that the electronic device itself directly stores and provides, or content that another device downloads and displays through a mirroring method, etc.

[0084] The processor (130) can first obtain probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres, by providing an image as input to a neural network model. For example, the second classification may not always be classified as a first genre, but may be a classification that has the potential to be classified as a first genre.

[0085] For example, the neural network model may be a model trained to output probability values ​​for each of the following: a first classification image associated with a first feature corresponding to a first genre, a second classification associated with a second feature that is likely to belong to the first genre but is not associated with the first feature, and a third classification not associated with either the first or second feature.

[0086] Here, the first genre (e.g., sports genre) may further include subgenres or subclassifications such as close-up footage of a specific individual, crowd scenes, or data graphic images. Accordingly, the aforementioned first feature may be a feature for identifying or distinguishing various sports, and may be, for example, the detection of the grass environment in soccer, the layout of the home plate in baseball, or the layout of the batter and catcher. Here, "cluster" may refer to a group of people whose faces can be recognized, and "crowded" may refer to a group of multiple people (e.g., dozens or hundreds) whose individual faces cannot be recognized.

[0087] Although the above description used the case where the first genre is sports as an example, the same can be applied to other genres besides sports during implementation. For example, the first genre could be a game genre, a music broadcast genre, a movie genre, etc. Additionally, the first genre can be identified during implementation using the metadata of the current content.

[0088] Here, a neural network model is a computing system implemented based on the neural networks of the human or animal brain, and may be referred to as a learning model, machine learning model, artificial intelligence model, or deep learning model. For example, a learning model can be implemented as a CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), etc., but is not limited to such examples.

[0089] The processor (130) obtains genre information for the current video based on the obtained probability information and genre information for the previous frame. For example, the processor (130) determines that the current video belongs to the first genre if the probability information corresponding to the first classification is greater than or equal to a predetermined value, and can maintain the genre determined in the previous frame if the probability information corresponding to the first classification is less than the predetermined value and the probability value corresponding to the second classification among the probability values ​​included in the obtained probability information is the highest.

[0090] In some embodiments, the processor (130) may determine the genre as the mode (e.g., mode) among the genres corresponding to each of the preset number of frames and the genre corresponding to the highest probability value among the acquired probability information. For example, the processor (130) may determine the current genre based on the judgment result of a certain period of time.

[0091] In some embodiments, the time information and usage method may be applied in various ways. For example, if the previous judgment result (e.g., the judgment result immediately before the current time) is a sports genre, the processor (130) may use the mode for 10 seconds. However, for example, if the previous judgment result is not determined to be a sports genre (e.g., the ad display period), the processor (130) may use the mode for a relatively short time (e.g., 1 to 3 seconds), or if the judgment result of the current video is a sports genre, it may immediately determine it to be a sports genre.

[0092] The processor (130) may acquire additional graphic information from the image and, based on the acquired probability information and genre information for the previous frame, whether additional graphic information was acquired in the previous frame, and whether additional graphic information was acquired in the current image, may acquire genre information for the current image.

[0093] For example, the processor (130) may use information related to the content (e.g., metadata) as well as the additional graphic information described above within the video. However, since metadata cannot be used at the point where an advertisement appears during the progress of a specific program, the genre based on such metadata can be used to determine what the first genre is.

[0094] Although the above-described example describes an example in which additional graphic information is considered in a separate step, the present embodiment is not limited thereto. For example, in some embodiments, the genre analysis network described above may perform genre analysis by considering additional graphic information. Here, the additional graphic information may include at least one of a broadcaster logo, a sports federation logo, and score status information placed at a specific location in the video.

[0095] The processor (130) may perform final classification by additionally utilizing audio information. For example, the processor (130) may obtain audio information corresponding to the video. For example, the processor (130) may obtain audio data corresponding to the current video from the content. In some embodiments, the processor (130) may obtain sound data delivered to a speaker rather than the content.

[0096] The processor (130) can obtain audio characteristic information based on audio information. For example, the processor (130) can identify frequency-specific components through frequency analysis of the audio information.

[0097] The processor (130) can obtain genre information corresponding to the current video based on the obtained probability information and genre information for the previous frame, audio characteristic information from the previous frame, and audio characteristic information corresponding to the current video. For example, if a high cheering sound is continuously maintained, the content can be considered as the same content being continuously played, so the processor (130) can determine that the genre of the previous screen is maintained if the audio characteristics described above are similar.

[0098] Although an example of determining the genre of the current screen using specific information, which is the operation described above, has been explained, the operation may be described differently. For example, the processor (130) may determine whether to maintain the existing genre or identify it as a changing genre using the information described above.

[0099] Although examples related to distinguishing whether a video is a sports video or another video have been described above, the embodiments are not limited thereto. For example, some embodiments may relate to distinguishing whether a video is a main video (e.g., content video) or an advertisement video. In other words, the electronic device can identify the genre of the current content based on metadata and perform image quality processing corresponding to that genre. The processor (130) can determine whether the current video is a follow-up to a sports video or whether the sports video has been converted into an advertisement video by using video classification, etc., as previously described.

[0100] For example, while a user is watching a sports video, if the game logo was displayed in the previous video but is absent in the current video, and the current frame is classified into Category 3 rather than Category 1 or Category 2, it can be determined that the video has switched from a sports video to an advertisement video.

[0101] However, after the advertisement video is confirmed, if a game logo is detected in the video and the probability value of the first classification is relatively high, the processor (130) can immediately determine that the video is a sports video again.

[0102] Then, the processor (130) performs image quality processing corresponding to the acquired genre information on the current video. For example, if the acquired genre is a sports genre, the processor (130) can perform image quality processing corresponding to the sports genre. If the acquired genre is a movie genre, the processor (130) may not perform any additional image quality processing or may only perform image quality processing pre-set by the user. Additionally, if the acquired genre is a game genre, the processor (130) may perform image quality processing using a method that enables high-speed image quality processing rather than high-quality image quality processing.

[0103] Here, image quality processing refers to the process of enhancing or adjusting the quality of images or videos, and may include improving the visual quality of videos or images by utilizing various algorithms. For example, image quality processing may include processing to enhance resolution, processing to remove noise, processing to correct color and contrast, processing to remove blur, processing to restore compressed images with loss, and processing to add special effects.

[0104] Although the examples described above explain examples related to image quality processing, the embodiments are not limited thereto. For example, in some embodiments, for specific genres, additional image quality processing corresponding to that genre may not be performed, and only the audio processing corresponding to that genre described below may be carried out. For example, for music-centered content such as music broadcasts, instrument performances, or orchestras, audio processing may be primarily performed instead of image quality processing.

[0105] Additionally, the processor (130) can perform audio processing corresponding to the acquired genre information on the current audio data. For example, if the current screen is a sports screen, the processor (130) can perform audio processing to output a sound that is more realistic (or three-dimensional). If the type of sports can be distinguished, the processor (130) can perform audio processing corresponding to the type of sports. For example, different sound processing can be performed for sports performed indoors and sports performed outdoors, or three-dimensionality corresponding to the stadium can be applied. For example, for sports performed in a very large area, such as rugby and soccer, three-dimensionality corresponding to the size of the stadium can be applied, and for sports performed in a small area, such as table tennis and fencing, sound processing can be performed to have three-dimensionality corresponding to that size.

[0106] The electronic device (100) according to the present embodiment as described above identifies close-up images, crowd images, and graphic images as separate items, and if it recognizes that the current image corresponds to one of the items described above, it can finally determine the genre by referring to the previous identification result. Accordingly, more accurate and consistent genre analysis becomes possible, and through this, more accurate and consistent image quality processing can be performed.

[0107] Meanwhile, although a relatively simple configuration of the electronic device (100) has been illustrated and described above, other configurations may be provided in addition to this during implementation, and some examples thereof are described below with reference to FIG. 3.

[0108] FIG. 3 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0109] Referring to FIG. 3, the electronic device (100') may include a communication interface (110), memory (120), processor (130), I / O interface (140), microphone (150), display (160), and speaker (170).

[0110] According to an embodiment, the communication interface (110), memory (120), and processor (130) may be the same or similar as those described above with reference to FIG. 2. Therefore, redundant or unnecessary descriptions may be omitted.

[0111] The configuration of the communication interface (110), memory (120), and processor (130) was previously described in FIG. 2, and below, only operations different from FIG. 2 will be described.

[0112] The I / O interface (140) may be any one of the following interfaces: HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface).

[0113] The I / O interface (140) can input and output at least one of audio and video signals. Depending on the implementation example, the I / O interface (140) may include separate ports for inputting and outputting only audio signals and for inputting and outputting only video signals, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0114] And the I / O interface (140) can provide or transmit a video signal corresponding to a screen generated by the electronic device (100') or an audio signal together with the video signal to an external device (e.g., a display device, an STB, etc.). For example, the video signal being transmitted may have been processed for image quality in a manner corresponding to the genre of the screen.

[0115] The microphone (150) can receive or detect the user's voice when active. For example, the microphone (150) may be integrally formed in at least one of the upper, front, or side directions of the electronic device (100'). The microphone (150) may include various configurations such as a microphone configured to collect user voice in an analog form, an amplifier circuit configured to amplify the collected user voice, an A / D conversion circuit configured to sample the amplified user voice and convert it into a digital signal, and a filter circuit configured to remove noise components from the converted digital signal.

[0116] When a user's voice is input through such a microphone (150), the processor (130) can check the content of the user's voice and perform an action corresponding to the content of the voice. For example, the content of the voice may be a command to change the content (or channel), a command to change the image quality processing method, etc.

[0117] In the example described above, an example in which user voice is input through a microphone (150) has been explained, but the present embodiment is not limited thereto. For example, the microphone (150) may be equipped in a remote control that controls an electronic device (100'), and user voice input through the microphone equipped in the remote control may be input and processed in the electronic device (100') through the communication interface (110) described above.

[0118] The electronic device (100') can operate not only based on the configuration or remote control provided in the electronic device (100'), but also according to the control commands of the terminal device. For example, if the electronic device (100') is a TV or a set-top box, recently, manufacturers provide applications for controlling the TV or set-top box. Such applications can provide a function that allows the terminal device to be used as a remote control for the electronic device.

[0119] Accordingly, when a user executes an application to control a TV or set-top box using a terminal device and inputs a voice command through the terminal device, the electronic device (100') can perform a voice recognition operation and a corresponding voice recognition result using the voice signal input through the terminal device.

[0120] Here, speech recognition includes the process of converting a user's voice into a form that can be processed by an electronic device (100'). For example, speech recognition includes converting an acoustic speech signal of voice acquired by the electronic device (100') into text such as words or sentences, and may be referred to as computer speech recognition or speech-to-text conversion (STT).

[0121] The display (160) can be implemented as a display of various forms, such as at least one of an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, and a PDP (Plasma Display Panel). The display (160) may also include a driving circuit, a backlight unit, etc., which can be implemented in a form such as at least one of an a-si TFT, an LTPS (low temperature poly silicon) TFT, and an OTFT (organic TFT). In an embodiment, the display (160) can be implemented as at least one of a touchscreen combined with a touch sensor, a flexible display, a 3D display, etc.

[0122] The display (160) can display various images. For example, the display (160) can display an image processed by the processor (130). According to an embodiment, if the image processing for a specific genre involves adjusting brightness values, the display (160) can receive setting information, such as brightness information, in response to the image processing described above. Based on this setting information, the display (160) may adjust the backlight operation or change the brightness state to display the image.

[0123] The speaker (170) can output sound. For example, the speaker (170) may be a component that outputs various audio data processed at the I / O interface, as well as various notification sounds or voice messages. The speaker (170) may also output result information (e.g., information about recommended content) corresponding to examples such as the voice recognition operation described later. In this way, the sound output from the speaker (170) may be sound processed by the processor (130) based on genre.

[0124] FIG. 3 illustrates an example in which an electronic device (100') includes a display (160), but the present invention is not limited thereto. For example, if the electronic device (100') is a device such as a set-top box that does not include a display, the display configuration may be omitted. Additionally, depending on the implementation form, the speaker and microphone described above may also be omitted. Furthermore, other configurations (e.g., camera, etc.) may be additionally included.

[0125] FIG. 4 is a drawing for explaining a classification method according to one embodiment of the present disclosure.

[0126] Referring to FIG. 4, an electronic device (e.g., at least one of electronic device (100) and electronic device (100')) can provide the current image as input to a neural network model and obtain probability information in operation 410. As illustrated in FIG. 4, the neural network model can output probability information for each of the first classification, the second classification, and the third classification. According to an embodiment, operation 410 can be performed on a frame-by-frame basis for each frame (e.g., frame-by-frame or for each frame of the image).

[0127] Here, the first classification corresponds to sports video, the second classification corresponds to close-up video, and the third classification may be other video that does not correspond to the first or second classifications. Although examples related to three classifications have been described, the embodiments are not limited thereto. For example, in some embodiments, only two classifications may be used, or four or more classifications may be used. Additionally, instead of classifying sports in general, the neural network model may distinguish individual sports events or sports types (e.g., baseball, soccer, volleyball, American football, etc.) and output probabilities for each sports event or sports type.

[0128] For example, a neural network model can output a probability value indicating that the video is likely to belong to the sports genre, provided that the video contains at least one specific game-related stadium feature or attire, such as the proportion of green in the video or the placement of white lines within the video.

[0129] The second category may correspond to the placement of close-up shots of specific individuals in most videos. For example, close-up shots of specific athletes are sometimes displayed in the sports genre. However, while close-up shots may be included in the sports genre, as explained above, they are screens that can also be used in other genres.

[0130] The electronic device can continuously accumulate and record the output results of this neural network model in memory at step 420. For example, the analysis described above can be performed four times per second, and the electronic device can continuously manage the output results within a certain time interval (e.g., 10 seconds). For example, the electronic device can perform the analysis operation described below based on approximately 40 analysis results. According to the embodiment, the analysis results may be referred to as judgment results and classification results, but the embodiment is not limited thereto. These times, frequency, and number are exemplary, and different values ​​may be used depending on the application environment and the specifications of the electronic device during implementation.

[0131] The electronic device aggregates the results of the current analysis and the results of the analysis within a specific period, and can determine whether the current video in operation 430 corresponds to the first genre or another genre.

[0132] For example, if the most common result among the analysis results within a specific time period indicates a classification corresponding to sports video (e.g., Classification 1), the electronic device may determine the current genre of the current video as the sports genre (e.g., classifying the current video as belonging to the sports genre). According to an embodiment, if the current analysis result indicates a classification corresponding to close-up video (e.g., Classification 2), the current genre may be determined as the sports genre if the previous analysis result is a classification corresponding to sports video (e.g., Classification 1), and the current genre may be determined as a different genre if the previous analysis result is a classification corresponding to a different video (e.g., Classification 3).

[0133] The electronic device can correct the image in operation 440 based on the previous classification result (e.g., perform image processing). For example, if the image corresponds to a sports genre, it can perform corrections such as increasing the brightness of the image, increasing sharpness, or increasing the factor corresponding to contrast, and can perform corrections using the control factor setting value corresponding to sports.

[0134] In the example described above, various types of sports are included in a single category, and an embodiment performing the correction described above has been illustrated and explained, but the embodiment is not limited thereto. For example, in some embodiments, specific types of sports may be distinguished individually, and image correction corresponding to a specific sports type may be performed. For example, an electronic device may classify an image into a specific subgenre, such as a baseball subgenre and a soccer subgenre. Accordingly, the electronic device may perform image correction using a first method based on the current image corresponding to the baseball subgenre, and perform image correction using a second method based on the current image corresponding to the soccer subgenre.

[0135] As described above, the electronic device according to one embodiment of the present disclosure classifies screens that are included in sports but do not have characteristics classified as sports into separate items, thereby enabling more accurate and consistent image classification.

[0136] Examples of classification characteristics including close-ups of the disclosed content are as follows.

[0137] FIG. 5 is a drawing illustrating classification items according to one embodiment of the present disclosure.

[0138] Referring to FIG. 5, two classification methods are illustrated. For example, the first classification method 510 corresponding to the first method classifies a video containing a first feature corresponding to a sports genre into a first category based on the first feature, a video containing a second feature corresponding to a movie genre into a second category based on the second feature, and a video not containing both the first feature and the second feature into a third category (e.g., other).

[0139] In the illustrated example, the video can be classified or distinguished into two genres (sports genre and movie genre) because there may be a method to correct specific image quality for sports, and another method to correct specific image quality for movies.

[0140] For example, since sports can undergo image processing such as increasing screen brightness or sharpness, videos classified as sports may be subject to such processing. As a general principle, however, no separate image quality correction is applied to movies. For instance, while there may be cases where the screen brightness is low or sharpness is poor, given that these issues are intentional on the part of the director, processing methods may be utilized to display the video in its original state without any quality correction whenever possible.

[0141] According to the first classification method 510, some videos that should be classified as sports but do not contain specific features corresponding to that sports genre may be classified into other genres. For example, if a close-up shot of a player or a shot of the audience is displayed during a sports match, features corresponding to that sports genre (e.g., green grass, stadium, etc.) may not be detected in the current video, and therefore the current video may be classified into another genre.

[0142] Therefore, while close-up shots of specific athletes or footage featuring crowds may frequently appear in sports videos, it may be beneficial to conduct additional classifications for the cases described above, even if they do not possess characteristics specific to the sports genre.

[0143] The following second classification method (520) is a method for performing such additional classification. For example, the second classification method 520 classifies videos that may be misclassified into other genres in the first classification method 510 into separate items using features related to the close-up videos, crowd videos, graphic images, etc. described above.

[0144] According to an embodiment, since the second classification method 520 may be identical to the first classification method 510 for videos classified into sports genre and movie genre, the second classification method 520 may be implemented using a combination of a first neural network model that performs the first classification method 510 and a second neural network model that further classifies videos classified into other genres by the first neural network model into close-up genre and other genres.

[0145] With reference to FIGS. 1 and 2, an example has been described in which a neural network is used to distinguish a first genre (e.g., sports genre), a second genre (e.g., movie genre), and a third genre (e.g., other genres), but the embodiments are not limited thereto. For example, in some embodiments, the neural network may distinguish only the first genre and other genres. For example, an electronic device may use a model trained for each specific genre. According to an embodiment, the electronic device may use a model trained to identify sports content using metadata, etc., and then classify the current video contained in the sports content into sports video, close-up video, and other video.

[0146] Although examples related to sports such as baseball and soccer have been described, the embodiments are not limited thereto. For example, in some embodiments, specific image processing may be performed by classifying game tournaments (e.g., electronic sports or e-sports) as a sports genre. Additionally, the video of a game tournament viewed by the user and the video of a game played by the user may be distinguished, and specific image processing corresponding to the game genre may be performed on the video of the game played by the user. For example, in the case of a game genre, the electronic device may be configured to perform image processing optimized for fast processing speed rather than image quality enhancement.

[0147] Although embodiments performing image analysis only have been described above, the operation is not limited to this. For example, in some embodiments, images may be classified using information other than image analysis, and examples thereof are described below with reference to FIGS. 5 to 8.

[0148] FIG. 6 is a drawing for explaining a classification method according to one embodiment of the present disclosure. Specifically, FIG. 6 is a drawing for explaining a case where additional information is used in the classification method described in FIG. 4.

[0149] Referring to FIG. 6, the electronic device can obtain probability information by inputting the current image into a neural network model in operation 610. Operation 610 may be identical or similar to operation 410 described earlier, and redundant descriptions thereof may be omitted.

[0150] The electronic device can continuously accumulate and record the output results of the neural network model in memory during operation 620. Subsequently, the electronic device can determine in operation 630 whether the current video belongs to the first genre or another genre by combining the currently analyzed results with the analysis results within a certain period.

[0151] In FIG. 6, the similarity of additional graphic information and sound similarity can be further analyzed in operation 640 and operation 650, and the image can be corrected in operation 660 by further using the two analysis results.

[0152] For example, video corresponding to the sports genre may continuously display additional graphic information, such as sports logos or game scores, at a specific location on the screen (e.g., top-left or top-right). The additional graphic information described may be displayed while the video containing the first feature is being displayed, and may continue to be displayed even while a close-up shot or a screen including the audience is being displayed.

[0153] Therefore, although the electronic device was classified as a close-up, if it is determined that additional graphic information such as game scores or sports logos was displayed on the previous screen and is continuously maintained on the current screen, the current video can be confirmed as a sports video.

[0154] The electronic device may also detect sound in operation 650. For example, when a specific player scores a goal during a soccer match, a close-up view of that player may be displayed, and a cheering crowd may be displayed. In this case, the sound of the crowd cheering may be continuously output in the content. Therefore, if the video displays a close-up view or a crowd view, the electronic device can identify the current video as sports video if the cheering sound was output along with the previous screen and continues to be output along with the current screen.

[0155] Although an example considering both additional graphic information and sound information has been described, it is not limited thereto, and in some embodiments, only one of the two may be considered.

[0156] In addition, although examples of analyzing additional graphic information as a separate item have been described, this is not limited thereto. For example, in some embodiments, the neural network model described above can calculate item-specific probability value information based on the presence or absence of additional graphic information during the genre analysis process.

[0157] FIG. 7 is a drawing for explaining another classification example in one embodiment of the present disclosure. Specifically, FIG. 7 is a drawing for explaining various screen examples in a baseball screen among sports genres and examples of classification thereof.

[0158] Referring to FIG. 7, various screens can be displayed in chronological order on the baseball screen. For example, the first screen (701), the third screen (703), the fourth screen (704), and the fifth screen (705) are close-up images of a specific person and are examples of screens that do not have features that would be recognized as sports. The second screen (702) is an image with a composition that can be recognized as a baseball screen.

[0159] The electronic device may determine that the current screen corresponds to the sports genre before displaying the first screen (701). Then, when the first screen (701) is displayed, the electronic device may classify the current screen as a close-up screen. According to some approach in which the close-up screen can be classified as belonging to a different genre, if the first screen (701) is displayed for a certain period of time or longer, the current video may be determined not to correspond to the sports genre.

[0160] However, according to the present disclosure, since the close-up screen can be classified and processed as a separate item from other screens (e.g., since the first screen (701) can be determined to correspond to the second classification), even if the first screen (701) is maintained for a certain period of time after the current video is determined to correspond to the sports genre, the electronic device can determine that the current video corresponds to the sports genre.

[0161] Subsequently, when a second screen (702) containing features classified as sports is output, the neural network model outputs probability value information including a probability value corresponding to the first genre (e.g., sports genre) for the screen, and the electronic device can classify the current video as corresponding to the sports genre.

[0162] According to some approach that classifies close-up shots as belonging to a different genre, if a close-up shot of a specific person (e.g., the third shot (703), the fourth shot (704), the fifth shot (705)) is subsequently displayed (e.g., if the close-up shot is maintained for a certain period of time or longer), the current video may be determined not to belong to the sports genre.

[0163] However, according to embodiments of the present disclosure, a close-up screen may be classified into a specific category (e.g., a second category), and in this case, since the genre identification of the previous screen is maintained, the current video may be identified as continuing to display a sports video even if the close-up screen is maintained for a certain period of time.

[0164] According to embodiments of the present disclosure, a close-up screen is classified as a separate item rather than another item as described above, and as the previous classification result is maintained when classifying the close-up, the classification result as a sports genre can be maintained even if the close-up screen is maintained for a certain period of time.

[0165] FIG. 8 is a drawing for explaining a different classification method in one embodiment of the present disclosure.

[0166] Referring to FIG. 8, the electronic device can obtain probability information by providing the current image in operation 810 as input to a neural network model. In the example illustrated in FIG. 8, classification into individual features can be performed not only on close-up screens but also on screens including clusters, crowds, data graphics, etc. The example illustrated in FIG. 8 includes these four situations, but the embodiment is not limited thereto and can be applied to other situations in addition to the examples described above.

[0167] The electronic device can continuously accumulate and record the output results of the neural network model in memory during operation 820.

[0168] Subsequently, the electronic device can determine whether the current image in operation 830 corresponds to the first genre or another genre by combining the currently analyzed result and the analysis result within a specific time in the past. For example, the electronic device can determine whether the current image corresponds to a close-up image, a cluster image, or a crowd image.

[0169] In addition, the electronic device can further analyze the similarity of additional graphic information in operation 840, further analyze sound similarity in operation 850, and further utilize the results of both analyses in operation 860 to perform final classification and image correction.

[0170] Although examples of applying various additional classifications to the scenario of FIG. 6 have been described, the embodiments are not limited thereto. For example, in some embodiments, various additional classifications may be applied to the scenario of FIG. 4.

[0171] FIG. 9 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0172] Refer to FIG. 9. In operation 910, an image may be provided as input to a neural network model, and probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres, may be obtained. According to an embodiment, the second classification is not always classified as the first genre, but there may be a possibility that it will be classified as the first genre. According to an embodiment, the neural network model may be a model trained to output probability values ​​for each of the first classification, the second classification, and the third classification. According to an embodiment, the first classification may be associated with a first feature corresponding to the first genre, the second classification may be associated with a second feature without being associated with the first feature, and the third classification may not be associated with the first feature or the second feature.

[0173] Here, the first genre may be a sports genre, and the second genre may be associated with at least one of a close-up video capturing a specific person, a crowd video showing multiple people gathered together, and a data graphic video. Therefore, the first feature described above may be a feature for identifying multiple sports; for example, in the case of soccer, it may be grass environment detection, and in the case of baseball, home plate layout detection and batter-catcher placement layout detection.

[0174] Subsequently, genre information corresponding to the current video can be obtained based on the probability information obtained in operation 920 and the genre information corresponding to the previous frame. For example, if the probability information corresponding to the first classification is greater than or equal to a predetermined value, the corresponding genre is determined as the first genre, and if the probability information corresponding to the first classification is less than the predetermined value and the probability value corresponding to the second classification is the highest among the probability values ​​included in the probability information, the genre determined in the previous frame may be maintained. In some embodiments, the genre information may be determined based on the maximum value among multiple probability values ​​related to one or more genres corresponding to a predetermined number of frames and the genre corresponding to the highest probability value included in the probability information.

[0175] According to an embodiment, the electronic device may additionally acquire additional graphic information from the image and acquire genre information corresponding to the current image based on the acquired probability information and genre information for the previous frame, whether the additional graphic information was acquired in the previous frame, and whether the additional graphic information was acquired in the current image. Although an example of considering the additional graphic information in a separate step has been described above, the embodiment is not limited thereto. For example, the genre analysis network described above may perform genre analysis by considering the additional graphic information. Here, the additional graphic information may include at least one of a broadcaster logo, a sports federation logo, and score status information placed at a predetermined location in the image.

[0176] Additionally, audio information may be used. For example, audio information corresponding to the video can be obtained, audio characteristic information can be obtained based on the audio information, and genre information corresponding to the current video can be obtained based on the obtained probability information, genre information of the previous frame, audio characteristic information of the previous frame, and audio characteristic information corresponding to the current video.

[0177] Then, image quality processing corresponding to the acquired genre information is performed on the current video.

[0178] The image processed in this manner can be displayed on a display device. If the electronic device is equipped with a display, the image processed in this manner can be displayed on an internal display. If the electronic device is a set-top box, the image processed in this manner can be transmitted to an external display device.

[0179] In the examples described above, examples in which history information is transmitted externally (e.g., outside the electronic device) have been illustrated and explained, but the embodiments are not limited thereto. For example, in some embodiments, the electronic device may obtain relevant information and a list of content from an external source opposite to the above and directly identify recommended content by performing an action such as that shown in Figure 9.

[0180] As described above, the control method according to the embodiment of the present disclosure can be analyzed into a first genre among close-up images, crowd images, and graphic images, but does not analyze a screen that does not have the characteristics of the first genre into another genre, and in the case described above, analyzes the final genre by additionally considering the previously analyzed genre, thereby enabling more accurate and consistent genre analysis and, accordingly, more accurate and consistent image quality processing.

[0181] According to an embodiment, methods according to at least some of the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0182] In addition, methods according to at least some of the various embodiments of the present disclosure described above may be implemented by software upgrades or hardware upgrades alone for existing electronic devices.

[0183] In addition, methods according to at least some of the various embodiments of the present disclosure described above may also be performed through an embedded server equipped in an electronic device, or through at least one external server among the electronic devices.

[0184] Meanwhile, according to one embodiment of the present disclosure, the various embodiments described above may be implemented as software containing instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions. When instructions are executed by a processor, the processor may perform a function corresponding to the instructions directly or by using other components under the control of the processor. Instructions may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily on the storage medium. For example, a 'non-transient storage medium' may include a buffer in which data is temporarily stored. According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store).TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., terminal devices). For online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0185] Various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include an electronic device (e.g., an electronic device (100)) according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0186] When the above-described instruction is executed by a processor, the processor may perform the function corresponding to the above-described instruction directly or by using other components under the control of the above-described processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0187] Although some embodiments of the present disclosure have been illustrated and described above, it should be understood that such embodiments are illustrative and not limiting. Those skilled in the art will understand that various modifications to the form and details are possible without departing from the true spirit and full scope of the present disclosure, including the appended claims and their equivalents.

Claims

1. In an electronic device, Communication interface; At least one processor; and Includes memory that stores at least one instruction, When the above at least one instruction is executed by the above at least one processor, the electronic device, Acquire an image using the above communication interface, and The above image is provided as input to a neural network model to obtain probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres, and Based on the above-mentioned probability information and previous genre information corresponding to the previous frame, current genre information corresponding to the current image is determined, and An electronic device that performs image quality processing corresponding to the genre information for the above current image.

2. In Paragraph 1, The above neural network model is, It is trained to output probability values ​​for each of the above first classification, the above second classification, and the above third classification, and The above first classification is associated with a first feature corresponding to the above first genre, and The above second classification is associated with the above first feature and is not associated with the second feature, The above third classification is an electronic device not associated with the above first feature and the above second feature.

3. In Paragraph 2, The above-mentioned first genre includes the sports genre, and The above second classification is an electronic device associated with at least one of a close-up image of a specific person, a crowd image of multiple people clustered together, and a data graphic image.

4. In Paragraph 1, The above neural network model is trained to classify sports events, and The above second classification is an electronic device associated with different video conditions corresponding to multiple types of sports.

5. In Paragraph 1, When the above at least one instruction is executed by the above at least one processor, the electronic device, Obtain additional graphic information from the above video, and An electronic device that determines genre information corresponding to the current image by additionally considering a determination of whether the additional graphic information was obtained from a previous frame and a determination of whether the additional graphic information was obtained from the current image.

6. In Paragraph 5, The above additional graphic information is an electronic device comprising at least one of a broadcast station logo, a sports federation logo, and score status information at a preset location of the above image.

7. In Paragraph 1, When the above at least one instruction is executed by the above at least one processor, the electronic device, Acquire audio information corresponding to the above video, and Based on the above audio information, audio characteristic information is obtained, and An electronic device that determines genre information corresponding to the current image by additionally considering previous audio characteristic information corresponding to the previous frame and current audio characteristic information corresponding to the current image.

8. In Paragraph 1, When the above at least one instruction is executed by the above at least one processor, the electronic device, Based on probability information indicating that the first probability value corresponding to the first classification is greater than or equal to a predetermined value, the genre of the current video is determined as the first genre, and An electronic device that maintains the genre of the current image as the previous genre corresponding to the previous frame based on probability information indicating that the first probability value is smaller than the predetermined value and the second probability value corresponding to the second classification is the highest probability value included in the probability information.

9. In Paragraph 1, When the above at least one instruction is executed by the above at least one processor, the electronic device, An electronic device that determines the current genre information based on the mode among the genre corresponding to a preset number of frames and the genre corresponding to the highest exchange rate value among the probability information obtained above.

10. In Paragraph 1, When the above at least one instruction is executed by the above at least one processor, the electronic device, An electronic device that controls the display to display the processed image.

11. In a method for controlling an electronic device, A step of providing an image as input to a neural network model to obtain probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres; A step of determining current genre information corresponding to the current image based on the above-mentioned acquired probability information and previous genre information corresponding to the previous frame; and A control method comprising the step of performing image quality processing corresponding to the genre information on the current image.

12. In Paragraph 11, The above neural network model is, It is trained to output probability values ​​for each of the above first classification, the above second classification, and the above third classification, and The above first classification is associated with a first feature corresponding to the above first genre, and The above second classification is associated with the above first feature and is not associated with the second feature, The above third classification is a control method not associated with the above first feature and the above second feature.

13. In Paragraph 12, The above-mentioned first genre includes the sports genre, and The above second classification is a control method associated with at least one of a close-up image of a specific person, a crowd image of multiple people clustered together, and a data graphic image.

14. In Paragraph 11, The method further includes the step of obtaining additional graphic information from the above image; and The above genre information is, A control method that makes a decision by additionally considering whether the additional graphic information was obtained from a previous frame and whether the additional graphic information was obtained from the current image.

15. A non-transient computer-readable recording medium storing a program for executing a method of controlling an electronic device, The above control method is, A step of providing an image as input to a neural network model to obtain probability information for a plurality of classifications, including a first classification corresponding to a first genre and a second classification corresponding to a plurality of genres; A step of determining current genre information corresponding to the current image based on the above-mentioned acquired probability information and previous genre information corresponding to the previous frame; and A computer-readable recording medium comprising: a step of performing image quality processing corresponding to the genre information on the current image.

Images