Electronic device and image processing method using the same

Abstract

According to one embodiment of this disclosure, an electronic device may include a processor and a memory for storing instructions. According to one embodiment, the instructions, when executed by the processor, may cause the electronic device to: identify similarity among a plurality of pixels constituting an image; based on the identified similarity among the plurality of pixels, control an artificial intelligence model driving module to apply a plurality of filters related to image quality improvement of the image to at least some pixels among the plurality of pixels that satisfy a specified similarity condition; and control the artificial intelligence model driving module to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition. Various other embodiments may exist besides those disclosed in this document.

Description

Technical Field

[0001] Embodiments of this disclosure relate to an electronic device and an image processing method using the electronic device. Background Technology

[0002] Electronic devices are becoming increasingly thinner, and their designs are constantly being improved to enhance their appearance and highlight the differentiation of functional elements. Electronic devices are increasingly taking on diverse forms, moving away from the monotonous rectangular design. For example, in recent years, electronic devices with large displays (such as foldable, rollable, or stretchable displays) have become widely popular.

[0003] When an image is output through a large-screen display, an electronic device can upscale a low-frame-resolution image to give it a higher frame (e.g., target frame) resolution before outputting the image through the display.

[0004] The above information is provided as related technology to assist in understanding the contents of this disclosure. No description above is intended to imply or identify it as prior art applicable to this disclosure. Summary of the Invention

[0005] Technical issues When an upscaled image is output to a large display, only the resolution of the image is increased, and therefore, the quality of the upscaled image may be lower.

[0006] Based on the similarity between multiple pixels included in an image, an electronic device according to embodiments of the present disclosure can perform image processing on at least some of the multiple pixels that satisfy a specified similarity condition, and can restrict the performance of image processing on at least some other pixels that do not satisfy the similarity condition.

[0007] An electronic device according to embodiments of the present disclosure may include an artificial intelligence model trained for image processing related to image quality improvement. The electronic device may, based on state information of the electronic device, partially activate multiple filters related to image quality improvement by using the artificial intelligence model. The electronic device may perform image processing on at least some pixels among a plurality of pixels included in an image that satisfy a specified similarity condition by using the partially activated filters.

[0008] Technical solution According to embodiments of this disclosure, an electronic device may include a processor and a memory storing instructions. According to one embodiment, when executed by the processor, the instructions enable the electronic device to identify the similarity between a plurality of pixels included in an image. According to another embodiment, when executed by the processor, the instructions enable the electronic device to control an artificial intelligence model-driven module to apply a plurality of filters related to image quality improvement to at least some pixels among the plurality of pixels that satisfy a specified similarity condition, based on the identified similarity between the plurality of pixels. According to yet another embodiment, when executed by the processor, the instructions enable the electronic device to control the artificial intelligence model-driven module to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition.

[0009] According to embodiments of this disclosure, an image processing method for an electronic device may include an operation of identifying the similarity between a plurality of pixels included in an image. According to an embodiment, the image processing method for the electronic device may include, based on the identified similarity between the plurality of pixels, controlling an artificial intelligence model driving module to apply a plurality of filters related to image quality improvement to at least some pixels among the plurality of pixels that satisfy a specified similarity condition. According to an embodiment, the image processing method for the electronic device may include controlling the artificial intelligence model driving module to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition.

[0010] According to embodiments of this disclosure, a non-transitory computer-readable storage medium (or computer program product) storing one or more programs is described. One or more programs according to embodiments may include instructions for identifying similarity between a plurality of pixels included in an image when executed by a processor of an electronic device. One or more programs according to embodiments may include instructions for controlling an artificial intelligence model driving module to apply a plurality of filters related to image quality improvement to at least some pixels among the plurality of pixels that satisfy a specified similarity condition, based on the identified similarity between the plurality of pixels when executed by a processor of an electronic device. One or more programs according to embodiments may include instructions for controlling an artificial intelligence model driving module to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition, when executed by a processor of an electronic device.

[0011] Beneficial effects of the present invention The electronic device according to embodiments of the present disclosure can not only output clear images on large displays, but also reduce current consumption by performing image processing only on at least some of the pixels that require image processing among the plurality of pixels included in the image.

[0012] An electronic device according to an embodiment of the present disclosure can use an artificial intelligence model to perform image processing on at least some pixels in an image that satisfy a specified similarity condition, based on the state information of the electronic device and using partially activated filters among a plurality of filters related to image quality improvement, thereby outputting a high-quality image in which predetermined regions (e.g., point regions and / or edge regions) in the image are clearly displayed. Attached Figure Description

[0013] Figure 1 This is a block diagram of an electronic device in a network environment according to embodiments of the present disclosure.

[0014] Figure 2 This is a block diagram of an electronic device according to an embodiment of the present disclosure.

[0015] Figure 3 This is a flowchart illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0016] Figure 4 This is a diagram illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0017] Figure 5 This is a diagram illustrating a method for identifying the similarity between multiple pixels included in an image according to an embodiment of the present disclosure.

[0018] Figure 6 This is a diagram illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0019] Figure 7 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure.

[0020] Figure 8 This is a diagram illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure.

[0021] Figure 9 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure.

[0022] Figure 10 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure.

[0023] Figure 11 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure.

[0024] Figure 12 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device, according to an embodiment of the present disclosure. Detailed Implementation

[0025] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement this disclosure. However, this disclosure can be implemented in various different forms and is not limited to the embodiments set forth herein. Regarding the description of the drawings, the same or similar reference numerals may be used to denote the same or similar elements. Furthermore, for clarity and brevity, descriptions of well-known functions and configurations may be omitted in the drawings and related descriptions.

[0026] Figure 1 This is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0027] Reference Figure 1 In network environment 100, electronic device 101 can communicate with electronic device 102 via a first network 198 (e.g., a short-range wireless communication network), or with at least one of electronic device 104 or server 108 via a second network 199 (e.g., a long-range wireless communication network). According to an embodiment, electronic device 101 can communicate with electronic device 104 via server 108. According to an embodiment, electronic device 101 may include a processor 120, memory 130, input module 150, sound output module 155, display module 160, audio module 170, sensor module 176, interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, user identification module (SIM) 196, or antenna module 197. In some embodiments, at least one of the above components (e.g., connection terminal 178) may be omitted from electronic device 101, or one or more other components may be added to electronic device 101. In some embodiments, some of the components described above (e.g., sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (e.g., display module 160).

[0028] Processor 120 may run software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of electronic device 101 connected to processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, processor 120 may store commands or data received from another component (e.g., sensor module 176 or communication module 190) in volatile memory 132, process the commands or data stored in volatile memory 132, and store the resulting data in non-volatile memory 134. According to embodiments, processor 120 may include a main processor 121 (e.g., central processing unit (CPU) or application processor (AP)) or an auxiliary processor 123 (e.g., graphics processing unit (GPU), neural processing unit (NPU), image signal processor (ISP), sensor central processor, or communication processor (CP)) that is operationally independent of or combined with the main processor 121. For example, when electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or may be implemented as part of the main processor 121.

[0029] When the main processor 121 is inactive (e.g., in sleep) state, the auxiliary processor 123 (rather than the main processor 121) can control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190), or when the main processor 121 is active (e.g., running an application), the auxiliary processor 123 can work with the main processor 121 to control at least some of the functions or states associated with at least one component of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190). According to embodiments, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 180 or communication module 190) functionally associated with the auxiliary processor 123. According to embodiments, the auxiliary processor 123 (e.g., a neural processing unit) may include hardware architecture dedicated to artificial intelligence model processing. Artificial intelligence models can be generated through machine learning. For example, such learning can be performed via electronic device 101 where artificial intelligence is performed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple layers of artificial neural networks. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q-network, or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include software structures in addition to hardware structures.

[0030] Memory 130 may store various data used by at least one component of electronic device 101 (e.g., processor 120 or sensor module 176). The various data may include, for example, software (e.g., program 140) and input or output data for commands associated with it. Memory 130 may include volatile memory 132 or non-volatile memory 134.

[0031] The program 140 may be stored as software in the memory 130, and the program 140 may include, for example, an operating system (OS) 142, middleware 144, or application 146.

[0032] The input module 150 can receive commands or data from outside the electronic device 101 (e.g., a user) that will be used by other components of the electronic device 101 (e.g., processor 120). The input module 150 may include, for example, a microphone, mouse, keyboard, keys (e.g., buttons), or digital pen (e.g., stylus).

[0033] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. According to an embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0034] Display module 160 can visually provide information to the outside of electronic device 101 (e.g., to a user). Display device 160 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. According to an embodiment, display module 160 may include a touch sensor adapted to detect touch or a pressure sensor adapted to measure the intensity of the force caused by touch.

[0035] The audio module 170 can convert sound into electrical signals and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or headphones of an external electronic device (e.g., electronic device 102) that is directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0036] Sensor module 176 can detect the operating state of electronic device 101 (e.g., power or temperature) or the environmental state outside electronic device 101 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. According to embodiments, sensor module 176 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0037] Interface 177 may support one or more specific protocols used to enable electronic device 101 to connect directly (e.g., wired) or wirelessly to external electronic devices (e.g., electronic device 102). According to embodiments, interface 177 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0038] 8] Connection end 178 may include a connector, wherein electronic device 101 may be physically connected to an external electronic device (e.g., electronic device 102) via the connector. According to an embodiment, connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0039] The haptic module 179 can convert electrical signals into mechanical stimuli (e.g., vibration or motion) or electrical stimuli that can be recognized by a user through his touch or kinesthesia. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0040] Camera module 180 can capture still or moving images. According to an embodiment, camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0041] The power management module 188 manages the power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0042] Battery 189 can power at least one component of electronic device 101. According to an embodiment, battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable rechargeable battery, or a fuel cell.

[0043] Communication module 190 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 101 and external electronic devices (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. Communication module 190 may include one or more communication processors capable of operating independently of processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) or wireless communication. According to embodiments, communication module 190 may include wireless communication module 192 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 194 (e.g., local area network (LAN) communication module or power line communication (PLC) module). One of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wi-Fi Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules can be implemented as a single component (e.g., a single chip) or as multiple components separate from each other (e.g., multiple chips). The wireless communication module 192 can identify and verify the electronic device 101 in the communication network (such as the first network 198 or the second network 199) using user information (e.g., the International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0044] Wireless communication module 192 can support 5G networks following 4G networks and next-generation communication technologies (such as new radio (NR) access technologies). NR access technologies can support enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), or ultra-reliable low-latency communication (URLLC). Wireless communication module 192 can support high-frequency bands (e.g., millimeter-wave bands) to achieve, for example, high data transmission rates. Wireless communication module 192 can support various technologies used to ensure performance in high-frequency bands, such as, for example, beamforming, massive MIMO, full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. Wireless communication module 192 can support various requirements specified in electronic device 101, external electronic devices (e.g., electronic device 104), or network systems (e.g., second network 199). According to an embodiment, the wireless communication module 192 may support peak data rates (e.g., 20 Gbps or greater) for implementing eMBB, lost coverage (e.g., 164 dB or less) for implementing mMTC, or U-plane latency (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip) for implementing URLLC.

[0045] Antenna module 197 can transmit or receive signals or power to or from the exterior of electronic device 101 (e.g., external electronic device). According to an embodiment, antenna module 197 may include an antenna comprising a radiating element formed of a conductive material or conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, antenna module 197 may include multiple antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as a first network 198 or a second network 199) can be selected from the multiple antennas by, for example, communication module 190 (e.g., wireless communication module 192). Signals or power can then be transmitted or received between communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, additional components besides the radiating element (e.g., a radio frequency integrated circuit (RFIC)) may be additionally incorporated into antenna module 197.

[0046] According to various embodiments, antenna module 197 may form a millimeter-wave antenna module. According to embodiments, the millimeter-wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and multiple antennas (e.g., an array antenna), wherein the RFIC is disposed on or adjacent to a first surface (e.g., a bottom surface) of the printed circuit board and is capable of supporting a specified high-frequency band (e.g., a millimeter-wave band), and the multiple antennas are disposed on or adjacent to a second surface (e.g., a top surface or a side surface) of the printed circuit board and are capable of transmitting or receiving signals in the specified high-frequency band.

[0047] At least some of the aforementioned components can be interconnected and communicate signals (e.g., commands or data) between them via an inter-peripheral communication scheme (e.g., bus, general purpose input / output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0048] According to an embodiment, commands or data can be sent or received between electronic device 101 and external electronic device 104 via server 108 connected to a second network 199. Each of electronic device 102 or electronic device 104 can be a device of the same type as electronic device 101, or a device of a different type. According to an embodiment, all or some operations that would be performed on electronic device 101 can be performed on one or more of external electronic devices 102, external electronic devices 104, or server 108. For example, if electronic device 101 is required to automatically perform a function or service, or is required to perform a function or service in response to a request from a user or another device, electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service, instead of running the function or service, or electronic device 101 may request the one or more external electronic devices to perform at least a portion of the function or service in addition to running the function or service. Upon receiving the request, one or more external electronic devices may perform at least a portion of the requested function or service, or perform additional functions or services related to the request, and transmit the result of the execution to electronic device 101. Electronic device 101 may provide the result as at least a partial response to the request, with or without further processing of the result. For this purpose, technologies such as cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing may be used. Electronic device 101 may use, for example, distributed computing or mobile edge computing to provide ultra-low latency services. In another embodiment, external electronic device 104 may include an Internet of Things (IoT) device. Server 108 may be an intelligent server using machine learning and / or neural networks. According to embodiments, external electronic device 104 or server 108 may be included in a second network 199. Electronic device 101 may be applied to intelligent services based on 5G communication technology or IoT-related technologies (e.g., smart homes, smart cities, smart cars, or healthcare).

[0049] Figure 2 This is a block diagram of an electronic device 101 according to an embodiment of the present disclosure.

[0050] Reference Figure 2 Electronic device 101 (e.g., Figure 1 The electronic device 101 in the middle may include a communication circuit 210 (e.g., Figure 1 The communication module 190 and the memory 220 (e.g., Figure 1 The memory 130 and the display 230 (e.g., in the memory 130) are shown in the Figure 1 The display module 160 and / or processor 240 (e.g., Figure 1The processor 120 in the middle.

[0051] According to embodiments of this disclosure, the communication circuit 210 (e.g., Figure 1 The communication module 190 in the processor 240 can control the electronic device 101 and at least one external electronic device (e.g., under the control of the processor 240) to communicate with the electronic device 101. Figure 1 Electronic devices 102 and 104 (and / or servers, for example, Figure 1 The communication connection between server 108 in the middle.

[0052] According to embodiments of this disclosure, memory 220 (e.g., Figure 1 The memory 130 in the memory can execute programs stored in the processor 240 for processing and controlling the electronic device 101 (e.g., Figure 1 Program 140), Operating System (OS) (e.g., Figure 1 The memory 142 contains the operating system 142, various application and / or input / output data functions, and can store programs for controlling the overall operation of the electronic device 101. The memory 220 can store various configuration information required when the electronic device 101 processes functions related to the various embodiments of this disclosure.

[0053] In an embodiment, memory 220 may store instructions for magnifying an image having a first resolution (e.g., low resolution) to a second resolution (e.g., target resolution or high resolution) higher than the first resolution. Memory 220 may store instructions for identifying the similarity between a plurality of pixels included in an image and for detecting (or analyzing) edge regions in the image. Memory 220 may store instructions for performing image processing on at least some pixels among the plurality of pixels that satisfy a specified similarity condition. Memory 220 may store instructions for limiting image processing to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition. Memory 220 may store instructions for selecting at least some filters for image processing from a plurality of filters related to image quality improvement based on state information of electronic device 101. Memory 220 may store instructions for performing image processing on at least some pixels among the plurality of pixels that satisfy the specified similarity condition using the selected at least some filters.

[0054] In one embodiment, memory 220 may include an artificial intelligence (AI) model 221. In another embodiment, operations based on AI model 221 may be performed in processor 240.

[0055] According to embodiments of this disclosure, the display 230 (e.g., Figure 1The display module 160 can display images under the control of the processor 240, and can be implemented using any of the following: liquid crystal display (LCD), light-emitting diode (LED) display, micro LED (μLED) display, organic light-emitting diode (OLED) display, active matrix organic light-emitting diode (AMOLED) display, microelectromechanical system (MEMS) display, electronic paper display, flexible display, foldable display, or rollable display. However, it is not limited to these.

[0056] In one embodiment, the display 230 can display an image processed by the processor 240 under the control of the processor 240.

[0057] According to embodiments of this disclosure, processor 240 may include, for example, a microcontroller unit (MCU), and can control multiple hardware components connected to processor 240 by driving an operating system (OS) or embedded software program. Processor 240 can, for example, operate according to instructions stored in memory 220 (e.g., Figure 1 The program 140 in the middle is used to control multiple hardware components.

[0058] In embodiments, processor 240 may include at least one component (or module) for operation according to embodiments of the present disclosure. For example, processor 240 may include at least one functional unit such as a magnification module 241, an image analysis module 243, an artificial intelligence model driving module 245, and / or an image merging module 247. According to embodiments, at least some of the functional units may be implemented as hardware modules (e.g., circuits) included in processor 240 and / or implemented as software including one or more instructions executable by processor 240. For example, operations performed by processor 240 may be performed by instructions stored in memory 220, and when executed, cause processor 240 to operate.

[0059] In an embodiment, the magnification module 241 of the processor 240 can magnify an image having a first resolution (e.g., low resolution) to a second resolution (e.g., target resolution or high resolution) higher than the first resolution. The image analysis module 243 of the processor 240 can identify the similarity between multiple pixels included in the image magnified by the magnification module 241. Identifying the similarity between multiple pixels included in an image can be an operation of detecting (or analyzing) edge regions in the image. For example, the image analysis module 243 of the processor 240 can detect (or analyze) edge regions in the image by using a specified filter. The artificial intelligence model driving module 245 of the processor 240 can perform image processing on at least some pixels among the multiple pixels that satisfy a specified similarity condition based on the similarity between the multiple pixels identified by the image analysis module 243. For example, the artificial intelligence model driving module 245 can apply multiple filters (e.g., multiple convolutional layers) related to image quality improvement to at least some pixels among the multiple pixels that satisfy the specified similarity condition. The artificial intelligence model driving module 245 of the processor 240 can restrict image processing to at least some other pixels among the multiple pixels that do not satisfy the specified similarity condition. For example, the AI ​​model driving module 245 may not apply (e.g., bypass) multiple filters (e.g., multiple convolutional layers) related to image quality improvement to at least some pixels among multiple pixels that satisfy a specified similarity condition. In an embodiment, at least some pixels among multiple pixels that satisfy the specified similarity condition may be pixels corresponding to (or belonging to) edge regions. At least some pixels among multiple pixels that do not satisfy the specified similarity condition may be pixels that do not correspond to (or belong to) edge regions.

[0060] In an embodiment, the artificial intelligence model driving module 245 of the processor 240 can select at least some filters for image processing from a plurality of filters related to image quality improvement based on the state information of the electronic device 101. In an embodiment, the state information of the electronic device 101 may include information related to the battery of the electronic device 101 (e.g., ...). Figure 1 Information related to the remaining capacity of the battery 189 in the device, information related to the characteristics of the application currently displayed (or requested to be displayed (or output)) on the display 230 of the electronic device 101 (e.g., display characteristics (e.g., resolution)), information related to the operating state of the electronic device 101 (e.g., power saving state (or low power state)), information related to the detection of the user's gaze on the electronic device 101, and / or information related to the detection of user input.

[0061] In one embodiment, based on information related to the remaining capacity of battery 189, when it is identified that the remaining capacity of battery 189 of electronic device 101 is less than or equal to a specified capacity, the artificial intelligence model driving module 245 of processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. However, this is not the only possibility. For example, based on information related to the characteristics of the application, when it is identified that the application does not require high image quality, the artificial intelligence model driving module 245 of processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the operating state of electronic device 101, when it is identified that electronic device 101 is operating in a power-saving state (or low-power state), the artificial intelligence model driving module 245 of processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user gaze on electronic device 101, when it is identified that a state without detected user gaze exceeds a specified time, the artificial intelligence model driving module 245 of processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user input, when it is identified that the screen scrolling speed exceeds a specified speed, the artificial intelligence model driving module 245 of the processor 240 can select a first number of filters from a plurality of filters related to image quality improvement.

[0062] In one embodiment, based on information related to the remaining capacity of battery 189, when it is identified that the remaining capacity of battery 189 of electronic device 101 exceeds a specified capacity, the artificial intelligence model driving module 245 of processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. However, this is not the only possibility. For example, based on information related to the characteristics of an application, when it is identified that the application requires high image quality, the artificial intelligence model driving module 245 of processor 240 can select a second number of filters from a plurality of filters related to image quality enhancement. As another example, based on information related to the operating state of electronic device 101, when it is identified that electronic device 101 is not operating in a power-saving state (or low-power state), the artificial intelligence model driving module 245 of processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user gaze on electronic device 101, when it is identified that the state of no detected user gaze is less than or equal to a specified time, the artificial intelligence model driving module 245 of processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user input, when the screen scrolling speed is identified to be less than a specified speed, the artificial intelligence model driving module 245 of the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. In an embodiment, the second number may be greater than the first number, or equal to the number of filters.

[0063] In an embodiment, the artificial intelligence model driving module 245 of the processor 240 can perform image processing on at least some pixels among a plurality of pixels that satisfy a specified similarity condition by using a selected first number of filters or a second number of filters (e.g., applying the selected first number of filters or the second number of filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition).

[0064] In an embodiment, the image merging module 247 of the processor 240 can assign weight values ​​to the feature map generated by the artificial intelligence model driving module 245 (e.g., a feature map based on at least some pixels among a plurality of pixels to which multiple filters have been applied and at least some other pixels among a plurality of pixels to which multiple filters have not been applied). The image merging module 247 of the processor 240 can merge the weighted feature map with the magnified image to generate a second image. The processor 240 can output (or display) the generated second image on the display 230.

[0065] An electronic device 101 according to an embodiment of the present disclosure may include a processor 240 and a memory 220 storing instructions. The instructions, when executed by the processor 240, may cause the electronic device 101 to identify the similarity between a plurality of pixels included in an image. The instructions, when executed by the processor 240, may cause the electronic device 101 to control an artificial intelligence model driving module 245 to apply a plurality of filters related to image quality improvement to at least some pixels among the plurality of pixels that satisfy a specified similarity condition, based on the identified similarity between the plurality of pixels included in the image. The instructions, when executed by the processor 240, may cause the electronic device 101 to control the artificial intelligence model driving module 245 to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels included in the image that do not satisfy the specified similarity condition.

[0066] In this embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be pixels corresponding to edge regions. At least some other pixels among a plurality of pixels that do not satisfy the specified similarity condition may be pixels that do not correspond to edge regions.

[0067] The plurality of filters according to embodiments may include a plurality of first filters and a plurality of second filters. The plurality of first filters according to embodiments may include filters related to image manipulation. The plurality of second filters according to embodiments may include filters related to image quality.

[0068] When executed by processor 240, the instructions according to the embodiment can cause electronic device 101 to select at least some filters from a plurality of filters based on the state information of electronic device 101. When executed by processor 240, the instructions according to the embodiment can cause electronic device 101 to apply the selected at least some filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition.

[0069] The status information of the electronic device 101 according to the embodiment may include at least one of the following: information related to the remaining capacity of the battery 189 of the electronic device 101, information related to the characteristics of the application displayed on the display 230, information related to the operating status of the electronic device 101, information related to the detection of the user's gaze on the electronic device 101, and information related to the detection of user input.

[0070] When executed by the processor 240, the instructions according to the embodiment can cause the electronic device 101 to identify whether the state information of the electronic device 101 meets a specified condition. When executed by the processor 240, the instructions according to the embodiment can cause the electronic device 101 to control the artificial intelligence model driving module 245 to select a first number of filters from a plurality of filters and apply the selected first number of filters to at least some pixels among the plurality of pixels that meet a specified similarity condition when the state information of the electronic device 101 meets the specified condition when the instructions according to the embodiment can control the artificial intelligence model driving module 245 to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity condition when the instructions according to the embodiment are executed by the processor 240. When executed by the processor 240, the instructions according to the embodiment can cause the electronic device 101 to select a second number of filters from a plurality of filters when the state information of the electronic device 101 does not meet the specified condition when the instructions according to the embodiment are executed by the processor 240. When executed by the processor 240, the instructions according to the embodiment can cause the electronic device 101 to control the artificial intelligence model driving module 245 to apply the selected second number of filters to at least some pixels among the plurality of pixels that meet the specified similarity condition.

[0071] The state information of the electronic device 101 according to the embodiment may satisfy at least one of the following conditions: based on information related to the remaining capacity of the battery 189, identifying that the remaining capacity of the battery 189 of the electronic device 101 is less than or equal to a specified capacity; based on information related to the characteristics of the application, identifying that the application is an application that does not require high image quality; based on information related to the operating state of the electronic device 101, identifying that the electronic device 101 is operating in a power-saving state; based on information related to the detection of the user's gaze on the electronic device 101, identifying that the state in which the user's gaze is detected is less than or equal to a specified time; or based on information related to the detection of user input, identifying that the screen scrolling speed exceeds a specified speed.

[0072] The situation where the state information of the electronic device 101 according to the embodiment does not meet the specified conditions may include at least one of the following: identifying a situation where the remaining capacity of the battery 189 exceeds a specified capacity based on information related to the remaining capacity of the battery 189 of the electronic device 101; identifying a situation where the application requires high image quality based on information related to the characteristics of the application; identifying a situation where the electronic device 101 is not operating in a power-saving state based on information related to the operating state of the electronic device 101; identifying a situation where the state in which the user's gaze is detected exceeds a specified time based on information related to the detection of the user's gaze on the electronic device 101; and identifying a situation where the screen scrolling speed is less than or equal to a specified speed based on information related to the detection of user input.

[0073] The instructions according to the embodiment, when executed by the processor 240, can, based on information related to the detection of user input, cause the electronic device 101 to detect edge regions in the scaled-down or scaled-up screen when user input for scaling down or scaling up the screen is detected. The instructions according to the embodiment, when executed by the processor 240, can cause the electronic device 101 to select a first number of filters from a plurality of filters when the proportion of the detected edge region does not exceed a specified proportion, and apply the selected first number of filters to at least some pixels among the plurality of pixels that satisfy a specified similarity condition. The instructions according to the embodiment, when executed by the processor 240, can cause the electronic device 101 to select a second number of filters from a plurality of filters when the proportion of the detected edge region exceeds a specified proportion, and apply the selected second number of filters to at least some pixels among the plurality of pixels that satisfy a specified similarity condition.

[0074] According to the embodiment, the second quantity can be greater than the first quantity, or equal to the number of multiple filters.

[0075] The electronic device 101 according to an embodiment may further include a display 230. The image according to an embodiment may include a magnified image. Instructions according to an embodiment, when executed by the processor 240, may cause the electronic device 101 to merge the magnified image with a feature map to generate a second image, the feature map being based on at least some pixels to which multiple filters are applied and at least some other pixels, wherein the application of multiple filters to the at least one other pixel is limited. Instructions according to an embodiment, when executed by the processor 240, may cause the electronic device 101 to display the generated second image on the display 230.

[0076] The image according to the embodiment may include an unenlarged image. Instructions according to the embodiment, when executed by processor 240, may cause electronic device 101 to merge the image with a feature map to generate a second image, the feature map being based on at least some pixels to which multiple filters are applied and at least some other pixels, wherein the application of multiple filters to the at least one other pixel is limited. Instructions according to the embodiment, when executed by processor 240, may cause electronic device 101 to enlarge the generated second image. Instructions according to the embodiment, when executed by processor 240, may cause electronic device 101 to display the enlarged second image on display 230.

[0077] Figure 3 This is a flowchart illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0078] In the following embodiments, Figure 3 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 3 The order of operations can be changed, and at least two operations can be executed in parallel.

[0079] According to an embodiment, Figure 3 Operations 305 to 315 in the above can be understood as being performed by an electronic device (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0080] Reference Figure 3 In operation 305, processor 240 can identify the similarity between multiple pixels included in an image.

[0081] In an embodiment, the image may be an enlarged image. For example, processor 240 (e.g., Figure 2 The magnification module 241 can magnify an image with a first resolution (e.g., low resolution) to a second resolution (e.g., target resolution or high resolution) that is higher than the first resolution. For example, the processor 240 (e.g., magnification module 241) can use interpolation methods such as bilinear, nearest neighbor, or bicubic to magnify an image with a first resolution (e.g., low resolution) to a second resolution (e.g., target resolution or high resolution) that is higher than the first resolution.

[0082] Not limited to this, and the image can be an unenlarged image. In this case, the processor 240 (e.g., Figure 2 The magnification module 241 in the image processing unit can perform the operation of magnifying the image after the operation 315 described below. Alternatively, the image can be a magnified image, and the processor 240 (e.g., Figure 2The magnification module 241 in the image can perform the operation of magnifying the image after the image processing operation 315 described below.

[0083] In an embodiment, processor 240 (e.g., Figure 2 The image analysis module 243 in the image can analyze an image and identify the similarity between multiple pixels. According to an embodiment, identifying the similarity between multiple pixels included in an image can be an operation of detecting (or analyzing) edge regions in the image. For example, the processor 240 (e.g., the image analysis module 243) can use a specified filter to detect (or analyze) edge regions in the image. For example, the processor 240 (e.g., the image analysis module 243) can configure one pixel among multiple pixels included in the image as a first reference pixel, identify the similarity between multiple first pixels adjacent to the first reference pixel, and detect edge regions based on the similarity. The processor 240 (e.g., the image analysis module 243) can repeatedly perform this operation. For example, the reference pixel can be changed, and the processor 240 (e.g., the image analysis module 243) can identify the similarity between multiple first pixels adjacent to the first reference pixel, and then configure a pixel located at a predetermined distance from the first reference pixel as a second reference pixel. The processor 240 (e.g., the image analysis module 243) can identify the similarity between multiple second pixels adjacent to the second reference pixel and detect edge regions based on the similarity. Various embodiments will be referred to below in this regard. Figure 5 Describe it.

[0084] In an embodiment, processor 240 (e.g., Figure 2 In operation 310, the AI ​​model-driven module 245 can apply multiple filters (e.g., multiple convolutional layers) included in the AI ​​model-driven module 245, which are related to image quality improvement, to at least some pixels among the multiple pixels that satisfy a specified similarity condition, based on the similarity between the identified multiple pixels. In operation 315, the processor 240 (e.g., the AI ​​model-driven module 245) can restrict the application of the multiple filters to at least some other pixels among the multiple pixels that do not satisfy the specified similarity condition.

[0085] In this embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be pixels that correspond to (or belong to) an edge region. At least some other pixels among a plurality of pixels that do not satisfy the specified similarity condition may be pixels that do not correspond to (or belong to) an edge region.

[0086] In an embodiment, processor 240 (e.g., Figure 2The AI ​​model-driven module 245 can apply multiple filters related to image quality improvement to pixels corresponding to edge regions among multiple pixels, and can also avoid applying (e.g., bypassing) multiple filters related to image quality improvement to pixels that do not correspond to edge regions.

[0087] In an embodiment, although not shown, processor 240 may generate a feature map based on at least some pixels among a plurality of pixels to which multiple filters have been applied and at least some other pixels to which multiple filters have not been applied. Processor 240 (e.g., Figure 2 The image merging module 247 can assign weight values ​​to the feature maps generated by the artificial intelligence model driving module 245. For example, the weight value of the feature map of at least some other pixels among the multiple pixels that have not applied multiple filters can be "0". Therefore, the values ​​of at least some other pixels among the multiple pixels that have not applied multiple filters can be the same as the values ​​of at least some other pixels among the multiple pixels included in the image of operation 305. The processor 240 (e.g., the image merging module 247) can merge the feature maps with the assigned weight values ​​with the image of operation 305 and output it as a second image to a display (e.g., ...). Figure 2 (In the display 230). For example, when the image of operation 305 is an enlarged image, processor 240 (e.g., image merging module 247) can merge a feature map with assigned weight values ​​with the enlarged image of operation 305 to generate a second image. Processor 240 can output (or display) the generated second image on display 230. As another example, when the image of operation 305 is not an enlarged image, processor 240 (e.g., image merging module 247) can merge a feature map with assigned weight values ​​with the unenlarged image of operation 305 to generate a second image. Processor 240 (e.g., enlargement module 241) can enlarge the generated second image. Processor 240 can output (or display) the enlarged second image to display 230.

[0088] Figure 4 This is a diagram illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0089] Reference Figure 4 Electronic devices (e.g.) Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 may include an amplification module 241, an image analysis module 243, an artificial intelligence model driving module 245, and / or an image merging module 247.

[0090] In this embodiment, the magnification module 241 can magnify the first image 405. For example, the magnification module 241 can magnify the first image 405, which has a first resolution, to a second resolution, which has a higher resolution than the first resolution.

[0091] In an embodiment, the image analysis module 243 can analyze the magnified first image 410 to identify the similarity between multiple pixels included in the first image. For example, identifying the similarity between multiple pixels included in the first image can be an operation of detecting (or analyzing) edge regions in the first image. For example, the image analysis module 243 can generate a second image 415 by analyzing the edge regions in the magnified first image 410. For example, edge regions in the second image 415 can be represented in white, and non-edge regions can be represented in black. The artificial intelligence model driving module 245 can perform image processing on pixels 420 that correspond to edge regions among multiple pixels, and can avoid (e.g., bypass) performing image processing on pixels 425 that do not correspond to edge regions. For example, the artificial intelligence model driving module 245 can include multiple filters (e.g., multiple convolutional layers). Performing image processing on pixels 420 that correspond to the aforementioned edge regions can include applying multiple filters to pixels 420 that correspond to edge regions among multiple pixels. Not performing image processing on pixels 425 that do not correspond to the aforementioned edge regions can include not applying multiple filters to pixels 425 that do not correspond to edge regions.

[0092] In an embodiment, the image merging module 247 can generate a third image 430 by merging a magnified first image with feature maps of at least some pixels to which multiple filters are applied by the artificial intelligence model-driven module 245 and at least some other pixels to which multiple filters are not applied. The processor 240 can be displayed on a monitor (e.g., Figure 2 The generated third image 430 is displayed on the monitor 230.

[0093] According to various embodiments Figure 4 In this description, the magnification module 241 is positioned before the image analysis module 243, and it is described that the first image 405 is magnified and subsequently the magnified first image 410 is analyzed, but this is not a limitation. For example, the magnification module 241 may be positioned after the image merging module 247. In this case, the magnification module 241 can magnify the third image 430 generated by the image merging module 247. As another example, in addition to the magnification module 241, the electronic device 101 may also include a second magnification module (not shown) positioned after the image merging module 247. In this case, the second magnification module (not shown) can again magnify the third image 430 generated by the image merging module 247.

[0094] Figure 5This is a diagram illustrating a method for identifying the similarity between multiple pixels included in an image according to an embodiment of the present disclosure.

[0095] Reference Figure 5 Electronic devices (e.g.) Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the image analysis module may include an image analysis module (e.g., Figure 2 Image analysis module 243 (as described in the embodiment). Image analysis module 243 can analyze an image and identify the similarity between multiple pixels. The operation of identifying the similarity between multiple pixels included in an image, according to an embodiment, can be the operation of detecting (or analyzing) edge regions in the image. For example, image analysis module 243 can detect (or analyze) edge regions in the image by using a specified filter. In an embodiment, the specified filter may include a 7×7 edge detection filter 510. However, it is not limited to this.

[0096] In an embodiment, the image analysis module 243 can configure one of the multiple pixels included in the image as a first reference pixel 515 by using a 7×7 edge detection filter 510. x,y (x and y represent the position of the pixel in the image). The image analysis module 243 can be relative to the first reference pixel (Pixel) as the center. x,y The image analysis module 243 can apply a 7×7 edge detection filter 510 based on the first reference pixel 515. x,y The nth pixel out of a total of 49 pixels is configured as the second reference pixel 520. x+i,y+j Image analysis module 243 can identify a plurality of first pixels 525 adjacent to the second reference pixel 520 to detect similarity with the second reference pixel 520. For example, image analysis module 243 can identify pixels located after the second reference pixel 520 within a 3×3 pixel area relative to the second reference pixel 520 that have not undergone image analysis (excluding pixels located before the second reference pixel 520) as a plurality of first pixels 525 adjacent to the second reference pixel 520. Image analysis module 243 can perform image analysis operations relative to the plurality of first pixels 525 adjacent to the second reference pixel 520 in sequence from the top left pixel to the right.

[0097] The image analysis operation according to the embodiment can be performed on multiple pixels included in the image. For example, the image analysis module 243 can calculate a boundary value (e.g., a detection count) for identifying whether multiple pixels included in the image satisfy a specified similarity condition based on the following Equation 1. The boundary value may refer to the value of the pixel corresponding to the edge region among the multiple pixels included in the image.

[0098] Equation 1

[0099] In this embodiment, the image analysis module 243 can analyze the second reference pixel 520. x+i,y+j The second reference pixel 520 is compared sequentially with four adjacent pixels (530). When the difference between the second reference pixel 520 and one of the four adjacent pixels is less than or equal to a predetermined threshold, the image analysis module 243 can determine that the second reference pixel 520 is a similar (or identical) pixel to that pixel, and can increment the boundary value (e.g., detection count) by "1". In an embodiment, when the difference between the second reference pixel 520 and all four adjacent pixels is less than or equal to the predetermined threshold, the image analysis module 243 can determine that the second reference pixel 520 is a similar (or identical) pixel to all four adjacent pixels, and can increment the boundary value (e.g., detection count) by "4".

[0100] In the embodiments, it has been described that the image analysis module 243 compares the second reference pixel 520 sequentially with four adjacent pixels 530, but this disclosure is not limited thereto. For example, depending on the position of the reference pixel among the plurality of pixels included in the image, there may be no adjacent pixels (e.g., neighboring pixels). For example, when the second reference pixel 520 corresponds to the 49th pixel in the 7×7 edge detection filter 510 (e.g., the pixel located at the bottom right of the 7×7 edge detection filter 510), there may be no comparison target pixel in the 7×7 edge detection filter 510. For example, the number of adjacent pixels may be less than 4 depending on the position of the reference pixel among the plurality of pixels included in the image. For example, when the second reference pixel 520 is the first pixel in the 7×7 edge detection filter 510 (e.g., the top left pixel in the 7×7 edge detection filter 510), there may be three pixels in the 7×7 edge detection filter 510 (e.g., the pixel to the right of the second reference pixel, the pixel to the bottom right of the second reference pixel, and the pixel below the second reference pixel) as comparison target pixels.

[0101] In this embodiment, the image analysis module 243 can apply the 7×7 edge detection filter 510 to multiple pixels included in the image and repeat the operation for multiple pixels. Therefore, the image analysis module 243 can perform the operation 196 times (e.g., 4 times the number of adjacent pixels) on some pixels among the multiple pixels included in the image where the 7×7 edge detection filter 510 has been applied. 7 7) Comparison with a predetermined threshold. Based on this, when the boundary value (e.g., detection count) is "196", the image analysis module 243 can determine some pixels (e.g., including the first reference pixel) to which the 7×7 edge detection filter 510 has been applied. x,y The pixels in the 7×7 edge detection filter 510 that are 515 are similar (or identical) pixels. In other words, the image analysis module 243 can determine some pixels to which the 7×7 edge detection filter 510 is applied (e.g., including the first reference pixel 515). x,y Pixels in the 7×7 edge detection filter 510 that do not meet the specified similarity criteria may not require image processing (e.g., Figure 3 In operation 315). When the boundary value (e.g., the detection count) is less than "196", the image analysis module 243 can determine that some pixels (e.g., the first reference pixel) have been applied to the 7×7 edge detection filter 510. x,y There are points or edge regions around 515. In other words, the image analysis module 243 can determine some pixels (e.g., including the first reference pixel) to which the 7×7 edge detection filter 510 has been applied. x,y The pixels in the 7×7 edge detection filter 510 of 515 satisfy the specified similarity condition, and image processing (e.g., the above) is performed. Figure 3 Operation 310 in the process is used to clearly output the corresponding pattern (e.g., dots or edge areas).

[0102] In various embodiments, although it has been described that the image analysis module 243 performs 196 (e.g., 4 (e.g., the number of adjacent pixels) operations on some pixels in the image to which the 7×7 edge detection filter 510 is applied among the plurality of pixels included in the image. 7 7) Comparison with a predetermined threshold, but not limited to this. For example, as mentioned above, depending on the position of a reference pixel among the multiple pixels included in the image, there may be no adjacent pixels (e.g., neighboring pixels). In this case, the operation of comparing some pixels among the multiple pixels included in the image to which the 7×7 edge detection filter 510 is applied with the predetermined threshold can be performed 156 times.

[0103] According to various embodiments Figure 5 In this document, the boundary value (e.g., detection count) used as a standard for determining the presence of edge regions has been described as "196," but is not limited to this. For example, the boundary value can be adjusted according to user configuration, the internal configuration of electronic device 101, or other environments. As another example, the boundary value can be automatically changed according to an algorithm in electronic device 101, and the sensitivity used to detect edge regions can be modified.

[0104] In this embodiment, when the image analysis module 243 determines the edge region after completing the image analysis operation, the processor 240 can generate an edge map, and the artificial intelligence model driving module (e.g., Figure 2 The AI ​​model-driven module 245 can perform image processing sequentially based on an edge map. In an embodiment, the edge map can be a graph in which pixels in the image are represented by gray levels (e.g., values ​​in the range of 0 to 255). For example, the AI ​​model-driven module 245 may not perform image processing on pixels represented as 0 (e.g., black) (e.g., limiting the application of multiple image quality-related filters included in the AI ​​model-driven module 245), and may perform image processing on pixels represented as 255 (e.g., white) (e.g., applying multiple image quality-related filters included in the AI ​​model-driven module 245).

[0105] In one embodiment, the image analysis module 243 can adjust a predetermined threshold to adjust the sensitivity used for detecting edge regions.

[0106] Not limited to this, the image analysis module 243 can enable the artificial intelligence model-driven module 245 to partially activate multiple filters related to image quality improvement based on boundary values ​​(e.g., detection counts) to progressively perform image processing. For example, when the boundary value (e.g., detection count) of a predetermined pixel is a value within a first range (e.g., a value within the range of 0 to 49), the image analysis module 243 can identify the pixel as a pixel corresponding to a point or edge region, and can map the first value (e.g., 255) to the pixel corresponding to the point or edge region. The artificial intelligence model-driven module 245 can perform image processing on the pixel having the first value (e.g., 255). When the boundary value (e.g., detection count) of a predetermined pixel is a value within a second range (e.g., a value within the range of 50 to 98), the image analysis module 243 can map the second value (e.g., 239) to the predetermined pixel. The artificial intelligence model-driven module 245 can perform image processing on the predetermined pixel having the second value (e.g., 239) by using a filter that is activated among multiple filters.

[0107] Figure 6 This is a diagram illustrating a method for performing image processing according to an embodiment of the present disclosure.

[0108] Reference Figure 6 In Figure 610, the processor (e.g., Figure 2 The image analysis module of the processor 240 in the processor (e.g., Figure 2The image analysis module 243 in the first image 615 can generate a second image 625 as an image in which edge regions are detected, based on the similarity between multiple pixels included in the first image 615. (See reference...) Figure 6 As shown in Figure 650, the image analysis module 243 can generate a fourth image 660 as an image in which edge regions are detected, based on the similarity between multiple pixels included in the third image 655.

[0109] In an embodiment, the second image 625 and the fourth image 660, which are images in which edge regions are detected, can be images representing pixels to which image processing is performed and pixels to which no image processing is performed. For example, white areas can correspond to areas due to boundary values ​​(e.g., Figure 5 A pixel whose boundary value (e.g., detection count) is less than "196" corresponds to a point or edge region, and can be a region representing a pixel to which image processing is performed. A black region can correspond to a pixel whose boundary value (e.g., Figure 5 The boundary value (e.g., detection count) in the image is equal to "196" and has a similar (or identical) pixel value, and can be a region representing pixels to which no image processing is performed. In an embodiment, the artificial intelligence model driving module of processor 240 (e.g., Figure 2 The AI ​​model driving module 245 can perform image processing (e.g., apply multiple filters related to image quality included in the AI ​​model driving module 245) on pixels with boundary values ​​less than "196" (e.g., pixels corresponding to points or edge regions), and can, based on the second image 625 or the fourth image 660 generated by the image analysis module 243, not perform image processing (e.g., restrict the application of multiple filters related to image quality included in the AI ​​model driving module 245) on pixels with boundary values ​​of 196 (e.g., pixels with similar or identical pixel values).

[0110] According to various embodiments Figures 3 to 6 In this device, the electronic device 101 can not only output clear images on a large display, but also reduce current consumption by performing image processing only on at least some of the pixels that require image processing among the multiple pixels included in the image (e.g., pixels corresponding to points or edge regions).

[0111] Figure 7 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0112] In the following embodiments, Figure 7 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 7The order of operations can be changed, and at least two operations can be executed in parallel.

[0113] According to an embodiment, Figure 7 Operations 705 and 710 in the text can be understood as being performed by an electronic device (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0114] Reference Figure 7 In operation 705, processor 240 can select at least some filters for image processing from a plurality of filters related to image quality improvement based on the status information of electronic device 101.

[0115] In an embodiment, the status information of the electronic device 101 may include information related to the battery of the electronic device 101 (e.g., ...). Figure 1 Information related to the remaining capacity of the battery 189 in the battery, and the display of the electronic device 101 (e.g., Figure 2 Information related to the characteristics of the application displayed (or requested to be displayed) on the display 230 (e.g., display characteristics (e.g., resolution)), information related to the operating state of the electronic device 101 (e.g., power saving state (or low power state)), information related to the detection of the user's gaze on the electronic device 101, and / or information related to the detection of user input.

[0116] In one embodiment, based on information related to the remaining capacity of battery 189, when it is identified that the remaining capacity of battery 189 of electronic device 101 is less than or equal to a specified capacity, processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. However, this is not a limitation. For example, based on information related to the characteristics of the application, when it is identified that the application does not require high image quality, processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the operating state of electronic device 101, when processor 240 identifies that electronic device 101 is operating in a power-saving state (or low-power state), processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user gaze on electronic device 101, when it is identified that the state in which user gaze is detected is less than or equal to a specified time, processor 240 can select a first number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user input, when a first user input exceeding a specified speed is detected, the processor 240 can select a first number of filters from a plurality of filters related to image quality improvement.

[0117] In one embodiment, based on information related to the remaining capacity of the battery 189 of the electronic device 101, when it is identified that the remaining capacity of the battery 189 exceeds a specified capacity, the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. However, this is not the only possibility. For example, based on information related to the characteristics of an application, when it is identified that the application requires high image quality, the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the operating state of the electronic device 101, when it is identified that the electronic device 101 is not operating in a power-saving state (or low-power state), the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of gaze on the electronic device 101, when it is identified that the state in which a user's gaze is detected exceeds a specified time, the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement. As another example, based on information related to the detection of user input, when no first user input exceeding a specified speed is detected, the processor 240 can select a second number of filters from a plurality of filters related to image quality improvement.

[0118] In an embodiment, the second quantity may be greater than the first quantity, or equal to the number of multiple filters.

[0119] In an embodiment, processor 240 may perform image processing in operation 710 on at least some pixels among a plurality of pixels that satisfy a specified similarity condition by using at least some selected filters. Performing image processing on at least some pixels among a plurality of pixels that satisfy a specified similarity condition by using at least some selected filters may involve applying the selected filters to the at least some pixels among a plurality of pixels that satisfy the specified similarity condition. For example, when a first number of filters are selected, processor 240 may perform image processing on at least some pixels among a plurality of pixels that satisfy the specified similarity condition by using the first number of selected filters. As another example, when a second number of filters are selected, processor 240 may perform image processing on at least some pixels among a plurality of pixels that satisfy the specified similarity condition by using the second number of selected filters.

[0120] Regarding the above Figure 7 The operations will be shown below. Figures 8 to 12 Various embodiments are described herein.

[0121] Figure 8 This is a diagram illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0122] Reference Figure 8 Electronic devices (e.g.) Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the process may include an artificial intelligence model-driven module (e.g., Figure 2 The artificial intelligence model-driven module 245 in the middle.

[0123] In an embodiment, the artificial intelligence model driving module 245 may include multiple filters (e.g., multiple convolutional filters (or layers)). For example, the multiple filters may include three 3×3 filters 805, 810, and 815 and three 1×1 filters 820, 825, and 830. However, it is not limited to this.

[0124] In an embodiment, the AI ​​model driving module 245 can activate or deactivate multiple filters. For example, the AI ​​model driving module 245 can partially activate at least some of the multiple filters related to image quality improvement used for image processing based on the state information of the electronic device 101. For example, the state information of the electronic device 101 may include information related to the battery of the electronic device 101 (e.g., ...). Figure 1 Information related to the remaining capacity of the battery 189 in the battery, and the display of the electronic device 101 (e.g., Figure 2 Information related to the characteristics of the application displayed (or requested to be displayed (or output)) on the display 230, information related to the operating state of the electronic device 101 (e.g., power saving state (or low power state)), information related to the detection of the user's gaze on the electronic device 101, and / or information related to the detection of user input.

[0125] In this embodiment, as described above, the artificial intelligence model driving module 245 may include six filters, for example, three 3×3 filters 805, 810, and 815 and three 1×1 filters 820, 825, and 830. In this embodiment, the three 3×3 filters 805, 810, and 815 may be filters related to image manipulation. The three 1×1 filters 820, 825, and 830 may be filters related to image quality.

[0126] In this embodiment, the AI ​​model-driven module 245 can, based on the state information of the electronic device 101, keep one of the six filters—a 3×3 filter 805, 810, or 815 and a 1×1 filter 820, 825, or 830—in an active state, and partially activate the other two 3×3 filters and the other two 1×1 filters, respectively. Therefore, the AI ​​model-driven module 245 can partially activate multiple filters in a total of 16 steps. However, it is not limited to this.

[0127] In an embodiment, when the status information of electronic device 101 is related to the power supply of electronic device 101, the artificial intelligence model driving module 245 can partially activate or deactivate the three 3×3 filters 805, 810, and 815 related to image operation (e.g., controlling each of the 3×3 filters 805, 810, and 815 to be on / off). When the status information of electronic device 101 is related to image quality, the artificial intelligence model driving module 245 can partially activate or deactivate the three 1×1 filters 820, 825, and 830 related to image quality (e.g., controlling each of the 1×1 filters 820, 825, and 830 to be on / off sequentially).

[0128] In the embodiments, although it has been described that multiple filters are partially activated or deactivated based on whether the state information of the electronic device 101 is related to the power supply of the electronic device 101 or related to image quality, this disclosure is not limited thereto. For example, the processor 240 may partially activate or deactivate multiple filters based on user selection. For example, the processor 240 may activate or deactivate multiple filters on a display (e.g., Figure 2 The processor 240 displays menus related to low power mode and high image quality mode on the monitor 230. When one of the menus related to low power mode and high image quality mode is selected, the processor 240 can perform image processing by partially activating or deactivating multiple filters according to the selected menu.

[0129] According to various embodiments Figure 8 The three 3×3 filters 805, 810 and 815 and the three 1×1 filters 820, 825 and 830 shown are merely for the purpose of describing embodiments of the present disclosure, and the present disclosure is not limited to the number and type of filters described above (e.g., three 3×3 filters 805, 810 and 815 and three 1×1 filters 820, 825 and 830).

[0130] Figure 9 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0131] In the following embodiments, Figure 9 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 9 The order of operations can be changed, and at least two operations can be executed in parallel.

[0132] According to an embodiment, Figure 9Operations 905 to 925 in the above can be understood as being performed by electronic devices (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0133] According to various embodiments Figure 9 The above can be explained in further detail. Figure 3 Additional operations of operation 305 or operation 310. Not limited to this. Figure 9 The operations in can be used as... Figure 3 The operations are executed separately.

[0134] Reference Figure 9 The processor 240 can identify whether the status information of the electronic device 101 meets the specified conditions in operation 905.

[0135] The status information of the electronic device 101 according to the embodiment may include information related to the battery of the electronic device 101 (e.g., ...). Figure 1 Information related to the remaining capacity of the battery 189 in the device, information related to the characteristics of the application displayed (or requested to be displayed (or output)) on the display 230 of the electronic device 101 (e.g., display characteristics (e.g., resolution)) and / or information related to the operating state of the electronic device 101 (e.g., power saving state (or low power state)).

[0136] For example, processor 240 can identify whether the remaining capacity of battery 189 is less than or equal to a specified capacity. For example, processor 240 can obtain this information from a power management module (e.g., Figure 1 The power management module 188 in the processor 240 obtains information related to the remaining capacity of the battery 189. The processor 240 can identify the information related to the remaining capacity of the battery 189 obtained from the power management module 188, and determine whether the remaining capacity of the battery 189 meets the specified conditions based on whether the remaining capacity is less than or equal to a specified capacity.

[0137] As another example, processor 240 can identify information related to the characteristics (e.g., display characteristics (e.g., resolution) of an application currently displayed (or requested to be displayed) on display 230 of electronic device 101. Processor 240 can identify whether the characteristics of an application meet specified conditions based on whether the application's characteristics require high image quality. For example, applications requiring high image quality may include applications related to visibility or image quality, such as photo applications or social networking service (SNS) applications, and / or applications configured by the user to be displayed in high image quality (e.g., applications related to video reproduction). However, it is not limited to this. As another example, processor 240 can identify information related to the operating state of electronic device 101 (e.g., power-saving state (or low-power state)). For example, a power-saving state (or low-power state) may refer to a state that restricts at least some functions that electronic device 101 can perform, thereby reducing the power consumption of battery 189 compared to a normal state. Processor 240 can identify whether the operating state of electronic device 101 meets specified conditions based on whether the operating state of electronic device 101 is a power-saving state (or low-power state).

[0138] In an embodiment, when the state information of electronic device 101 is identified as meeting a specified condition (e.g., yes in operation 905), processor 240 can, in operation 910, select from multiple filters (e.g., those related to image quality improvement) in relation to image quality improvement. Figure 8 A first number of filters is selected from multiple filters 805, 810, 815, 820, 825, and 830. For example, as in... Figure 8 As described in [the document], in multiple filters (e.g., Figure 8 Of the multiple filters 805, 810, 815, 820, 825, and 830 in the image, the three 3×3 filters 805, 810, and 815 can be filters related to image operations. Furthermore, multiple filters (e.g., Figure 8Three of the 1×1 filters 820, 825, and 830 among the multiple filters 805, 810, 815, 820, 825, and 830 in the image processing unit may be image quality-related filters. The processor 240 may, when the state information of the electronic device 101 is recognized as meeting a specified condition, deactivate at least two of the three 3×3 filters 805, 810, and 815 related to image processing (e.g., control at least two 3×3 filters to be off), and / or deactivate at least two of the three 1×1 filters 820, 825, and 830 related to image quality (e.g., control at least two 1×1 filters to be off). By deactivating at least two 3×3 filters (e.g., control at least two 3×3 filters to be off) and / or deactivating at least two 1×1 filters (e.g., control at least two 1×1 filters to be off), the processor 240 can perform fewer image operations and operate with lower power, thereby reducing battery consumption 189. However, this is not the only limitation.

[0139] In an embodiment, during operation 915, processor 240 can perform image processing on at least some pixels among a plurality of pixels that satisfy a specified similarity condition using a selected first number of filters. For example, processor 240 can apply the selected first number of filters to at least some pixels among a plurality of pixels that satisfy the specified similarity condition.

[0140] In an embodiment, when the status information of electronic device 101 is information related to the remaining capacity of battery 189 of electronic device 101, a state satisfying a specified condition may include a state in which the remaining capacity of battery 189 is less than or equal to a specified capacity. In an embodiment, when the status information of electronic device 101 is information related to the characteristics (e.g., display characteristics, such as resolution) of an application displayed (or requested to be displayed (or output)) on display 230 of electronic device 101, a state satisfying a specified condition may include a state in which the application does not require high image quality (e.g., an application to which image quality is not important). In an embodiment, when the status information of electronic device 101 is information related to the operating state of electronic device 101 (e.g., a power-saving state (or low-power state)), a state satisfying a specified condition may include a state in which the operating state of electronic device 101 is a power-saving state (or low-power state).

[0141] In this embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be... Figure 3 The pixels corresponding to the edge regions have been described in operation 310.

[0142] In an embodiment, when the state information of electronic device 101 is identified as not meeting a specified condition (e.g., No in operation 905), processor 240 can, in operation 920, select from multiple filters related to image quality improvement (e.g., ... Figure 8 The second number of filters is selected from the multiple filters 805, 810, 815, 820, 825 and 830.

[0143] In an embodiment, when the status information of electronic device 101 is information related to the remaining capacity of battery 189 of electronic device 101, a state that does not meet the specified condition may include the following state: the remaining capacity of battery 189 exceeds a specified capacity. In an embodiment, when the status information of electronic device 101 is information related to the characteristics (e.g., display characteristics (e.g., resolution)) of an application displayed (or requested to be displayed (or output) on display 230 of electronic device 101, a state that does not meet the specified condition may include the following state: the application requires high image quality. In an embodiment, when the status information of electronic device 101 is information related to the operating state of electronic device 101 (e.g., power-saving state (or low-power state)), a state that does not meet the specified condition may include the following state: the operating state of electronic device 101 is not a power-saving state (or low-power state).

[0144] In an embodiment, during operation 925, processor 240 can perform image processing on at least some pixels among a plurality of pixels that satisfy a specified similarity condition using a selected second number of filters. For example, processor 240 can apply the selected second number of filters to at least some pixels among a plurality of pixels that satisfy the specified similarity condition.

[0145] In an embodiment, the second number may be greater than the first number or equal to the number of multiple filters (e.g., 6). For example, when it is determined that the state information of the electronic device 101 does not meet the specified conditions, the processor 240 may perform image processing on at least some pixels among the multiple pixels that meet the specified similarity conditions by using a second number of filters that is greater than the first number or equal to the number of multiple filters, in order to sharpen the edge regions in the image and output a high-quality image.

[0146] According to various embodiments Figure 9 As described above, multiple filters (e.g., based on whether the remaining capacity of battery 189 is less than or equal to a specified capacity, whether the application requires high image quality, or whether the operating state of electronic device 101 is a power-saving state (or low-power state), are associated with image quality improvement. Figure 8The multiple filters 805, 810, 815, 820, 825 and 830 in the filter can be partially controlled (e.g., activated or deactivated). However, this disclosure is not limited thereto.

[0147] For example, processor 240 may partially control (e.g., activate or deactivate) multiple filters (e.g., based on whether the brightness of display 230 decreases when the time for turning off display 230 after activation of display 230 arrives.) that are related to image quality improvement. Figure 8 The processor 240 can execute operations 910 and 915 when it detects a decrease in the brightness of the display 230 after the time for the display 230 to turn off has arrived following its activation. When it does not detect a decrease in the brightness of the display 230 after its activation because the time for the display 230 to turn off has not arrived, the processor 240 can execute operations 920 and 925.

[0148] As another example, processor 240 can, based on sensor information related to the movement of electronic device 101 and / or electronic device 101 via communication circuits (e.g., Figure 1 The communication module 190 in the processor is connected to an external electronic device to identify whether the user is driving, walking, or exercising. Based on the identified state indicating whether the user is driving, walking, or exercising, the processor 240 can partially control (e.g., activate or deactivate) multiple filters (e.g., ...) related to image quality improvement. Figure 8 The processor 240 uses multiple filters 805, 810, 815, 820, 825, and 830. For example, when the user is driving, walking, or moving, the processor 240 can identify this as a situation where visibility is less important than a stationary state and execute operations 910 and 915. When the user is not driving, walking, or moving, the processor 240 can identify this as a situation where visibility is more important and execute operations 920 and 925.

[0149] According to various embodiments Figure 9The present disclosure has described the execution of image processing operations when the status information of electronic device 101 is information related to the remaining capacity of the battery 189 of electronic device 101, information related to the characteristics (e.g., display characteristics (e.g., resolution)) of an application displayed (or requested to be displayed (or output) on the display 230 of electronic device 101, or information related to the operating state of electronic device 101 (e.g., power-saving state (or low-power state)), but this disclosure is not limited thereto. For example, the status information of electronic device 101 may also include information related to the detection of the user's gaze on electronic device 101 and / or information related to the detection of user input. In this regard, reference will be made to the following Figures 10 to 12 Various embodiments are described.

[0150] Figure 10 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0151] In the following embodiments, Figure 10 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 10 The order of operations can be changed, and at least two operations can be executed in parallel.

[0152] According to an embodiment, Figure 10 Operations 1005 to 1040 in the above can be understood as being performed by electronic devices (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0153] According to various embodiments Figure 10 The above can be explained in further detail. Figure 3 Additional operations of operation 305 or operation 310. Not limited to this, and... Figure 10 The operations in can be used as... Figure 3 The operations within are executed separately.

[0154] In the embodiments, in Figure 10 In this context, the status information of the electronic device 101 may include information related to the detection of the user's gaze on the electronic device 101.

[0155] Reference Figure 10 In operation 1005, processor 240 can identify whether a user's gaze has been detected. For example, processor 240 can detect this through a camera (e.g., Figure 1The camera module 180 in the processor 240 acquires an image of the user's face. The processor 240 can detect the direction of the user's gaze from the acquired facial image. For example, the processor 240 can identify whether the user is looking at a display based on the direction of their gaze (e.g., ...). Figure 2 (The monitor 230 in the middle).

[0156] In an embodiment, when a user's gaze is not detected (e.g., No in operation 1005), the processor 240 may determine in operation 1010 whether the state in which the user's gaze is not detected has exceeded a specified time. For example, when it is not recognized that the user is looking at the display 230 (e.g., the state in which the user's gaze is detected), the processor may determine whether the state in which the user is not looking at the display 230 (e.g., the state in which the user's gaze is not detected) has exceeded a specified time.

[0157] In an embodiment, when a state where a user's gaze has not been detected exceeds a specified time (e.g., Yes in operation 1010), processor 240 may partially deactivate multiple filters associated with image quality improvement (e.g., ...) in operation 1015. Figure 8 Multiple filters (805, 810, 815, 820, 825, and 830) are used, and a first number of filters are selected. For example, such as Figure 8 As shown, in multiple filters (e.g., Figure 8 Of the multiple filters 805, 810, 815, 820, 825, and 830 in the image processing unit, the three 3×3 filters 805, 810, and 815 can be filters related to image operation, and the three 1×1 filters 820, 825, and 830 can be filters related to image quality. When it is determined that a user's gaze has not been detected for more than a specified time, the processor 240 can identify this as a situation of low visibility importance or a situation where the user is not using the electronic device 101, and deactivate at least two of the three 1×1 filters 820, 825, and 830 related to image quality (e.g., control at least two 1×1 filters to be in a closed state). Not limited to this, when it is determined that a user's gaze has not been detected for more than a specified time, the processor 240 can deactivate at least two of the three 3×3 filters 805, 810, and 815 related to image operation (e.g., control at least two 3×3 filters to be in a closed state).

[0158] In an embodiment, processor 240 may perform image processing in operation 1020 on at least some pixels among a plurality of pixels that satisfy a specified similarity condition using a selected first number of filters. For example, processor 240 may apply a selected second number of filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition.

[0159] In this embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be similar to the above. Figure 3 The pixels corresponding to the edge regions described in operation 310.

[0160] In an embodiment, processor 240 may identify in operation 1025 whether the number of deactivated filters is the same as the specified number of filters. For example, the specified number of filters may include the number of filters in a default configuration. For example, one of the three 3×3 filters 805, 810, and 815 and one of the three 1×1 filters 820, 825, and 830 may be active by default. In other words, the maximum number of deactivated filters may be 4 by deactivating at least two of the three 3×3 filters 805, 810, and 815 (e.g., controlling at least two 3×3 filters to be off) and deactivating at least two of the three 1×1 filters 820, 825, and 830 (e.g., controlling at least two 1×1 filters to be off). Alternatively, the three 3×3 filters 805, 810, and 815 and the three 1×1 filters 820, 825, and 830 may be controlled to be off, and in this case, the specified number of filters may be a plurality of filters (e.g., 6).

[0161] In an embodiment, when it is determined that the number of deactivated filters is equal to the specified number of filters (e.g., Yes in operation 1025), processor 240 may terminate the operation of selecting (or controlling) at least some filters for image processing. When it is determined that the number of deactivated filters is different from the specified number of filters (e.g., No in operation 1025), processor 240 may initialize the time associated with the state where the user's gaze is not detected in operation 1030, and then determine in operation 1010 whether the state where the user's gaze is not detected has exceeded a specified time. Not limited thereto, when it is determined that the number of deactivated filters is different from the specified number of filters (e.g., No in operation 1025), processor 240 may initialize the time associated with the state where the user's gaze is not detected in operation 1030, and then determine in operation 1005 whether the user's gaze has been detected.

[0162] In an embodiment, the first number of filters may correspond to a number of filters that is less than the number of multiple filters (e.g., one of 1 to 5).

[0163] In an embodiment, processor 240 may repeatedly execute operations 1010, 1015, 1020, and 1030 until it is identified that the number of deactivated filters equals the number of filters. Since operations 1010, 1015, 1020, and 1030 can be repeated, the operation of partially deactivating multiple filters and selecting a first number of filters (e.g., operation 1015) can control the 3×3 filters 805, 810, and 815 and / or the 1×1 filters 820, 825, and 830 to be sequentially turned off. For example, each time a user's gaze is not detected for a specified time, processor 240 can control the 3×3 filters 805, 810, and 815 and / or the 1×1 filters 820, 825, and 830 to be sequentially turned off.

[0164] In an embodiment, when a user's gaze is detected (e.g., Yes in operation 1005), or when the state that a user's gaze has not been detected has not exceeded a specified time (e.g., No in operation 1010), the processor 240 may, in operation 1035, select from multiple filters associated with image quality improvement (e.g., ... Figure 8 A second number of filters are selected from a plurality of filters 805, 810, 815, 820, 825, and 830. For example, when a user's gaze is detected, or when it is determined that a state in which a user's gaze has not been detected has not exceeded a specified time, the processor 240 may identify this as a situation of high visibility importance or a situation in which a user is using the electronic device 101, and select the second number of filters. In operation 1040, the processor 240 may use the selected second number of filters to perform image processing on at least some of the pixels among the plurality of pixels that meet a specified similarity condition.

[0165] In an embodiment, the second number may be greater than the first number, or equal to the number of filters. For example, when it is determined that a user's gaze has been detected for more than a specified time, the processor 240 may identify it as a situation of high visibility importance, and when it is determined that the characteristics of the application indicate that the application requires high image quality, the processor 240 may perform image processing on at least some of the pixels that meet a specified similarity condition among the multiple pixels by using a second number of filters that is greater than the first number or equal to the number of filters, thereby sharpening the edge regions in the image and outputting a high-quality image.

[0166] In an embodiment, although not shown, the number of deactivated filters can be 0 to 4 (or 0 to 6) by repeatedly performing operations 1010, 1015, 1020, and 1030. In this case, when a user's gaze is detected, the processor 240 can control at least two of the deactivated 3×3 filters 805, 810, and 815 and / or at least two of the deactivated 1×1 filters 820, 825, and 830 to be activated sequentially. In an embodiment, by controlling at least two of the deactivated 3×3 filters 805, 810, and 815 and / or at least two of the deactivated 1×1 filters 820, 825, and 830 to be activated sequentially, the processor 240 can prevent abrupt changes in the image and provide the user with improved visibility and readability.

[0167] According to various embodiments Figure 10 The text describes several filters (e.g., based on whether the user's gaze was detected for a specified time, which are related to image quality improvement) Figure 8 Multiple filters 805, 810, 815, 820, 825, and 830 in the device are partially controlled (e.g., activated or deactivated), but are not limited thereto. For example, electronic device 101 may include wearable electronic devices. Wearable electronic devices may include augmented reality (AR) glasses, which are of the type of glasses, smart glasses, or head-mounted displays (HMDs).

[0168] In an embodiment, electronic device 101 may include a plurality of gaze-tracking cameras (not shown) for tracking the movement of a user's pupils. The plurality of gaze-tracking cameras may include a first gaze-tracking camera and a second gaze-tracking camera. The first gaze-tracking camera may track the movement of the user's left eye, and the second gaze-tracking camera may track the movement of the user's right eye. Processor 240 may detect the user's pupils (e.g., left and right eyes) through the plurality of gaze-tracking cameras and track the direction of the user's gaze. Processor 240 may perform image processing on pixels constituting a portion of the image, corresponding to the direction of the tracked user's gaze in the image (or screen) displayed on display 230 (or on the image (or screen) displayed on a display in an extended reality (XR) environment), and may restrict image processing to pixels constituting another portion of the image, which does not correspond to the direction of the tracked user's gaze.

[0169] Figure 11 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0170] In the following embodiments, Figure 11 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 11 The order of operations can be changed, and at least two operations can be executed in parallel.

[0171] According to an embodiment, Figure 11 Operations 1105 to 1130 in the above can be understood as being performed by electronic devices (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0172] According to various embodiments Figure 11 The above can be explained in further detail. Figure 3 Additional operations of operation 305 or operation 310. Not limited to this. Figure 11 The operations in can be used as... Figure 3 The operations within are executed separately.

[0173] According to various embodiments Figure 11 Operations 1115 to 1130 are the same as those mentioned above. Figure 9 The operations from 910 to 925 are essentially the same; therefore, for a detailed description, please refer to [reference needed]. Figure 9 The description.

[0174] In an embodiment, Figure 11 The status information of the electronic device 101 may include information related to the detection of user input.

[0175] Reference Figure 11 The processor 240 can detect scroll input in operation 1105. For example, the processor 240 can detect scroll input from an object. For example, the object may include a user's hand or an electronic pen. The processor 240 can detect scroll input based on touch input from the object. Not limited to this, the processor 240 can also detect scroll input from buttons located in at least a portion of the housing of the electronic pen.

[0176] In this embodiment, the processor 240 may identify whether the screen scrolling speed exceeds a specified speed during operation 1110.

[0177] For example, processor 240 may be based on a display (e.g., Figure 2 The detection of user input (e.g., scrolling input) as the first touch position of the display (230) moves to the second touch position, and the identification of whether the screen scrolling speed exceeds a specified speed.

[0178] In an embodiment, when it is detected that the screen scrolling speed exceeds a specified speed (e.g., yes in operation 1110), the processor 240 may, in operation 1115, select from multiple filters associated with image quality improvement (e.g., Figure 8 The processor 240 selects a first number of filters from a plurality of filters 805, 810, 815, 820, 825, and 830. When the screen scrolling speed exceeds a specified speed, the screen displayed on the monitor 230 can scroll rapidly. In this case, the processor 240 can identify it as a situation of low visibility importance (e.g., a situation where a clear image output is not required) and deactivate at least two of the three 1×1 filters 820, 825, and 830 related to image quality to a closed state (e.g., controlling at least two 1×1 filters to be closed). However, it is not limited to this. For example, the processor 240 can deactivate multiple filters (e.g., Figure 8 At least two of the 3×3 filters 805, 810, and 815 that are related to image operation (e.g., controlling at least two 3×3 filters to be off) among the multiple filters 805, 810, 815, 820, 825, and 830 in the image (e.g., controlling at least two 3×3 filters to be off).

[0179] In an embodiment, processor 240 may perform image processing in operation 1120 on at least some pixels among a plurality of pixels that satisfy a specified similarity condition using a selected first number of filters. For example, processor 240 may apply the selected first number of filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition.

[0180] In this embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be similar to the above. Figure 3 The pixels corresponding to the edge regions described in operation 310.

[0181] In an embodiment, when it is determined that the screen scrolling speed does not exceed a specified speed (e.g., No in operation 1110), the processor 240 may, in operation 1125, select from multiple filters related to image quality improvement (e.g., Figure 8The processor 240 selects a second number of filters from a plurality of filters 805, 810, 815, 820, 825, and 830. For example, when the screen scrolling speed does not exceed a specified speed, the screen displayed on the display 230 can scroll slower than when the screen scrolling speed exceeds the specified speed. Based on this, the processor 240 can identify this as a case of high visibility importance (e.g., a case requiring clear image output) and select the second number of filters. The processor 240 can perform image processing on at least some pixels among the plurality of pixels that satisfy a specified similarity condition using the selected second number of filters in operation 1130. For example, the processor 240 can apply the selected second number of filters to at least some pixels among the plurality of pixels that satisfy the specified similarity condition.

[0182] In an embodiment, the second number may be greater than the first number, or equal to the number of multiple filters. For example, when it is determined that the screen scrolling speed does not exceed a specified speed, the processor 240 may identify it as a case of high visibility importance, and use a second number of filters greater than the first number or equal to the number of multiple filters to perform image processing on at least some of the pixels among the multiple pixels that meet the specified similarity condition, so as to sharpen the edge regions in the image and output a high-quality image.

[0183] Figure 12 This is a flowchart illustrating a method for selecting at least some filters for image processing based on the state information of an electronic device 101 according to an embodiment of the present disclosure.

[0184] In the following embodiments, Figure 12 Operations within the context can be executed sequentially, but they are not necessarily executed in order. For example, Figure 12 The order of operations can be changed, and at least two operations can be executed in parallel.

[0185] According to an embodiment, Figure 12 Operations 1205 to 1235 in the above can be understood as being performed by electronic devices (e.g., Figure 1 The processor of the electronic device 101 in the device (e.g., Figure 2 The processor 240 in the middle executes.

[0186] According to various embodiments Figure 12 The above can be explained in further detail. Figure 3 Additional operations of operation 305 or operation 310. Not limited to this. Figure 12 The operations in can be used as... Figure 3 The operations within are executed separately.

[0187] According to various embodiments Figure 12 Operations 1220 to 1235 and Figure 9 The operations from 910 to 925 are basically the same; therefore, for a detailed description, please refer to [link / reference needed]. Figure 9 The description.

[0188] In an embodiment, Figure 12 The status information of the electronic device 101 may include information related to the detection of user input.

[0189] Reference Figure 12 In operation 1205, processor 240 can identify whether user input for zooming in or out of the screen is detected. For example, processor 240 can identify whether input for zooming in or out of the display is detected (e.g., ...). Figure 2 User input displayed on the screen of the monitor 230.

[0190] In an embodiment, when user input for zooming in or out of the screen is detected (e.g., yes in operation 1205), processor 240 may detect edge regions in the zoomed-out or zoomed-out screen in operation 1210. For example, as described above. Figure 3 As described in operations 305 and 310, processor 240 can identify at least some pixels among the plurality of pixels that satisfy a specified similarity condition based on the similarity between the plurality of pixels included in the image being reduced or enlarged. For example, at least some pixels among the plurality of pixels that satisfy the specified similarity condition may be pixels corresponding to edge regions.

[0191] In an embodiment, processor 240 may identify in operation 1215 whether the proportion of the detected edge region exceeds a specified proportion. For example, processor 240 may identify whether the proportion of the edge region relative to the entire frame of the scaled-down or scaled-up image exceeds a specified proportion. When no proportion of the detected edge region is identified to exceed a specified proportion (e.g., no in operation 1215), processor 240 may, in operation 1220, select from multiple filters associated with image quality improvement (e.g., ...). Figure 8 The processor 240 selects a first number of filters from a plurality of filters 805, 810, 815, 820, 825, and 830. In operation 1225, the processor 240 may use the selected first number of filters to perform image processing on at least some pixels among the plurality of pixels that satisfy a specified similarity condition. For example, the processor 240 may apply the selected first number of filters to at least some pixels among the plurality of pixels that satisfy a specified similarity condition.

[0192] In an embodiment, when the proportion of detected edge regions identified exceeds a specified proportion (e.g., yes in operation 1215), processor 240 may, in operation 1230, select from multiple filters associated with image quality improvement (e.g., ...). Figure 8A second number of filters (805, 810, 815, 820, 825, and 830) are selected from the plurality of filters in operation 1435. In operation 1435, processor 240 may use the selected second number of filters to perform image processing on at least some of the pixels among the plurality of pixels that satisfy a specified similarity condition. For example, processor 240 may apply the selected second number of filters to at least some of the pixels among the plurality of pixels that satisfy the specified similarity condition.

[0193] In an embodiment, the second number may be greater than the first number, or equal to the number of multiple filters. For example, when the proportion of detected edge regions exceeds a specified proportion, the processor 240 may use a second number of filters, greater than the first number or equal to the number of multiple filters, to perform image processing on at least some pixels among the multiple pixels that meet a specified similarity condition, so as to make the edge regions in the image clear and output a high-quality image.

[0194] In an embodiment, when it is recognized that no user input for zooming in or out of the screen is detected (e.g., No in operation 1205), the processor 240 may perform the operations 1230 and 1235 described above.

[0195] In various embodiments, although not shown, the processor 240 may choose not to perform the operation of analyzing the similarity between multiple pixels included in an image, depending on the type of application. For example, in the case of an image that needs to be updated, such as in a game application, photo application, video playback application, and / or SNS application, there may be many edge regions. Therefore, the time required for the processor 240 to perform the operation of analyzing the similarity between multiple pixels included in the image may be long, and the current consumption may be high. Based on this, the processor 240 may choose not to analyze the similarity between multiple pixels included in an image displayed as a game application, photo application, video playback application, or SNS application is executed.

[0196] Furthermore, depending on the application's display mode, the operation of analyzing the similarity between multiple pixels included in an image may or may not be performed. For example, when one of multiple images is displayed full-screen on display 230 after executing a photo application, processor 240 may not analyze the similarity between multiple pixels included in the image. When multiple images are displayed as thumbnails after executing a photo application, processor 240 may analyze the similarity between multiple pixels included in the thumbnails. As another example, when a video is reproduced via a video playback application, if the video is displayed full-screen on display 230, processor 240 may not analyze the similarity between multiple pixels included in the video. When a video is displayed on a portion of display 230 in picture-in-picture (PIP) mode, processor 240 may analyze the similarity between multiple pixels included in the video displayed in PIP mode.

[0197] exist Figures 7 to 12 In various embodiments, the electronic device 101 may partially activate multiple filters related to image quality improvement based on the state information of the electronic device 101, and perform image processing on at least some pixels among the multiple pixels included in the image that meet a specified similarity condition by using the activated filters, thereby sharpening a predetermined region (e.g., point region and / or edge region) in the image and outputting a high-quality image.

[0198] The image processing method of the electronic device 101 according to an embodiment of the present disclosure may include an operation of identifying the similarity between a plurality of pixels included in an image. The image processing method of the electronic device 101 according to an embodiment may include an operation of controlling an artificial intelligence model driving module 245 to apply a plurality of filters related to image quality improvement of the image to at least some pixels among the plurality of pixels that satisfy a specified similarity condition, based on the identified similarity between the plurality of pixels in the image. The image processing method of the electronic device 101 according to an embodiment may include an operation of controlling the artificial intelligence model driving module 245 to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not satisfy the specified similarity condition.

[0199] According to the embodiment, at least some pixels among a plurality of pixels that satisfy a specified similarity condition may be pixels corresponding to an edge region. At least some other pixels among a plurality of pixels that do not satisfy the specified similarity condition may be pixels that do not correspond to an edge region.

[0200] The plurality of filters according to embodiments may include a plurality of first filters and a plurality of second filters. The plurality of first filters according to embodiments may include filters related to image manipulation. The plurality of second filters according to embodiments may include filters related to image quality.

[0201] The operation of applying multiple filters related to image quality improvement of an image to at least some pixels that satisfy a specified similarity condition, according to an embodiment, may include the operation of selecting at least some filters from the multiple filters related to image quality improvement based on the state information of the electronic device 101. The operation of applying multiple filters related to image quality improvement of an image to at least some pixels that satisfy a specified similarity condition, according to an embodiment, may include the operation of applying the selected at least some filters to at least some pixels among the multiple pixels that satisfy the specified similarity condition.

[0202] The status information of the electronic device 101 according to the embodiment may include at least one of the following: information related to the remaining capacity of the battery 189 of the electronic device 101, information related to the characteristics of the application displayed on the display 230, information related to the operating status of the electronic device 101, information related to the detection of the user's gaze on the electronic device 101, or information related to the detection of user input.

[0203] The image processing method of the electronic device 101 according to an embodiment may include an operation of identifying whether the state information of the electronic device (101) meets specified conditions. The image processing method of the electronic device 101 according to an embodiment may include: when the state information of the electronic device 101 meets the specified conditions, controlling the artificial intelligence model driving module 245 to select a first number of filters from a plurality of filters and applying the selected first number of filters to at least some pixels among the plurality of pixels that meet the specified similarity conditions. The image processing method of the electronic device 101 according to an embodiment may include: when the state information of the electronic device 101 meets the specified conditions, controlling the artificial intelligence model driving module 245 to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity conditions. The image processing method of the electronic device 101 according to an embodiment may include: when the state information of the electronic device (101) does not meet the specified conditions, selecting a second number of filters from a plurality of filters. The image processing method of the electronic device 101 according to an embodiment may include applying the selected second number of filters to at least some pixels among the plurality of pixels that meet the specified similarity conditions.

[0204] The state information of the electronic device 101 according to the embodiment may satisfy at least one of the following conditions: identifying a condition where the remaining capacity of the battery 189 is less than or equal to a specified capacity based on information related to the remaining capacity of the battery 189 of the electronic device 101; identifying a condition where the application does not require high image quality based on information related to the characteristics of the application; identifying a condition where the electronic device 101 is operating in a power-saving state based on information related to the operating state of the electronic device 101; identifying a condition where the state in which the user's gaze is detected is less than or equal to a specified time based on information related to the detection of the user's gaze on the electronic device 101; or identifying a condition where the screen scrolling speed exceeds a specified speed based on information related to the detection of user input.

[0205] The situation where the state information of the electronic device (101) according to the embodiment does not meet the specified conditions may include at least one of the following: identifying a situation where the remaining capacity of the battery 189 of the electronic device (101) exceeds a specified capacity based on information related to the remaining capacity of the battery 189 of the electronic device (101); identifying a situation where the application is an application requiring high image quality based on information related to the characteristics of the application; identifying a situation where the electronic device 101 is not operating in a power-saving state based on information related to the operating state of the electronic device 101; identifying a situation where the state in which the user's gaze is detected exceeds a specified time based on information related to the detection of the user's gaze on the electronic device 101; or identifying a situation where the screen scrolling speed is less than or equal to a specified speed based on information related to the detection of user input.

[0206] The operation of applying selected filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition, according to an embodiment, may include: detecting edge regions from the scaled-down or scaled-up screen when user input for zooming in or out is detected based on detection information related to user input. The operation of applying selected filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition, according to an embodiment, may include: selecting a first number of filters from a plurality of filters when the proportion of detected edge regions does not exceed a specified proportion, and applying the selected first number of filters to at least some pixels among a plurality of pixels that satisfy the specified similarity condition. The operation of applying selected filters to at least some pixels among a plurality of pixels that satisfy a specified similarity condition, according to an embodiment, may include: selecting a second number of filters from a plurality of filters when the proportion of detected edge regions exceeds a specified proportion, and applying the selected second number of filters to at least some pixels among a plurality of pixels that satisfy the specified similarity condition.

[0207] According to the embodiment, the second quantity can be greater than the first quantity, or equal to the number of multiple filters.

[0208] The electronic device according to various embodiments can be one of a variety of types of electronic devices. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. According to embodiments of this disclosure, the electronic device is not limited to those described above.

[0209] It should be understood that the various embodiments of this disclosure and the terminology used therein are not intended to limit the technical features set forth herein to the specific embodiments, but rather to include various changes, equivalents, or substitutions to the respective embodiments. In the description of the drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that nouns in the singular form corresponding to terms may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “at least one of A or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least one of A, B, or C” may include any one or all possible combinations of the items enumerated together with the corresponding phrase among the plurality of phrases. As used herein, terms such as “first” and “second” or “first” and “second” may be used to simply distinguish the respective component from another component and do not limit the component in other respects (e.g., importance or order). It will be understood that, whether the terms “operably” or “communically” are used or not, if an element (e.g., a first element) is referred to as “combined with another element (e.g., a second element),” “combined to another element (e.g., a second element),” “connected to another element (e.g., a second element),” or “connected to another element (e.g., a second element)”, it means that the element can be directly (e.g., wiredly) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0210] As used in connection with various embodiments of this disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "part," or "circuit." A module may be a single integrated component adapted to perform one or more functions, or the smallest unit or part of such a single integrated component. For example, according to embodiments, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0211] The various embodiments set forth herein can be implemented as software (e.g., program 140) containing one or more instructions readable by a machine (e.g., electronic device 101) stored in a storage medium (e.g., internal memory 136 or external memory 138). For example, under the control of a processor, the processor (e.g., processor 120) of the machine (e.g., electronic device 101) can invoke and execute at least one of the one or more instructions stored in the storage medium, with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media may be provided in the form of non-transitory storage media. The term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between data being stored semi-permanently in the storage medium and data being temporarily stored in the storage medium.

[0212] According to embodiments, methods according to various embodiments of this disclosure may be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TM The computer program product may be published online (e.g., downloaded or uploaded), or may be distributed directly between two user devices (e.g., smartphones) (e.g., downloaded or uploaded). If published online, at least a portion of the computer program product may be temporarily generated, or at least a portion of the computer program product may be temporarily stored in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a forwarding server).

[0213] According to various embodiments, each of the above-described components (e.g., a module or program) may include a single entity or multiple entities, and some of the multiple entities may be separately disposed in different components. According to various embodiments, one or more of the above-described components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as the corresponding component of the multiple components performed one or more functions before integration. According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations may be run in a different order or omitted, or one or more other operations may be added.

Claims

1. An electronic device (101), comprising: Processor (240); as well as Memory (220), the memory (220) stores instructions, The instructions, when executed by the processor (240), cause the electronic device (101) to: Identify the similarity between multiple pixels in an image; Based on the similarity between the identified plurality of pixels, the artificial intelligence model driving module (245) is controlled to apply a plurality of filters related to the quality improvement of the image to at least some of the plurality of pixels that satisfy a specified similarity condition; as well as The artificial intelligence model driving module (245) is controlled to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity condition.

2. The electronic device according to claim 1, wherein, Among the plurality of pixels, at least some pixels that satisfy the specified similarity condition are pixels corresponding to edge regions. Among the plurality of pixels, at least some other pixels that do not meet the specified similarity condition are pixels that do not correspond to the edge region. The plurality of filters includes a plurality of first filters and a plurality of second filters. The plurality of first filters include filters related to image operations, and The plurality of second filters include filters related to image quality.

3. The electronic device according to claim 1 or claim 2, wherein, When the instruction is executed by the processor (240), it causes the electronic device (101) to: Based on the state information of the electronic device (101), at least some filters are selected from the plurality of filters; as well as The selected at least some filters are applied to at least some pixels among the plurality of pixels that satisfy the specified similarity condition. The status information of the electronic device (101) includes at least one of the following: information related to the remaining capacity of the battery (189) of the electronic device (101); information related to the characteristics of the application displayed on the display (230); information related to the operating status of the electronic device (101); information related to the detection of the user's gaze on the electronic device (101); or information related to the detection of user input.

4. The electronic device according to claim 3, wherein, When the instruction is executed by the processor (240), it causes the electronic device (101) to: Identify whether the status information of the electronic device (101) meets the specified conditions; When the state information of the electronic device (101) satisfies the specified condition, the artificial intelligence model driving module (245) is controlled to select a first number of filters from the plurality of filters and apply the selected first number of filters to at least some pixels among the plurality of pixels that satisfy the specified similarity condition. Control the artificial intelligence model driving module (245) to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity condition; as well as If the state information of the electronic device (101) does not meet the specified conditions, the artificial intelligence model driving module (245) is controlled to select a second number of filters from the plurality of filters and apply the selected second number of filters to at least some of the pixels that meet the specified similarity conditions.

5. The electronic device according to claim 4, wherein, The states of the electronic device (101) satisfying the specified conditions include at least one of the following: identifying, based on information related to the remaining capacity of the battery (189), that the remaining capacity of the battery (189) is less than or equal to a specified capacity; identifying, based on information related to the characteristics of the application, that the application does not require high image quality; identifying, based on information related to the operating state of the electronic device (101), that the electronic device (101) is operating in a power-saving state; identifying, based on information related to the detection of the user's gaze on the electronic device (101), that the state in which the user's gaze is detected is less than or equal to a specified time; or identifying, based on information related to the detection of the user's input, that the screen scrolling speed exceeds a specified speed, and The situation in which the status information of the electronic device (101) does not meet the specified conditions includes at least one of the following: based on information related to the remaining capacity of the battery (189) of the electronic device (101), it is identified that the remaining capacity of the battery (189) exceeds the specified capacity; based on information related to the characteristics of the application, it is identified that the application requires high image quality; based on information related to the operating state of the electronic device (101), it is identified that the electronic device (101) is not operating in the power-saving state; based on information related to the detection of the user's gaze on the electronic device (101), it is identified that the state in which the user's gaze is detected exceeds the specified time; or based on information related to the detection of the user's input, it is identified that the screen scrolling speed is less than or equal to the specified speed.

6. The electronic device according to any one of claims 3 to 5, wherein, When the instruction is executed by the processor (240), it causes the electronic device (101) to: When user input for zooming in or out of the screen is detected based on information related to the detection of the user input, edge regions are detected from the zoomed-out or zoomed-out screen. If the proportion of the detected edge region does not exceed a specified proportion, a first number of filters are selected from the plurality of filters, and the selected first number of filters are applied to at least some of the pixels that satisfy the specified similarity condition among the plurality of pixels; as well as If the proportion of the detected edge region exceeds the specified proportion, a second number of filters are selected from the plurality of filters, and the selected second number of filters are applied to at least some of the pixels that satisfy the specified similarity condition among the plurality of pixels.

7. The electronic device according to any one of claims 4 to 6, wherein, The second quantity is greater than the first quantity, or equal to the number of the plurality of filters.

8. The electronic device according to any one of claims 1 to 7, wherein the electronic device further comprises a display (230). in, The image includes a magnified image, and The instructions, when executed by the processor (240), cause the electronic device (101) to: A second image is generated by merging the magnified image and a feature map, the feature map being based on at least some pixels to which the plurality of filters have been applied and at least some other pixels to which the plurality of filters have been applied; and The generated second image is displayed on the display (230).

9. The electronic device according to any one of claims 1 to 7, wherein the electronic device further comprises a display (230). in, The image includes an unmagnified image, and The instructions, when executed by the processor (240), cause the electronic device (101) to: A second image is generated by merging the image and the feature map, and the generated second image is magnified, wherein the feature map is based on at least some pixels to which the plurality of filters are applied and at least some other pixels to which the plurality of filters are applied; and An enlarged version of the second image is displayed on the display (230).

10. A method for processing an image by an electronic device (101), the method comprising: Identify the similarity between multiple pixels in an image; Based on the similarity between the identified plurality of pixels, the artificial intelligence model driving module (245) is controlled to apply a plurality of filters related to the quality improvement of the image to at least some of the plurality of pixels that satisfy a specified similarity condition; as well as The artificial intelligence model driving module (245) is controlled to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity condition.

11. The method according to claim 10, wherein, Among the plurality of pixels, at least some pixels that satisfy the specified similarity condition are pixels corresponding to edge regions. Among the plurality of pixels, at least some other pixels that do not meet the specified similarity condition are pixels that do not correspond to the edge region. The plurality of filters includes a plurality of first filters and a plurality of second filters. The plurality of first filters include filters related to image operations, and The plurality of second filters include filters related to image quality.

12. The method according to claim 10 or claim 11, wherein, Applying the plurality of filters associated with image quality improvement of the image to at least some pixels that satisfy the specified similarity condition includes: Based on the state information of the electronic device (101), at least some filters are selected from the plurality of filters; and The selected filters are applied to at least some pixels among the plurality of pixels that satisfy the specified similarity condition, and The status information of the electronic device (101) includes at least one of the following: information related to the remaining capacity of the battery (189) of the electronic device (101); information related to the characteristics of the application displayed on the display (230); information related to the operating status of the electronic device (101); information related to the detection of the user's gaze on the electronic device (101); or information related to the detection of user input.

13. The method according to claim 12, further comprising: Identify whether the status information of the electronic device (101) meets the specified conditions; When the state information of the electronic device (101) satisfies the specified condition, the following operations are performed: the artificial intelligence model driving module (245) is controlled to select a first number of filters from the plurality of filters and apply the selected first number of filters to at least some of the pixels among the plurality of pixels that satisfy the specified similarity condition; as well as Control the artificial intelligence model driving module (245) to restrict the application of the plurality of filters to at least some other pixels among the plurality of pixels that do not meet the specified similarity condition; as well as If the state information of the electronic device (101) does not meet the specified condition, the artificial intelligence model driving module (245) is controlled to select a second number of filters from the plurality of filters and apply the selected second number of filters to at least some of the pixels that meet the specified similarity condition. Wherein, the second quantity is greater than the first quantity, or equal to the number of the plurality of filters.

14. The method according to claim 13, wherein, The states of the electronic device (101) satisfying the specified conditions include at least one of the following: identifying, based on information related to the remaining capacity of the battery (189) of the electronic device (101), that the remaining capacity of the battery (189) is less than or equal to a specified capacity; identifying, based on information related to the characteristics of the application, that the application does not require high image quality; identifying, based on information related to the operating state of the electronic device (101), that the electronic device (101) is operating in a power-saving state; identifying, based on information related to the detection of the user's gaze on the electronic device (101), that the state in which the user's gaze is detected is less than or equal to a specified time; or identifying, based on information related to the detection of the user's input, that the screen scrolling speed exceeds a specified speed, and The situation in which the status information of the electronic device (101) does not meet the specified conditions includes at least one of the following: based on information related to the remaining capacity of the battery (189) of the electronic device (101), it is identified that the remaining capacity of the battery (189) exceeds the specified capacity; based on information related to the characteristics of the application, it is identified that the application requires high image quality; based on information related to the operating state of the electronic device (101), it is identified that the electronic device (101) is not operating in the power-saving state; based on information related to the detection of the user's gaze on the electronic device (101), it is identified that the state in which the user's gaze is detected exceeds the specified time; or based on information related to the detection of the user's input, it is identified that the screen scrolling speed is less than or equal to the specified speed.

15. The method according to any one of claims 12 to 14, wherein, The at least some pixels that satisfy the specified similarity condition when the selected filters are applied to the plurality of pixels include: When user input for zooming in or out of the screen is detected based on information related to the detection of the user input, edge regions are detected from the zoomed-out or zoomed-out screen. If the proportion of the detected edge regions does not exceed a specified proportion, a first number of filters are selected from the plurality of filters, and the selected first number of filters are applied to at least some pixels among the plurality of pixels that satisfy the specified similarity condition; and If the proportion of the detected edge region exceeds the specified proportion, a second number of filters are selected from the plurality of filters, and the selected second number of filters are applied to at least some pixels among the plurality of pixels that satisfy the specified similarity condition. Wherein, the second quantity is greater than the first quantity, or equal to the number of the plurality of filters.

Images