Electronic device and method for image processing

Abstract

Disclosed are an electronic device and a method. The electronic device may comprise: a memory including at least one storage medium for storing instructions; a display; a communication unit; and one or more processors including processing circuitry. The one or more processors may be configured to: acquire at least one of ambient condition information or positional relationship information; acquire at least one intermediate image by performing first image processing; acquire at least one simulation image by performing second image processing on the basis of at least one of the ambient condition information or the positional relationship information; and display the at least one simulation image by means of the display.

Description

Electronic device and method for image processing

[0001] The present disclosure relates to an electronic device for image processing and a method related to the control thereof.

[0002] Unlike conventional displays, reflective display technology offers the advantages of high visibility and low power consumption. Because reflective displays primarily display images by reflecting external light, they can minimize or reduce power consumption and are widely used in e-books, smart tags, and electronic billboards. Reflective displays can include e-paper.

[0003] Technologies that reflect user choices in the image processing process are also developing significantly to provide the viewing experience desired by users. This technology enables the delivery of screen quality by incorporating settings such as brightness, color contrast, and sharpness preferred by the user. Such technologies that reflect user choices contribute to increasing user satisfaction and providing consistent quality across various viewing environments.

[0004] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art related to the present disclosure.

[0005] According to an exemplary embodiment of the present disclosure, an electronic device may be provided. The electronic device may include a display, a memory comprising at least one storage medium storing at least one instruction, and a processing circuit, and may include at least one processor configured to execute the at least one instruction individually and / or collectively and to enable the electronic device to perform various operations. The at least one processor may control the operation of the electronic device by executing at least one instruction.

[0006] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: perform a first image processing on an original image to obtain at least one intermediate image, perform a second image processing on at least one intermediate image based on at least one of ambient condition information or positional relationship information to obtain at least one simulated image, and display at least one simulated image through a display of the electronic device. The intermediate image may include an image for display through an external reflective display device, the ambient condition information may include ambient illuminance information, and the positional relationship information may include information regarding at least one of the distance or direction of the reflective display device to the electronic device.

[0007] According to an exemplary embodiment, the second image processing is performed based on ambient condition information, positional relationship information, and optical characteristic information of the reflective display device, and the optical characteristic information of the reflective display device may include at least one of the reflectance of the display of the reflective display device or the display luminance of the reflective display device.

[0008] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: generate optical characteristic information of the reflective display device based on RGB (red-green-blue) data and ambient illumination information captured from the display of the reflective display device.

[0009] According to an exemplary embodiment, the second image processing is performed based on ambient condition information, positional relationship information, optical characteristic information of a reflective display device, display performance information of a reflective display device, and display performance information of an electronic device, wherein the ambient condition information further includes at least one of position information of an ambient light source or ambient color temperature information, and the performance information may include at least one of size or resolution.

[0010] According to an exemplary embodiment, the first image processing includes image processing that acquires a plurality of different intermediate images based on different image processing setting information for an original image, wherein the image processing setting information includes information regarding an image processing algorithm, and the plurality of intermediate images may include a plurality of images acquired based on different types of image processing algorithms or a plurality of images acquired based on the same type of image processing algorithm and different attribute values. The second image processing may be image processing that acquires a plurality of simulation images from a plurality of intermediate images based on at least one of the surrounding condition information or the positional relationship information.

[0011] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to transmit a signal to a reflective display device to display at least one intermediate image corresponding to at least one selected intermediate image among a plurality of intermediate images as a final image, based on identifying that at least one image is selected from a plurality of simulated images. The reflective display device may be configured to display the final image based on the received signal.

[0012] According to an exemplary embodiment, the simulation image may be an image related to the appearance seen at the position of an electronic device corresponding to positional relationship information, based on the intermediate image being displayed through the display of a reflective display device.

[0013] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: identify a change in ambient condition information and display a simulation image updated based on the change in ambient condition information through a display.

[0014] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to adjust the brightness of displaying a simulated image on the display based on at least one of ambient condition information or display brightness setting information of the electronic device, individually and / or collectively.

[0015] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: capture an image of a real space including a reflective display device, identify the location and shape of the reflective display device in the image of the real space, overlay a simulation image on the shape of the reflective display device on the image of the real space, and display the image of the real space and the overlaid simulation image on the display.

[0016] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: perform a first image processing on a first original image to obtain at least one first intermediate image and a second intermediate image. A first simulated image obtained through second image processing based on ambient condition information and a first positional relationship with respect to the first intermediate image has the same visual appearance as a second simulated image obtained through second image processing based on ambient condition information and a second positional relationship with respect to the second intermediate image, and the same visual appearance may refer to, for example, an appearance having the same or similar shape and the same perceptual image features within a defined error range.

[0017] According to an exemplary embodiment, at least one processor may be configured to cause the electronic device to: perform a first image processing on a second original image to obtain at least one third intermediate image. A third simulated image obtained through a second image processing based on ambient condition information and a second positional relationship with respect to the third intermediate image may have a visual appearance consistent with the first simulated image. A consistent visual appearance may refer, for example, to an appearance having the same perceptual image characteristics within a defined error range.

[0018] According to an exemplary embodiment, a method of operation of an electronic device may be provided. The method of operation of the electronic device may include: an operation of obtaining at least one intermediate image by performing a first image processing on an original image; an operation of obtaining at least one simulated image by performing a second image processing on at least one intermediate image based on at least one of ambient condition information or positional relationship information; and an operation of displaying at least one simulated image through a display of the electronic device.

[0019] A method of operation of an electronic device according to an exemplary embodiment of the present disclosure may include: an operation of calculating optical characteristic information of a reflective display device based on RGB data and ambient illumination information captured from a display of a reflective display device.

[0020] A method of operation of an electronic device according to an exemplary embodiment of the present disclosure may include: an operation of identifying that at least one image is selected from a plurality of simulation images; and an operation of transmitting a signal to a reflective display device to display at least one intermediate image corresponding to the selected at least one image among a plurality of intermediate images as a final image based on identifying that at least one image is selected.

[0021] A method of operation of an electronic device according to an exemplary embodiment of the present disclosure may include: capturing an image of a real space including a reflective display device; identifying the location and shape of the reflective display device in the image of the real space; overlaying a simulation image onto the shape of the reflective display device on the image of the real space; and displaying the image of the real space and the overlaid simulation image on a display.

[0022] Identical or similar reference numerals may be used throughout this disclosure, including the specification and drawings, to refer to identical or similar components. Additionally, various aspects, features, and advantages according to specific embodiments of this disclosure may become more apparent from the following detailed description, which is considered together with the accompanying drawings.

[0023] FIG. 1a is a block diagram showing an exemplary configuration of an electronic device according to various embodiments.

[0024] FIG. 1b is a block diagram showing an exemplary configuration of a display device according to various embodiments.

[0025] FIG. 2 is a drawing for illustrating an exemplary surrounding environment including an electronic device and an external display according to various embodiments.

[0026] FIGS. 3a and 3b include a flowchart and a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires a simulated image.

[0027] FIGS. 4a and 4b include a flowchart and a diagram for explaining an exemplary operation in which an electronic device according to various embodiments acquires and uses a plurality of simulation images.

[0028] FIG. 5 is a diagram showing an exemplary electronic device according to various embodiments implementing a simulation image into augmented reality.

[0029] FIG. 6 is a flowchart illustrating the exemplary operation of an electronic device according to various embodiments.

[0030] FIG. 7 is a drawing for explaining examples of screens displayed on the display of an electronic device according to various embodiments.

[0031] FIG. 8 is a drawing showing an exemplary electronic device and several external displays according to various embodiments.

[0032] FIG. 9 is a drawing showing multiple external displays according to various embodiments forming a video wall.

[0033] Hereinafter, various exemplary embodiments of the present disclosure are described in detail with reference to the drawings. However, the present disclosure may be embodied in various different forms and is not limited to the various embodiments described herein, and should be understood to include various modifications, equivalents, or substitutions. The present disclosure is capable of various modifications by those skilled in the art without departing from the essence claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

[0034] In the following drawings and related descriptions, functions, configurations, technical terms, and technical details well known in the art to which this disclosure pertains may be omitted. This is intended to convey the essentials of this disclosure more clearly and concisely by minimizing or reducing unnecessary detailed descriptions.

[0035] In the drawings, each block of the flowcharts and combinations of the flowcharts may be performed by at least one instruction. The instruction may be loaded into a processor of a computer or other programmable data processing equipment to generate means for performing the functions described in the drawings. The instruction may also provide steps for performing the functions described in the drawings by being executed on a computer or other programmable data processing equipment.

[0036] Various elements and regions in the drawings are depicted schematically, and the technical concept of the present disclosure is not limited by the relative sizes, spacing, or arrangements depicted in the attached drawings. The electronic device of the present disclosure is not limited to the configuration and / or operation shown in the drawings and may include all other configurations capable of performing the same or similar functions.

[0037] The individual components depicted in the drawings are not necessarily physically separated but are separated to aid in the explanation and understanding of the present disclosure. The present disclosure may include configurations in which the individual components shown in the drawings are merged, modified, or have some components deleted and / or added. Likewise, the operations depicted in the drawings are illustrative for the purpose of explanation and understanding, and the present disclosure may be modified by merging, changing the order of, or deleting and / or adding parts of the operations depicted in the drawings. For example, two or more operations depicted consecutively in the drawings may be performed substantially simultaneously or, if necessary, in reverse order.

[0038] In the drawings and the following description, 'image' or words containing 'image' may include video data in addition to static images.

[0039] FIG. 1a is a block diagram showing an exemplary configuration of an electronic device according to various embodiments.

[0040] The electronic device (100) of FIG. 1a may be a smartphone, tablet PC, PC, TV, smart TV, mobile phone, PDA (personal digital assistant), laptop, media player, micro server, digital broadcasting terminal, navigation, kiosk, home appliance, and other mobile or non-mobile computing devices, but is not limited thereto. The electronic device (100) may perform various computing functions such as real-time video viewing and communication. Various embodiments of the electronic device (100) of the present disclosure described below may be equally applicable to other electronic devices.

[0041] According to one embodiment, the electronic device (100) may include at least one processor (110) (e.g., including a processing circuit), memory (120), display (130), communication unit (140) (e.g., including a communication circuit), detection unit (150) (e.g., including at least one sensor) and / or camera (160).

[0042] According to one embodiment, the memory (120) may include a storage medium used by the electronic device (100) and may store data such as at least one instruction (121) corresponding to at least one program or setting information. The program may include an operating system (OS) program and various application programs. When the at least one instruction (121) stored in the memory (120) is executed by at least one processor (110), the electronic device (100) may be able to perform at least one operation.

[0043] According to one embodiment, the memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), RAM (random access memory, RAM), SRAM (static random access memory), ROM (read only memory, ROM), EEPROM (electrically erasable programmable ROM), PROM (programmable ROM), magnetic memory, a magnetic disk, and an optical disk.

[0044] According to one embodiment, the display (130) can perform functions for outputting information in the form of numbers, characters, images, and / or graphics. The display (130) may include at least one hardware module for output. The at least one hardware module may include, for example, at least one of an LCD (Liquid Crystal Display), an LED (Light Emitting Diode), an LPD (Light Emitting Polymer Display), an OLED (Organic Light Emitting Diode), an AMOLED (Active Matrix Organic Light Emitting Diode), or an FLED (Flexible LED). The display (130) can display a screen corresponding to data received from the processor (110). The display (130) may be referred to as an 'output unit', a 'display unit', or other terms having an equivalent technical meaning.

[0045] According to one embodiment, the display (130) can perform various display operations according to the function of the electronic device (100). For example, the display (130) can display various forms of information such as numbers, characters, images, graphics, or tables. The display (130) may be configured as a touchscreen by forming a layer structure with a touch pad. In this case, the display (130) can be used as an input interface as well as an output interface. The display (130) may be a single independent display or may include multiple displays. Multiple displays may be placed at different locations.

[0046] According to one embodiment, the electronic device (100) may include a communication unit (140). The communication unit (140) may include various communication circuits and may perform functions for the electronic device (100) to receive data from the outside or transmit data to the outside. The communication unit (140) may be connected to a network or exchange data with another device via wired and / or wireless communication methods. For example, the communication unit (140) may support various communication protocols such as Wi-Fi, Bluetooth, NFC (near field communication), 4G / 5G mobile communication networks, Ethernet, or USB. Additionally, the communication unit (140) may manage the connection status with an external device and allow the external device to remotely control the operation of the electronic device (100).

[0047] According to one embodiment, the communication unit (140) can support a 5G network after a 4G network and next-generation communication technology, for example, new radio access technology. The NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The communication unit (140) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The communication unit (140) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beamforming, or large-scale antenna. The communication unit (140) can support various requirements specified in the electronic device (100), external electronic device, or network system. According to one embodiment, the communication unit (140) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

[0048] According to one embodiment, the communication unit (140) may receive data transmitted from an external sensor. The data transmitted from the sensor may include ambient conditions information detected by the sensor. Ambient conditions information is a term that comprehensively refers to various physical characteristics measured or analyzed in the surrounding environment, and may include at least one of factors such as illuminance, light source position, color temperature, ambient reflection, temperature, humidity, noise level, or air pressure. Ambient conditions may also be referred to by various terms such as surrounding conditions, ambient environmental factors, or environmental variables.

[0049] According to one embodiment, the electronic device (100) may include a sensing unit (150). The sensing unit (150) may include at least one sensor and may perform a function for detecting various external signals interacting with the electronic device (100). The sensing unit (150) may include various types of sensors capable of measuring physical states such as distance, direction, illuminance, the position of a light source, or color temperature.

[0050] According to one embodiment, the sensing unit (150) can detect the operating state of the electronic device (100) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensing unit (150) can perform a function to detect various external signals interacting with the electronic device (100). For example, it may include at least one of an illuminance sensor, a color temperature sensor, an RGB (red-green-blue) sensor, a distance sensor, a direction sensor, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, or a humidity sensor.

[0051] According to one embodiment, the electronic device (100) may include a camera (160). The camera (160) may capture still images and video and may include at least one of one or more lenses, image sensors, image signal processors, or flashes.

[0052] According to one embodiment, at least one processor (110) may include various processing circuits and, by executing at least one instruction (121) stored in memory (120), may perform control, image processing, computation, and / or data processing of at least a part of the electronic device (100).

[0053] According to one embodiment, at least one processor (110) may include at least one processing circuit and / or multiple processors. One or more of the at least one processor (110) may be configured to perform various functions described in the present disclosure individually and / or collectively. Where in the present disclosure, "processor," "at least one processor," or "one or more processors" are described as being configured to perform various functions, these terms may cover, for example, a situation in which one processor performs some of the cited functions and other processor(s) perform other parts of the cited functions, and may also cover, but are not limited to, a situation in which a single processor can perform all of the cited functions. Additionally, at least one processor (110) may include a combination of processors performing the cited / disclosed various functions, for example, in a distributed manner. At least one processor (110) may execute program instructions to achieve or perform various functions.

[0054] According to one embodiment, at least one processor (110) may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), a digital signal processor (DSP), a microcontroller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA), and may have a plurality of cores.

[0055] According to one embodiment, at least one processor (110) can identify the presence of an external device connected through a communication unit (140) and receive information related to the external device. For example, it can identify an external reflective display and receive information regarding the reflective display.

[0056] According to one embodiment, at least one processor (110) can obtain a positional relationship between an external device and an electronic device (100) through a communication unit (140) and / or a sensing unit. The positional relationship may include a distance and / or direction between the external device and the electronic device (100).

[0057] According to one embodiment, at least one processor (110) can perform image processing. For example, at least one processor (110) can analyze an input image to identify a subject or extract specific attributes of the input image. Image processing may be referred to, for example, as image processing.

[0058] According to one embodiment, image processing that can be performed by at least one processor (110) may include at least one of the following examples: dithering, edge detection, noise reduction, color correction, resolution adjustment, brightness adjustment, sharpness adjustment, contrast adjustment, or image stitching.

[0059] According to one embodiment, dithering may include a technique for expressing a natural gradient by smoothly adjusting the difference in color or brightness of an image. Dithering may be used, for example, to improve the visual quality of an image in an environment where the color palette or resolution is limited. For example, dithering may form a visible gradient by mixing and arranging adjacent colors as small dots in areas lacking color, thereby expressing color or gradient.

[0060] According to one embodiment, in a reflective display, dithering can serve to enable the expression of image details and gradations within a limited range of colors and brightness. Since a reflective display displays images by reflecting external light, it is subject to fixed color limitations and is significantly affected by ambient lighting. In such an environment, applying dithering allows the reflective display to enhance image sharpness and / or visual clarity even with limited colors.

[0061] According to one embodiment, image processing may be associated with at least one image processing setting information. At least one processor (110) may adjust the quality of an input image or enhance specific attributes based on this image processing setting information during the image processing process. The image processing setting information may include information regarding image quality parameters and / or image processing algorithms.

[0062] According to one embodiment, image processing setting information can be used to optimize the visual quality and processing effects of an image, and can be used for various adjustment tasks to improve the user experience or to match the original image to a display intended for output.

[0063] According to one embodiment, image quality parameters may include variables related to the overall visual characteristics of an image. For example, image quality parameters may include variables such as brightness, contrast, color saturation, sharpness, and color gamut.

[0064] According to one embodiment, information regarding an image processing algorithm may include settings related to the type, configuration, and / or application intensity of a specific algorithm used in the image processing process. For example, information regarding an image processing algorithm may include information such as the type of image processing, parameters (e.g., blur intensity in a Gaussian blur algorithm), dithering intensity, and edge detection sensitivity.

[0065] According to one embodiment, dithering intensity may be a parameter that determines how strongly dithering is applied. For example, a higher dithering intensity may result in smaller dots being more densely arranged to form a smooth gradient, but the detail of the image may be reduced. Conversely, for example, a lower dithering intensity may result in a less smooth gradient expression, but the detail of the image may appear more distinct. Dithering intensity may be adjusted to suit the characteristics of the display, the viewing environment, and / or the user's requirements (e.g., requirements based on the user's preferences or choices).

[0066] According to one embodiment, at least one processor (110) may perform image processing for simulation. The simulation may be related to implementing an appearance, that is, how an actual object would look at a specific location or an actual image would be displayed at a specific location, at the location of the electronic device (100). Through the simulation, at least one processor (110) may allow a user to experience the actual environment in advance. The image implemented through the simulation may be referred to as a "simulation image" in this disclosure.

[0067] According to one embodiment, at least one processor (110) can analyze an image (e.g., an image input through a camera (160)) to obtain related data (e.g., RGB data, illuminance data). In this way, at least one processor (110) can perform the role of a sensor (e.g., RGB sensor, illuminance sensor) using an image captured by the camera (160).

[0068] According to one embodiment, at least one processor (110) may enable a user interface (UI) to be displayed on a display (130). The UI may include, for example, buttons, menus, or icons for receiving user input. The UI may provide information, for example, the current system status, warning notifications, and work progress. Additionally, the UI may be changed or updated in real time according to user operation, and may be adjustable in color, layout, size, etc. through user custom settings.

[0069] According to one embodiment, a UI displayed on a display (130) by at least one processor (110) may include an interface associated with a plurality of sample images and / or selection options to allow the user to select an image to be displayed on an external display (e.g., a reflective display). In this case, if the external display is a display with a relatively slow screen update speed (e.g., electronic paper), the time required for screen updates can be saved by checking various selection options in advance on the electronic device (100) and then displaying only the final selected image on the external display. Accordingly, the user experience can be improved.

[0070] FIG. 1b is a block diagram showing an exemplary configuration of a display device according to various embodiments.

[0071] Referring to FIG. 1b, the display device (101) may include a display (131), a communication unit (141) (e.g., including a communication circuit), a memory (122), and a processor (111) (e.g., including a processing circuit). According to one embodiment, the electronic device (120) may include additional components (e.g., a user interface) in addition to the illustrated components, or at least one of the illustrated components may be omitted.

[0072] According to one embodiment, the display (131) can visually provide information (e.g., text and / or images) to the outside of the display device (101). The display (131) may include, for example, a control circuit for controlling the display (131).

[0073] According to one embodiment, the display (131) may be a reflective display, that is, a display that reflects ambient light to display an image, but is not limited thereto. The display (131) may include, for example, an e-paper display, an e-ink display, a mirasol display, an electrophoretic display, an electrochromic display, a reflective liquid crystal display (RLCD), and micro-electro-mechanical systems (MEMS). A reflective display does not require a backlight and can be operated at low power, and because it reflects ambient light, it can maintain visibility even when the external illumination is high. In addition, a reflective display does not use an artificial light source such as a backlight but reflects natural light and can maintain a static image for a long time, so eye strain caused by blue light or flicker can be reduced.

[0074] According to one embodiment, a reflective display may include a reflective surface and a driving element. The reflective surface may be a physical surface that reflects an external light source to form an image, and the driving element may be a device that controls the intensity, color, or transparency of light reflected from the reflective surface according to an electrical signal. The brightness of the reflective display may be proportional to the reflectance of the reflective surface.

[0075] According to one embodiment, the luminance of a reflective display may be proportional to the ambient illuminance. For example, if the ambient illuminance is E, the reflectance of the reflective surface is R, and the relative luminance of an image displayed on the reflective display is In this case, the luminance L of the reflective display can be expressed as follows:

[0076]

[0077] According to one embodiment, the relative luminance is a value between 0 and 1, which may be 1 for white (R:255, G:255, B:255) and 0 for black (R:0, G:0, B:0). The relative luminance can be calculated using a relative luminance formula that applies weights to RGB data and the CIE 1931 XYZ color space (giving a high weight to green to reflect human visual sensitivity). Specifically, the relative luminance It can be obtained by acquiring RGB data, performing an inverse gamma correction if the acquired RGB data is a gamma-corrected value to convert it into linear RGB data, and substituting the linear RGB value into the formula below:

[0078]

[0079] According to one embodiment, if there is information on the reflectivity of the reflective surface of a reflective display (e.g., display (131)) and external illumination according to the above equations, the luminance when a specific image with RGB data is displayed on the reflective display can be known without actually displaying the specific image.

[0080] According to one embodiment, an electrophoretic display (e.g., an electrophoretic electronic paper display), which is a type of reflective display, may include charged particles and a fluid medium. The charged particles and the fluid medium have colors that are clearly distinguishable from each other for image representation; for example, the charged particles may be white and the fluid medium may be black. The electrophoretic display can display a desired pattern and / or image by moving the charged particles by applying an electric field in a direction perpendicular to its surface. For example, the electrophoretic display may include other charged particles charged with a charge opposite to that of the charged particles instead of the fluid medium, and in this case (e.g., white negatively charged particles and black positively charged particles), the principle of image display may be the same.

[0081] According to one embodiment, the image refresh rate of an electrophoretic display may be relatively slower compared to a non-reflective display (e.g., an LCD). This may be because the time for charged particles to move within a fluid medium is physically limited. Additionally, according to one embodiment, to prevent and / or reduce ghosting (residual images) when updating an image, the electrophoretic display may perform a full-screen refresh operation (e.g., displaying the entire screen in black and then back to white) prior to the image refresh operation. Consequently, more time may be required.

[0082] According to one embodiment, the image refresh rate may be slow in reflective displays based on principles other than electrophoretic displays. For example, a cholesteric liquid crystal display (C-LCD) uses a principle in which liquid crystal molecules selectively reflect light through a specific arrangement (e.g., selectively reflecting light of a specific wavelength according to the pitch between helical periods in a helical molecular arrangement), but the image refresh rate may be limited because the rearrangement of liquid crystal molecules takes time. Similarly, an electrochromic display uses a principle in which a specific material changes color in response to a specific electrical signal (e.g., It uses a material that changes from transparent to blue when a specific voltage is applied, but the image refresh rate may be limited because the material's color change reaction takes time during this process. As such, generally, reflective displays may have the characteristic of having a relatively slow refresh rate to maintain energy efficiency.

[0083] According to one embodiment, the constraint on the image refresh rate in the above-described reflective display can degrade the visual quality or the responsiveness of the user interface, thereby worsening the user experience. For example, when attempting to change image processing setting information (e.g., dithering intensity) to be output to the reflective display, one may have to wait for the image refresh time each time the setting is changed.

[0084] According to one embodiment, the communication unit (141) can support the establishment of a wireless communication channel or a wired communication channel between the display device (101) and an external electronic device (e.g., electronic device (120) or source device (130)), and / or the performance of communication through the established communication channel.

[0085] According to one embodiment, the communication unit (141) may operate independently of the processor (e.g., application processor) (250) and may include one or more communication processors that support wireless communication or wired communication. According to one embodiment, the communication unit (141) may include a wireless communication module (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules may communicate with an external electronic device through a first network (e.g., a short-range communication network such as Bluetooth, BLE (Bluetooth Low Energy), Wi-Fi, Wi-Fi Direct (WFD), or IrDA (infrared data association)) or a second network (e.g., a legacy cellular network, 5G network, next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0086] According to one embodiment, the memory (122) may store various data or information used by at least one component of the display device (101) (e.g., display (131), communication unit (141), or processor (111)). For example, the memory (122) may store at least one program for processing and controlling the processor (111), and may store input and / or output data (e.g., images or content). The memory (122) may store at least one artificial intelligence (AI) model and may include volatile memory or non-volatile memory.

[0087] According to one embodiment, the processor (111) includes various processing circuits and can control the overall operation of the display device (101). The processor (111) can perform operations or data processing regarding the control and / or communication of at least one other component of the display device (101). For example, the processor (111) can be electrically connected to the display (131), the communication unit (141), and the memory (122), and can control the operations of the display device (101) by executing instructions of a program stored in the memory (122).

[0088] According to one embodiment, the processor (111) can perform a residual image compensation operation based on a first application. For example, the processor (111) can determine a residual image compensation operation based on information received from an electronic device (120) and control a display (131) or a control circuit included in the display (131) so that the determined residual image compensation operation is performed. According to one embodiment, the processor (111) can perform operations of the display device (101) to be described later.

[0089] According to one embodiment, the processor (111) may correspond to a plurality of processors that perform a plurality of operations individually or collectively by dividing them among the processors. As set forth in relation to the processor (110) of FIG. 1A above, the processor (111) may include various processing circuits and / or a plurality of processor(s). For example, in this specification, including claims, the term “processor” may include various processing circuits including at least one processor, and at least one of the at least one processor may be configured to perform the various functions described in this specification individually and / or collectively in a distributed manner. In this specification, when “one processor,” “at least one processor,” and “one or more processors” are described as being configured to perform a plurality of functions, these terms include, exemplarily and without limitation, situations in which one processor performs some of the described functions and other processor(s) perform other parts of the described functions, and also situations in which a single processor can perform all of the described functions. Additionally, at least one processor may include a combination of processors that perform various described / disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0090] According to one embodiment, the processor (111) may include a processing circuit that executes instructions of a program stored in memory (122). The processor (111) may include at least one of a CPU (central processing unit), NPU (neural processing unit), GPU (graphics processing unit), MPU (micro processing unit), MCU (micro controller unit), AP (application processor), CP (communication processor), SoC (system on chip), or IC (integrated circuit) sensor hub, supplementary processor, communication processor, ASIC (application specific integrated circuit), or FPGA (field programmable gate arrays), and may have multiple cores.

[0091] FIG. 2 is a drawing showing an electronic device (100) and an exemplary surrounding environment of the electronic device (100) according to various embodiments.

[0092] Referring to FIG. 2, according to one embodiment, an electronic device (100) may be connected to an external display (210) wirelessly and / or via a wire. For example, the electronic device (100) may be connected to the external display (210) via a wireless communication method such as Bluetooth or NFC and / or a wired communication method such as USB or HDMI (high-definition multimedia interface) to exchange information with each other.

[0093] According to one embodiment, the electronic device (100) can receive information about the external display (210) from the external display (210). The information about the external display (210) may include, for example, performance information of the external display (210) (e.g., resolution, brightness range, color gamut, viewing angle, etc.).

[0094] According to one embodiment, the electronic device (100) can obtain positional relationship information with an external display (210). The positional relationship information with the external display (210) may include at least one of distance information or direction information of the external display (210) with respect to the electronic device (100).

[0095] According to one embodiment, an electronic device (100) can acquire distance information to an external display (210) based on a distance sensing technology. The distance sensing technology may be a technology utilizing at least one of ultra-wideband (UWB), the time of arrival or reflection of a signal (e.g., communication signal, laser, infrared) (e.g., LiDAR technology), received signal strength indicator (RSSI), frequency modulation, camera-based distance measurement (e.g., triangulation, depth from defocus), or phase shift.

[0096] According to one embodiment, the electronic device (100) can obtain direction information to an external display (210) based on direction sensing technology. The direction sensing technology may be a technology that utilizes at least one of UWB, angle of arrival, beamforming, magnetometer, camera-based image analysis (e.g., shape analysis, feature matching), inertial measurement unit (IMU), or phase information.

[0097] According to one embodiment, the electronic device (100) can obtain performance information of an external display (210). The performance information may include, for example, at least one of size, brightness range, contrast ratio, ambient contrast ratio (ACR), refresh rate, resolution, color gamut, reflectance, color depth, viewing angle, or surface reflectance.

[0098] According to one embodiment, the external display (210) may be an independent display device (e.g., the display device (101) of FIG. 1b) rather than a simple display panel. For example, the external display (210) may operate independently by including its own processor (e.g., the processor (111) of FIG. 1b) and memory (e.g., the memory (122) of FIG. 1b), and may maintain a screen or perform information display functions without being connected to the electronic device (100).

[0099] According to one embodiment, an external display (210) receives a specific signal from an electronic device (100) and can perform an operation based on the received signal (e.g., an operation of displaying a specific image on its own display (e.g., the display (131) of FIG. 1b).

[0100] According to one embodiment, the electronic device (100) can obtain information about ambient conditions. For example, referring to FIG. 2, the electronic device (100) can obtain illuminance and / or location information of the ambient light source (220) due to the ambient light source (220).

[0101] According to one embodiment, the electronic device (100) may receive ambient condition information from an external sensor through a communication unit (140), and if the electronic device (100) includes a sensing unit (e.g., an internal light sensor), it may also directly detect ambient condition information using the sensing unit.

[0102] According to one embodiment, depending on the positional relationship with the external display (210) and / or ambient conditions (e.g., ambient illumination, location of the ambient light source (220)), the actual appearance of the image displayed on the external display (210) at the location of the electronic device (100) may differ. For example, if the external display (210) is a reflective display, as the distance between the electronic device (100) and the external display (210) increases, the details of the image seen at the electronic device (100) may become fainter and the brightness and / or contrast may be perceived as decreasing. Additionally, for example, as the ambient illumination increases, the readability of the image may be improved as it is more affected by ambient light due to the characteristics of the reflective display.

[0103] According to one embodiment, when the external display (210) is a reflective display, the electronic device (100) can obtain the luminance of the external display (210) when a specific image is displayed on the external display (210) by using the reflectance of the reflective surface, ambient illumination information and / or RGB data of the external display (210) (e.g., by substituting into the equations described above in the description of the display (131) of FIG. 1b) to know the relative luminance information per pixel on the external display (210).

[0104] According to one embodiment, when the external display (210) is a reflective display, the electronic device (100) can obtain relative luminance from the RGB data of the external display (210) even without knowing the reflectance of the reflective surface, and can estimate the luminance of the entire external display (210) when a specific image with pixel-by-pixel relative luminance information (or RGB data) is displayed on the external display (210) by assuming that the absolute luminance of the part where the relative luminance of the external display (210) is 1 is equal to the external illuminance (provided unit conversion is required), that is, by assuming that the brightest part reflects 100% of the ambient light. When the total luminance is estimated, the reflectance can also be estimated, and the luminance when a specific image is displayed on the external display (210) can be predicted even when the ambient illuminance changes.

[0105] According to one embodiment, when a specific image is displayed on an external display (210), the brightness to be perceived at the location of the electronic device (100) can be estimated based on the luminance when a specific image is displayed on the external display (210) and / or information on the positional relationship between the electronic device (100) and the external display (210). For example, the perceived brightness may be proportional to the luminance, inversely proportional to the square of the distance according to the inverse square law, and proportional to the cosine value of the angle between the direction of the electronic device (100) relative to the external display (210) and the normal of the surface of the external display (210).

[0106] FIG. 3a is a flowchart illustrating an exemplary operation of an electronic device acquiring and displaying a simulation image according to various embodiments.

[0107] FIG. 3b is a diagram illustrating an exemplary operation of an electronic device according to various embodiments acquiring and displaying a simulation image.

[0108] In the following, for the sake of consistency of explanation and convenience of understanding, terms may be as follows:

[0109] - Original image: The original image received by the electronic device (100).

[0110] - Intermediate image: An image obtained by performing image processing on an original image so that the electronic device (100) can display it through an external display (e.g., an external reflective display). The intermediate image may be displayed through an external display, but it is not necessarily displayed in reality. If a specific image among the intermediate images is displayed on an external display, that specific image may be referred to as the 'final image'.

[0111] - Simulation image: An image obtained by performing image processing on an intermediate image to simulate the image that would be seen if the intermediate image were displayed through an external display on the display (130) of the electronic device (100). For example, the simulation image may be an image that implements the appearance that would actually be seen at the location of the electronic device (100) if the intermediate image were displayed on an external display.

[0112] - First image processing: This does not refer to image processing of a specific algorithm, but comprehensively refers to all image processing steps involved in obtaining an intermediate image from an arbitrary original image. For example, the first image processing may include dithering of the original image.

[0113] - Second image processing: This does not refer to image processing of a specific algorithm, but comprehensively refers to all image processing in the step of obtaining a simulation image from an arbitrary intermediate image. For example, the second image processing may include brightness adjustment for the intermediate image.

[0114] In operation 301, according to one embodiment, the electronic device (100) may acquire ambient condition information and / or positional relationship information with an external display. The method (or process or operation) for acquiring ambient condition information and / or positional relationship information may be as described, for example, in FIG. 2. The acquired information may be used for a second image processing in a subsequent process.

[0115] In operation 302, according to one embodiment, the electronic device (100) may receive an original image (e.g., original image (310)). The electronic device (100) may receive the original image in the form of digital data from a display (130), a network server, or user input.

[0116] Referring to FIGS. 3a and 3b, according to one embodiment, the original image (310) is unsuitable for display as is through an external display, so a first image processing may be required for display on the external display. For example, the original image (310) is a high-resolution image (e.g., an image having 256 levels of grayscale and 10-bit color information), and if the display performance of the external display is exceeded, a first image processing including an algorithm to reduce the resolution may be required.

[0117] In operation 303, according to one embodiment, the electronic device (100) may perform a first image processing to obtain an intermediate image (e.g., an intermediate image (320)). The first image processing may include, for example, adjusting the resolution of the original image, correcting brightness and contrast, and applying a dithering algorithm. For example, the electronic device (100) may apply the first image processing to a high-resolution original image (e.g., an image having 256 levels of grayscale and 10 bits of color information) to obtain an intermediate image (e.g., an image having 16 levels of grayscale and 8 bits of color information) that has relatively low resolution and can be displayed through an external display.

[0118] According to one embodiment, there may be one or more intermediate images for one original image, and the electronic device (100) may allow only some of the one or more intermediate images to be displayed through an external display.

[0119] In operation 304, according to one embodiment, the electronic device (100) can obtain a simulation image (e.g., simulation image (330)) by performing a second image processing based on ambient condition information and position relationship information.

[0120] According to one embodiment, if the image display speed and / or image update speed of an external display is slow (e.g., if the external display is a reflective display), the electronic device (100) can increase user convenience by displaying or updating a simulated image (330) through the display (130).

[0121] According to one embodiment, when an intermediate image (320) is simulated without special image processing (e.g., using the image as is or only undergoing simple image quality adjustment processing), there may be a difference between the user experience when the intermediate image (320) is actually displayed on an external display and the visual appearance through the simulated image (330). For example, even if the same image is displayed on an external display, the visual appearance of the actual image may differ if the ambient light changes or if the distance and / or angle from the external display changes.

[0122] According to one embodiment, as there is a difference in the visual appearance when the intermediate image (320) is actually displayed on an external display and when the simulation image (330) is displayed on the display (330), it may be difficult to properly achieve the purpose of the simulation. For example, if the purpose of the simulation is to adjust the image characteristics (e.g., brightness, contrast, sharpness) of the intermediate image (320), and there is a difference in the visual appearance when the actual image is displayed on the external display (210) and when the simulation image is displayed on the display (130), the accuracy of the simulation may be reduced and the purpose may not be achieved. Accordingly, the user experience may be deteriorated.

[0123] According to one embodiment, a difference in visual appearance may refer to, for example, a difference in perceptual image features that exceeds a certain range. Perceptual image features may refer to, for example, image features that are actually detected at a specific location (e.g., brightness visible to the eye or detected through a camera) rather than image features of the image itself (e.g., brightness) when a specific image is displayed through a specific display (e.g., an external display, the display (130) of the electronic device (100)).

[0124] According to one embodiment, perceptual image characteristics can be objectively measured through objective numerical values ​​and / or parameters regarding measurable visual characteristics such as color, contrast ratio, brightness, saturation, and sharpness. Whether the visual appearance of two images that are identical or very similar in shape and associated with the same original image is identical can be determined, for example, based on whether the perceptual image characteristics of the two images are identical within a defined error range. For example, the color difference between the two images ( When E) is 2 or less, the contrast difference is 5% or less, and the brightness difference (nits) is ±10 or less, it may be difficult to perceive a significant difference with the naked eye. In this way, when perceptual image characteristics are identical within a defined margin of error, they can be defined as having 'identical visual appearance.' Perceptual image characteristics, when quantified, may be referred to as 'perceptual image parameters,' which can be used as objective evaluation indicators to determine visual identity.

[0125] According to one embodiment, two images associated with different original images can have a consistent visual impression if their perceptual image characteristics are similar. For example, if the perceptual image characteristics between two images with different shapes are identical within a defined error range, it can be defined as having a 'consistent visual appearance.' Two or more images having a consistent visual appearance can provide a visually consistent feeling to the user. For example, if different images displayed through multiple external displays differ in shape but have the same actual brightness, contrast ratio, etc., within a specific error range, those images can provide a consistent visual experience.

[0126] Identical visual appearance and / or consistent visual appearance may include differences in other perceptual image characteristics that are identical within a defined error range, in addition to the color differences, contrast differences, and brightness differences of the examples described above. The defined error range is not limited to the examples described above and may be appropriately selected to suit the purposes of various embodiments.

[0127] In the above description of perceptual image characteristics, it was assumed that 'shape' is not included in perceptual image characteristics in order to distinguish between cases where two or more images relate to the same source image and cases where they relate to different source images. Shape can refer to a comprehensive concept that defines, for example, the form of the appearance of an object and / or its structure in physical space. For example, shape can include size, the contour of a two-dimensional image, and / or the shape of the surface of a three-dimensional image.

[0128] In operation 304, according to one embodiment, the electronic device (100) may perform a second image processing that takes into account ambient condition information and / or positional relationship information with an external display. The positional relationship information with the external display may include the distance and / or direction of the external display relative to the electronic device (100).

[0129] According to one embodiment, the electronic device (100) may consider optical characteristic information of an external display during second image processing. The optical characteristic information may include at least one of brightness or reflectance.

[0130] According to one embodiment, when an electronic device (100) performs a second image processing based on ambient condition information and positional relationship information with an external display during the process of acquiring a simulation image (e.g., intermediate image (420) of FIG. 3b) from an intermediate image (e.g., intermediate image (420) of FIG. 3b), the user's visual experience can be improved and an accurate simulation can be provided compared to a case where the simulation is performed without special image processing. For example, by acquiring the simulation image (330) through the second image processing based on ambient condition information and positional relationship information with an external display, the same visual appearance can be achieved when the intermediate image (320) is displayed on an external display and when the simulation image (330) is displayed on an external display.

[0131] According to one embodiment, the second image processing may have different image processing setting information based on ambient condition information. For example, if the external display is a reflective display (e.g., electronic paper), due to its characteristics, the brightness, contrast, and contrast ratio of the displayed image may be perceived as high in a high-illumination environment (e.g., 1000 lux or more) and as low in a low-illumination environment (e.g., 100 lux or less). For example, the electronic device (100) can obtain a simulated image reflecting ambient illumination from an intermediate image by adjusting the settings of the second image processing according to the ambient illumination. In addition to ambient illumination, the influence of ambient environmental factors such as ambient color temperature and the location of an ambient light source may also be reflected.

[0132] According to one embodiment, the second image processing may have different image processing setting information based on positional relationship information with the external display. For example, when viewing an image displayed on the external display from close up, detailed elements (e.g., boundary lines, patterns of dithering points) may be perceived in detail, and when viewing from a distance, detailed elements may become faint and distortion of brightness or color due to ambient conditions (e.g., illuminance, color temperature) may be perceived more significantly. Similarly, as the viewing angle deviates from the front, the brightness and / or contrast of the displayed image may be perceived as lower depending on the viewing angle of the external display. The electronic device (100) can obtain a simulated image reflecting positional relationship information by adjusting the settings of the second image processing to reflect these changes in perceptual image characteristics based on the positional relationship information.

[0133] According to one embodiment, the electronic device (100) may perform a second image processing by considering the reflectance and / or brightness of the external display, in addition to ambient condition information and the positional relationship with the external display. For example, the second image processing may include an algorithm that adjusts the brightness of an intermediate image to match the brightness expected to be perceived at the position of the electronic device (100) by considering ambient illuminance, reflectance, and the positional relationship.

[0134] According to one embodiment, the electronic device (100) may change the setting information of the second image processing by further considering the performance information of the external display (e.g., resolution, size) and / or the performance information of the display (130). For example, if the resolution of the intermediate image (320) and the maximum resolution of the display (130) are higher than the maximum resolution of the external display, the second image processing may include an algorithm that lowers the resolution of the intermediate image (320) (e.g., linear interpolation downscaling).

[0135] In operation 305, according to one embodiment, the electronic device (100) may perform a second image processing to obtain a simulation image and display it through a display (e.g., the display (130) of FIG. 1a). For example, the electronic device (100) may identify changes in ambient condition information in real time or periodically, and update the simulation image (430) by applying a second image processing of an algorithm that has changed based on changes in ambient condition information, and display the updated simulation image through the display (130).

[0136] According to one embodiment, the electronic device (100) can adjust the brightness of the final image displayed on an external display based on ambient condition information and / or brightness setting information of the display (130). For example, if the ambient environment is bright, the overall brightness of the final image may be increased and output, or if the brightness of the display (130) is set to maximum, the brightness of the final image may be decreased and output.

[0137] FIG. 4a is a flowchart illustrating an exemplary operation in which an electronic device according to various embodiments acquires and displays a plurality of simulation images.

[0138] FIG. 4b is a diagram illustrating an exemplary operation in which an electronic device according to various embodiments acquires and displays a plurality of simulation images.

[0139] In Fig. 4b, images are represented by alphabets (e.g., letters) for intuitive understanding. Each A represents an example of the original image, (i=1,2,3) are examples of intermediate images, (i=1,2,3) represents examples of simulation images.

[0140] In operation 401, an electronic device (e.g., the electronic device (100) of FIG. 1a) can obtain ambient condition information and position relationship information. This can be obtained directly through a sensor (e.g., the sensing unit (150) of FIG. 1a) embedded in the electronic device (100) or obtained from an external sensor through a communication unit (e.g., the communication unit (140) of FIG. 1a).

[0141] In operation 402, according to one embodiment, the electronic device (100) may receive an original image (e.g., original image (411)). The original image may be input, for example, through user input.

[0142] In operation 403, according to one embodiment, the electronic device (100) performs a first image processing with various image processing settings applied to an original image A (411) to obtain a plurality of different intermediate images. (421), (422), and (423) can be obtained. The image processing settings may include information about the image processing algorithm. Different image processing settings may be, for example, different types of image processing algorithms or the same type of image processing algorithm and different attribute values. For example, regarding A (411), (421) is an ordered dithering, (422) Floyd-Steinberg dithering, (423) may be an intermediate image obtained by applying Jarvis dithering.

[0143] According to one embodiment, various image processing settings in the first image processing may be determined arbitrarily or may be determined by considering various factors. For example, various image processing settings may be determined based on at least one of user input, user preference data, pre-adjusted settings, ambient condition information, characteristics of the original image A (411), or the performance of an external display to display the final image.

[0144] Referring to FIGS. 4a and 4b, according to one embodiment, an electronic device (100) can acquire a plurality of intermediate images by applying various image processing settings, thereby enabling the output of an intermediate image having image characteristics desired by the user as a final image. For example, (421), (422), and If the brightness and sharpness of (423) are different, the user can output an image with the desired level of brightness and / or sharpness.

[0145] In operation 404, according to one embodiment, the electronic device (100) can obtain a plurality of simulation images by performing a second image processing on a plurality of intermediate images based on ambient condition information and positional relationship information with an external display. For example, referring to FIG. 4b, By applying the second image processing to (421) (431) By applying the second image processing to (422) (432) By applying the second image processing to (423) (433) can be obtained.

[0146] Referring to FIGS. 4a and 4b, according to one embodiment, (431), (432) and / or (433) are each (421), (422), and / or (423) may be an image related to the implementation of the appearance that will be seen when displayed on an external display.

[0147] In operation 405, according to one embodiment, the electronic device (100) may display a plurality of simulation images through a display (e.g., the display (130) of FIG. 1a). For example, referring to FIG. 4b, the electronic device (100) (431), (432) and / or (433) can be displayed.

[0148] In operation 406, according to one embodiment, the electronic device (100) can identify a selected image among a plurality of simulation images. The selection may be, for example, a selection by user input. The selected image may be one or a plurality of images.

[0149] In operation 407, according to one embodiment, the electronic device (100) may output a final image to an external display based on identifying a selected image. The final image may be at least one image among a plurality of intermediate images, and may be an image corresponding to at least one selected image among a plurality of simulation images. For example, referring to FIG. 4b, among the simulation images of the electronic device (100) (432) Based on identifying what was selected, (432) corresponding to (422) can be output to an external display as a final image.

[0150] According to one embodiment, the electronic device (100) outputting or displaying a specific image on an external display may refer, for example, to transmitting a signal to display a specific image on an external display and to display the specific image based on the external display receiving the signal. The external display may be an independent display device (e.g., the display device (101) of FIG. 1b) and may be configured to display a specific image based on receiving the signal from the electronic device (100).

[0151] FIG. 5 is a drawing showing an exemplary electronic device according to various embodiments displaying a simulation image.

[0152] Referring to FIG. 5, according to one embodiment, the simulation image (430) may be an image in which an intermediate image is implemented in augmented reality (AR). That is, the electronic device (100) may acquire an image of a real space (500) including an external display (210) (e.g., acquired through the camera (160) of FIG. 1a), identify the location and shape of the external display (210) portion on the image of the real space (500), and overlay the simulation image (530) onto the shape (510) of the external display (210) on the image of the real space and display it through the display (130).

[0153] According to one embodiment, the electronic device (100) may display a plurality of simulation images (e.g., a plurality of simulation images (520) of FIG. 4b) on a display (130). The electronic device (100) may output only some of the plurality of simulation images (520) as simulation images (530) in augmented reality. For example, the simulation images (530) in augmented reality may be displayed larger than each of the plurality of simulation images (520) to allow the user to check the image quality in more detail.

[0154] According to one embodiment, the operation of identifying the location and shape of the external display (210) portion on an image of the actual space (500) can be implemented through an artificial intelligence model, such as a deep learning-based object detection algorithm. The deep learning-based object detection algorithm can automatically identify the external display (210) within an image captured by a camera and extract the contour and location data of the external display (210) to enable accurate overlay of the simulation image (530).

[0155] According to one embodiment, compared to a general simulation that displays only a simulation image, the simulation image (530) in augmented reality can provide a more immersive and realistic user experience. For example, since the augmented reality simulation includes an image of the actual space (500), which is a physical environment where the external display (210) is actually installed, the user can intuitively understand how the result of the final image being displayed on the external display (210) will look in the actual space (500).

[0156] FIG. 6 is a flowchart illustrating the exemplary operation of an electronic device according to various embodiments.

[0157] FIG. 7 is a drawing showing an exemplary screen displayed on a display of an electronic device according to operation according to various embodiments.

[0158] Referring to FIGS. 6 and 7, according to one embodiment, an electronic device (100) can select and connect an external display to be connected (e.g., an external display (210) of FIG. 2) (operation 610). To this end, the electronic device (100) can display a screen (710) for selecting an external display through a display (130). The screen (710) for connecting an external display may include a list (611) of surrounding external displays recognized by wireless communication and / or a confirmation button. The electronic device (100) can select an external display and connect to the external display to be selected, for example, through interaction between the user and the UI on the display (130).

[0159] In operation 620, according to one embodiment, the electronic device (100) can select an original image. The electronic device (100) may display a relevant UI on the display (130) that supports selecting an original image or uploading an original image based on input provided through UI interaction with a user, for example. For example, if a user uploads a specific image through the UI, the electronic device (100) can select that image as the original image.

[0160] Referring to FIGS. 6 and 7, in operation 630, according to one embodiment, the electronic device (100) can select different image processing setting information to apply to the first image processing (operation 620). To this end, the electronic device (100) can display a screen (720) for selecting image processing setting information through a display (130). The screen (720) for selecting image processing setting information may include a list (721) associated with at least one image processing setting information and / or a confirmation button. The list (721) may be text that directly displays, for example, the name of an image processing algorithm type or the numerical value of an image processing parameter, or simplified text (e.g., 'clearly') for intuitive understanding by the user.

[0161] In operation 640, according to one embodiment, the electronic device (100) may obtain a plurality of intermediate images by performing a first image processing according to selected different image processing setting information. Subsequently, the electronic device (100) may perform a second image processing based on ambient environment information and positional relationship information with an external display (e.g., the external display (210) of FIG. 2). According to one embodiment, to determine the image processing setting information for the second image processing, the electronic device (100) may further consider at least one of the reflectance of the external display, the brightness of the external display, performance information of the external display, or performance information of the display (130).

[0162] Referring to FIGS. 6 and 7, in operation 650, according to one embodiment, an electronic device (100) may display a plurality of simulation images through a display (130). To this end, the electronic device (100) may display a screen (730) that supports viewing and selecting simulation image samples through the display (130). The user may compare a plurality of simulation images at once, as shown in the screen (730), or view the simulation images one at a time. The screen (730) for viewing simulation image samples may include a plurality of simulation images (732) and a simulation image (731) that is displayed large among them for detailed viewing. The simulation image (731) that is displayed large may be an image selected by user input (e.g., touch) among the simulation images (732). The electronic device (100) can display an image corresponding to the selection among the intermediate images as a final image on an external display (e.g., the external display (210) of FIG. 2) in response to the selection of one of the simulation images (e.g., a selection based on user input).

[0163] In operation 660, according to one embodiment, the electronic device (100) can identify a selected image among the simulation images. For example, the electronic device (100) can identify a selected image based on user input through the display (130).

[0164] In operation 670, according to one embodiment, the electronic device (100) may transmit a signal related to the display of a final image corresponding to a selected image to an external display (e.g., the external display (210) of FIG. 2). The external display may be configured to display the final image based on receiving the signal from the electronic device (100).

[0165] FIG. 8 is a drawing showing an electronic device according to various embodiments and several external displays.

[0166] Referring to FIG. 8, according to one embodiment, an electronic device (100) can be simultaneously connected to a first external display (211), a second external display (212), and / or a third external display (213). That is, the electronic device (100) can be simultaneously connected to a plurality of external display devices (e.g., the display device (101) of FIG. 1b). In this case, since the position of each external display is different, the positional relationship information between the electronic device (100) and each external display may all be different. For example, even if the same final image is displayed through each external display, the visual appearance at the position of the electronic device (100) may differ. For example, even if the same image is displayed, it may be perceived as darker when viewed at an angle than when viewed from the front, and may be perceived as clearer and brighter when viewed up close than when viewed from a distance.

[0167] According to one embodiment, the original image (810), different intermediate image (820), and / or different simulation image (830) shown in FIG. 8 may be displayed on the display (130), but may exist or be acquired in data and may not actually be displayed on the display (830).

[0168] According to one embodiment, the electronic device (100) can obtain different intermediate images (820) by performing a first image processing based on a plurality of image processing setting information on the same original image (810). The different intermediate images (820) may include a first intermediate image (821), a second intermediate image (822), and / or a third intermediate image (823).

[0169] According to one embodiment, the electronic device (100) can obtain a first simulation image (831) by performing second image processing based on ambient condition information and positional relationship information with a first external display on a first intermediate image (821), a second simulation image (832) by performing second image processing based on ambient condition information and positional relationship information with a second external display on a second intermediate image (822), and a third simulation image (833) by performing second image processing based on ambient condition information and positional relationship information with a third external display on a third intermediate image (823).

[0170] According to one embodiment, the first simulation image (831), the second simulation image (832), and the third simulation image (833) may have the same visual appearance. That is, a plurality of simulation images (830) obtained by performing a second image processing on each of the different intermediate images (820) may all have the same or similar shape and have the same perceptual image characteristics within a defined error range.

[0171] According to one embodiment, in order to make each simulation image of the electronic device (100) have the same appearance, different image processing setting information can be appropriately set to perform a first image processing. The appropriate setting can be determined through the inverse calculation of the second image processing.

[0172] For example, for the convenience of understanding and explanation, it is assumed that the first external display (211) is far from the electronic device (100) and at a large angle (e.g., viewed at an angle), and the second external display (212) is close and at a small angle (e.g., viewed close to the front), and the third external display (213) will be excluded from the explanation. Due to the influence of distance and angle, when an image is displayed on the first external display (211), the image seen from the position of the electronic device (100) may be perceived as having lower brightness and clarity than the displayed image. For example, considering each positional relationship information under the assumption that ambient condition information is the same, the second image processing for the first intermediate image (821) may include more algorithms related to brightness and / or clarity reduction relative to the second image processing for the second intermediate image (822). For example, in order for the first simulation image (831) and the second simulation image (832) to have the same visual appearance, it can be seen that, conversely, the first intermediate image (821) must have higher brightness and sharpness compared to the second intermediate image (822). That is, the first image processing that obtains the first intermediate image (821) from the same original image may include more algorithms related to increasing brightness and / or sharpness relatively compared to the first image processing that obtains the second intermediate image (822).

[0173] Explained through formulas containing exemplary numbers, it is as follows (1) to (7):

[0174] (1) The image displayed on the first external display is perceived as having a brightness of 0.5 times at the location of the electronic device, and the image displayed on the second external display is perceived as having a normal brightness (1.0 times) at the location of the electronic device (100). This is reflected in the second image processing.

[0175] (2) The brightness of the original image is am.

[0176] (3) Finally, the same brightness ( We intend to perceive the image as ). To do this, both the first simulation image and the second simulation image must be displayed with a brightness of 50.

[0177] (4) The brightness of the first intermediate image is and the brightness of the second intermediate image is am.

[0178] (5) According to item 1 above, the brightness of the first simulation image obtained by applying the second image processing to the first intermediate image is And, the brightness of the second simulation image obtained by applying second image processing to the second intermediate image is am.

[0179] (6) since, am.

[0180] (7) Therefore, the original image( From ) the first intermediate image( The first image processing for obtaining ) includes a brightness of 1.0x and a second intermediate image ( It can be seen that the first image processing for obtaining ) must include a brightness 0.5x algorithm. That is, the brightness adjustment algorithm of the first image processing for obtaining the first intermediate image, for example, compared to the brightness adjustment algorithm of the first image processing for obtaining the second intermediate image, may have an attribute value related to the brightness adjustment multiplier that is double.

[0181] According to one embodiment, when different intermediate images (820) are displayed as final images on corresponding external displays so that each of the plurality of simulated images (830) has the same visual appearance (e.g., when the first intermediate image (821) is displayed on the first external display (211)), the same visual experience can be provided to a user who actually views each external display at the location of the electronic device (100). According to one embodiment, the principle for simulating the same visual appearance described above can be applied in the same way to simulate a consistent visual appearance even when different original images are displayed on each external display.

[0182] FIG. 9 is a drawing showing an example in which multiple external displays according to various embodiments form a video wall.

[0183] According to one embodiment, a video wall (900) may refer to a display system in which, for example, a plurality of displays are physically arranged or synchronized through a network to function as one integrated screen.

[0184] According to one embodiment, the video wall may include an environment in which different external displays are present at different locations. The electronic device (100) may perform a first image processing that appropriately sets different image processing setting information by using the inverse of a second image processing based on each location relationship information and surrounding environment information (e.g., described in FIG. 8) so that each simulated image associated with the appearance when an image is displayed on each external display has a consistent visual appearance. Through this, the electronic device (100) can provide a consistent visual experience to a user viewing the video wall at the location of the electronic device (100).

[0185] According to one embodiment, at least some of the operations described as being performed by the electronic device (100) in the description of the drawings above can be implemented by at least one processor (110) executing at least one instruction (121).

[0186] The various embodiments described above may be implemented in software containing instructions stored on a device-readable storage medium, included in a computer program product in the form of a device-readable storage medium or distributed online through an application store, or implemented within a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

[0187] Each component according to the various embodiments described above may be composed of a single or multiple entities, and some auxiliary components may be omitted or additionally included. Some components may be integrated into a single entity to perform the same or similar functions as those performed by each corresponding component prior to integration.

[0188] The operations according to the various embodiments described above may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0189] Although the present disclosure has been illustrated and described with reference to various exemplary embodiments, it will be understood that the various exemplary embodiments are illustrative rather than limiting. It will be further understood by those skilled in the art that various modifications, alternatives, and / or variations may be made to the various exemplary embodiments without departing from the true technical spirit and the entire technical scope of the present disclosure, including the appended claims and their equivalents. It will also be understood that any embodiment(s) described herein may be used in conjunction with any other embodiment(s) described herein.

Claims

1. In an electronic device, Memory including at least one storage medium for storing instructions; Display; and at least one processor including a processing circuit; comprising, At least one processor, individually and / or collectively, causes the electronic device: A first image processing is performed on an original image to obtain at least one intermediate image, and the intermediate image includes an image for displaying through an external reflective display device. Based on at least one of ambient condition information or positional relationship information, a second image processing is performed on the at least one intermediate image to obtain at least one simulation image, wherein the ambient condition information includes ambient illuminance information, and the positional relationship information includes information on at least one of the distance and / or direction of the reflective display device to the electronic device. Configured to cause at least one simulation image to be displayed through the display, Electronic device.

2. In Paragraph 1, The second image processing described above is performed based on the surrounding condition information, the positional relationship information, and the optical characteristic information of the reflective display device, and The optical characteristic information of the reflective display device comprises at least one of the reflectance of the display of the reflective display device or the display luminance of the reflective display device. Electronic device.

3. In Paragraph 2, At least one processor, individually and / or collectively, causes the electronic device: A configuration configured to cause optical characteristic information of the reflective display device to be generated based on RGB (red-green-blue) data and ambient illumination information captured from the display of the reflective display device. Electronic device.

4. In Paragraph 2, The second image processing is performed based on the surrounding condition information, the positional relationship information, the optical characteristic information of the reflective display device, the display performance information of the reflective display device, and the display performance information of the electronic device. The above ambient condition information further includes at least one of location information of an ambient light source or ambient color temperature information, and the above performance information includes at least one of size or resolution. Electronic device.

5. In Paragraph 1, The first image processing described above includes image processing that obtains a plurality of different intermediate images based on different image processing setting information for an original image, wherein the image processing setting information includes information regarding an image processing algorithm, and the plurality of intermediate images include a plurality of images obtained based on a different type of image processing algorithm and / or a plurality of images obtained based on the same type of image processing algorithm and different attribute values. The second image processing described above includes image processing that obtains a plurality of simulation images from the plurality of intermediate images based on at least one of the surrounding condition information and / or the position relationship information. Electronic device.

6. In Paragraph 5, At least one processor, individually and / or collectively, causes the electronic device: Based on identifying that at least one image is selected among the plurality of simulation images, the system is configured to cause a signal to be transmitted to the reflective display device to display at least one intermediate image corresponding to the selected at least one image among the plurality of intermediate images as a final image. The reflective display device is configured to display the final image based on the received signal, Electronic device.

7. In Paragraph 1, The simulation image above is, Based on the fact that the above intermediate image is displayed through the display of the above reflective display device, the image includes an image related to the appearance to be seen at the position of the electronic device corresponding to the positional relationship information. Electronic device.

8. In Paragraph 1, At least one processor, individually and / or collectively, causes the electronic device: Identify changes in the above surrounding condition information, and Configured to cause an updated simulation image based on changes in the above surrounding condition information to be displayed through the above display, Electronic device.

9. In any one of paragraphs 1 through 7, At least one processor, individually and / or collectively, causes the electronic device: Configured to cause the brightness of displaying the simulation image on the display to be adjusted based on at least one of the above ambient condition information and / or the display brightness setting information of the electronic device, Electronic device.

10. In any one of paragraphs 1 through 7, At least one processor, individually and / or collectively, causes the electronic device: An image of the actual space including the above-mentioned reflective display device is captured, and Identify the location and shape of the reflective display device in the image of the actual space above, and The above simulation image is overlaid on the shape of the reflective display device on the image of the actual space, and Configured to cause the image of the actual space and the overlaid simulation image to be displayed on the display, Electronic device.

11. In Paragraph 1, The above position relationship information includes first position relationship information between the electronic device and an external first reflective display device and second position relationship information between an external second reflective display device, and At least one processor, individually and / or collectively, causes the electronic device: A first image processing is performed on a first original image to obtain at least one first intermediate image and a second intermediate image, wherein the first intermediate image includes an image for display through the first reflective display device and the second intermediate image includes an image for display through the second reflective display device, and is configured to cause such that the first intermediate image includes an image for display through the second reflective display device. The first simulation image obtained through second image processing based on the surrounding condition information and the first positional relationship with respect to the first intermediate image has the same visual appearance as the second simulation image obtained through second image processing based on the surrounding condition information and the second positional relationship with respect to the second intermediate image, and The above identical visual appearance includes an appearance having the same or similar shape and the same perceptual image features within a specified error range. Electronic device.

12. In Paragraph 10, At least one processor, individually and / or collectively, causes the electronic device: It is configured to cause at least one third intermediate image to be obtained by performing a first image processing on a second original image, and said third intermediate image includes an image to be displayed through the second reflective display device, With respect to the third intermediate image, the third simulation image obtained through second image processing based on the surrounding condition information and the second positional relationship has a visual appearance consistent with the first simulation image, and The above consistent visual appearance includes an appearance having the same perceptual image characteristics within a specified error range. Electronic device.

13. In a method of operating an electronic device, The operation of obtaining at least one intermediate image by performing a first image processing on the original image; An operation to obtain at least one simulation image by performing a second image processing on the at least one intermediate image based on at least one of surrounding condition information or positional relationship information; and The operation of displaying at least one simulation image through the display of the electronic device; is included, The above intermediate image includes an image for displaying through an external reflective display device, the above ambient condition information includes ambient illuminance information, and the above positional relationship information includes information on at least one of the distance and / or direction of the reflective display device to the electronic device. method.

14. In Paragraph 13, The second image processing described above is performed based on the surrounding condition information, the positional relationship information, and the optical characteristic information of the reflective display device, and The optical characteristic information of the reflective display device comprises at least one of the reflectance of the display of the reflective display device or the display brightness of the reflective display device. method.

15. In Paragraph 14, The operation of calculating optical characteristic information of the reflective display device based on RGB data and ambient illumination information captured from the display of the reflective display device; further comprising method.

Images