Electronic device and control method therefor

Abstract

This electronic device comprises: an image projection unit; a first sensor; a second sensor; a memory for storing one or more instructions; and at least one processor including a processing circuit and operatively connected to the image projection unit, the first sensor, the second sensor, and the memory. The at least one processor is configured to individually and / or collectively execute the one or more instructions so as to: acquire information on a projection distance to a projection surface on the basis of sensing data acquired through the first sensor; acquire illuminance information on the basis of sensing data acquired through the second sensor; acquire gain information for each region included in the projection surface on the basis of the projection distance information and the illuminance information; correct the gain information for each region on the basis of information on the correlation between the projection distance information and image quality information, and information on the correlation between the illumination information and the image quality information; acquire an output image by correcting an input image on the basis of the corrected gain information for each region; and control the image projection unit to project the output image onto the projection surface.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for projecting an image and a method for controlling the same.

[0002] Advances in electronic technology have led to the development and proliferation of various types of electronic devices. In particular, electronic devices used in various settings, such as homes, offices, and public spaces, have been continuously evolving in recent years.

[0003] In particular, beam projectors are being used in a variety of settings, including offices, theaters, homes, and stores, and the market is continuously expanding. The increasing efficiency and power savings of LED light sources have led to a proliferation of diverse form factors for home / portable beam projectors, and the market is growing by 10-15% annually.

[0004] Additionally, with the recent increase in wired and wireless interface capabilities between beam projector devices and external devices, it has become possible to select and output various contents in real time from various locations via Wi-Fi and 5G networks.

[0005] According to one embodiment, an electronic device includes: an image projector; a first sensor; a second sensor; a memory storing one or more instructions; and at least one processor including a processing circuit operatively connected to the image projector, the first sensor, the second sensor, and the memory; wherein the at least one processor is configured to individually and / or collectively execute the one or more instructions to: obtain projection distance information to a projection surface based on sensing data obtained through the first sensor; obtain illuminance information based on sensing data obtained through the second sensor; obtain gain information for each area included in the projection surface based on the projection distance information and the illuminance information; correct the gain information for each area based on correlation information between the projection distance information and image quality information and correlation information between the illuminance information and the image quality information; correct an input image based on the corrected gain information for each area to obtain an output image; and control the image projector to project the output image onto the projection surface.

[0006] According to one embodiment, the image quality information includes at least one of brightness information, color information, and sharpness information, and the at least one processor may be configured to individually and / or collectively correct the gain information for each region based on correlation information between the projection distance information and the brightness information and correlation information between the illuminance information and the brightness information to obtain first gain information corresponding to the brightness information, correct the gain information for each region based on correlation information between the projection distance information and the color information and correlation information between the illuminance information and the color information to obtain second gain information corresponding to color information, correct the gain information for each region based on correlation information between the projection distance information and the sharpness information and correlation information between the illuminance information and the sharpness information to obtain third gain information corresponding to sharpness information, and correct the input image based on the first gain information, the second gain information, and the third gain information to obtain the output image.

[0007] According to one embodiment, the at least one processor may be configured to individually and / or collectively analyze the input image to obtain image characteristic information, and correct the input image based on the image characteristic information, the first gain information, the second gain information, and the third gain information to obtain the output image.

[0008] According to one embodiment, the image characteristic information includes at least one of luminance histogram information, color histogram information, and outline information, and the at least one processor may be configured to individually and / or collectively correct the region-specific brightness information based on the luminance histogram information and the first gain information, correct the region-specific color information based on the color histogram information and the second gain information, and correct the region-specific sharpness information based on the outline information and the third gain information.

[0009] According to one embodiment, the at least one processor may individually and / or collectively correct brightness information and contrast information for each region based on the luminance histogram information and the first gain information, and correct saturation information and color temperature information for each region based on the color histogram information and the second gain information.

[0010] According to one embodiment, the at least one processor may be configured to, individually and / or collectively, identify maximum luminance information of the input image based on the luminance histogram information, adjust contrast gain for contrast information correction based on the identified maximum luminance information, identify maximum RGB information of the input image based on the color histogram information, adjust saturation gain for saturation information correction based on the identified maximum R / G / B information, and identify amount information and intensity information of an outline included in the input image based on the outline information, and adjust gain for sharpness correction based on the amount information and intensity information of the identified outline.

[0011] According to one embodiment, the at least one processor may be configured to individually and / or collectively analyze a change in the image quality information according to the projection distance information for each of a plurality of areas included in the projection surface to obtain local gain information for each area, analyze a change in the image quality information according to the illuminance information to obtain global gain information corresponding to the entire area of ​​the projection surface, and obtain gain information for each area based on the local gain information and the global gain information.

[0012] According to one embodiment, the local gain information for each region may include a gain map in the form of a gray image in which a gain value increases as a projection distance corresponding to each of the plurality of regions increases, and the global gain information may include a gain map in the form of a gray image in which a gain value increases as brightness included in the illuminance information increases.

[0013] According to one embodiment, the at least one processor may be configured to individually and / or collectively adjust the gain information such that the gain information increases from a center region to an edge region of the projection surface based on at least one of the projection distance information or the size information of the projection surface.

[0014] In one embodiment, the at least one processor may be configured to, individually and / or collectively, identify a region of interest within the input image and compensate for the region of interest based on the compensated region-specific gain information to obtain the output image.

[0015] A control method of an electronic device according to one embodiment includes the steps of: obtaining projection distance information and illuminance information to a projection surface; obtaining gain information for each area included in the projection surface based on the projection distance information and the illuminance information; correcting the gain information for each area based on correlation information between the projection distance information and the image quality information and correlation information between the illuminance information and the image quality information; correcting an input image based on the corrected gain information for each area to obtain an output image; and projecting the output image onto the projection surface.

[0016] A non-transitory computer-readable medium storing instructions that, when individually and / or collectively executed by at least one processor of an electronic device according to one embodiment, cause the electronic device to perform an operation, the operation includes: obtaining projection distance information and illuminance information to a projection surface; obtaining gain information for each area included in the projection surface based on the projection distance information and the illuminance information; correcting the gain information for each area based on correlation information between the projection distance information and the image quality information and correlation information between the illuminance information and the image quality information; correcting an input image based on the corrected gain information for each area to obtain an output image; and projecting the output image onto the projection surface.

[0017] Aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0018] FIG. 1 is a drawing for explaining an implementation example of an electronic device according to one embodiment.

[0019] FIG. 2A is a block diagram illustrating an exemplary electronic device according to various embodiments.

[0020] FIG. 2b is a block diagram illustrating an exemplary configuration of an exemplary electronic device according to various embodiments.

[0021] FIG. 3 is a flowchart illustrating an exemplary control method of an electronic device according to various embodiments.

[0022] FIG. 4 is a drawing for explaining an exemplary video signal correction method according to various embodiments.

[0023] FIG. 5 is a drawing for explaining in detail an exemplary video signal correction method according to various embodiments.

[0024] FIGS. 6A and 6B are drawings for explaining exemplary methods for obtaining gain information by region according to various embodiments.

[0025] FIG. 7 is a flowchart detailing an exemplary area-specific gain information correction method according to various embodiments.

[0026] FIGS. 8A and 8B are drawings for explaining a method for correcting gain information by region of a modeling module (540) according to various embodiments.

[0027] FIG. 9 is a flowchart illustrating an image correction method based on example region-specific gain information and image characteristics according to various embodiments.

[0028] FIG. 10 is a drawing for explaining in detail an exemplary image analysis method of an image analysis module (550) according to various embodiments.

[0029] FIG. 11 is a drawing for explaining in detail an exemplary image correction method of an image correction module (560) according to various embodiments.

[0030] FIG. 12 is a drawing for explaining an exemplary sharpness correction processing method according to various embodiments.

[0031] FIG. 13 is a drawing for explaining an exemplary brightness difference compensation method according to projection distance according to various embodiments.

[0032] FIG. 14 is a diagram illustrating an exemplary region of interest compensation method according to various embodiments.

[0033] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0034] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0035] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0036] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0037] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0038] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0039] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware, for example.

[0040] In some contexts, the phrase "a device configured to" may mean, for example, that the device is "capable of" doing something in conjunction with other devices or components. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those actions, or a general-purpose processor (e.g., a CPU or application processor) that can perform those actions by executing one or more software programs stored in a memory device.

[0041] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this disclosure, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0042] In various examples, a "module" or "part" performs at least one function or operation and may be implemented in hardware, software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented by at least one processor (not shown), excluding any "modules" or "parts" that require specific hardware implementation.

[0043] The various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacings drawn in the attached drawings.

[0044] FIG. 1 is a drawing for explaining an implementation example of an electronic device according to various embodiments.

[0045] An electronic device (100) according to one embodiment may have a function of projecting an image, for example, a projector function. For example, the electronic device (100) may be a projector device that projects an image onto a wall or a projection surface, and the projector device may be an LCD projector or a DLP (digital light processing) projector using a DMD (digital micromirror device).

[0046] In addition, the electronic device (100) may be implemented as a home or industrial display device, a lighting device used in daily life, an audio device including an audio module, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a wearable device, or a home appliance device. However, the electronic device (100) is not limited to the above-described devices, and the electronic device (100) may be implemented as an electronic device (100) having two or more functions of the above-described devices. For example, the electronic device (100) may be used as a display device, a lighting device, or an audio device by turning off the projector function and turning on the lighting function or the speaker function according to the operation of the processor, and may be used as an AI speaker including a microphone or a communication device.

[0047] As illustrated in FIG. 1, a problem may arise in which the visibility of the projector screen is reduced depending on the distance between the electronic device (100) and the projection surface, or / and the ambient lighting.

[0048] Accordingly, below, various embodiments that can improve the visibility of a projector screen by considering the influence of image quality according to the viewing environment will be described.

[0049] FIG. 2A is a block diagram showing an exemplary configuration of an electronic device according to various embodiments.

[0050] According to FIG. 2A, the electronic device (100) may include an image projection unit (110), a memory (120), and one or more processors (150) (e.g., including a processing circuit). The electronic device (100) may be implemented as a projector that projects an image onto a wall, a projection surface, or a projection surface, or various types of devices having an image projection function.

[0051] According to FIG. 2a, the electronic device (100) may include an image projector (110), a sensor (120), a memory (120), and one or more processors (140).

[0052] The image projection unit (110) can perform a function of projecting light to the outside to express an image and outputting the image on a projection surface. Here, the projection surface may be part of the physical space where the image is output or a separate projection surface. The image projection unit (110) may include various detailed components such as a light source of at least one of a lamp, an LED, and a laser, a projection lens, and a reflector.

[0053] The image projection unit (110) can project an image using one of various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.). The image projection unit (110) can include at least one light source.

[0054] The image projection unit (110) can output images with a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or the user's settings, and can output images with various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the screen ratio.

[0055] The image projector (110) can perform various functions for adjusting the projected image under the control of the processor (120). For example, the image projector (110) can perform a zoom in / out function, a lens shift function, etc. The zoom in / out function can include a hardware method for adjusting the screen size by moving the lens and a software method for adjusting the screen size by cropping the image, etc. Meanwhile, when the zoom in / out function is performed, it is necessary to adjust the focus of the image. For example, the method for adjusting the focus can include a manual focus method, an electric method, etc.

[0056] The image projection unit (110) can automatically analyze the surrounding environment and projection environment without user input to provide zoom / keystone / focus functions. For example, the image projection unit (110) can automatically provide zoom / keystone / focus functions based on the distance between the electronic device (100) and the projection surface, information about the space where the electronic device (100) is currently located, information about the amount of ambient light, etc.

[0057] The memory (120) can store data required for various embodiments. Depending on the purpose of data storage, the memory (120) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100). In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)). In addition, in the case of memory that can be attached or detached to the electronic device (100'), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented.

[0058] According to an example, the memory (120) may store various information for correcting image quality according to an embodiment. For example, the memory (120) may store correlation information between projection distance information and image quality information and correlation information between illuminance information and image quality information.

[0059] The first sensor (130) may include a distance sensor. A distance sensor is a component for measuring the distance from a projection surface. For example, the distance sensor may be implemented in various types, such as a ToF (Time of Flight) sensor, an ultrasonic sensor, an infrared sensor, a LIDAR sensor, a RADAR sensor, or a photodiode sensor.

[0060] The second sensor (140) may include a light sensor. The light sensor is a component for measuring ambient brightness and / or color temperature. For example, the light sensor may be implemented so that its resistance value changes depending on the ambient brightness. For example, the light sensor may be implemented so that the resistance value of the light sensor decreases when the surroundings are bright, and the resistance of the light sensor increases when the surroundings are dark, thereby measuring the ambient brightness.

[0061] In addition, the electronic device (100) may include various types of sensors such as an image sensor, a touch sensor, a proximity sensor, an acceleration sensor, a geomagnetic sensor, a gyro sensor, a pressure sensor, a position sensor, etc.

[0062] One or more processors (150) include various processing circuits and control the overall operation of the electronic device (100). Specifically, one or more processors (150) may be connected to each component of the electronic device (100) and control the overall operation of the electronic device (100). For example, one or more processors (150) may be operatively connected to the display (130) and the memory (120). The processor (150) may include one or more processors.

[0063] One or more processors (150) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (120).

[0064] The one or more processors (150) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (150) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (150) may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform methods according to various embodiments of the present disclosure by executing one or more instructions stored in a memory.

[0065] When a method according to various embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to various embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0066] One or more processors (150) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (150) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to various embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to various embodiments of the present disclosure.

[0067] When a method according to various embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to various embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0068] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto. Hereinafter, for convenience of description, one or more processors (150) will be referred to as processor (150). That is, the processor (150) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in the present disclosure, including the claims, may include various processing circuits including at least one processor, and one or more of the at least one processor may be configured to individually and / or collectively perform various functions described herein in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms may encompass, for example, without limitation, a single processor performing some of the recited functions, other processors performing other functions of the recited functions, and even situations where a single processor can perform all of the recited functions. Additionally, the at least one processor may comprise a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0069] According to one embodiment, the processor (150) may obtain projection distance information to the projection surface based on sensing data obtained through the first sensor (130). According to one example, the projection distance information may include projection distance information for each of a plurality of areas included in the projection surface. For example, the plurality of areas may be classified as areas that can be sensed by the first sensor (130). For example, when the first sensor (130) is implemented as a ToF sensor, the plurality of areas may be identified as a plurality of areas in which a plurality of lights emitted by the ToF sensor are reflected.

[0070] According to one embodiment, the processor (150) may obtain illuminance information based on sensing data obtained through the second sensor (130). According to one example, the illuminance information may include at least one of brightness information and color temperature information. The brightness information may represent the brightness of the ambient illuminance caused by the light source. For example, the brightness information may represent the brightness of light on a surface of 1 m2 when light of 1 lm (lumen) is evenly irradiated on that surface. The color temperature information may be information that represents the color of the light source numerically using absolute temperature. For example, the redder the light source, the lower the color temperature, and the bluer the light source, the higher the color temperature. The temperature can be expressed using Kelvin (K), which is a traditional absolute temperature unit.

[0071] According to one embodiment, the processor (150) can obtain gain information for each area included in the projection surface based on projection distance information and illuminance information.

[0072] For example, the processor (150) may obtain local gain information for each region based on the projection distance for each region of the projection surface. For example, the processor (150) may analyze the change in image quality information according to the projection distance for each of the plurality of regions included in the projection surface to obtain local gain information for each region. For example, the local gain information for each region may include a gain map in the form of a gray image in which the gain value increases as the projection distance corresponding to each of the plurality of regions increases. However, the present invention is not limited thereto, and the local gain information for each region may be implemented in the form of a gain map including gain values ​​such as % information, ratio information between 0 and 1, etc.

[0073] For example, the processor (150) may obtain global gain information corresponding to the entire projection surface area based on illuminance information. For example, the processor (150) may obtain global gain information corresponding to the entire projection surface area based on a change in image quality information according to at least one of brightness information and color temperature information. For example, the global gain information may include a gain map in the form of a gray image in which a gain value increases as the brightness included in the illuminance information increases. For example, the gain map may include a gain value for each pixel block. In the present disclosure, a “pixel block” refers to, for example, one pixel or a set of adjacent pixels including at least one pixel, and a “region” refers to, for example, a part of an image and may mean at least one pixel block or a set of pixel blocks. Hereinafter, for convenience of explanation, a pixel block will be referred to as a “pixel.”

[0074] For example, the processor (150) can obtain region-specific gain information based on region-specific local gain information obtained based on projection distance information and global gain information obtained based on illumination information.

[0075] According to one embodiment, the processor (150) may correct gain information for each region based on correlation information between projection distance information and image quality information, and correlation information between illuminance information and image quality information. According to one example, the image quality information may include at least one of brightness information, color information, and sharpness information.

[0076] According to one embodiment, the processor (150) can control the image projection unit (110) to obtain an output image by correcting the input image based on the corrected area-specific gain information, and to project the output image onto a projection surface.

[0077] According to one embodiment, the processor (150) may obtain first gain information corresponding to the brightness information by correcting gain information for each area based on correlation information between projection distance information and brightness information and correlation information between illuminance information and brightness information. For example, the first gain information may be in the form of a gain map corresponding to the brightness information.

[0078] According to one embodiment, the processor (150) may obtain second gain information corresponding to color information by correcting gain information for each area based on correlation information between projection distance information and color information and correlation information between illuminance information and color information. For example, the first gain information may be in the form of a gain map corresponding to color information.

[0079] According to one embodiment, the processor (150) may obtain third gain information corresponding to the sharpness information by correcting gain information for each area based on correlation information between projection distance information and sharpness information and correlation information between illuminance information and sharpness information. For example, the third gain information may be in the form of a gain map corresponding to the sharpness information.

[0080] According to one embodiment, the processor (150) can obtain an output image by correcting an input image based on the first gain information, the second gain information, and the third gain information.

[0081] According to one embodiment, the processor (150) may analyze an input image to obtain image characteristic information, and may correct the input image based on the image characteristic information, the first gain information, the second gain information, and the third gain information to obtain an output image. According to one example, the image characteristic information may include at least one of luminance histogram information, color histogram information, and contour information. For example, the luminance histogram information may include a graph that displays luminance values ​​on the horizontal axis and the number of pixels corresponding to each luminance value on the vertical axis, and is represented in a rectangular shape. For example, the color histogram information may include a graph that displays color values ​​on the horizontal axis and the number of pixels corresponding to each color value on the vertical axis, and is represented in a rectangular shape. For example, the contour information may include a contour map image that represents the distribution of contours in the image in the form of a map. For example, the contour map image may include at least one of position information (or coordinate information), magnitude information, and direction information for pixels detected as contours (or boundaries). In the present disclosure, a contour can be distinguished from a complex edge with various directions in that it refers to an edge with a clear direction and a straight edge, and / or an edge with a clear direction and a thickness greater than a threshold. For example, the processor (150) can obtain a contour map image by applying a predetermined filter to an input image.For example, the filter may include, but is not limited to, at least one of a second-order differential filter, a Laplacian filter, a Roberts filter, a Sobel filter, a directional filter, a gradient filter, a difference filter, or a Prewitte filter. Applying a filter to an input image may mean convolving the filter with the input image. Convolution is an image processing technique that uses a filter with weights, and refers to a technique of multiplying the pixel values ​​of the input image by the corresponding weights (or coefficients) included in the filter and then obtaining the sum. Here, the filter is also called a mask, a window, or a kernel.

[0082] According to one embodiment, the processor (150) can correct brightness information for each region based on luminance histogram information and first gain information. According to one example, the processor (150) can correct at least one of brightness information and contrast information for each region based on luminance histogram information and first gain information. For example, the processor (150) can identify maximum luminance information of an input image based on luminance histogram information, and adjust contrast gain for contrast information correction based on the identified maximum luminance information.

[0083] According to one embodiment, the processor (150) can correct color information for each region based on color histogram information and second gain information. According to one example, the processor (150) can correct at least one of saturation information and color temperature information for each region based on color histogram information and second gain information. For example, the processor (150) can identify maximum RGB information of an input image based on color histogram information, and adjust saturation gain for saturation information correction based on the identified maximum RGB information.

[0084] According to one embodiment, the processor (150) may correct sharpness information for each region based on outline information and third gain information. For example, the processor (150) may identify the amount information and intensity information of the outline included in the input image based on the outline information, and adjust the gain for sharpness correction based on the amount information and intensity information of the identified outline.

[0085] According to one embodiment, the processor (150) may adjust gain information so that the gain information for each area increases from the center area of ​​the projection surface to the edge area based on at least one of projection distance information and projection surface size information.

[0086] According to one embodiment, the processor (150) can identify a region of interest within an input image and obtain an output image by correcting the region of interest based on gain information for each corrected region.

[0087] FIG. 2b is a drawing for explaining a detailed configuration of an electronic device according to various embodiments.

[0088] According to FIG. 2b, the electronic device (100') includes an image projector (110), a memory (120), a first sensor (130), a second sensor (140), and one or more processors (150) (e.g., including a processing circuit), a communication interface (160) (e.g., including a communication circuit), a user interface (170) (e.g., including a circuit), and a camera (180). A detailed description of the configurations illustrated in FIG. 2b that overlap with the configuration illustrated in FIG. 2a may not be repeated.

[0089] The communication interface (160) includes various communication circuits and can perform communication with external devices (servers or user terminals). For example, the processor (150) can receive various data or information from an external device connected via the communication interface (160) and can also transmit various data or information to the external device.

[0090] The communication interface (160) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band). At this time, the wireless communication module may perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc.

[0091] In addition, the communication interface (160) may perform communication according to a communication method such as Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc., depending on the implementation example of the electronic device (100').

[0092] The user interface (170) includes various circuits and may be implemented as devices such as buttons, touch pads, mice, and keyboards, or as a touch projection surface or remote control transmitter / receiver that can also perform the above-described display function and operation input function. The remote control transmitter / receiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one of infrared communication, Bluetooth communication, and Wi-Fi communication.

[0093] The camera (180) can be turned on and perform shooting according to a preset event. The camera (180) can convert the captured image into an electrical signal and generate image data based on the converted signal. For example, the subject can be converted into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), and the converted image signal can be amplified and converted into a digital signal and then signal processed.

[0094] In addition, the electronic device (100') may include a microphone or the like, depending on the implementation example. The microphone is configured to receive user voice or other sounds and convert them into audio data. However, according to another embodiment, the electronic device (100') may receive user voice input via an external device through the communication interface (160).

[0095] FIG. 3 is a flowchart for explaining a method of controlling an electronic device according to various embodiments.

[0096] According to FIG. 3, in operation 310, the electronic device (100) may obtain projection distance information to the projection surface. In one example, the electronic device (100) may obtain projection distance information to the projection surface based on sensing data obtained through the first sensor (130). For example, the projection distance information may include projection distance information for a plurality of areas included in the projection surface.

[0097] In operation 320, the electronic device (100) may obtain illumination information. In one example, the electronic device (100) may obtain illumination information based on sensing data obtained through the second sensor (130). For example, the illumination information may include at least one of brightness information and color temperature information.

[0098] In operation 330, the electronic device (100) may obtain gain information for each region included in the projection surface based on projection distance information and illuminance information. In one example, the processor (150) may obtain local gain information for each region based on projection distances for each region of the projection surface, and may obtain global gain information corresponding to the entire projection surface area based on illuminance information. In one example, the processor (150) may obtain gain information for each region based on local gain information for each region obtained based on projection distance information and global gain information obtained based on illuminance information.

[0099] In operation 340, the electronic device (100) may correct gain information for each region based on correlation information between projection distance information and image quality information and correlation information between illuminance information and image quality information. According to an example, the image quality information may include at least one of brightness information, color information, and sharpness information.

[0100] In operation 350, the electronic device (100) can obtain an output image by correcting the input image based on the corrected area-specific gain information.

[0101] In operation 360, the electronic device (100) can project the output image onto a projection surface.

[0102] In Fig. 3, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0103] FIG. 4 is a drawing for explaining a video signal correction method according to various embodiments.

[0104] According to Fig. 4, if the input image (10) is projected as is onto the projection surface, the visibility of the projection screen may decrease depending on the surrounding lighting and the projection distance of the projector. Accordingly, as illustrated in Fig. 3, the image signal (420) of the input image (10) is corrected based on the region-specific gain map information (410) obtained by analyzing the image quality influence of each region of the projection screen to obtain an output image (20), and the obtained output image (20) is projected onto the projection surface to improve the visibility of the projection screen.

[0105] Below, we will explain the video signal correction method in more detail.

[0106] FIG. 5 is a drawing for explaining in detail a video signal correction method according to various embodiments.

[0107] As illustrated in FIG. 5, the processor (150) may obtain an output image using a distance sensing module (510) (e.g., including various circuits and / or executable program instructions), an illuminance / color temperature sensing module (520) (e.g., including various circuits and / or executable program instructions), a screen area calculation module (530) (e.g., including various circuits and / or executable program instructions), a modeling module (540) (e.g., including various circuits and / or executable program instructions), an image analysis module (550) (e.g., including various circuits and / or executable program instructions), and an image correction module (560) (e.g., including various circuits and / or executable program instructions). Each module may be implemented with at least one software, at least one hardware, and / or a combination thereof.

[0108] For example, at least one of the distance sensing module (510), the illuminance / color temperature sensing module (520), the screen area calculation module (530), the modeling module (540), the image analysis module (550), and the image correction module (560) may be implemented to use a predefined algorithm, a predefined formula, and / or a learned artificial intelligence model. The distance sensing module (510), the illuminance / color temperature sensing module (520), the screen area calculation module (530), the modeling module (540), the image analysis module (550), and the image correction module (560) may be included within the electronic device (100), but may be distributed to at least one external device according to an example.

[0109] In one example, the processor (150) can obtain projection distance information to a projection surface using a distance sensing module (510). In one example, the projection distance information can include projection distance information for a plurality of areas included in the projection surface.

[0110] For example, the distance sensing module (510) can measure the projection distance between the electronic device (100) and the projection surface by area of ​​the projection surface (for example, by area of ​​5x4 or 50x40 units) using the first sensor (120). For example, when the first sensor (120) is implemented as a ToF sensor, a plurality of areas are identified as a plurality of areas in which a plurality of lights emitted by the ToF sensor are reflected, and the projection distance for each area can be measured according to the depth value of the ToF sensor.

[0111] According to one example, the processor (120) can obtain illuminance information using the illuminance / color temperature sensing module (520). According to one example, the illuminance information can include at least one of brightness information and color temperature information.

[0112] For example, the illuminance / color temperature sensing module (520) can measure the illuminance brightness and color temperature of the surrounding environment as continuous values ​​or by dividing them into intervals using the second sensor (130).

[0113] According to an example, the processor (120) can obtain gain information for each area using the screen area calculation module (530).

[0114] For example, the screen area calculation module (530) can obtain a gain map by analyzing information on changes in image quality of the projection surface according to the illuminance brightness / color temperature and projection distance by region. For example, the screen area calculation module (530) can obtain a gain map in the form of a gray image including a gain value for each pixel. For example, the gray value may be a bitmap having a value of 0 to 255 in the case of an 8-bit image, but is not limited thereto. For example, the gain map may include a gain map in the form of a gray image in which the gain value according to the projection distance increases for each region and the gain value increases as the illuminance brightness increases. This is because the image quality may deteriorate and visibility may decrease as the projection distance increases and the illuminance brightness increases.

[0115] FIGS. 6A and 6B are drawings for explaining a method for obtaining gain information by region according to various embodiments.

[0116] FIG. 6a is a drawing for explaining a gain map obtained in a viewing environment of illuminance brightness A lux, B lux (e.g., 50 lux, 100 lux), color temperature CK (e.g., 3000 K), and projection distance d1 (e.g., 2 m).

[0117] For example, the first sensor data (610) is data acquired through a ToF sensor, and the depth value for each area may represent the projection distance.

[0118] For example, the first gain map (621) may be a gain map measured at an illuminance of 50 lux, a color temperature of 3000 K, and a projection distance of 2 m.

[0119] For example, the second gain map (622) may be a gain map measured at an illuminance of 100 lux, a color temperature of 3000 K, and a projection distance of 2 m.

[0120] For example, according to FIG. 6a, since the projection distances are the same for each area when projected frontally at a distance of 2 m, the first gain map (621) and the second gain map (622) may be gain maps that include the same gain value for each area.

[0121] Fig. 6b is a diagram for explaining a gain map obtained in a viewing environment of illuminance A lux, B lux (e.g., 50 lux, 100 lux), color temperature DK (e.g., 5000 K), and projection distance d2 (e.g., 4 m). For example, since projection is performed from bottom to top at a distance of 4 m and the projection distance is different for each area, the gain according to the projection distance may be a local gain that is different for each area.

[0122] For example, the second sensor data (630) is data acquired through a ToF sensor, and the depth value for each area may represent the projection distance.

[0123] For example, the first gain map (641) may be a gain map measured at an illuminance of 50 lux, a color temperature of 3000 K, and a projection distance of 4 m.

[0124] For example, the second gain map (642) may be a gain map measured at an illuminance of 100 lux, a color temperature of 3000 K, and a projection distance of 4 m.

[0125] For example, according to FIG. 6b, since the projection distances are different for each area when projected from the bottom up at a distance of 4 m, the third gain map (641) and the second gain map (642) may be gain maps that include different gain values ​​for each area.

[0126] FIG. 7 is a flowchart for explaining in detail a method for compensating gain information by region according to various embodiments.

[0127] According to one embodiment, the electronic device (100) can correct gain information for each region according to correlation using the modeling module (540).

[0128] According to FIG. 7, in operation 710, the electronic device (100) may obtain first gain information corresponding to the brightness information by correcting gain information for each area based on correlation information between projection distance information and brightness information and correlation information between illuminance information and brightness information. For example, the first gain information may be in the form of a gain map corresponding to the brightness information.

[0129] In operation 720, the electronic device (100) may obtain second gain information corresponding to color information by correcting gain information for each region based on correlation information between projection distance information and color information and correlation information between illuminance information and color information. For example, the second gain information may be in the form of a gain map corresponding to color information.

[0130] In operation 730, the electronic device (100) may obtain third gain information corresponding to the sharpness information by correcting gain information for each area based on correlation information between projection distance information and sharpness information and correlation information between illuminance information and sharpness information. For example, the third gain information may be in the form of a gain map corresponding to the sharpness information.

[0131] In operation 740, the electronic device (100) can obtain an output image by correcting the input image based on the first gain information, the second gain information, and the third gain information.

[0132] Meanwhile, in Fig. 7, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0133] FIGS. 8A and 8B illustrate graphs and tables for explaining exemplary methods of area-specific gain information correction of a modeling module (540) according to various embodiments.

[0134] According to one embodiment, each piece of image characteristic information may have a correlation with each piece of viewing environment information, as illustrated in FIG. 8A. For example, the viewing environment information may include projection distance, illuminance brightness, and color temperature information. For example, the image characteristic information may include contrast information, color information, and sharpness information.

[0135] According to an example, the modeling module (540) can derive an image correction function and calculate an image signal correction amount by using a correlation model between illuminance, color temperature, projection distance, and image characteristic information (e.g., contrast, color, sharpness) that affect visibility.

[0136] For example, the graphs illustrated in Figure 8a may be correlation models representing the correlation between each image characteristic information and viewing environment information. The correlation model may be obtained based on experimental data obtained through preliminary experiments.

[0137] For example, a correlation model can be obtained based on proportional coefficients corresponding to each correlation calculated based on experimental data, as shown in the table in Fig. 8b. For example, as the illuminance brightness / projection distance increases, the contrast ratio decreases, so the compensation gain can be increased.

[0138] According to an example, the modeling module (540) can calculate a gain value for image quality correction using the acquired correlation model.

[0139] For example, the modeling module (540) can derive functional formulas for illuminance, color temperature, projection distance, contrast gain, saturation gain, sharpness gain, and white balance gain. For example, in the case of saturation, a functional formula for calculating a gain value for image quality correction can be obtained as follows using the saturation component in the HSV (hue saturation value) color space. The HSV color space refers to a method of specifying a specific color using the coordinates of hue, saturation, and value.

[0140] Contrast Gain = f1 (illuminance, projection distance)

[0141] Saturation Gain = f2 (illuminance, projection distance)

[0142] Sharpness Gain = f3 (illuminance, projection distance)

[0143] White Balance Gain = f4 (color temperature)

[0144] FIG. 9 is a flowchart illustrating an example of an image correction method based on region-specific gain information and image characteristics according to various embodiments.

[0145] According to FIG. 9, in operation 910, the electronic device (100) can analyze the input image to obtain at least one of luminance histogram information, color histogram information, and outline information.

[0146] In operation 920, the electronic device (100) can correct the brightness information for each region based on the brightness histogram information and the gain information for each region corresponding to the brightness information.

[0147] In operation 930, the electronic device (100) can correct color information for each region based on color histogram information and gain information for each region corresponding to the color information.

[0148] In operation 940, the electronic device (100) can correct the sharpness information for each region based on the gain information for each region corresponding to the outline information and the sharpness information.

[0149] In operation 950, the electronic device (100) can obtain an output image based on the corrected area-specific brightness information, color information, and sharpness information.

[0150] Meanwhile, in Fig. 9, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0151] FIG. 10 is a drawing for explaining an example of an image analysis method of an image analysis module (550) according to various embodiments.

[0152] According to FIG. 10, the image analysis module (550) can analyze the input image (10) to obtain image characteristic information. For example, the image characteristic information can include at least one of luminance histogram information, color histogram information, and contour distribution information. For example, the luminance histogram information can include a graph that displays luminance values ​​on the horizontal axis and the number of pixels corresponding to each luminance value on the vertical axis, representing the graph in a rectangular shape. For example, the color histogram information can include a graph that displays color values ​​on the horizontal axis and the number of pixels corresponding to each color value on the vertical axis, representing the graph in a rectangular shape. For example, the contour information can include a contour map image that represents the distribution of contours in the image in the form of a map.

[0153] For example, the image analysis module (550) can adjust the gain correction amount for each region based on the image characteristic information of the input image for precise image correction.

[0154] For example, the image analysis module (550) may adjust the contrast gain according to the maximum luminance value of the acquired image based on luminance histogram information. For example, the gain adjustment range may increase as the maximum luminance value decreases.

[0155] For example, the image analysis module (550) can adjust the saturation gain according to the maximum R / G / B value of the acquired image based on color histogram information. For example, the smaller the maximum R / G / B value, the larger the gain adjustment range. For example, the image analysis module (550) can adjust the saturation gain using the saturation component in the HSV color space.

[0156] For example, the image analysis module (550) can adjust the sharpness gain according to the amount and intensity of the outline included in the outline distribution information. For example, the gain value can be increased as the intensity of the outline increases for each pixel. For example, the image analysis module (550) can adjust the sharpness gain by using Y-component separation after converting the RGB value to YUV. YUV is a component signal that constitutes an image signal with luminance and chrominance.

[0157] FIG. 11 is a drawing for explaining an example of an image correction method of an image correction module (560) according to various embodiments.

[0158] According to one embodiment, the image correction module (560) can combine gain information obtained by the screen area calculation module (530), the modeling module (540), and the image analysis module (550), and correct the brightness, color, and sharpness of the input image (10) based on the combined gain information to obtain an output image (20).

[0159] According to one example, when the area-specific gain information obtained by the screen area calculation module (530) is corrected by the modeling module (540), the image signal of the input image (10) can be corrected based on the area-specific gain information corrected by the modeling module (540) and the image characteristic information obtained by the image analysis module (550).

[0160] For example, the image correction module (560) can correct brightness information for each region of the input image signal based on the brightness histogram information and the first gain information. For example, the image correction module (560) can obtain a brightness gain map by multiplying the gain map obtained by the screen area calculation module (530) and the contrast gain obtained by the modeling module (540), and perform brightness correction processing for each region based on the brightness gain map. Here, the contrast gain can have a gain range adjusted by the image analysis module (550).

[0161] For example, the image correction module (560) can correct color information for each region of the input image signal based on color histogram information and second gain information. For example, the image correction module (560) can obtain a color gain map by multiplying the gain map obtained by the screen region calculation module (530) and the saturation gain obtained by the modeling module (540), and perform color correction processing for each region based on the color gain map. Here, the saturation gain can have a gain range adjusted by the image analysis module (550). For example, the image correction module (560) can apply the same method to the White Balance gain.

[0162] For example, the image correction module (560) can correct the sharpness information of each region of the input image signal based on the outline information and the third gain information. For example, the image correction module (560) can obtain a sharpness gain map by multiplying the gain map obtained by the screen area calculation module (530) and the sharpness gain obtained by the modeling module (540), and perform sharpness correction processing for each region based on the sharpness gain map. Here, the sharpness gain can have a gain range adjusted by the image analysis module (550).

[0163] FIG. 12 is a drawing for explaining an example of a sharpness correction processing method according to various embodiments.

[0164] According to FIG. 12, the sharpness of the outline area can be corrected by processing the pixel value (1210) in the outline area using at least one of the first method (1220), the second method (1230), and the third method (1240).

[0165] FIG. 13 is a drawing for explaining a method for compensating for brightness differences according to projection distance according to various embodiments.

[0166] According to one embodiment, the electronic device (100) can adjust gain information so that the gain information increases from the center area of ​​the projection surface to the edge area based on at least one of the projection distance or the size information of the projection surface.

[0167] For example, the electronic device (100) may adjust the gain map (1310, 1320) so that the gain of brightness and / or color, etc. in the edge area increases by radially adjusting the gain map as the projection distance (d1, d2) or the size of the projection surface increases. This is because the screen size increases as the projection distance increases and the difference in brightness between the center area of ​​the screen and the edge area can be relatively more noticeable.

[0168] FIG. 14 is a diagram illustrating an example of a region of interest compensation method according to various embodiments.

[0169] According to one embodiment, the electronic device (100) can identify a region of interest within an input image, correct the region of interest based on gain information for each corrected region, and obtain an output image. For example, the region of interest can be determined according to various region of interest setting methods based on the arrangement of objects, the type of objects, the number of objects, the size of objects, etc. The region of interest can be set in various shapes such as a square, a circle, an oval, and an irregular shape.

[0170] For example, the electronic device (100) may apply image quality enhancement to a region of interest, and leave no processing or apply image quality degradation to a region of non-interest. For example, since visibility is reduced under bright lighting (1410), the electronic device (100) may enhance the contrast, color, and sharpness of the region of interest and degrade the contrast of the region of non-interest (1420) to improve relative visibility within the image. Accordingly, the relative contrast ratio and / or sharpness of the region of interest may be improved due to the out-of-focus effect of the region of non-interest.

[0171] According to various embodiments, the electronic device (100) may input an input image, a corrected gain map, and image characteristic information into a learned artificial intelligence model to obtain an output image. In one example, the artificial intelligence model may be trained to correct the input image based on the corrected gain map and image characteristic information to obtain an output image.

[0172] When an AI model is trained, it means that a basic AI model (e.g., an AI model including arbitrary random parameters) is trained using a learning algorithm using a large amount of training data, thereby creating a predefined set of operating rules or an AI model that is set to perform a desired characteristic (or purpose). This learning may be performed through a separate server and / or system, but is not limited thereto, and may also be performed in a cooking device. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0173] The artificial intelligence model can be implemented as, but is not limited to, a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), or a Deep Q-Network.

[0174] According to the various embodiments described above, the problem of reduced visibility of the projector screen depending on the ambient lighting and the projector projection distance can be improved through image signal correction.

[0175] The methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0176] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0177] According to an example embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., electronic device A) according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When an instruction is executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instruction using other components. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. 'Non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0178] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0179] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0180] While the present disclosure has been illustrated and described with reference to various exemplary embodiments, it will be understood that the various exemplary embodiments are illustrative and not limiting. It will be further understood by those skilled in the art that various changes in form and detail may be made without departing from the true spirit and scope of the present disclosure, including the appended claims and their equivalents. It will also be understood that any embodiment(s) described herein can be used in conjunction with any other embodiment(s) described herein.

Claims

1. In electronic devices, Video projection unit; Sensor 1; Second sensor; A memory that stores one or more instructions; and At least one processor including processing circuitry operatively connected to the image projector, the first sensor, the second sensor and the memory; The at least one processor individually and / or collectively executes the one or more instructions, Obtain projection distance information to the projection surface based on the sensing data acquired through the first sensor, Obtaining illumination information based on sensing data acquired through the second sensor, Based on the projection distance information and the illuminance information, gain information for each area included in the projection surface is obtained, The gain information for each area is corrected based on the correlation information between the projection distance information and the image quality information and the correlation information between the illuminance information and the image quality information. Correct the input image based on the gain information for each area corrected above to obtain an output image, An electronic device configured to control the image projector to project the output image onto the projection surface.

2. In paragraph 1, The above image quality information is, Contains at least one of brightness information, color information, and sharpness information, The at least one processor, individually and / or collectively, corrects the gain information for each area based on the correlation information between the projection distance information and the brightness information and the correlation information between the illuminance information and the brightness information to obtain first gain information corresponding to the brightness information, Based on the correlation information between the projection distance information and the color information, and the correlation information between the illuminance information and the color information, the gain information for each area is corrected to obtain second gain information corresponding to the color information, Based on the correlation information between the projection distance information and the sharpness information and the correlation information between the illuminance information and the sharpness information, the gain information for each area is corrected to obtain third gain information corresponding to the sharpness information. An electronic device configured to obtain the output image by correcting the input image based on the first gain information, the second gain information, and the third gain information.

3. In paragraph 2, The at least one processor, individually and / or collectively, By analyzing the above input image, image characteristic information is obtained, An electronic device configured to obtain the output image by correcting the input image based on the image characteristic information, the first gain information, the second gain information, and the third gain information.

4. In paragraph 3, The above video characteristic information is, Contains at least one of luminance histogram information, color histogram information, and outline information, The at least one processor, individually and / or collectively, Corrects the brightness information for each area based on the above brightness histogram information and the first gain information, Correcting color information for each region based on the color histogram information and the second gain information, An electronic device configured to correct the sharpness information for each region based on the outline information and the third gain information.

5. In paragraph 4, The at least one processor, individually and / or collectively, Correcting brightness information and contrast information for each area based on the above brightness histogram information and the first gain information, An electronic device configured to correct saturation information and color temperature information for each area based on the color histogram information and the second gain information.

6. In paragraph 5, The at least one processor, individually and / or collectively, identifies maximum luminance information of the input image based on the luminance histogram information, Adjust the contrast gain for contrast information correction based on the identified maximum luminance information, Based on the color histogram information, the maximum RGB (red, green, blue) information of the input image is identified, Adjust the saturation gain for saturation information compensation based on the identified maximum R / G / B information, Based on the above outline information, the amount information and intensity information of the outline included in the input image are identified, An electronic device configured to adjust gain for sharpness correction based on quantity information and intensity information of the identified outline.

7. In paragraph 1, The at least one processor, individually and / or collectively, The change in the image quality information according to the projection distance information is analyzed for each of the plurality of areas included in the projection surface to obtain local gain information for each area, By analyzing the change in the image quality information according to the above illuminance information, global gain information corresponding to the entire area of ​​the projection surface is obtained, An electronic device configured to obtain region-specific gain information based on the local gain information and the global gain information.

8. In paragraph 7, Local gain information for each of the above areas is: It includes a gain map in the form of a gray image in which the gain value increases as the projection distance corresponding to each of the above multiple areas increases, The above global gain information is, An electronic device including a gain map in the form of a gray image in which a gain value increases as the brightness included in the above illuminance information increases.

9. In paragraph 1, The at least one processor, individually and / or collectively, An electronic device configured to adjust the gain information so that the gain information for each area increases from the center area to the edge area of ​​the projection surface based on at least one of the projection distance information and the size information of the projection surface.

10. In paragraph 1, The at least one processor, individually and / or collectively, Identifying a region of interest within the input image, An electronic device that obtains the output image by correcting the region of interest based on the gain information for each corrected region.

11. In a method for controlling an electronic device, A step of obtaining projection distance information and illuminance information to a projection surface; A step of obtaining gain information for each area included in the projection surface based on the projection distance information and the illuminance information; A step of correcting the gain information for each area based on the correlation information between the projection distance information and the image quality information and the correlation information between the illuminance information and the image quality information; A step of obtaining an output image by correcting an input image based on the above-mentioned corrected gain information for each area; and A control method, comprising: a step of projecting the output image onto the projection surface.

12. In paragraph 11, The above image quality information is, Contains at least one of brightness information, color information, and sharpness information, The step of obtaining the above output image is: A step of correcting the gain information for each area based on the correlation information between the projection distance information and the brightness information and the correlation information between the illuminance information and the brightness information to obtain first gain information corresponding to the brightness information; A step of obtaining second gain information corresponding to color information by correcting gain information for each area based on correlation information between the projection distance information and the color information and correlation information between the illuminance information and the color information; A step of obtaining third gain information corresponding to the sharpness information by correcting the gain information for each area based on the correlation information between the projection distance information and the sharpness information and the correlation information between the illuminance information and the sharpness information; and A control method, comprising: a step of obtaining the output image by correcting the input image based on the first gain information, the second gain information, and the third gain information.

13. In paragraph 12, The step of obtaining the above output image is: A step of analyzing the input image to obtain image characteristic information; and A control method, comprising: a step of correcting the input image based on the image characteristic information, the first gain information, the second gain information, and the third gain information to obtain the output image.

14. In paragraph 13, The above video characteristic information is, Contains at least one of luminance histogram information, color histogram information, and outline information, The step of obtaining the above output image is: A step of correcting brightness information for each area based on the brightness histogram information and the first gain information; A step of correcting color information for each region based on the color histogram information and the second gain information; and A control method, comprising: a step of correcting the sharpness information for each region based on the outline information and the third gain information.

15. A non-transitory computer-readable medium storing instructions that, when individually or collectively executed by at least one processor of an electronic device, cause the electronic device to perform an operation, The above actions are, A step of obtaining projection distance information and illuminance information to a projection surface; A step of obtaining gain information for each area included in the projection surface based on the projection distance information and the illuminance information; A step of correcting the gain information for each area based on the correlation information between the projection distance information and the image quality information and the correlation information between the illuminance information and the image quality information; A step of obtaining an output image by correcting an input image based on the above-mentioned corrected gain information for each area; and A non-transitory computer-readable medium, comprising: a step of projecting the output image onto the projection surface.

Images