Electronic device and control method therefor

Abstract

Disclosed is an electronic device. The electronic device comprises: an image projection unit; a memory for storing one or more instructions; and one or more processors including a processing circuit and operatively connected to the image projection unit and the memory. The one or more processors individually and / or collectively execute one or more instructions to: project a test image onto a screen to perform keystone correction; identify screen reference coordinates corresponding to each of a plurality of pixels included in the test image on the basis of the keystone correction; identify a screen reference area corresponding to each of the plurality of pixels on the basis of the identified screen reference coordinates; identify a reference pixel among the plurality of pixels on the basis of the screen reference area corresponding to each of the plurality of pixels; and identify brightness correction information corresponding to each of the plurality of pixels on the basis of the screen reference area of the identified reference pixel and the screen reference area of each of the plurality of pixels.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for projecting an image and a method for controlling the same.

[0002] Recent advancements in electronic and optical technology have led to the use of a variety of projectors. A projector is an electronic device that projects light onto a screen (or projection surface) to form an image on the screen.

[0003] When projecting an image using a projector, if the projector is placed flat and level, facing the screen, a rectangular image is displayed on the screen. Otherwise, the image may appear distorted vertically or horizontally, or rotated. This distortion is called the keystone effect.

[0004] Accordingly, the projector projected an image with keystone correction on the screen. However, this caused a problem of brightness imbalance on the screen.

[0005] According to one embodiment, an electronic device includes an image projector, a memory storing one or more commands, and a processing circuit, and includes one or more processors operatively connected to the image projector and the memory, wherein the one or more processors are configured to individually and / or collectively execute the one or more commands to project a test image onto a screen to perform keystone correction, identify screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on the keystone correction, identify a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates, identify a reference pixel among the plurality of pixels based on the screen reference area corresponding to each of the plurality of pixels, and identify brightness correction information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

[0006] In one embodiment, the one or more processors may be configured, individually and / or collectively, to identify a value obtained by dividing the screen reference area of ​​each of the plurality of pixels by the screen reference area of ​​the reference pixel as brightness compensation information corresponding to each of the plurality of pixels.

[0007] In one embodiment, the one or more processors may be configured, individually and / or collectively, to identify a pixel corresponding to a maximum area among the screen reference areas corresponding to each of the plurality of pixels as the reference pixel.

[0008] According to one embodiment, the plurality of screen-based coordinates corresponding to each of the plurality of pixels may include four screen-based vertex coordinates corresponding to four vertices of each of the plurality of pixels included in the test image. The one or more processors may be individually and / or collectively configured to calculate a screen area of ​​each of the plurality of pixels based on the four screen-based vertex coordinates of each of the plurality of pixels.

[0009] According to one embodiment, the one or more processors may be configured, individually and / or collectively, to identify an area where a first test image projected on the screen and a second test image projected on the screen from an external projector device overlap according to the keystone correction, identify the overlap area identified on the screen as a plurality of virtual pixel areas, identify projector reference coordinates corresponding to each of the plurality of virtual pixel areas, identify screen reference coordinates corresponding to the identified projector reference coordinates, obtain third brightness information corresponding to each of the plurality of virtual pixel areas based on first brightness information corresponding to the identified screen reference coordinates and second brightness information corresponding to the external projector device, and identify first brightness correction information corresponding to each of a plurality of pixels included in the first test image based on the first brightness information and the third brightness information.

[0010] According to one embodiment, the one or more processors may be configured, individually and / or collectively, to obtain the third brightness information by adding the first brightness information of the first pixel corresponding to the identified screen reference coordinates and the second brightness information corresponding to the second pixel corresponding to the identified screen reference coordinates based on the external projector device.

[0011] According to one embodiment, the one or more processors may be individually and / or collectively configured to identify a first pixel area having the lowest brightness among the plurality of virtual pixel areas based on the third brightness information, identify a brightness compensation value corresponding to each of the plurality of virtual pixel areas based on the fourth brightness information corresponding to the first pixel area and the first brightness information of each of the plurality of virtual pixel areas, and identify the first brightness compensation information corresponding to each of the plurality of pixels included in the first test image based on the identified brightness compensation value.

[0012] According to one embodiment, the area where the first test image and the second test image overlap may be a rectangular area of ​​the largest size identified based on an aspect ratio of the first test image and an aspect ratio of the second test image in the area where the first test image and the second test image overlap.

[0013] According to one embodiment, the apparatus further comprises a communication interface, wherein the one or more processors are configured to individually and / or collectively receive the second brightness information from the external projector device through the communication interface, obtain second brightness correction information corresponding to each of a plurality of pixels included in the second test image based on the second brightness information and the third brightness information, and transmit the obtained second brightness correction information to the external projector device through the communication interface.

[0014] According to one embodiment, the test image includes a plurality of markers, and the one or more processors may be configured to, individually and / or collectively, obtain third information indicating a vertex position of the test image from the captured image based on first information indicating a position of the plurality of markers in the test image and second information indicating a position of the plurality of markers in a captured image captured by an external device, correct the third information based on attitude information of the external device, and perform keystone correction based on the corrected third information.

[0015] A method for controlling an electronic device according to one embodiment includes the steps of: projecting a test image onto a screen to perform keystone correction; identifying screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on the keystone correction; identifying a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates; identifying a reference pixel among the plurality of pixels based on the screen reference area corresponding to each of the plurality of pixels; and identifying brightness correction information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

[0016] A non-transitory computer-readable recording medium storing computer instructions that, when individually and / or collectively executed by a processor of an electronic device according to an embodiment, cause the electronic device to perform an operation, the operation includes the steps of: projecting a test image onto a screen to perform keystone correction; identifying screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on the keystone correction; identifying a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates; identifying a reference pixel among the plurality of pixels based on the screen reference area corresponding to each of the plurality of pixels; and identifying brightness correction information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

[0017] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0018] Figure 1a is a drawing to explain an example of a keystone correction method and the concept of a coordinate system to help understanding.

[0019] Figure 1b is a drawing to explain an example of a keystone correction method and the concept of a coordinate system to help understanding.

[0020] FIG. 2 is a block diagram showing an example of a configuration of a projector according to various embodiments.

[0021] FIG. 3 is a flowchart illustrating an example of a method for controlling an electronic device according to various embodiments.

[0022] FIGS. 4A and 4B are drawings illustrating examples of coordinate information according to various embodiments.

[0023] FIG. 5 is a drawing for explaining an example of third coordinate information according to various embodiments.

[0024] FIG. 6a is a diagram illustrating an example of a method for obtaining roll information and pitch information according to various embodiments.

[0025] FIG. 6b is a diagram for explaining an example of a method for obtaining roll information and pitch information according to various embodiments.

[0026] FIG. 7 is a drawing for explaining an example of a method for obtaining key information according to various embodiments.

[0027] FIG. 8A is a diagram illustrating an example of a method for obtaining a distance between a user terminal and a screen according to various embodiments.

[0028] FIG. 8b is a diagram illustrating an example of a method for obtaining a distance between a user terminal and a screen according to various embodiments.

[0029] FIG. 9 is a drawing illustrating an example of a method for identifying a maximum size rectangular area according to various embodiments.

[0030] FIG. 10 is a drawing for explaining an example of a projection image according to keystone correction according to various embodiments.

[0031] FIG. 11 is a drawing for explaining an example of the relationship between coordinate systems according to various embodiments.

[0032] FIG. 12 is a diagram illustrating an example of a method for obtaining a coordinate transformation matrix according to various embodiments.

[0033] FIG. 13 is a drawing for explaining an example of a method for calculating the area of ​​a rectangle according to various embodiments.

[0034] FIG. 14 is a flowchart illustrating an example of a method for controlling an electronic device according to various embodiments.

[0035] FIG. 15 is a drawing for explaining an example of a method for calculating correction information in a stack situation according to various embodiments.

[0036] FIG. 16 is a block diagram illustrating an example of a detailed configuration of an electronic device according to various embodiments.

[0037] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0038] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0039] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0040] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0041] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0042] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0043] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0044] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0045] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0046] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "module" or "part" that needs to be implemented as specific hardware.

[0047] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0048] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0049] Figures 1a and 1b are drawings to explain an example of a keystone correction method and the concept of a coordinate system to help understanding.

[0050] An electronic device (100) having a function of projecting an image, i.e., a projector function, displays a screen with a relatively accurate ratio when the projector is positioned in a straight line with the screen (or projection surface). However, if this is not satisfied due to space conditions, the screen may go beyond the screen or project a distorted diamond-shaped screen in all directions. In this case, keystone correction may be necessary. Here, keystone correction refers to a function of adjusting the screen to be viewed, i.e., the projected screen, so as to be closer to its original rectangular shape by forcibly moving the corners of the projected screen.

[0051] According to one embodiment, keystone correction can be performed using a user terminal as illustrated in FIG. 1A. For example, a screen (10) on which an image is projected can be captured using a camera provided in the user terminal, and keystone correction can be performed based on the captured image. Here, Projective Transformation can be used. Projective Transformation is a transformation that projects an image in 3D space into 2D space. In other words, it is a method of transforming two images viewed from two different viewpoints in 3D space. At this time, a matrix expressing the relationship between two different images is called a homography matrix (hereinafter referred to as H matrix). For example, the size of the H matrix can be 3x3. To obtain the H matrix, four corresponding pair coordinates are required. In one example, the four corresponding pair coordinates can be coordinates on a world coordinate system.

[0052] Figure 1b is a drawing to explain an example of the concept of a coordinate system to help understanding.

[0053] As illustrated in Figure 1b, there are four coordinate systems in image geometry: the world coordinate system, the camera coordinate system (or screen coordinate system), the normal coordinate system, and the pixel coordinate system. Here, the world coordinate system and the camera coordinate system (or screen coordinate system) are three-dimensional coordinate systems, and the normal coordinate system and the pixel coordinate system are two-dimensional coordinate systems.

[0054] The World Coordinate System (WCS) is a coordinate system used as a reference when expressing the position of an object. The World Coordinate System is a coordinate system that can be arbitrarily selected and used. For example, one corner of space can be taken as the origin, the direction toward one wall can be set as the X-axis, the direction toward the other wall can be set as the Y-axis, and the direction facing the sky can be set as the Z-axis. A point in the World Coordinate System can be expressed as P(X, Y, Z).

[0055] The camera coordinate system (or screen coordinate system) is a coordinate system based on the camera (or screen). As illustrated in Fig. 4, the camera coordinate system (or screen coordinate system) takes the camera's focus (center of the lens) as the origin, the camera's front optical axis direction as the Z-axis, the downward direction of the camera as the Y-axis, and the rightward direction as the X-axis. A point on the camera coordinate system (or screen coordinate system) can be expressed as Pc (Xc, Yc, Zc).

[0056] The pixel image coordinate system is also called the image coordinate system. The pixel coordinate system is a coordinate system for an image that is actually seen with the eyes, and as shown in Figure 4, the upper left corner of the image is the origin, the right direction is the x-axis increasing direction, and the downward direction is the y-axis increasing direction. The plane determined by the x-axis and y-axis of the pixel coordinate system is called the image plane.

[0057] Geometrically, a point P = (X, Y, Z) in 3D space is a point p on the image plane that passes through the focus of the camera (or the focus of the lens). img=(x,y) is projected onto point P and point p img All 3D points on the ray connecting p img is projected onto . Therefore, from 3D point P to p img can be determined uniquely, but conversely, the image pixel p img Finding P from is impossible without additional information. The unit of the pixel coordinate system is pixel, and p img It can be expressed as =(x,y).

[0058] A normalized image coordinate system (NOCS) can be viewed as an image coordinate system that removes the influence of the camera's internal parameters, for example. Furthermore, a NOCS is, for example, a coordinate system that eliminates the units of the coordinate system (normalized) and defines a virtual image plane whose distance from the camera focal point is 1. In other words, it can be an image plane that has been translated from the original image plane to a point whose distance from the camera focal point is 1. The origin of the NOCS is the center of the image plane (the intersection with the optical axis Zc). A point on the NOCS can be expressed as p'=(u, v). Even if the same scene is captured from the same position and angle, different images are obtained depending on the camera used or camera settings. It is more effective to analyze common geometric characteristics and establish theories in a normalized image plane that removes these factors, and thus, a normalized image plane can be used.

[0059] Meanwhile, when projecting images using a projector, keystone correction can be performed to correct the resulting keystone effect. However, while keystone correction adjusts the screen's projection ratio, it has the drawback of not compensating for the inherent brightness unevenness that occurs during keystone correction.

[0060] Accordingly, various embodiments for compensating for screen brightness unevenness will be described below.

[0061] FIG. 2 is a block diagram showing an example of a configuration of an electronic device according to one embodiment.

[0062] According to FIG. 2, the electronic device (100) may include an image projection unit (110), a memory (120), and one or more processors (130) (e.g., including a processing circuit). The electronic device (100) may be implemented as a projector that projects an image onto a wall, a screen, or a screen, or various types of devices having an image projection function.

[0063] The image projection unit (110) can perform the function of projecting light to the outside to express an image and outputting the image on a screen. Here, the screen can be part of the physical space where the image is output or a separate screen. The image projection unit (110) can include various detailed components such as a light source such as at least one of a lamp, an LED, and a laser, a projection lens, and a reflector.

[0064] The image projection unit (110) can project an image using one of various projection methods (e.g., CRT (cathode-ray tube) method, LCD (Liquid Crystal Display) method, DLP (Digital Light Processing) method, laser method, etc.). The image projection unit (110) can include at least one light source.

[0065] The image projection unit (110) can output images with a 4:3 screen ratio, a 5:4 screen ratio, or a 16:9 wide screen ratio depending on the purpose of the electronic device (100) or the user's settings, and can output images with various resolutions such as WVGA (854*480), SVGA (800*600), XGA (1024*768), WXGA (1280*720), WXGA (1280*800), SXGA (1280*1024), UXGA (1600*1200), and Full HD (1920*1080) depending on the screen ratio.

[0066] In addition, the image projector (110) can perform various functions for adjusting the projection image under the control of the processor (130). For example, the image projector (110) can perform a zoom function, a lens shift function, etc.

[0067] The memory (120) can store data required for various embodiments. Depending on the purpose of data storage, the memory (120) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100). In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)). In addition, in the case of memory that can be attached or detached to the electronic device (100'), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented.

[0068] For example, the memory (120) may store various information related to keystone correction and various information related to brightness correction. For example, the memory (120) may store various information obtained during the keystone correction process, such as a transformation matrix. For example, the memory (120) may store various information obtained during the brightness correction process, such as a brightness correction coefficient.

[0069] One or more processors (130) include various processing circuits and control the overall operation of the electronic device (100). Specifically, one or more processors (130) may be connected to each component of the electronic device (100) and control the overall operation of the electronic device (100). For example, one or more processors (130) may be operatively connected to the image projector (110) and the memory (120). The processor (130) may be composed of one or more processors.

[0070] One or more processors (130) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (120).

[0071] The one or more processors (130) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (130) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (130) may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform methods according to various embodiments of the present disclosure by executing one or more instructions stored in a memory.

[0072] When a method according to various embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to various embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0073] One or more processors (130) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (130) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to various embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to various embodiments of the present disclosure.

[0074] When a method according to various embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to various embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0075] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but various embodiments of the present disclosure are not limited thereto. Hereinafter, for convenience of description, one or more processors (130) will be referred to as processor (130). That is, the processor (130) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in this document, including the claims, may include various processing circuits including at least one processor, and one or more of the at least one processor may be configured to individually and / or collectively perform various functions described herein in a distributed manner. As used herein, when "processor," "at least one processor," and "one or more processors" are described as being configured to perform various functions, these terms may encompass, for example, without limitation, a single processor performing some of the recited functions, other processors performing other functions of the recited functions, and even a single processor performing all of the recited functions. Additionally, the at least one processor may comprise a combination of processors that perform the various functions enumerated / disclosed, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0076] According to one embodiment, the processor (130) may perform keystone correction by projecting a test image onto a screen (or projection surface). In one example, the test image may be a white image, but is not limited thereto.

[0077] The processor (130) can control the image projection unit (110) to project a test image, each of which includes a plurality of markers (or tags) in different areas, onto a screen. For example, the test image may be an image including only the plurality of markers. However, the test image may include other images in addition to the plurality of markers, but the other images (e.g., background images) may be included so as not to overlap with the positions where the plurality of markers are included.

[0078] For example, each of the plurality of markers may be in the form of a pattern composed of a black area and a white area in a preset ratio in each of the plurality of directions.

[0079] In one example, the plurality of markers may be positioned at predetermined (e.g., specific) locations, for example, within a threshold distance from four vertices of a test image of the image. For example, the plurality of markers may be positioned within a predetermined ratio of the size of the entire image relative to the four vertices of the test image.

[0080] According to one embodiment, the processor (130) may identify screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on keystone correction. For example, the plurality of screen reference coordinates corresponding to each of the plurality of pixels may include four screen reference vertex coordinates corresponding to four vertices of each of the plurality of pixels included in the test image.

[0081] According to one embodiment, the processor (130) may identify a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates. For example, the processor (130) may calculate the screen area of ​​each of the plurality of pixels based on the screen reference coordinates of four vertices of each of the plurality of pixels.

[0082] According to one embodiment, the processor (130) may identify a reference pixel among a plurality of pixels based on a screen reference area corresponding to each of the plurality of pixels. For example, the processor (130) may identify a pixel corresponding to a maximum area among the screen reference areas corresponding to each of the plurality of pixels as the reference pixel.

[0083] According to one embodiment, the processor (130) may identify brightness correction information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels. For example, the processor (130) may identify a value obtained by dividing the screen reference area of ​​each of the plurality of pixels by the screen reference area of ​​the reference pixel as brightness correction information corresponding to each of the plurality of pixels. For example, the brightness correction information may be a correction coefficient that applies a gain by multiplying the grayscale value of the pixel.

[0084] According to one embodiment, the processor (130) can identify brightness correction information corresponding to each of a plurality of pixels in a stacked situation. Here, a stacked situation may refer to a case where a single screen is implemented using multiple projectors. For example, this may be a case where two projectors are used to project two projected images onto a single screen to create a single screen. In this case, a relatively brighter screen can be implemented compared to a case where a single projector is used.

[0085] According to one embodiment, the processor (130) can identify an area where a first test image projected on a screen and a second test image projected on a screen from an external projector device overlap according to keystone correction in a stack situation.

[0086] According to one embodiment, the processor (130) may identify an overlapping area identified on the screen as a plurality of virtual pixel areas. For example, the area where the first test image and the second test image overlap may be a rectangular area of ​​the largest size identified based on the aspect ratio of the first test image and the aspect ratio of the second test image in the area where the first test image and the second test image overlap.

[0087] According to one embodiment, the processor (130) can identify projector reference coordinates corresponding to each of a plurality of virtual pixel areas.

[0088] According to one embodiment, the processor (130) can identify screen reference coordinates corresponding to the identified projector reference coordinates.

[0089] According to one embodiment, the processor (130) may obtain third brightness information corresponding to each of a plurality of virtual pixel areas based on first brightness information corresponding to the identified screen reference coordinates and second brightness information corresponding to the external projector device.

[0090] For example, the processor (130) may obtain third brightness information by adding first brightness information of a first pixel corresponding to the identified screen reference coordinates and second brightness information corresponding to a second pixel corresponding to the identified screen reference coordinates based on an external projector device.

[0091] For example, the processor (130) may receive second brightness information from an external projector device through a communication interface (140), obtain second brightness correction information corresponding to each of a plurality of pixels included in a second test image based on the second brightness information and the third brightness information, and transmit the obtained second brightness correction information to the external projector device through the communication interface (140).

[0092] According to one embodiment, the processor (130) may identify first brightness compensation information corresponding to each of a plurality of pixels included in the first test image based on the first brightness information and the third brightness information. For example, the processor (130) may identify a first pixel area having the lowest brightness among the plurality of virtual pixel areas based on the third brightness information. The processor (130) may identify a brightness compensation value corresponding to each of the plurality of virtual pixel areas based on the fourth brightness information corresponding to the first pixel area and the first brightness information of each of the plurality of virtual pixel areas. The processor (130) may identify the first brightness compensation information corresponding to each of the plurality of pixels included in the first test image based on the identified brightness compensation values.

[0093] FIG. 3 is a flowchart illustrating an example of a method for controlling an electronic device according to various embodiments.

[0094] According to FIG. 3, the electronic device (100) can perform keystone correction by projecting a test image onto a screen (S310).

[0095] The electronic device (100) can identify screen reference coordinates corresponding to each of a plurality of pixels included in a test image based on keystone correction (S320). For example, the plurality of screen reference coordinates corresponding to each of the plurality of pixels may include four screen reference vertex coordinates corresponding to four vertices of each of the plurality of pixels included in the test image.

[0096] The electronic device (100) can identify a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates (S330). For example, the electronic device (100) can calculate the screen area of ​​each of the plurality of pixels based on the coordinates of four screen reference vertices of each of the plurality of pixels.

[0097] The electronic device (100) can identify a reference pixel among a plurality of pixels based on a screen reference area corresponding to each of the plurality of pixels (S340). For example, the electronic device (100) can identify a pixel corresponding to the largest area among the screen reference areas corresponding to each of the plurality of pixels as the reference pixel.

[0098] The electronic device (100) can identify brightness correction information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels (S350). For example, the electronic device (100) can identify a value obtained by dividing the screen reference area of ​​each of the plurality of pixels by the screen reference area of ​​the reference pixel as brightness correction information corresponding to each of the plurality of pixels.

[0099] Meanwhile, in Fig. 3, the order of various steps is mapped for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0100]

[0101] Hereinafter, the keystone correction method will be described in detail with reference to FIGS. 4a to 10.

[0102] According to one embodiment, the electronic device (100) may obtain third information indicating the positions of vertices of the test image from the captured image based on first information indicating the positions of a plurality of markers in the test image and second information indicating the positions of the plurality of markers in the captured image captured by an external device (e.g., a user terminal equipped with a camera). Subsequently, the electronic device (100) may correct the third information based on the attitude information of the external device and perform keystone correction based on the corrected third information.

[0103] For example, the electronic device (100) may control the image projection unit (110) to project a test image, in which each of a plurality of markers (or tags) is included in a different area, onto a screen. For example, the test image may be an image including only the plurality of markers. However, the test image may include other images in addition to the plurality of markers, but the other images (e.g., a background image) may be included so as not to overlap the positions where the plurality of markers are included.

[0104] For example, each of the plurality of markers may be in the form of a pattern composed of a black area and a white area in a preset ratio in each of the plurality of directions.

[0105] For example, multiple markers may be positioned at predetermined locations, for example, within a threshold distance from four vertices of a test image of the image. For example, multiple markers may be positioned within a predetermined ratio of the size of the entire image relative to the four vertices of the test image.

[0106] According to an example, the electronic device (100) can obtain first information indicating the positions of a plurality of markers in a test image and second information indicating the positions of a plurality of markers in an image captured by an external device (hereinafter, “captured image”). Here, the external device may be a user terminal equipped with a camera, and the following description will assume that the external device is a user terminal.

[0107] As an example, the electronic device (100) can obtain first information indicating the positions of a plurality of markers in an original test image projected through an image projector (110).

[0108] The electronic device (100) may receive second information from the user terminal or acquire second information based on a captured image received from the user terminal. For example, the user terminal may directly identify second information indicating the positions of multiple markers in the captured image and transmit it to the electronic device (100), and the processor (130) may also acquire second information directly from the received captured image.

[0109] For convenience of explanation, the coordinates in the original test image will be described as the projector coordinate system, and the coordinates in the captured image will be described as the screen coordinate system. Accordingly, the first information may correspond to the projector coordinate system, and the second information may correspond to the screen coordinate system. For convenience of explanation, the first information and the second information will be referred to as the first coordinate information and the second coordinate information, respectively.

[0110] FIGS. 4A and 4B are drawings illustrating examples of coordinate information according to various embodiments.

[0111] Fig. 4a is a diagram showing the first coordinate information of a plurality of markers (411, 412, 413, 414) in the original test image, i.e., the projector coordinate system. The first coordinate information may be P1, P2, P3, or P4. For example, the first coordinate information may be calculated based on a specific point of the plurality of markers (411, 412, 413, 414), for example, the center point.

[0112] FIG. 4b is a drawing showing second coordinate information of a plurality of markers (411, 412, 413, 414) in a captured image, i.e., a screen coordinate system, where the second coordinate information may be C1, C2, C3, or C4. For example, the second coordinate information may be calculated based on a specific point (the same point as the first coordinate information) of the plurality of markers (411, 412, 413, 414), for example, a center point.

[0113] The electronic device (100) can acquire third information indicating the positions of vertex areas of a test image in a captured image based on the first coordinate information and the second coordinate information. Here, the vertex areas may be four points where each corner area meets. Here, the third information may be coordinate information in the screen coordinate system, and for convenience of explanation, it will be referred to as third coordinate information.

[0114] According to an example, the electronic device (100) may obtain fourth information indicating the position of a vertex of a test image from a test image based on the first coordinate information and the position information of the marker, and may obtain third coordinate information from the captured image based on the fourth information and a transformation matrix, i.e., the first H matrix. Here, the fourth information may be coordinate information of a projector coordinate system, and for the convenience of explanation, it is named fourth coordinate information.

[0115] In this case, the first H matrix can be obtained based on the mapping relationship between the first coordinate information and the second coordinate information. For example, the electronic device (100) knows four coordinate pairs based on the first coordinate information (P1, P2, P3, P4) and the second coordinate information (C1, C2, C3, C4), and thus can obtain the first H matrix.

[0116] The electronic device (100) can convert the four vertex coordinates, i.e., the fourth coordinate information, in the projector coordinate system into the third coordinate information, i.e., the screen coordinate system, using the first H matrix. For example, the electronic device (100) can store the four vertex coordinates, i.e., the fourth coordinate information, in the test image, or can calculate the four vertex coordinates, i.e., the fourth coordinate information, in the projector coordinate system based on the first coordinate information.

[0117] For example, each of the plurality of markers may be positioned in the inner region at a preset ratio based on the four vertices of the test image. In this case, the electronic device (100) may obtain first coordinate information indicating the position of the marker and fourth coordinate information indicating the position of the vertices of the test image based on the preset ratio. Here, the fourth coordinate information may correspond to the projector coordinate system.

[0118] Next, the electronic device (100) can obtain third coordinate information by converting the coordinates of four vertices (fourth coordinate information) in the projector coordinate system into the screen coordinate system using the first H matrix. For example, as illustrated in FIG. 5, the processor (130) can obtain third coordinate information (C5, C6, C7, C8) corresponding to the four vertices (511, 512, 513, 514) of the projected image in the captured image, i.e., third coordinate information in the screen coordinate system.

[0119] The electronic device (100) obtains third coordinate information indicating the position of the vertex of the test image from the captured image, but since it is difficult to assume that the user terminal was captured in the correct posture, correction of the third coordinate information may be necessary.

[0120] Accordingly, the electronic device (100) can correct the third coordinate information based on the posture information of the user terminal. Here, the posture information may include at least one of roll information, pitch information, or yaw information. According to an example, the roll information and pitch information may be acquired through an acceleration sensor (or acceleration sensors) provided in the user terminal. Additionally, the yaw information may be acquired based on the angle of view information of the camera used to capture the screen in the user terminal.

[0121] Additionally, the electronic device (100) can correct third coordinate information based on distance information between the user terminal and the screen as well as the posture information during correction. For example, the electronic device (100) can perform rotational correction of the third coordinate information based on the three pieces of information, and projection correction of the third coordinate information based on the distance information.

[0122] Below, the rotation correction and projection correction methods are sequentially explained.

[0123] FIGS. 6A and 6B are drawings for explaining an example of a method for obtaining roll information and pitch information according to various embodiments.

[0124] For example, if the Xc, Yc, and Zc axes are defined as shown in FIG. 6a based on the user terminal, the roll angle (φ) for rotation around the y-axis and the pitch angle (θ) for rotation around the x-axis can be defined as follows.

[0125]

[0126]

[0127] A in mathematical expression 1 X , A Y , A Z are the x, y, and z-axis acceleration values ​​of the acceleration sensor equipped in the user terminal, respectively. For example, as shown in Fig. 6b, the pitch angle (θ) can be calculated based on the relationship.

[0128] FIG. 7 is a drawing for explaining an example of a method for obtaining key information according to various embodiments.

[0129] As described above, attitude information related to the direction of gravity, that is, roll information and pitch information, can be obtained using the output value of the acceleration sensor (or gravity sensor), but yaw information unrelated to the direction of gravity can be obtained using a geomagnetic sensor or a gyro sensor based on a direction arbitrarily designated by the user. However, if a gyro sensor is not used, yaw information can be obtained based on the angle of view information of the camera. For example, the electronic device (100) can obtain the center point coordinates of the projected image in the screen coordinate system based on the third coordinate information (C5, C6, C7, C8) corresponding to the four vertices (511, 512, 513, 514) of the projected image in the zero image. Then, the processor (130) can obtain the pixel distance value between the center point coordinates of the projected image and the center point coordinates of the captured image. The electronic device (100) can obtain the camera rotation angle based on the following equation: total angle of view: total pixels = camera rotation angle: pixel distance value. For example, if the total angle of view is 80', the total pixels are 4000px, and the pixel distance value is 500px, then based on 80':4000px = camera rotation angle: 500px, the camera rotation angle is 10', that is, the following information can be obtained.

[0130] Meanwhile, according to one embodiment, when the screen of the user terminal is identified as a preset area, the electronic device (100) can obtain at least one of roll information, pitch information, or yaw information by changing a gravity direction reference value among the output values ​​of the acceleration sensor.

[0131] For example, there may be a case where the screen is not a general wall in the same / similar direction as the gravity direction, but a ceiling, or even if it is a wall, it may be rotated 90 degrees to project an image. In this case, the electronic device (100) may change the gravity direction reference value among the output values ​​of the acceleration sensor to obtain at least one of roll information, pitch information, or yaw information. For example, if the screen is a general wall and the gravity direction reference value among the output values ​​of the acceleration sensor is an x-axis value, if the screen is a ceiling, the gravity direction reference value may be changed to a y-axis value or a z-axis value to obtain at least one of roll information, pitch information, or yaw information. In this case, if the x-axis value, which is the gravity direction reference value, is greater than a threshold value, the electronic device (100) may determine that the screen is not a general wall but a ceiling, or even if it is a wall, it may be rotated 90 degrees to project an image. Accordingly, when the screen is not a regular wall but a ceiling, or when the screen is rotated 90 degrees to project an image on a wall, calculation errors due to the user terminal's posture information can be prevented.

[0132] According to one embodiment, the electronic device (100) can obtain distance information between the user terminal and the screen.

[0133] According to one embodiment, the electronic device (100) can obtain distance information to a virtual plane in pixel units on which a camera image is projected, rather than an actual screen. The virtual plane in pixel units may be the pixel coordinate system described in FIG. 1b.

[0134] For example, if a user terminal is equipped with a distance sensor (e.g., a ToF sensor), and the distance (Z-axis value) of each vertex can be known from the user terminal, the Z-axis physical distance can be calculated in units of px and the Z-axis value can be scaled to units of px. Since the distance between pixels on the x and y axes is identifiable through the captured image, and the corresponding physical distance (real world distance) can be identifiable based on the ToF sensor, the ratio between the px unit and the physical distance can be calculated, and the z-axis can be calculated in px using this.

[0135] In another example, if the user terminal does not have a distance sensor and the camera's field of view information is known, distance information can be obtained based on the lens (sensor) field of view information. For example, the lens' field of view information can be obtained from the EXIF ​​(exchangeable image file format).

[0136] For example, as illustrated in Fig. 8a, the physical ratio of the focal length and the screen diagonal is fixed according to the field of view. The screen diagonal can be obtained based on the number of diagonal pixels, and the distance to the target can correspond to the focal length. That is, if two points of the target to be photographed are 1000px apart on the xy plane and the screen diagonal is 2000px, and the focal length and the screen diagonal are in a 2:1 ratio, the value of the z-axis of the distance value from the camera of the two points on the xy plane can be 2:1 = x: 2000, so the distance value can be 4000px. That is, the xy plane can be a plane that is 4000px away from the camera in the z-axis.

[0137] In another example, if the camera does not have a TOF sensor and does not have information such as focal length, and thus the angle of view is not known at all, a lens angle of about 75 degrees, which is generally used in user terminals, can be input and calculations can be performed while taking into account errors. In one example, information about the camera can be received through an external server. For example, a camera manufacturer or a keystone correction service provider can store information about the camera on a cloud server, etc. In this case, the electronic device (100) can receive information about the camera angle of view from the external server. For example, the camera angle of view information can include information such as sensor size and focal length. As illustrated in FIG. 8b, the focal length and the camera angle of view can be inversely proportional. That is, the shorter the focal length, the wider the angle of view, and the longer the focal length, the narrower the angle of view.

[0138] When the electronic device (100) acquires the user terminal's posture information and the distance information between the user terminal and the screen, it can correct the third coordinate information based on the acquired information.

[0139] For example, the electronic device (100) can rotate and correct third coordinate information based on the posture information of the user terminal, and can obtain corrected third coordinate information by projecting and correcting the rotationally corrected third coordinate information based on distance information.

[0140] Although the captured image recognizes the screen coordinates of the camera, it does not know how the plane on which the projector image is projected is positioned in three dimensions, so three-dimensional rotation correction is required. However, although this can be determined if a ToF sensor is present, let's assume that the ToF sensor is not used. In addition, a method of creating a virtual image can be used assuming that the screen is not tilted after correction and is perpendicular to the user's gravity. For example, four virtual points a1, a2, a3, and a4 are generated and it is assumed that the Z-axis values ​​of these points are all the same. In this case, the attitude information, that is, the inverse of the roll, pitch, and yaw values, are applied as correction values ​​to obtain the points b1, b2, b3, and b4 of the plane in an inclined relationship with the camera shooting plane. Next, a transformation equation is obtained from the plane including the points b1, b2, b3, and b4 to the plane containing the points a1, a2, a3, and a4. Specifically, a transformation equation for rotation transformation such as the following mathematical expression 3 can be obtained.

[0141]

[0142] The electronic device (100) can obtain rotation-corrected third coordinate information by performing rotation correction on the third coordinate information based on mathematical expression 3. Accordingly, rotation-corrected third coordinate information, i.e., coordinates for a point in three dimensions, can be obtained.

[0143] Next, the electronic device (100) can calculate how the 3D coordinates obtained through the above-described rotation correction will be projected onto the actual camera shooting plane. That is, the electronic device (100) can obtain the final corrected third coordinate information by projecting and correcting the 3D coordinates obtained through the rotation correction. Referring to FIG. 1B, a 3D point on the screen coordinate system is projected onto the shooting plane along a virtual vanishing point line passing through the camera sensor. Accordingly, the electronic device (100) can calculate how the 3D point will be projected onto a 2D shooting plane, for example, a 2D regular coordinate system, on the 3D screen coordinate system.

[0144] For example, if the projection reference point is set as the origin, the transformation equation for projecting a point P on a three-dimensional surface to p' can be given as in the following mathematical equation 4.

[0145]

[0146] According to mathematical expression 4, when the projection plane is Zc = d, (Xc, Yc, Zc, 1) can be projected as (Xc, Yc, Zc / d) = (d*Xc / Zc, d*Yc / Zc, 1).

[0147] As described above, the electronic device (100) can obtain the final corrected third coordinate information by projecting the rotationally corrected third coordinate information.

[0148] However, in the above-described embodiment, projection correction is described as being performed after rotation correction, but it is of course possible to perform rotation correction after projection correction.

[0149] The electronic device (100) can obtain a transformation matrix, for example, a second H matrix, based on the final corrected third coordinate information and the vertex coordinates of the test image. Here, the final corrected third coordinate information and the vertex coordinates of the test image may each be regular coordinate system coordinates (or pixel coordinate system coordinates). For example, if the final corrected third coordinate information, that is, the four vertex coordinates are d1, d2, d3, and d4, and the four vertex coordinates of the actual projection point of the test image are e1, e2, e3, and e4, the second H matrix can be obtained based on four pairs of (d1, e1), (d2, e2), (d3, e3), and (d4, e4). For example, in the case of an FHD resolution projector, e1, e2, e3, and e4 may be (0,0), (1920,0), (0,1080), and (1920,1080).

[0150] Additionally, the electronic device (100) can identify a rectangular area of ​​the maximum size corresponding to the aspect ratio of the input image within the area identified based on the corrected third coordinate information, and obtain fifth information corresponding to the identified rectangular area. For example, the fifth information may include coordinate information of each vertex of the identified rectangular area, and for convenience of explanation, it will be referred to as fifth coordinate information hereinafter.

[0151] In this case, the electronic device (100) can expand a rectangle to the same size in all directions from a center point where the vertices of the first region obtained based on the corrected third coordinate information diagonally meet, and check whether the vertices of the rectangle meet the corners of the first region. In addition, when the vertices of the expanded rectangle meet the corners of the first region, the electronic device (100) can expand the small side of the rectangle by a preset pixel unit and expand the large side of the rectangle to correspond to the aspect ratio. Subsequently, the electronic device (100) can identify the largest square area at the location where the vertices corresponding to the diagonals of the expanded rectangle meet the corners of the first region.

[0152] FIG. 9 is a drawing illustrating an example of a method for identifying a maximum size rectangular area according to various embodiments.

[0153] As shown in Fig. 9, when corrected third coordinate information (d1, d2, d3, d4), for example, vertices (911, 912, 913, 914), is obtained, the square is expanded using the center point (920) where each vertex is diagonally connected as the starting point.

[0154] For example, a rectangle is expanded to the same size in all directions from a starting point (920) to check if there is any part that extends beyond the projector screen (910). If there is no part that extends beyond the screen (910), the rectangle is expanded by a preset ratio of the screen (e.g., 5%) and the position is checked to see if the vertices of the expanded rectangle intersect with the corners of the projector screen.

[0155] Next, if any of the corners of the screen (910) and the vertices (931, 932, 933, 934) of the rectangle (930) meet, the rectangle can be expanded in proportion to the larger side, with a preset pixel unit (e.g., 1px) as the smaller side standard. For example, if the upper left, upper right, lower left, lower right corners, etc. of the rectangle (930) meet a point of the screen (910), move 1px to the opposite corner to check whether the vertex and the corner of the screen (910) meet, expand the size by 1px, and check whether there is a point of contact. If the upper left, upper right, lower left, lower right corners, etc. of the expanded rectangle (930) meet non-diagonal vertices, move 1px horizontally in the opposite vertical and horizontal directions to check again whether there is a point of contact, expand the size by 1px, and check whether there is a point of contact.

[0156] Next, when the vertices (942, 943) existing on the diagonal of the expanded rectangle (930) meet the boundary of the screen (910), the expansion is terminated and the coordinates (g1, g2, g3, g4) of the last vertex (941, 942, 943, 944) are acquired. In addition, in order to prevent the rectangle position from moving infinitely, the process can be terminated when the rectangle moves back to the point it moved to while the size of the rectangle is the same.

[0157] Returning to FIG. 2, the electronic device (100) can perform keystone correction by applying the inverse matrix of the second H matrix to the acquired fifth coordinate information. For example, if the coordinate information of the vertices corresponding to the maximum-sized square is g1, g2, g3, and g4, the coordinates of the projection area to be projected by the electronic device (100) can be acquired by applying the inverse matrix of the second H matrix to (g1, g2, g3, g4). That is, when the electronic device (100) projects an image based on the corresponding coordinates, the user can view the maximum-sized square area.

[0158] However, although the above-described embodiment described correcting the vertex coordinates of the projected image based on the camera's attitude information, it is also possible to correct the marker coordinates based on the camera's attitude information. In this case, after correcting the marker coordinates, the vertex coordinates of the projected image can be obtained based on the corrected marker coordinates. In other words, if the marker coordinates are corrected based on the camera's attitude information, there is no need to correct the vertex coordinates based on the camera's attitude information.

[0159] FIG. 10 is a drawing for explaining an example of a projection image according to keystone correction according to various embodiments.

[0160] In FIG. 10, vertices 941, 942, 943, and 944 correspond to the fifth coordinate information, and the area identified by the vertices means the rectangular area of ​​the maximum size obtained in FIG. 9. In this case, the processor (130) can determine the coordinates of the image to be projected by applying the inverse matrix of the second H matrix to the fifth coordinate information. That is, the processor (130) can determine the coordinates of the vertices 951, 952, 953, and 954 of the keystone-corrected image by applying the inverse matrix of the second H matrix to the coordinates of the vertices 941, 942, 943, and 944. In this case, the processor (130) projects the image based on the vertices 951, 952, 953, and 954, so a distorted image (950) is projected, but the user can view a rectangular image (960).

[0161]

[0162] As described above, when keystone correction is performed, the projection area may be any rectangle based on the projector, but may be rectangular based on the screen. For example, assuming an FHD resolution projector as shown in FIG. 11, the projector reference coordinate values ​​for rectangular projection based on the screen are assumed to be (200, 0), (1919, 170), (190, 1079), and (1880, 950) in the order of upper left, upper right, lower left, and lower right, and the screen reference coordinates of the four corners can be set to (0, 0), (1920, 0), (0, 1080), and (1920, 1080) in accordance with the ratio of 16:9.

[0163] According to one embodiment, the electronic device (100) can identify the screen reference coordinates corresponding to each pixel included in the test image. For example, the electronic device (100) needs to transform the coordinates of each pixel included in the test image to the coordinates projected relative to the screen in order to identify the size at which each pixel is projected relative to the screen reference coordinates relative to the projector. Accordingly, the electronic device (100) can obtain a transformation matrix for identifying the screen reference coordinates corresponding to each pixel relative to the projector.

[0164] For example, the electronic device (100) may obtain a coordinate transformation matrix (or a mutual planar projection relationship transformation matrix) based on a pair of projector reference coordinates and screen reference coordinates. For example, as illustrated in FIG. 12, the electronic device (100) may obtain a coordinate transformation matrix (or a mutual planar projection relationship transformation matrix) based on a pair of projector reference coordinates P P and screen coordinates P s A coordinate transformation matrix can be obtained based on a pair of coordinates P. For example, the projector coordinates P P The four vertex coordinates are d1, d2, d3, d4, and the screen coordinate is P. s If the four vertex coordinates are e1, e2, e3, and e4, the transformation matrix can be obtained based on four pairs of (d1, e1), (d2, e2), (d3, e3), and (d4, e4). For example, in the case of an FHD resolution projector, e1, e2, e3, and e4 can be (0,0), (1920,0), (0,1080), and (1920,1080). As an example, the coordinate transformation matrix can be the second H matrix described above.

[0165] For example, the coordinate transformation matrix can be expressed by the following mathematical expression 5.

[0166]

[0167] According to one embodiment, the electronic device (100) can obtain screen reference coordinates corresponding to each pixel based on the coordinate transformation matrix (S320).

[0168] For example, the projection area based on the projector in FIG. 11 may be an irregular rectangle ranging from 200,0 to 1880,950. The electronic device (100) may sequentially calculate screen-based transformation coordinates for each pixel from (0,0) to (1919,1079) based on the FHD resolution projector. For example, the electronic device (100) may sequentially calculate screen-based transformation coordinates for each pixel from (0,0) to (1919,1079) excluding the black area outside the keystone correction area. For example, the electronic device (100) may calculate four vertex coordinates of each pixel based on the screen by applying the second H matrix to the four vertex coordinates of each pixel based on the projector.

[0169] For example, a pixel based on the projector may be 1*1 in size, but may be an irregular square (e.g., 1101, 1102) based on the screen. For example, the upper left pixel (200,0) based on the projector may be 1 unit wide to (201,1), but may be a square with corner coordinates (0,0), (1.01, 0.04), (0.007, 1.0), (1.004, 1.05) based on the screen. As illustrated in Fig. 11, each pixel is a square (e.g., 1*1) in the projector coordinate system, but each pixel may have an arbitrary shape that is not necessarily a square or rectangle in the screen coordinate system. For example, in frontal projection, an area relatively close to the projector becomes smaller and brighter, and an area relatively far away becomes larger and darker.

[0170] For example, when the screen reference coordinates corresponding to each pixel relative to the projector are calculated, the electronic device (100) can calculate the screen reference area of ​​each pixel relative to the projector based on the calculated coordinates (S330). For example, the electronic device (100) can calculate the screen reference area of ​​each pixel relative to the projector based on the coordinates of the four corners of each pixel relative to the screen.

[0171] For example, if you know the coordinates of the four vertices of a quadrilateral, you can calculate the area of ​​the quadrilateral. For example, referring to Fig. 13, the area of ​​a quadrilateral can be calculated using the triangle area formula. Suppose there is a line connecting the point between sides a and b to the point between sides c and d, and that line divides the quadrilateral into two different triangles. The area of ​​the triangle is ab*sinC, and since angle C is the angle between sides a and b, since there are a total of two triangles, we can calculate the area of ​​the quadrilateral by using this formula twice and combining them. In conclusion, the area of ​​the quadrilateral can be calculated based on the formula half the length of side 1 * length of side 4 * sin(angle between sides 1 and 4) + 0.5 * length of side 2 * length of side 3 * sin(angle between sides 2 and 3), that is, 0.5 * a * d * sin A + 0.5 * b * c * sin C. Accordingly, assuming a=12, b=9, c=5, d=12, A=80°, C=110°, the area of ​​the rectangle can be calculated as follows.

[0172] Area of ​​a square = 0.5 (12 x 14) x sin (80) + 0.5 x (9 x 5) x sin (110)

[0173] = 84 x sin (80) + 22.5 x sin (110)

[0174] = 84 x 0.984 + 22.5 x 0.939

[0175] = 82.66 + 21.13 = 103.79 square cm

[0176] When trying to find the area of ​​a parallelogram, the formula for finding the area can be shortened to Area = 0.5*(ad + bc) * sin A because the values ​​of the opposite angles are the same.

[0177] According to one embodiment, the electronic device (100) can identify a reference pixel based on the screen reference area of ​​each pixel (S340). For example, the electronic device (100) can identify a pixel corresponding to the largest area among the screen reference areas corresponding to each pixel as the reference pixel. Since the area and brightness of a pixel are inversely proportional, the pixel with the largest area has the lowest brightness. Generally, the maximum brightness that each projector can output is preset, so it is easy to adjust the brightness downward, but in some cases, it may be difficult to adjust the brightness upward, so the purpose is to match the brightness uniformity to the brightness of the corresponding pixel.

[0178] According to one embodiment, the electronic device (100) can identify brightness correction information corresponding to each pixel based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each pixel (S350). According to one example, the electronic device (100) can identify a value obtained by dividing the screen reference area of ​​each pixel by the screen reference area of ​​the reference pixel as brightness correction information corresponding to each pixel. For example, the electronic device (100) can identify a brightness correction ratio (or brightness adjustment gain) of each pixel based on 'pixel area / reference pixel area'.

[0179] According to one embodiment, the electronic device (100) may store brightness correction information calculated for each pixel in the memory (120). For example, the electronic device (100) may store brightness correction information corresponding to each pixel in the form of a lookup table (LUT) including a brightness correction ratio (or brightness adjustment gain) corresponding to each pixel in the memory (120). Here, the lookup table is a data structure composed of keys and values ​​so that previously stored values ​​can be quickly retrieved without separate calculation, and may be a table in which a brightness correction ratio corresponding to the position (or coordinate) of each pixel is mapped.

[0180] According to one embodiment, the electronic device (100) can correct the grayscale of an image based on brightness correction information for each pixel stored in the memory (120) and project an image with the grayscale corrected. For example, when the electronic device (100) projects a specific image, the grayscale value of each pixel constituting the image is multiplied by a corresponding brightness adjustment ratio to obtain a corrected grayscale value, and the image including the corrected grayscale value can be projected onto a screen. Accordingly, since the pixel brightness of the projected image becomes uniform, the user can view the image uniformly.

[0181] FIG. 14 is a flowchart illustrating an example of a method for controlling an electronic device according to various embodiments.

[0182] According to one embodiment, the electronic device (100) can identify brightness correction information corresponding to each of a plurality of pixels in a stack situation. A stack situation may refer to a case where a single screen is implemented using a plurality of projectors.

[0183] According to FIG. 14, the electronic device (100) can identify an area where a first test image projected on a screen and a second test image projected on a screen from an external projector device overlap according to keystone correction (S1410). In one example, the area where the first test image and the second test image overlap may be a maximum-sized rectangular area identified based on an aspect ratio of the first test image and an aspect ratio of the second test image in the area where the first test image and the second test image overlap. For example, when the electronic device (100) and the external projector device are implemented as an FHD resolution projector, the area may be a maximum-sized rectangular area corresponding to the FHD aspect ratio.

[0184] The electronic device (100) can identify the overlap area identified on the screen as a plurality of virtual pixel areas (S1420).

[0185] The electronic device (100) can identify projector reference coordinates corresponding to each of a plurality of virtual pixel areas (S1430).

[0186] The electronic device (100) can identify screen reference coordinates corresponding to the identified projector reference coordinates (S1440).

[0187] The electronic device (100) may obtain third brightness information corresponding to each of a plurality of virtual pixel areas based on the first brightness information corresponding to the identified screen reference coordinates and the second brightness information corresponding to the external projector device (S1450). In one example, the electronic device (100) may obtain the third brightness information by adding the first brightness information of the first pixel corresponding to the identified screen reference coordinates and the second brightness information corresponding to the second pixel corresponding to the identified screen reference coordinates based on the external projector device. In one example, the electronic device (100) may receive the second brightness information from the external projector device.

[0188] The electronic device (100) can identify first brightness compensation information corresponding to each of the plurality of pixels included in the first test image based on the first brightness information and the third brightness information (S1460). According to an example, the electronic device (100) can identify a first pixel area having the lowest brightness among the plurality of virtual pixel areas based on the third brightness information, and can identify a brightness compensation value corresponding to each of the plurality of virtual pixel areas based on the fourth brightness information corresponding to the first pixel area and the first brightness information of each of the plurality of virtual pixel areas. According to an example, the electronic device (100) can identify the first brightness compensation information corresponding to each of the plurality of pixels included in the first test image based on the identified brightness compensation values.

[0189] According to one embodiment, the electronic device (100) may obtain second brightness correction information corresponding to each of a plurality of pixels included in a second test image based on second brightness information and third brightness information received from an external projector device, and transmit the obtained second brightness correction information to the external projector device.

[0190] In Figure 14, the order of the steps is mapped for convenience of explanation, but the order is not necessarily limited.

[0191] FIG. 15 is a drawing for explaining an example of a method for obtaining brightness correction information in a stack situation according to various embodiments.

[0192] According to FIG. 15, the electronic device (100) can identify an area (1530) where a first test image (1510) projected on a screen and a second test image (1520) projected on a screen from an external projector device (200) overlap according to keystone correction. For example, a projector reference image corresponding to the first test image (1510) projected by the electronic device (100) may be a first test image (1540), and a projector reference image corresponding to the second test image (1520) projected by the external projector device (200) may be a first test image (1550).

[0193] According to one embodiment, the electronic device (100) may identify an overlap area (1530) identified on the screen as a plurality of virtual pixel areas. For example, the electronic device (100) may divide the virtual pixel areas included in the overlap area (1530) into grid areas m in width and n in height.

[0194] According to one embodiment, the electronic device (100) can identify projector reference coordinates corresponding to each of a plurality of virtual pixel areas, and identify screen reference coordinates corresponding to the identified projector reference coordinates. For example, the electronic device (100) can identify projector reference coordinates corresponding to coordinates corresponding to the center point of each virtual pixel area, and identify screen reference coordinates corresponding to the identified projector reference coordinates. For example, the electronic device (100) can identify screen reference coordinates corresponding to the projector reference coordinates of the center point (C) of the first virtual pixel area (1531) in FIG. 15.

[0195] According to one embodiment, the electronic device (100) may identify first brightness correction information corresponding to each of a plurality of pixels based on first brightness information corresponding to the identified screen reference coordinates and second brightness information corresponding to the external projector device. For example, the electronic device (100) may identify a first brightness value according to the area of ​​the screen reference pixel based on the screen reference coordinates corresponding to the projector reference pixel as described above. In addition, the electronic device (100) may receive a second brightness value according to the area of ​​the screen reference pixel from the external projector device (200). In this case, the external projector device (200) may calculate the second brightness value according to the area of ​​the screen reference pixel in the same manner as the electronic device (100).

[0196] According to one embodiment, the electronic device (100) may calculate a brightness value of each virtual pixel area by adding a first brightness value and a second brightness value corresponding to each virtual pixel area. For example, the electronic device (100) may calculate a brightness value from the coordinate of the virtual coordinate system grid (0,0) to the coordinate position of (m,n).

[0197] According to one embodiment, when the brightness value of each virtual pixel area is calculated, the electronic device (100) may identify a reference virtual pixel area among the virtual pixel areas, and identify first brightness correction information corresponding to each virtual pixel area based on the brightness value (or area value) of the identified reference virtual pixel area. In this case, the first brightness correction information may be brightness correction information based on the electronic device (100) in a stack situation. According to one example, the electronic device (100) may identify a value obtained by dividing the brightness value of each virtual pixel area by the brightness value of the reference virtual pixel area as the first brightness correction information corresponding to each virtual pixel area. For example, the electronic device (100) may identify a brightness correction ratio (or brightness adjustment gain) of each virtual pixel area based on 'brightness value of each virtual pixel area / brightness value of the reference virtual pixel area'.

[0198] According to one embodiment, the electronic device (100) may perform brightness correction by applying first brightness correction information corresponding to each virtual pixel area to a projector reference pixel corresponding to each virtual pixel area. For example, the electronic device (100) may store the first brightness correction information calculated for the projector reference pixel corresponding to each virtual pixel area in the memory (120). For example, the electronic device (100) may store the first brightness correction information in the memory (120) in the form of a lookup table (LUT) including the first brightness correction ratio (or brightness adjustment gain) corresponding to each pixel.

[0199] According to one embodiment, the electronic device (100) may project an image with corrected grayscale by correcting the grayscale of the image based on the first brightness correction information for each pixel stored in the memory (120) in a stacked state. For example, when the electronic device (100) projects a specific image, the electronic device (100) may obtain a corrected grayscale value by multiplying the grayscale value of each pixel constituting the image by a corresponding first brightness adjustment ratio, and may project an image including the corrected grayscale value onto a screen.

[0200] According to one embodiment, the electronic device (100) may calculate second brightness correction information based on the external projector device (200) in the same manner as the first brightness correction information, and transmit the calculated second brightness correction information to the external projector device (200).

[0201] In FIGS. 14 and 15, it is described that the electronic device (100) calculates brightness correction information for the electronic device (100) and the external projector device (200) in a stack situation, but this is not limited thereto. For example, it is also possible to calculate brightness correction information for the electronic device (100) and the external projector device (200) in a stack situation by receiving necessary information from another external device that communicates with the electronic device (100) and the external projector device (200).

[0202] In addition, although various embodiments have been described as calculating brightness correction information based on uniformly adjusting the brightness of the entire screen, other embodiments may calculate brightness correction information for each pixel so that the central portion has relatively high brightness and the peripheral portion has relatively low brightness in order to improve the average brightness of the entire screen. In one example, brightness correction information for each pixel may be calculated to some extent to reduce the gap between bright pixels and dark pixels.

[0203] In addition, although various embodiments have been described as calculating the screen-based width of each pixel to calculate brightness correction information, according to other embodiments, brightness correction information may be calculated by sampling some pixels to calculate the screen-based width, and brightness correction information may be calculated by applying weights to pixels between the sampled pixels according to an interpolation method. In this case, the amount of computation required to calculate brightness correction information can be reduced. For example, since the brightness of a pixel adjusted in a projector is ultimately proportional to the distance, the distance value can be calculated to calculate the weight.

[0204] FIG. 16 is a block diagram illustrating an example of a detailed configuration of an electronic device according to various embodiments.

[0205] According to FIG. 16, the electronic device (100') includes an image projector (110), a memory (120), one or more processors (130) (e.g., including a processing circuit), a communication interface (140) (e.g., including a communication circuit), a user interface (150) (e.g., including an interface circuit), and a sensor (160).

[0206] The image projector (110) can enlarge or reduce the image depending on the distance from the screen (projection distance). In other words, a zoom function can be performed depending on the distance from the screen. At this time, the zoom function may include a hardware method that adjusts the screen size by moving the lens, and a software method that adjusts the screen size by cropping the image, etc. When the zoom function is performed, the focus of the image needs to be adjusted. For example, the method of adjusting the focus may include a manual focus method, an electric method, etc.

[0207] In addition, the image projection unit (110) can automatically analyze the surrounding environment and projection environment without user input to provide zoom / keystone / focus functions. Specifically, the projection unit (111) can automatically provide zoom / keystone / focus functions based on the distance between the electronic device (100) and the screen detected through a sensor (depth camera, distance sensor, infrared sensor, light sensor, etc.), information about the space where the electronic device (100) is currently located, information about the amount of ambient light, etc.

[0208] It goes without saying that at least one communication interface (140) (hereinafter, communication interface) can be implemented as various interfaces including various communication circuits according to an implementation example of the electronic device (100'). For example, the communication interface (120) can communicate with an external device (e.g., a user terminal), an external storage medium (e.g., a USB memory), an external server (e.g., a web hard drive), etc. through communication methods such as various types of digital interfaces, AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc.

[0209] The user interface (150) may be implemented as a device including various interface circuits such as buttons, a touch pad, a mouse, and a keyboard, or may be implemented as a touch screen, a remote control transmitter / receiver, etc. that can also perform the above-described display function and operation input function. The remote control transmitter / receiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one communication method among infrared communication, Bluetooth communication, and Wi-Fi communication.

[0210] The sensor (160) may include various types of sensors, such as an acceleration sensor and a distance sensor.

[0211] According to an implementation example of the electronic device (100'), a speaker, a tuner, and a demodulator may be additionally included. The tuner (not shown) can receive an RF (Radio Frequency) broadcast signal by tuning a channel selected by a user or all pre-stored channels among RF broadcast signals received through an antenna. The demodulator (not shown) can receive and demodulate a digital IF signal (DIF) converted by the tuner, and can also perform channel decoding, etc. According to an embodiment, an input image received through the tuner may be processed through the demodulator (not shown) and then provided to the processor (130) for tone mapping processing according to an embodiment of the present disclosure.

[0212] According to the various embodiments described above, it is possible to resolve the problem of screen brightness imbalance that occurs when performing keystone correction in a projector device.

[0213] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of applications installable on existing electronic devices. The methods according to the various embodiments of the present disclosure described above can be performed using a deep learning-based artificial neural network (or deep artificial neural network), i.e., a learning network model.

[0214] Additionally, the methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0215] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0216] According to an exemplary embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., electronic device A) according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When an instruction is executed by a processor, the processor may directly or under the control of the processor perform a function corresponding to the instruction using other components. The instruction may include code generated or executed by a compiler or an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, and does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0217] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0218] Each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0219] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by those skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure. It will also be understood that any embodiment(s) described in the present disclosure may be used in conjunction with any other embodiment(s) described herein.

Claims

1. In electronic devices, Video projection unit; A memory storing one or more instructions; and A processing circuit comprising: one or more processors operatively connected to the image projector and the memory; The one or more processors individually and / or collectively execute the one or more instructions, Perform keystone correction by projecting a test image onto a screen. Based on the above keystone correction, the screen reference coordinates corresponding to each of the plurality of pixels included in the test image are identified, Identifying a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates, Identifying a reference pixel among the plurality of pixels based on a screen reference area corresponding to each of the plurality of pixels, An electronic device configured to identify brightness compensation information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

2. In paragraph 1, An electronic device wherein said one or more processors are individually and / or collectively configured to identify a value obtained by dividing a screen reference area of ​​each of said plurality of pixels by a screen reference area of ​​said reference pixel as brightness compensation information corresponding to each of said plurality of pixels.

3. In paragraph 1, An electronic device wherein said one or more processors are individually and / or collectively configured to identify a pixel corresponding to a maximum area of ​​a screen reference area corresponding to each of said plurality of pixels as said reference pixel.

4. In paragraph 1, The plurality of screen reference coordinates corresponding to each of the plurality of pixels are, Contains four screen-based vertex coordinates corresponding to four vertices of each of the plurality of pixels included in the above test image, An electronic device wherein said one or more processors are individually and / or collectively configured to calculate a screen area of ​​each of said plurality of pixels based on coordinates of four vertices relative to the screen of each of said plurality of pixels.

5. In paragraph 1, The one or more processors, individually and / or collectively, identify an area where a first test image projected on the screen and a second test image projected on the screen from an external projector device overlap according to the keystone correction, Identifying the overlapping area identified on the above screen as a plurality of virtual pixel areas, Identifying the projector reference coordinates corresponding to each of the above multiple virtual pixel areas, Identify the screen reference coordinates corresponding to the above identified projector reference coordinates, Obtain third brightness information corresponding to each of the plurality of virtual pixel areas based on first brightness information corresponding to the identified screen reference coordinates and second brightness information corresponding to the external projector device, An electronic device configured to identify first brightness correction information corresponding to each of a plurality of pixels included in the first test image based on the first brightness information and the third brightness information.

6. In paragraph 5, The one or more processors, individually and / or collectively, An electronic device configured to obtain the third brightness information by adding the first brightness information of the first pixel corresponding to the identified screen reference coordinate and the second brightness information corresponding to the second pixel corresponding to the identified screen reference coordinate based on the external projector device.

7. In paragraph 5, The one or more processors, individually and / or collectively, Based on the third brightness information, the first pixel area having the lowest brightness among the plurality of virtual pixel areas is identified, Identifying a brightness compensation value corresponding to each of the plurality of virtual pixel areas based on the fourth brightness information corresponding to the first pixel area and the first brightness information of each of the plurality of virtual pixel areas, An electronic device configured to identify the first brightness compensation information corresponding to each of the plurality of pixels included in the first test image based on the identified brightness compensation value.

8. In paragraph 5, The area where the first test image and the second test image overlap is, An electronic device, wherein the first test image and the second test image overlap in an area where the first test image and the second test image are identified based on an aspect ratio of the first test image and an aspect ratio of the second test image.

9. In paragraph 5, A communication interface including a communication circuit; further comprising: The one or more processors, individually and / or collectively, Receive the second brightness information from the external projector device through the communication interface, Based on the second brightness information and the third brightness information, second brightness correction information corresponding to each of the plurality of pixels included in the second test image is obtained, An electronic device configured to transmit the acquired second brightness correction information to the external projector device through the communication interface.

10. In paragraph 1, The above test image contains multiple markers, The one or more processors, individually and / or collectively, Based on first information indicating the positions of the plurality of markers in the test image and second information indicating the positions of the plurality of markers in the captured image captured by an external device, third information indicating the positions of the vertices of the test image is acquired in the captured image, Correcting the third information based on the detailed information of the external device, An electronic device configured to perform keystone correction based on the above-mentioned corrected third information.

11. In a method for controlling an electronic device, A step of performing keystone correction by projecting a test image onto a screen; A step of identifying screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on the keystone correction; A step of identifying a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates; A step of identifying a reference pixel among the plurality of pixels based on a screen reference area corresponding to each of the plurality of pixels; and A control method comprising: a step of identifying brightness compensation information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

12. In paragraph 11, The step of identifying the above brightness compensation information is: A control method, comprising: a step of identifying a value obtained by dividing the screen reference area of ​​each of the plurality of pixels by the screen reference area of ​​the reference pixel as brightness correction information corresponding to each of the plurality of pixels.

13. In paragraph 11, The step of identifying the above reference pixel is: A control method, comprising: a step of identifying a pixel corresponding to a maximum area among screen reference areas corresponding to each of the plurality of pixels as the reference pixel.

14. In paragraph 11, The plurality of screen reference coordinates corresponding to each of the plurality of pixels are, Contains four screen-based vertex coordinates corresponding to four vertices of each of the plurality of pixels included in the above test image, The step of identifying the above reference pixel is: A control method, comprising: a step of calculating a screen area of ​​each of the plurality of pixels based on coordinates of four vertices relative to the screen of each of the plurality of pixels.

15. A non-transitory computer-readable recording medium storing computer instructions that, when individually and / or collectively executed by one or more processors of an electronic device, cause the electronic device to perform an operation, The above actions are, A step of performing keystone correction by projecting a test image onto a screen; A step of identifying screen reference coordinates corresponding to each of a plurality of pixels included in the test image based on the keystone correction; A step of identifying a screen reference area corresponding to each of the plurality of pixels based on the identified screen reference coordinates; A step of identifying a reference pixel among the plurality of pixels based on a screen reference area corresponding to each of the plurality of pixels; and A non-transitory computer-readable recording medium, comprising: a step of identifying brightness compensation information corresponding to each of the plurality of pixels based on the screen reference area of ​​the identified reference pixel and the screen reference area of ​​each of the plurality of pixels.

Images