Method and electronic device for generating predistorted image by using lookup table

Abstract

A method performed by an electronic device may comprise the steps of: identifying the value of a scaling parameter associated with an image output size; calculating coordinates of a second pixel corresponding to coordinates of a first pixel on the basis of the value of the scaling parameter; calculating a vector corresponding to the first pixel on the basis of the coordinates of the first pixel and the coordinates of the second pixel; and generating a lookup table on the basis of the calculated vector.

Description

Method and electronic device for generating a pre-distorted image using a look-up table

[0001] The present disclosure relates to a method and an electronic device for generating a pre-distorted image using a look-up table.

[0002] 3D image display technology is being applied to a wide range of fields, and recently, it has been applied to imaging devices related to virtual reality (VR) and augmented reality (AR). Magnifying optical devices can be used as imaging devices that provide VR and AR.

[0003] A virtual reality device may be implemented as a wearable device such as glasses, head-mounted devices, or goggle-type devices, or as a non-wearable device. In the case of a wearable device, since the user wears the wearable device and uses it, the user's position with respect to the wearable device may be fixed. That is, the relative position (e.g., distance, angle, etc.) between the hardware (e.g., display, lens, etc.) that outputs images on the wearable device and the user's visual field may be fixed. On the other hand, in the case of a non-wearable device, the user's position with respect to the wearable device may not be fixed. That is, the relative position (e.g., distance, angle, etc.) between the hardware (e.g., display, lens, etc.) that outputs images on the non-wearable device and the user's visual field may not be fixed.

[0004] The present disclosure can be implemented in various ways, including as a method, system, device, or computer program stored on a computer-readable storage medium.

[0005] According to one embodiment of the present disclosure, a method performed by an electronic device may include a step of identifying a target image including a plurality of pixels. According to one embodiment of the present disclosure, the method may include a step of identifying coordinates of a second pixel corresponding to coordinates of a first pixel using a target look-up table generated based on a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, the method may include a step of generating a pre-distorted image based on data of a first pixel of the target image and coordinates of the second pixel. According to one embodiment of the present disclosure, the method may include a step of outputting the pre-distorted image. According to one embodiment of the present disclosure, data of the second pixel in the pre-distorted image may be determined based on data of the first pixel in the target image.

[0006] According to one embodiment of the present disclosure, a method performed by an electronic device may include a step of identifying a value of a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, the method may include a step of calculating coordinates of a second pixel corresponding to coordinates of a first pixel based on the value of the scaling parameter. According to one embodiment of the present disclosure, the method may include a step of calculating a vector corresponding to the first pixel based on the coordinates of the first pixel and the coordinates of the second pixel. According to one embodiment of the present disclosure, the method may include a step of generating a look-up table for generating a pre-distorted image for a target image based on the calculated vector.

[0007] According to one embodiment of the present disclosure, an electronic device may include a memory that stores one or more commands and at least one processor that executes the one or more commands stored in the memory. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify a target image including a plurality of pixels. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify coordinates of a second pixel corresponding to coordinates of a first pixel using a target lookup table generated based on a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may generate a pre-distorted image based on data of the first pixel of the target image and the coordinates of the second pixel. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may output the pre-distorted image. According to one embodiment of the present disclosure, data of the second pixel in the pre-distorted image can be determined based on data of the first pixel in the target image.

[0008] According to one embodiment of the present disclosure, an electronic device may include a memory that stores one or more commands and at least one processor that executes the one or more commands stored in the memory. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify a value of a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may calculate coordinates of a second pixel corresponding to coordinates of a first pixel based on the value of the scaling parameter. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may calculate a vector corresponding to the first pixel based on the coordinates of the first pixel and the coordinates of the second pixel. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may generate a look-up table for generating a pre-distorted image for a target image based on the calculated vector.

[0009] FIG. 1a is a drawing showing an example of distortion that occurs when an image is output by an electronic device in one embodiment of the present disclosure.

[0010] FIG. 1b is a drawing showing an example of distortion that occurs when an image is output by an electronic device in one embodiment of the present disclosure.

[0011] FIG. 1c is a drawing showing an example of distortion that occurs when an image is output by an electronic device in one embodiment of the present disclosure.

[0012] FIG. 2 is a drawing showing an example of a method for generating a pre-distorted image according to one embodiment of the present disclosure.

[0013] FIG. 3 is a drawing showing an example of generating a pre-distorted image according to one embodiment of the present disclosure.

[0014] FIG. 4 is a diagram illustrating an example of a method for generating a pre-distorted image using a look-up table according to one embodiment of the present disclosure.

[0015] FIG. 5 is a diagram illustrating an example of determining values ​​of one or more parameters according to one embodiment of the present disclosure.

[0016] FIG. 6 is a diagram illustrating an example of generating a look-up table according to one embodiment of the present disclosure.

[0017] FIG. 7 is a diagram illustrating an example of generating a line distortion image using a look-up table according to one embodiment of the present disclosure.

[0018] FIG. 8 is a diagram illustrating an example of generating a pre-distorted image based on a user's viewing position according to one embodiment of the present disclosure.

[0019] FIG. 9 is a flowchart illustrating a method for an electronic device according to one embodiment of the present disclosure to generate a pre-distorted image using a look-up table.

[0020] FIG. 10 is a flowchart illustrating a method for an electronic device to generate a look-up table according to one embodiment of the present disclosure.

[0021] FIG. 11 is a block diagram illustrating an example of an electronic device for generating a look-up table and an electronic device for generating a pre-distorted image according to one embodiment of the present disclosure.

[0022] This disclosure may be subject to various modifications and various embodiments. Specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the disclosure.

[0023] When describing embodiments, detailed descriptions of related known technologies are omitted if they are deemed to unnecessarily obscure the main point. Furthermore, numbers (e.g., "first," "second," etc.) used in the description of embodiments are merely identifiers used to distinguish one component from another. Furthermore, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" are understood to include plural references.

[0024] It should also be understood that the blocks and combinations of flowcharts in each flowchart can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory, or may be stored in separate portions across multiple different memories.

[0025] All functions or operations described in this document may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuitry that performs processing, and may include circuitry such as an Application Processor (AP), a Communication Processor (CP), a Graphical Processing Unit (GPU), a Neural Processing Unit (NPU), a Microprocessor Unit (MPU), a System on Chip (SoC), or an Integrated Chip (IC).

[0026] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail to facilitate implementation by those skilled in the art. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. Before proceeding with a detailed description of the invention, the terms used herein are defined or understood as follows.

[0027] When a component is referred to herein as being "connected" or "connected" to another component, it should be understood that the component may be directly connected to or connected to the other component, but may also be connected or connected via another component in between, unless otherwise specifically stated. Furthermore, "connection" may mean a wireless connection or a wired connection.

[0028] In addition, in this specification, components expressed as 'unit', 'module', etc. may be two or more components combined into one component, or one component may be divided into two or more components with more detailed functions. In addition, each component described below may additionally perform some or all of the functions performed by other components in addition to its own main function, and of course, some of the main functions performed by each component may be performed exclusively by other components.

[0029] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0030] In the present disclosure, the 'size of the image' may mean, but is not limited to, the size in pixels. The 'resolution of the image' may mean, but is not limited to, the number of pixels included in the image. In one embodiment, the size in pixels of the image may be used interchangeably with the resolution of the image. In the present disclosure, the 'resolution of the lookup table' may mean, but is not limited to, the number of matrix elements of the lookup table. The 'size of the lookup table' may mean, but is not limited to, the size in matrix element units of the lookup table. In one embodiment, the size in matrix element units of the lookup table may be used interchangeably with the resolution of the lookup table.

[0031] FIGS. 1A to 1C are drawings showing examples of distortion that occurs when an image is output by an electronic device in one embodiment of the present disclosure.

[0032] The display (110) and lens (120) illustrated in FIGS. 1A to 1C are components of an electronic device and may be included in the electronic device. In one embodiment of the present disclosure, the electronic device is an magnifying optical electronic device and may provide virtual reality (VR), augmented reality (AR), mixed reality (MR), extended reality (XR), substitutional reality (SR), etc. For example, the magnifying optical electronic device may include a wearable magnifying optical device and a non-wearable magnifying optical device (e.g., a virtual screen).

[0033] FIG. 1A is a diagram illustrating radial distortion due to a curvature difference between a display (110) and a lens (120). The radial distortion may include negative radial distortion, i.e., pincushion-shaped distortion. The radial distortion may include positive radial distortion, i.e., barrel-shaped distortion. In one embodiment, the electronic device may apply linear distortion to the output target image to compensate for the distortion due to the curvature difference between the display (110) and the lens (120). For example, the electronic device may apply linear distortion to the output target image using one or more radial distortion parameters (e.g., k1, k2, k3).

[0034] FIG. 1B is a diagram illustrating tangential distortion due to a difference in inclination between a display (110) and a lens (120). In one embodiment, the electronic device may apply linear distortion to an output target image to compensate for the distortion due to the difference in inclination between the display (110) and the lens (120). For example, the electronic device may apply linear distortion to the output target image using one or more tangential distortion parameters (e.g., p1, p2).

[0035] In one embodiment, the electronic device may apply linear distortion to the output target image to compensate for radial and tangential distortion. For example, the electronic device may apply linear distortion to the output target image using the Brown-Conrady distortion model (see Equation 1 below).

[0036] Mathematical formula 1

[0037]

[0038] In the above mathematical expression 1, x and y represent pixel locations (e.g., components of pixel coordinates) in the target image, u and v represent values ​​for pixel locations (e.g., components of pixel coordinates) to which linear distortion is applied, k1, k2, and k3 represent radial distortion parameters, and p1 and p2 may represent tangential distortion parameters. For example, u and v may represent values ​​that can derive pixel locations (or pixel coordinates) to which linear distortion is applied.

[0039] An electronic device can provide an output image without radial distortion and tangential distortion (or with reduced radial distortion and tangential distortion) by outputting an image with applied linear distortion to a display (110).

[0040] FIG. 1C is a diagram showing distortion due to a difference in distance between a lens (120) and a user (130) (i.e., the user's visual field, eyes). The distance between the lens (120) and the user (130) may refer to the distance between the lens (120) and the viewing position. Unlike a wearable device in which the distance between the lens (120) and the user (130) is fixed, in the case of a non-wearable magnifying optical device, the distance between the lens (120) and the user (130) may vary depending on the position and movement of the user (130). For example, if the distance between the lens (120) and the user (130) increases, a phenomenon in which a part of the image is cut off and invisible to the user (130) may occur. That is, if the distance between the lens (120) and the user (130) increases further than a reference distance (or an optimal distance), distortion may occur in which an outer area of ​​the image is cut off and invisible to the user (130).

[0041] In one embodiment, the electronic device may apply pre-distortion to the output target image to compensate for distortion due to the distance between the lens (120) and the user (130). For example, when the distance between the lens (120) and the user (130) increases, the electronic device may downscale the size of the output target image to compensate for distortion in which the outer area of ​​the image appears to be cut off. For example, when the distance between the lens (120) and the user (130) decreases, the electronic device may upscale the size of the output target image.

[0042] In one embodiment, the electronic device may apply predistortion to the output target image to compensate for distortion due to the distance between the lens (120) and the user (130) (i.e., the viewing position), radial distortion, and tangential distortion. For example, the electronic device may apply predistortion to the output target image using a distortion model according to Equation 2.

[0043] Mathematical formula 2

[0044]

[0045] In the above mathematical expression 2, x and y represent pixel locations (e.g., components of pixel coordinates) in the target image, u' and v' represent pixel locations (e.g., components of pixel coordinates) to which linear distortion is applied, k1, k2, and k3 represent radial distortion parameters, p1 and p2 represent tangential distortion parameters, and m and n may represent scaling parameters. For example, the scaling parameters may be parameters for adjusting the size of the output image. In addition, the round function in the above mathematical expression 2 may represent any function (e.g., rounding up, rounding down, etc.) that causes u' and v' to be output as integer values. The values ​​of the parameters may be applied equally to a plurality of pixels of the target image, but are not limited thereto. For example, the values ​​of the parameters for at least some pixels of the target image may be different from the values ​​of the parameters for other pixels of the target image.

[0046] Although FIGS. 1A to 1C illustrate a display (110) and a lens (120) of an electronic device, the present invention is not limited thereto. For example, the electronic device may include a display, a curved mirror, and a beam splitter. The positions and structures of each component of the electronic device are not limited to those illustrated in FIGS. 1A to 1C.

[0047] FIG. 2 is a drawing showing an example of a method for generating a pre-distorted image according to one embodiment of the present disclosure.

[0048] Electronic devices can generate pre-distorted images to compensate for / compensate for distortions that occur during image output. When electronic devices output the original image as is, distortion can occur due to hardware and / or software architecture, conditions, and environmental factors. In this case, even if the electronic device displays the original image intact on the display, the image actually perceived by the user through a lens or other means may be distorted.

[0049] To compensate for this, electronic devices can generate and output a pre-distorted image that transforms the original image, taking into account the distortion that will occur. For example, the electronic device can generate a pre-distorted image by applying a distortion opposite to the distortion that would be applied to the original image. In this case, even if the electronic device outputs a pre-distorted image on the display, the image actually perceived by the user through a lens or other means can be an image without distortion (or with reduced distortion).

[0050] Referring to FIG. 2, an electronic device may identify values ​​of one or more distortion parameters (220). The one or more distortion parameters may include, but are not limited to, radial distortion parameters, tangential distortion parameters, etc. In one embodiment, the electronic device may obtain values ​​of one or more distortion parameters. In one embodiment, the electronic device may determine values ​​of one or more distortion parameters. For example, the value of at least one distortion parameter may be determined based on hardware and / or software architecture, conditions, environments, etc. For example, the value of at least one distortion parameter may be determined based on viewing location information (210).

[0051] The viewing position information (210) may include information on the upper, lower, left, and right positions of the user's visual field (or eyes) with respect to the image output device, information on the angle of gaze with respect to the image output device, etc. The image output device may be included in an input / output device of an electronic device that outputs a pre-distorted image. For example, the image output device may include a display, a lens, etc., and the viewing position information (210) may refer to the position of the user's visual field with respect to the lens of the image output device, but is not limited thereto.

[0052] The electronic device can generate predistortion coordinates based on the values ​​of the identified distortion parameters (230). For example, the electronic device can generate predistortion coordinates (u, v) using Equation 1 above based on the values ​​of the identified radial distortion parameters and the values ​​of the tangential distortion parameters. The f(·) function of FIG. 2 is a function according to Equation 1 above, and the predistortion coordinates (u, v) can be normalized values ​​ranging from 0 to 1. That is, the predistortion coordinates can be non-integer values.

[0053] An electronic device can identify a target image (240) to be output on a display (i.e., a pre-distorted target image). For example, the target image (240) may be an original image. In one embodiment, the electronic device can obtain the target image (240). For example, the electronic device can receive the target image (240) from another electronic device. For example, the electronic device can identify a target image (240) input through an input device. For example, the electronic device can generate the target image (240).

[0054] An electronic device can generate a scaled pre-distorted image (260) by applying pre-distortion coordinates to a target image (240) and performing interpolation processing (250). In one embodiment, the electronic device can generate a pre-distorted image without loss of quality by applying the pre-distortion coordinates to the target image (240). An image pre-distorted without quality loss may have a size (or a higher resolution) larger than the display size (or resolution) as an image that does not take the display size (or resolution) into consideration. In this case, an image pre-distorted without quality loss (i.e., an image generated by applying pre-distortion coordinates to the target image (240)) may not be directly output to a display, or even if an image pre-distorted without quality loss is directly output to a display, a phenomenon may occur in which some areas that exceed the display size (or resolution) are not output (e.g., a cropping phenomenon).

[0055] Therefore, in order to generate an image that can be output on a display without a phenomenon in which some areas of the image are not output, the electronic device can perform interpolation processing on a pre-distorted image generated by applying pre-distortion coordinates to a target image (240). For example, the electronic device can generate a scaled pre-distorted image (260) by scaling (e.g., down-scaling) the pre-distorted image to a size (or resolution) less than or equal to the display size (or resolution). The electronic device can use any interpolation operation, method, or algorithm to perform the interpolation operation.

[0056] The electronic device can output the scaled pre-distorted image (260) on the display of an image output device (e.g., a virtual screen). The image output device can be included as an output device as a component of the electronic device that generates the scaled pre-distorted image (260). For example, the image output device can be a device that is connected to or can communicate with the electronic device that generates the scaled pre-distorted image (260). The image output device can include, but is not limited to, a display, a lens, etc. By distorting the scaled pre-distorted image (260) output on the display again by a lens, etc., an image without distortion (or with less distortion) can be provided to the user's eyes.

[0057] When the method illustrated in FIG. 2 is applied to an image comprising multiple image frames rather than a single image, a relatively long processing time may be required. For example, in order to generate a pre-distorted image by applying pre-distortion to an image comprising multiple image frames, the electronic device needs to apply pre-distortion coordinates to each of the multiple image frames and perform interpolation processing. In other words, since the electronic device must perform interpolation processing for each image frame, a relatively long time may be required to generate the pre-distorted image, which may result in latency.

[0058] In the description of FIG. 2 described above, the operations illustrated in FIG. 2 are described as being performed by a single electronic device, but are not limited thereto. In one embodiment, the operations illustrated in FIG. 2 may be performed by multiple electronic devices. For example, a first electronic device may identify values ​​of one or more distortion parameters, generate pre-distortion coordinates, and provide the generated pre-distortion coordinates to a second electronic device. The second electronic device may use the provided pre-distortion coordinates to generate a pre-distortion image and / or a scaled pre-distortion image (260) from the target image (240).

[0059] The operations performed by the electronic device to generate the scaled pre-distorted image (260) are not limited to those illustrated in FIG. 2. In one embodiment, the electronic device may further perform any processing operations omitted in FIG. 2 to generate the scaled pre-distorted image (260). For example, the electronic device may perform black pixel padding processing on the target image (240) to apply pre-distortion coordinates to the target image (240) (e.g., a target image with a resolution of 3840x2160), thereby generating a padding image (e.g., a padding image with a resolution of (3840+b)x(2160+a)) by adding a border area of ​​black pixels (e.g., zero pixels, zero data pixels) to the target image (240). In this case, the electronic device may generate the pre-distorted image by applying the pre-distortion coordinates to the padding image.

[0060] FIG. 3 is a drawing showing an example of generating a pre-distorted image according to one embodiment of the present disclosure.

[0061] FIG. 3 illustrates an example of an operation (i.e., 250 of FIG. 2) in which an electronic device applies predistortion coordinates to a target image and performs interpolation processing in the method for generating a predistorted image described above in FIG. 2. Referring to FIG. 3, the electronic device may apply predistortion coordinates to a target image (310) having a resolution (or size) of 3840x2160 (i.e., 4k). For example, as illustrated in FIG. 3, the electronic device may generate a barrel predistortion image (320) by applying barrel predistortion coordinates to the target image (310). The barrel predistortion image (320) may be generated with a resolution (or size) of (3840+b)x(2160+a). That is, the barrel distortion image (320) is a distortion image without loss of image quality, and can be generated with a higher resolution (or larger size) than the resolution (or size) of the target image (310).

[0062] A barrel predistortion image (320) with a resolution of (3840+b)x(2160+a) may not be output on a display with a resolution (or size) of 3840x2160, or even if output, some areas of the barrel predistortion image (320) exceeding the resolution (or size) of 3840x2160 may not be output, and an image with some areas cut off may be output on the display. Therefore, the electronic device may downscale the barrel predistortion image (320) to the display resolution (or size) (i.e., 3840x2160) to output a complete image without any cut-off areas. In one embodiment, the electronic device may generate a predistortion image (330) scaled to a resolution (or size) of 3840x2160 by performing interpolation processing on the barrel predistortion image (320).

[0063] Although FIG. 3 illustrates a target image (310) having a resolution (or size) of 3840x2160, the resolution (or size) of the target image is not limited to 3840x2160, and the target image may have a higher resolution (or larger size) or, conversely, a lower resolution (or smaller size). In addition, although FIG. 3 illustrates an example in which an electronic device scales a pre-distorted image (320) into an image having a resolution (or size) of 3840x2160, the present invention is not limited thereto. For example, the electronic device may up / downscale the pre-distorted image (320) to an arbitrary resolution (or size) in consideration of the resolution (or size) of the display.

[0064] As described above with reference to FIGS. 2 and 3, when a pre-distorted image without image quality degradation is generated by applying pre-distortion coordinates determined based on radial distortion parameters and / or tangential distortion parameters and then scaling is performed, the electronic device can output an image without a jaggie phenomenon at the image border, i.e., an image without image degradation. However, due to the interpolation process for scaling, a long processing time may be required to process a single target image and output it on a display. For example, it may take about 3.17 seconds to process one image with a resolution of 3840x2160, and it may take about 159.97 seconds to process an image containing 50 image frames. Therefore, when processing an image containing a large number of image frames, it may take more time, which may cause latency and may make immediate image output difficult.

[0065] FIG. 4 is a diagram illustrating an example of a method for generating a pre-distorted image using a look-up table according to one embodiment of the present disclosure.

[0066] In the detailed description of FIG. 4, the description overlapping with FIG. 2 may be omitted. FIG. 4 may illustrate an example of a low-complexity lens distortion compensation image processing method according to an embodiment of the present disclosure. Referring to FIG. 4, an electronic device may identify values ​​of one or more distortion parameters and values ​​of one or more scaling parameters (420). In one embodiment, the electronic device may obtain values ​​of one or more scaling parameters. In one embodiment, the electronic device may determine values ​​of one or more scaling parameters. For example, the one or more scaling parameters may include m and n of the above mathematical expression 2. According to the above mathematical expressions 1 and 2, u' may be calculated by multiplying the scaling parameter m by u of the above mathematical expression 1, and v' may be calculated by multiplying the scaling parameter n by v of the above mathematical expression 1.

[0067] In one embodiment, the electronic device can determine the value of one or more scaling parameters based on the display resolution (400) (or size). For example, the electronic device can determine the scaling parameter m based on the horizontal resolution value (e.g., 3840) of the display, and can determine the scaling parameter n based on the vertical resolution value (e.g., 2160) of the display. The scaling parameters can cause the pre-distorted image to be generated at a lower resolution (or smaller size) than the display resolution (400) (or size). Accordingly, the electronic device can output the pre-distorted image generated based on the scaling parameters on the display without additional scaling processing.

[0068] In one embodiment, the electronic device may determine values ​​of one or more scaling parameters based on viewing position information (410). The viewing position information (410) may include information on the upper, lower, left, and right positions of the user's visual field (or eyes) with respect to the image output device, information on the gaze angle and direction with respect to the image output device, etc. The viewing position information (410) may include information on the distance between the image output device and the viewing position. For example, the viewing position information (410) may include the distance of the user's visual field (or eyes) from the image output device. For example, the image output device may include a display, a lens (or a curved mirror), etc., and the viewing position information (410) may mean, but is not limited to, the distance of the user's visual field from the lens (or curved mirror) of the image output device.

[0069] In one embodiment, the electronic device may determine the values ​​of one or more scaling parameters to compensate for distortion that occurs as the distance of the user's visual field from the lens of the image output device increases. For example, the electronic device may determine the values ​​of one or more scaling parameters based on the resolution (or size) of the display for a reference distance. For example, the electronic device may determine the values ​​of the scaling parameters to be smaller for a distance greater than the reference distance to compensate for a phenomenon in which a portion of the image is cut off and not visible to the user. In this case, the electronic device may generate a pre-distorted image of a smaller size (or lower resolution) using the determined values ​​of the scaling parameters.

[0070] The electronic device may generate a look-up table (LUT) (432) based on at least one of the values ​​of one or more distortion parameters or the values ​​of one or more scaling parameters (430). In one embodiment, the electronic device may generate the look-up table (432) based on the u' and v' values ​​calculated based on the mathematical expression 2 above. The operation (430) of the electronic device of FIG. 4 generating the look-up table (432) may be described below with reference to FIG. 6.

[0071] A lookup table (432) may be generated to map a plurality of pixels included in a target image (440) to pixels of a pre-distorted image. In one embodiment, the lookup table (432) may be associated with viewing location information (410). For example, a lookup table (432) generated based on viewing location information (410) may be used when a user is located at the corresponding viewing location.

[0072] The resolution (or size) of the lookup table (432) may be determined based on at least one of the resolution (or size) of the image to be distorted or the resolution (or size) of the display. The resolution (or size) of the lookup table (432) may mean the number of pixels that can be mapped to the lookup table (432). For example, in FIG. 4, the width (or column) of the lookup table (432) includes W pixels from 0 to W-1, and the height (or row) of the lookup table (432) includes H pixels from 0 to H-1, so the resolution (or size) of the lookup table (432) may be a resolution (or size) of WxH. For example, the horizontal resolution (or size) of the lookup table (432) may be the same as the horizontal resolution (or size) of the image to be distorted, and the vertical resolution (or size) of the lookup table (432) may be the same as the vertical resolution (or size) of the image to be distorted. The W value of the lookup table (432) of FIG. 4 may be the horizontal resolution (or number of horizontal pixels) (e.g., 3840) of at least one of the images or displays to be distorted, and the H value may be the vertical resolution (or number of vertical pixels) (e.g., 2160) of at least one of the images or displays to be distorted.

[0073] The electronic device can generate a pre-distorted image (460) by performing a mapping operation (450) on a target image (440) based on a lookup table (432). For example, the target image (440) may be an original image. For example, the target image (440) may be an image in which the original image is converted / scaled to a resolution (or size) of at least one of the lookup table (432) or the display.

[0074] The electronic device can generate a pre-distorted image (460) by mapping (450) a plurality of pixels included in a target image (440) based on vector data included in a look-up table (432). In one embodiment, the electronic device maps vector data (s) corresponding to a (1, 1) pixel in the look-up table. 1,1 , t 1,1 ) and identify the identified vector data (s 1,1 , t 1,1 ) can map the (1,1) pixel of the target image (440) to the (u'1, v'1) pixel. For example, the electronic device can map vector data (s) at the coordinates of the (1, 1) pixel of the target image (440). 1,1 , t 1,1 ) can be used to determine the coordinates of the (u'1, v'1) pixel based on the operation of subtracting the (u'1, v'1) pixel. In this case, the electronic device can determine the data of the (u'1, v'1) pixel of the pre-distorted image (460) based on the data of the (1,1) pixel of the target image (440).

[0075] When performing a mapping operation based on the lookup table (432), the electronic device can map two or more pixels of the target image (440) to one pixel. In this case, the electronic device can generate a pre-distorted image (460) by determining data of one pixel to which the two or more pixels are mapped based on data of two or more pixels of the target image (440). In one embodiment, the electronic device can apply any operation, algorithm, or rule to the data of two or more pixels of the target image (440) to produce / determine data of one pixel to which the two or more pixels are mapped. For example, the electronic device can determine an average value of data of two or more pixels of the target image (440) as data of one pixel to which the two or more pixels are mapped. For example, the electronic device can determine data of a pixel selected by any rule from among two or more pixels of the target image (440) as data of one pixel to which the two or more pixels are mapped.

[0076] Since the lookup table (432) generated according to one embodiment of the present disclosure is generated based on the values ​​of one or more scaling parameters, the pre-distorted image (460) generated using the lookup table (432) can be output on the display without additional scaling processing. Therefore, unlike FIG. 2, the electronic device may not need to perform additional interpolation processing after generating the pre-distorted image (460).

[0077] FIG. 4 illustrates a lookup table (432) containing (s, t) vector data for each pixel, but the structure, composition, form, data type, etc. of the lookup table are not limited thereto.

[0078] In the description of FIG. 4 described above, the operations illustrated in FIG. 4 are described as being performed by a single electronic device, but this is not limited thereto. In one embodiment, the operations illustrated in FIG. 4 may be performed by multiple electronic devices. For example, a first electronic device may generate a look-up table and provide the generated look-up table to one or more second electronic devices. The one or more second electronic devices may then use the provided look-up table to generate a pre-distorted image from the target image.

[0079] Although FIG. 4 illustrates an electronic device identifying one or more parameter values ​​based on display resolution (400) and viewing position information (410), this is not a limitation. For example, the electronic device may identify one or more parameter values ​​based on only one of display resolution (400) and viewing position information (410).

[0080] FIG. 5 is a diagram illustrating an example of determining values ​​of one or more parameters according to one embodiment of the present disclosure.

[0081] The electronic device may determine a value of at least one of one or more distortion parameters or one or more scaling parameters to apply a line distortion to an image. In one embodiment, the value of at least one of the radial distortion parameter, the tangential distortion parameter, or the scaling parameter may be determined based on a difference between an image input by a camera positioned at a target viewing position with respect to the image output device and a reference image. In one embodiment, the value of at least one of the radial distortion parameter, the tangential distortion parameter, or the scaling parameter may be determined based on a structure of the image output device, such as a size and position of the display (520), a size and position of a curved mirror (540), an angular difference between a lens (or curved mirror (540)) and the display (520), and the like. In one embodiment, the value of at least one of the radial distortion parameter, the tangential distortion parameter, or the scaling parameter used to generate a lookup table for a target viewing position may be determined based on the target viewing position with respect to the image output device.

[0082] Referring to FIG. 5, the image output device may include a display (520) (e.g., an image forming module), a beam splitter (530), and a curved mirror (540) (e.g., a concave mirror). The output image may be output on the display (520), reflected by the beam splitter (530), reaches the curved mirror (540), focused by the curved mirror (540), and may pass through the beam splitter (530) to reach the user's visual field. The image output device may be an output device included as a component in the electronic device (510), or may be a device connected to or capable of communicating with the electronic device (510).

[0083] To determine the values ​​of one or more parameters used to generate a lookup table for a target viewing position, a camera (550), which is an image input device, may be positioned at the target viewing position with respect to an image output device. The value of the scaling parameter may be associated with a distance between a curved mirror (540) of the image output device and the target viewing position. The camera (550) may be an input device included as a component in the electronic device (510), or may be a device connected to or capable of communicating with the electronic device (510).

[0084] In one embodiment, the electronic device (510) can output a reference image through an image output device. For example, the electronic device (510) can generate an output image by performing a pre-distortion process on the reference image and output the generated output image on the display (520). In one embodiment, the electronic device (510) can obtain an image input to the camera (550). The output image on the display (520) passes through the beam splitter (530) and the curved mirror (540), and a distorted image can be input to the camera (550), and the electronic device can obtain the distorted image.

[0085] In one embodiment, the values ​​of one or more parameters may be determined based on the difference between the original reference image and the image input to the camera (550). In one embodiment, according to the following mathematical expression 3, optimal values ​​of the parameters that minimize the difference between the image input to the camera and the original reference image may be determined, estimated, and inferred.

[0086] Mathematical formula 3

[0087]

[0088] In the above mathematical expression 3, k1, k2, and k3 represent radial distortion parameters, p1 and p2 represent tangential distortion parameters, and m and n represent scaling parameters. In addition, represents the original reference image, is a parameter When a reference image distorted by the value of can be output to an image processing device, it can represent an image input to a camera. According to the above mathematical expression 3, the parameter The values ​​of can be determined simultaneously, but are not limited thereto. For example, the parameter At least some of the values ​​may be determined by other factors or methods, and the values ​​of the remaining parameters may be determined by the above mathematical expression 3.

[0089] In one embodiment, the values ​​of one or more parameters may be experimentally / empirically determined. In one embodiment, the electronic device may use any model, method, algorithm, etc. to determine the values ​​of one or more parameters that can reduce the difference between the original reference image and the image input by the camera. In one embodiment, the values ​​of one or more parameters calculated / determined by the electronic device based on any factor or method may be experimentally fine-tuned.

[0090] One or more of the embodiments described above in FIG. 5 may be performed in the operation of identifying the value of the distortion parameter of FIG. 2 (i.e., 220 of FIG. 2), the operation of identifying the value of the distortion parameter and the value of the scaling parameter of FIG. 4 (i.e., 420 of FIG. 4), but are not limited thereto.

[0091] FIG. 6 is a diagram illustrating an example of generating a look-up table according to one embodiment of the present disclosure.

[0092] The electronic device may generate a lookup table (610) based on the values ​​of one or more parameters. In one embodiment, the electronic device may determine the coordinates of a second pixel corresponding to the coordinates of a first pixel based on the values ​​of the one or more parameters. The coordinates of the second pixel corresponding to the coordinates of the first pixel may be the coordinates of the pixel to which the first pixel of the target image is mapped when the target image is pre-distorted. For example, the electronic device may determine the coordinates of the (u', v') pixels corresponding to the coordinates of the (x, y) pixels of the target image based on the above mathematical expression 2. For example, the (x, y) pixels may represent a location in the target image, and the (u', v') pixels may represent a location in the display (or a location in the pre-distorted output image).

[0093] The electronic device can calculate a vector corresponding to the first pixel to be included in the lookup table (610) based on the coordinates of the first pixel and the coordinates of the second pixel. The electronic device can generate a lookup table (610) including a vector (e.g., a mapping vector) corresponding to the calculated first pixel. The vector corresponding to the first pixel can be included in the lookup table (610) for the first pixel, i.e., in a cell corresponding to the first pixel in the lookup table (610).

[0094] In one embodiment, the electronic device may calculate a vector corresponding to the first pixel based on an operation that subtracts the coordinates of the second pixel from the coordinates of the first pixel. For example, the electronic device may calculate vectors included in the lookup table (610) based on the following mathematical expression (4).

[0095] Mathematical formula 4

[0096]

[0097] In the above mathematical expression 4, (x, y) represents the coordinates of the pixel to be mapped, (u', v') represents the coordinates of the pixel corresponding to the (x, y) pixel, which are calculated based on the values ​​of one or more parameters, (s, t) may represent a vector corresponding to the coordinates of the (x, y) pixel, and (s, t) may be included in the lookup table (610) as data for the (x, y) pixel.

[0098] FIG. 6 illustrates a lookup table (610) including a table (610_1) for the first component (s) of the (s, t) vector and a table (610_2) for the second component (t), but the structure, composition, form, data type, etc. of the lookup table are not limited thereto.

[0099] One or more of the embodiments described above in FIG. 6 may be performed in the operation of generating the lookup table of FIG. 4 (i.e., 430 of FIG. 4), but is not limited thereto.

[0100] FIG. 7 is a diagram illustrating an example of generating a line distortion image using a look-up table according to one embodiment of the present disclosure.

[0101] FIG. 7 illustrates an example of the mapping operation described above in FIG. 4 (i.e., 450 of FIG. 4). Referring to FIG. 7, an electronic device may identify a target image (710) having a resolution (or size) of 3840x2160 (i.e., 4k). The electronic device may identify a target lookup table (610) to be used to generate a pre-distorted image. In one embodiment, the electronic device may obtain a target lookup table (610) generated by another electronic device. In one embodiment, the electronic device may identify a user's viewing location and identify a target lookup table (610) for the identified viewing location.

[0102] The electronic device can perform a mapping operation on the target image (710) using the lookup table (610). In one embodiment, the electronic device can identify a vector (s, t) corresponding to the coordinates (x, y) of the first pixel of the target image (710) in the lookup table (610). The electronic device can obtain a value s corresponding to the coordinates (x, y) of the first pixel from the table (610_1) for the first element of the lookup table (610). x,y Obtain the value t corresponding to the coordinates (x, y) of the first pixel from the table (610_2) for the second element x,y can be obtained. Accordingly, the electronic device may obtain a vector (s) containing values ​​obtained from the lookup table (610). x,y, t x,y ) can be identified by the vector (s, t) corresponding to the coordinates (x, y) of the first pixel.

[0103] The electronic device identifies a vector (s, t) corresponding to the coordinates (x, y) of the first pixel at the coordinates (x, y) of the first pixel. x,y, t x,y ) can be used to determine the coordinates (u', v') of the second pixel based on the operation of subtracting the first pixel from the second pixel. Referring to FIG. 7, the electronic device can determine the coordinates of the second pixel corresponding to the coordinates of the first pixel based on the following mathematical expression 5.

[0104] Mathematical Formula 5

[0105]

[0106] In the above mathematical expression 5, (x, y) represents the coordinates of the target pixel to be mapped in the target image, (u', v') represents the coordinates of the pixel corresponding to the coordinates of the (x, y) pixel, and (s, t) may represent a vector corresponding to the coordinates of the (x, y) pixel included in the lookup table (610). For example, (u', v') may represent the coordinates of the pixel in the pre-distorted image to be generated.

[0107] The electronic device can generate a 3840x2160 pre-distorted image (720) based on data (e.g., image data) of a first pixel (x, y) of a target image (710) and coordinates (u', v') of a second pixel. In one embodiment, data of the second pixel (u', v') in the pre-distorted image (720) can be determined based on data of the first pixel (x, y) in the target image. For example, the electronic device can determine data of the first pixel (x, y) in the target image as data of the second pixel (u', v') in the pre-distorted image (720). The electronic device can generate the pre-distorted image (720) by performing the above-described operations on a plurality of pixels included in the target image (710).

[0108] The 3840x2160 pre-distorted image (720) generated in FIG. 7 can be directly output on a display with a resolution (or size) of 3840x2160, and can be output as a complete image without any clipping. As described above in FIG. 7, when the pre-distorted image (720) is generated using the look-up table (610) generated based on the scaling parameters, the electronic device can output the pre-distorted image (720) on the display without additional scaling processing. Therefore, the electronic device may not need to perform interpolation processing on the pre-distorted image (720).

[0109] Since no interpolation processing is performed, it may take about 0.1 second for the electronic device to process one image with a resolution of 3840x2160 and about 1.29 seconds to process an image containing 50 image frames. Compared to the case where a pre-distorted image is generated without image quality degradation by applying pre-distortion coordinates and then scaling is performed, the time for processing one image with a resolution of 3840x2160 can be reduced by about 2.07 seconds and the time for processing an image containing 50 image frames can be reduced by 158.58 seconds. Therefore, when a pre-distorted image (720) is generated using a look-up table (610) generated based on a scaling parameter according to an embodiment of the present disclosure, the image processing time can be reduced, and even more time can be reduced when processing an image containing a large number of image frames.

[0110] Although FIG. 3 illustrates a target image (710) having a resolution (or size) of 3840x2160, the resolution (or size) of the target image is not limited to 3840x2160, and the target image may have a higher resolution (or larger size) or, conversely, a lower resolution (or smaller size).

[0111] In addition, although FIG. 3 illustrates an example in which an electronic device generates a pre-distorted image (720) of the same resolution (or size) for a target image (710) of 3840x2160 resolution (or size), the present invention is not limited thereto. In one embodiment, the resolution of the target image (710) and the resolution of the pre-distorted image (720) may be different from each other. For example, the electronic device may generate a pre-distorted image (720) of a lower resolution (or smaller size) than the resolution (or size) of the target image (710). In one embodiment, by using a look-up table for a viewing position that is further from a reference position from an image output device, the electronic device may generate a pre-distorted image (720) of a lower resolution (or smaller size) than the display resolution (or size). The electronic device may output the pre-distorted image (720) of a lower resolution (or smaller size) than the display resolution (or size) to a portion of the display (e.g., a middle area).

[0112] FIG. 7 illustrates a lookup table (610) including a table (610_1) for the first component (s) of the (s, t) vector for each pixel and a table (610_2) for the second component (t), but the structure, composition, form, data type, etc. of the lookup table used by the electronic device to generate a pre-distorted image are not limited thereto.

[0113] FIG. 8 is a diagram illustrating an example of generating a pre-distorted image based on a user's viewing position according to one embodiment of the present disclosure.

[0114] In the detailed description of FIG. 8, descriptions overlapping with those of FIGS. 4 to 7 may be omitted. In one embodiment, the electronic device may identify information on multiple viewing positions (810). The information on each viewing position may be 6-DOF (Degree Of Freedom) position information and may include (c1, c2, c3, c4, c5, c6) vector data. The 6-DOF (Degree Of Freedom) position information may indicate information on the user's head direction, pose, gaze direction, position, etc.

[0115] An electronic device may identify values ​​of one or more parameters for each viewing position. In one embodiment, the electronic device may identify values ​​of a distortion parameter (e.g., a radial distortion parameter, a tangential distortion parameter) and a scaling parameter for each viewing position (820). For example, the electronic device may determine values ​​of at least one of a radial distortion parameter, a tangential distortion parameter, or a scaling parameter for each viewing position based on information about a plurality of viewing positions (810) and / or a display resolution (800) (or size). For example, the electronic device may determine values ​​of at least one of a radial distortion parameter, a tangential distortion parameter, or a scaling parameter for each viewing position by performing one or more of the embodiments described above with reference to FIGS. 4 and / or 5 .

[0116] At least one value of the radial distortion parameter, the tangential distortion parameter, or the scaling parameter for the viewing position (c1,c2,c3,c4,c5,c6) can be determined, estimated, or inferred based on the following mathematical expression 6.

[0117] Mathematical formula 6

[0118]

[0119] In the above mathematical expression 6, k1, k2, and k3 represent radial distortion parameters, p1 and p2 represent tangential distortion parameters, and m and n represent scaling parameters. In addition, represents the original reference image, is a parameter When a reference image distorted by the value of is output to an image processing device, it can represent an image input to a camera located at a location (c1,c2,c3,c4,c5,c6). According to the above mathematical expression 6, the parameter The values ​​of can be determined simultaneously, but are not limited thereto. For example, the parameter At least some of the values ​​may be determined by other factors or methods, and the values ​​of the remaining parameters may be determined by the above mathematical expression 6.

[0120] The electronic device may generate a lookup table for each viewing position based on the values ​​of parameters for each viewing position (830). In one embodiment, the electronic device may generate multiple lookup tables for multiple viewing positions (810). For example, the electronic device may generate a lookup table for a first viewing position based on the values ​​of parameters determined based on the first viewing position, and may generate a lookup table for a second viewing position based on the values ​​of parameters determined based on the second viewing position.

[0121] The electronic device can identify a target image (840) and information on a user's viewing position (850). In one embodiment, the electronic device can identify the user's viewing position with respect to an image output device by tracking the user's viewing position. The electronic device can determine, from among a plurality of lookup tables, a lookup table for the identified user's viewing position as a target lookup table to use. The lookup table for the user's viewing position can be a lookup table generated based on the values ​​of one or more parameters determined based on information about the corresponding viewing position. The electronic device can generate a pre-distorted image (870) by performing a mapping operation (860) on the target image (840) based on the lookup table for the user's viewing position.

[0122] The electronic device can identify the user's viewing location information (850) periodically or aperiodically. As the user's viewing location information (850) changes, the electronic device can determine a lookup table to use for the mapping operation (860). For example, when the first viewing location information is identified at a first time point, the electronic device can perform the mapping operation (860) using the lookup table for the first viewing location to generate a pre-distorted image, and thereafter, when the second viewing location information is identified at a second time point, the electronic device can perform the mapping operation (860) using the lookup table for the second viewing location to generate a pre-distorted image (870).

[0123] To reduce computer memory usage and complexity, the electronic device may generate or store lookup tables for only some viewing positions. In one embodiment, if the electronic device identifies a first viewing position as a user viewing position but does not have a lookup table stored for the first viewing position, the electronic device may use lookup tables for other viewing positions close to the first viewing position, or may derive and use a lookup table for the first viewing position based on lookup tables for one or more other viewing positions. For example, if the electronic device identifies the first viewing position as a user viewing position but does not have a lookup table stored for the first viewing position, the electronic device may perform a mapping operation (860) for the target image (840) using a lookup table for a second viewing position. For example, if the first viewing location is identified as the user viewing location but the lookup table for the first viewing location is not stored, the electronic device may derive a lookup table for the first viewing location based on the lookup table for the second viewing location and the lookup table for the third viewing location and perform a mapping operation (860) for the target image (840). For example, if the first viewing location is identified as the user viewing location but the lookup table for the first viewing location is not stored, the electronic device may perform a mapping operation (860) for the target image (840) based on data derived by performing any operation (e.g., average, median) on data included in the lookup table for the second viewing location and / or the lookup table for the third viewing location.

[0124] In FIG. 8, each operation is illustrated as being performed for multiple viewing positions, but this is not limited thereto. For example, the electronic device may generate a lookup table for a first viewing position and then generate a lookup table for a second viewing position.

[0125] In the description of FIG. 8 described above, the operations illustrated in FIG. 8 are described as being performed by a single electronic device, but this is not limited thereto. In one embodiment, the operations illustrated in FIG. 8 may be performed by multiple electronic devices. For example, one or more first electronic devices may generate a look-up table and provide the generated look-up table to one or more second electronic devices. The one or more second electronic devices may use the provided look-up table to generate a pre-distorted image from a target image.

[0126] FIG. 9 is a flowchart illustrating a method for an electronic device according to one embodiment of the present disclosure to generate a pre-distorted image using a look-up table.

[0127] Referring to FIG. 9, a method (900) for generating a pre-distorted image using a look-up table by an electronic device according to one embodiment may include steps 910 to 940. In one embodiment, steps 910 to 940 may be executed by at least one processor included in the electronic device. In one or more embodiments, the method (900) for generating a pre-distorted image using a look-up table by an electronic device is not limited to that illustrated in FIG. 9, and may further include steps not illustrated in FIG. 9, or some steps may be omitted.

[0128] In step 910, the electronic device can identify a target image including a plurality of pixels. In one embodiment, the target image can be an original image. In one embodiment, the target image can be an image that is an upscaled / downscaled image of the original image. For example, based on the fact that the resolution of the original image is different from the resolution of the target lookup table, the electronic device can generate a target image having the same resolution as the resolution of the target lookup table from the original image. That is, the electronic device can convert the original image to the resolution (or size) of the target lookup table and then use the target lookup table.

[0129] In step 920, the electronic device may identify the coordinates of the second pixel corresponding to the coordinates of the first pixel using a target lookup table generated based on a scaling parameter associated with the image output size. In one embodiment, the target lookup table may be generated based on other parameters such as a radial distortion parameter and a tangential distortion parameter and the scaling parameter. In one embodiment, the target lookup table may be a lookup table selected by the electronic device from among a plurality of lookup tables based on information about the user's viewing position. That is, the electronic device may identify information about the user's viewing position, and determine, select, and identify a lookup table for the user's viewing position as the target lookup table from among the plurality of lookup tables.

[0130] In step 930, the electronic device can generate a pre-distorted image based on the data of the first pixel and the coordinates of the second pixel of the target image. In one embodiment, the data of the second pixel in the pre-distorted image can be determined based on the data of the first pixel in the target image. In one embodiment, if the coordinates of the first pixel and the coordinates of the third pixel both correspond to the coordinates of the second pixel by the target lookup table, the electronic device can determine the data of the second pixel in the pre-distorted image based on the data of the first pixel and the data of the third pixel in the target image. For example, the electronic device can determine the data of the second pixel in the pre-distorted image based on the data of the first pixel and the data of the third pixel in the target image using any rule, algorithm, operation, or model.

[0131] In step 940, the electronic device may output the generated pre-distorted image. In one embodiment, the electronic device may output the pre-distorted image without performing additional interpolation processing for scaling on the pre-distorted image generated using the look-up table, but is not limited thereto.

[0132] In one embodiment, the electronic device may perform steps 910 to 940 for each image frame as a target image (or source image) to process an image including a plurality of image frames. In one embodiment, the electronic device may periodically or aperiodically identify the user's viewing position information, and determine a target lookup table from among a plurality of lookup tables based on the user's viewing position information. In this case, as the user's viewing position information changes, the target lookup table may also change, and different pre-distorted images may be generated and output for the same target image. For example, as the user's viewing position information changes, a first lookup table may be used to process a first image frame of the image, and a second lookup table may be used to process a second image frame.

[0133] FIG. 10 is a flowchart illustrating a method for an electronic device to generate a look-up table according to one embodiment of the present disclosure.

[0134] Referring to FIG. 10, a method (1000) for generating a look-up table by an electronic device according to one embodiment may include steps 1010 to 1040. In one embodiment, steps 1010 to 1040 may be executed by at least one processor included in the electronic device. In one or more embodiments, the method (1000) for generating a look-up table by an electronic device is not limited to that illustrated in FIG. 10, and may further include steps not illustrated in FIG. 10, or some steps may be omitted.

[0135] In step 1010, the electronic device may identify a value of a scaling parameter associated with an image output size. In one embodiment, the value of the scaling parameter may be determined based on at least one of a distance between the image output device and a viewing position or a display resolution.

[0136] In step 1020, the electronic device may calculate the coordinates of the second pixel corresponding to the coordinates of the first pixel based on the value of the scaling parameter. In one embodiment, the electronic device may identify values ​​of other parameters, such as a radial distortion parameter and a tangential distortion parameter, and calculate the coordinates of the second pixel corresponding to the coordinates of the first pixel based on the values ​​of the identified other parameters and the value of the scaling parameter.

[0137] In step 1030, the electronic device may calculate a vector corresponding to the first pixel based on the coordinates of the first pixel and the coordinates of the second pixel. In one embodiment, the electronic device may calculate the vector corresponding to the first pixel based on an operation of subtracting the coordinates of the second pixel from the coordinates of the first pixel.

[0138] In step 1040, the electronic device may generate a lookup table based on the generated vector. In one embodiment, the generated lookup table may be used to generate a pre-distorted image for the target image. In one embodiment, the electronic device may provide the generated lookup table to another electronic device.

[0139] The electronic device can generate multiple lookup tables for multiple viewing locations by performing steps 1010 to 1040 for each of the multiple viewing locations. In one embodiment, the electronic device can generate multiple lookup tables by performing the method (1000) sequentially or in parallel for each of the multiple viewing locations.

[0140] FIG. 11 is a block diagram illustrating an example of an electronic device for generating a look-up table and an electronic device for generating a pre-distorted image according to one embodiment of the present disclosure.

[0141] The electronic device (1110) illustrated in FIG. 11 is a device that generates a look-up table and may be a user terminal or a server device. The electronic device (1120) illustrated in FIG. 11 is a device that generates a pre-distorted image using a look-up table and may be a user terminal or a server device. For example, the electronic devices (1110, 1120) may be user terminals such as a desktop computer, a laptop computer, a notebook, a smart phone, a tablet PC, a mobile phone, a smart watch, a wearable device, an augmented reality device (AR device), or a virtual reality device (VR device).

[0142] Although FIG. 11 illustrates an electronic device (1110) that generates a look-up table and an electronic device (1120) that generates a pre-distorted image using the look-up table as different devices, the present invention is not limited thereto. In one embodiment of the present disclosure, the electronic device (1110) that generates a look-up table and the electronic device (1120) that generates a pre-distorted image using the look-up table may be the same user terminal or server device. For example, one electronic device may generate a look-up table and generate a pre-distorted image using the generated look-up table.

[0143] In one embodiment of the present disclosure, the electronic device (1110) can provide one or more lookup tables it has generated to the electronic device (1120). The electronic device (1120) can generate a pre-distorted image using the one or more lookup tables it has received. For example, the electronic device (1110) can update a lookup table included in the electronic device (1120). In one embodiment of the present disclosure, the electronic device (1110) can provide one or more lookup tables it has generated to a plurality of electronic devices (1120). In one embodiment of the present disclosure, the electronic device (1120) can receive one or more lookup tables from a plurality of electronic devices (1110).

[0144] In one embodiment of the present disclosure, the electronic device (1110, 1120) may include, but is not limited to, at least one processor (1112, 1122) and memory (1114, 1124).

[0145] The processor (1112, 1122) is electrically connected to the components included in the electronic device (1110, 1120) and can execute operations or data processing related to control and / or communication of the components included in the electronic device (1110, 1120). In one embodiment, the processor (1112, 1122) can load and process requests, commands, or data received from at least one of the other components into a memory and store the processing result data in the memory. The processor (1112, 1122) is a component that controls a series of processes so that the electronic device (1110, 1120) operates according to the above-described embodiments, and may be configured with one or more processors.

[0146] One or more processors included in the processor (1112, 1122) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. One or more processors included in the processor (1112, 1122) may include at least one of a general-purpose processor such as a central processing unit (CPU), a Micro Processor Unit (MPU), an application processor (AP), a Digital Signal Processor (DSP), a graphics-only processor such as a GPU (graphic processing unit) or a VPU (vision processing unit), an artificial intelligence-only processor such as an NPU (neural processing unit), or a communication-only processor such as a CP (communication processor). When one or more processors included in the processor (1112, 1122) are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0147] Processors (1112, 1122) may include various processing circuits and / or multiple processors. For example, the term "processor" as used in this disclosure, including the claims, may include various processing circuits including at least one processor. One or more of the at least one processors may be configured to perform one or more functions of the present disclosure, individually and / or collectively in a distributed manner. In this disclosure, when "processor," "at least one processor," or "one or more processors" is described as being configured to perform multiple functions, this may include situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor performs all of the functions. Furthermore, at least one processor may include a combination of processors that perform various functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0148] The processor (1112, 1122) can write data to the memory (1114, 1124) or read data stored in the memory (1114, 1124), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (1114, 1124). The processor (1112, 1122) can be controlled to process input data according to predefined operation rules, algorithms, methods, or models stored in the memory (1114, 1124). The processor (1112, 1122) can be controlled to process input data (e.g., a target image) based on data (e.g., a look-up table) stored in the memory (1114, 1124).

[0149] The memory (1114, 1124) is electrically connected to the processor (1112, 1122) and may store one or more modules, algorithms, operating rules, models, programs, instructions, or data related to the operation of components included in the electronic device (1110, 1120). For example, the memory (1114, 1124) may store one or more modules, algorithms, operating rules, models, programs, instructions, or data for processing and controlling the processor (1112, 1122).

[0150] The memory (1114, 1124) may be configured as a storage medium or a combination of storage media such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, an optical disk, etc., but is not limited thereto. The memory (1114, 1124) may not exist separately and may be configured to be included in the processor (1112, 1122). The memory (1114, 1124) may be configured as a volatile memory, a nonvolatile memory, or a combination of a volatile memory and a nonvolatile memory. The memory (1114, 1124) may store a program or at least one instruction for performing operations according to the above-described embodiments. The memory (1114, 1124) may also provide stored data to the processor (1112, 1122) upon request of the processor (1112, 1122).

[0151] In one embodiment of the present disclosure, the memory (1114, 1124) may store data or information identified, acquired, or generated by the electronic device (1110, 1120). For example, the memory (1114, 1124) may store one or more look-up tables, etc. For example, the memory (1114, 1124) may store one or more look-up tables in association with viewing location information. For example, the memory (1114, 1124) may store reference images, original images, target images, pre-distorted images, etc. For example, the memory (1114, 1124) may store values ​​of one or more parameters. For example, the memory (1114, 1124) may store information (e.g., specifications, etc.) about an image output device. The memory (1114, 1124) can store data or information identified, acquired, or generated by the electronic device (1110, 1120) in a compressed form.

[0152] Some modules that perform at least one operation of the electronic device (1110, 1120) may be implemented as hardware modules, software modules, and / or a combination thereof. The memory (1114, 1124) may include software modules that perform at least some of the operations of the electronic device (1110, 1120) described above. In one embodiment, the modules included in the memory (1114, 1124) may perform the operations by being executed by the processor (1112, 1122). For example, the modules (i.e., software modules) included in the memory (1114, 1124) may include programs, models, or algorithms that are executed according to the control or command of the processor (1112, 1122) and are configured to perform operations that derive output data for input data. Some modules that perform at least one operation of the electronic device (1110, 1120) may be composed of a plurality of sub-modules or may constitute a single module.

[0153] The electronic device (1110, 1120) may include more components than those illustrated in FIG. 11. In one embodiment, the electronic device (1110, 1120) may further include a communication interface (or communication module) for communicating with an external device. In one embodiment, the electronic device (1110, 1120) may further include an input / output device and / or an input / output interface. For example, the electronic device (1120) may further include a camera sensor for acquiring a target image (or an original image). For example, the electronic device (1120) may further include an image output device (e.g., a display, a virtual screen) for outputting a pre-distorted image.

[0154] In this disclosure, overlapping descriptions in FIGS. 1A to 11 may be omitted, and one or more of the above-described embodiments may be applied / implemented in combination with each other. In this disclosure, one or more embodiments described with respect to images may also be applied with respect to video, and one or more embodiments described with respect to video may also be applied with respect to images.

[0155] According to one embodiment of the present disclosure, a method performed by an electronic device may include a step of identifying a target image including a plurality of pixels. According to one embodiment of the present disclosure, the method may include a step of identifying coordinates of a second pixel corresponding to coordinates of a first pixel using a target look-up table generated based on a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, the method may include a step of generating a pre-distorted image based on data of the first pixel of the target image and the coordinates of the second pixel. According to one embodiment of the present disclosure, the method may include a step of outputting the pre-distorted image. According to one embodiment of the present disclosure, data of the second pixel in the pre-distorted image may be determined based on data of the first pixel in the target image. According to one embodiment of the present disclosure, a scaling operation (e.g., an interpolation operation) for the pre-distorted image may be omitted, thereby reducing image / video processing time.

[0156] According to one embodiment of the present disclosure, the value of the scaling parameter may be determined based on at least one of the distance between the image output device and the viewing position or the display resolution. According to one embodiment of the present disclosure, a pre-distorted image suitable for the display resolution (or size) may be generated. According to one embodiment of the present disclosure, by applying pre-distortion suitable for the viewing position to the target image, a user located at the viewing position can view an image without distortion or with minimal distortion.

[0157] According to one embodiment of the present disclosure, the target lookup table may be generated based on radial distortion parameters and tangential distortion parameters. According to one embodiment of the present disclosure, linear distortion that compensates for distortion due to curvature differences between a display and a lens and distortion due to tilt differences between the display and the lens may be applied to the target image.

[0158] According to one embodiment of the present disclosure, at least one of the value of the scaling parameter, the value of the radial distortion parameter, or the value of the tangential distortion parameter for the target lookup table can be determined based on a difference between an original reference image and an image input by a camera positioned at a target location with respect to an image output device. According to one embodiment of the present disclosure, the target lookup table can be associated with the target location. According to one embodiment of the present disclosure, a pre-distorted image can be generated and output based on optimal values ​​of the parameters, and a user can be provided with an original image without distortion.

[0159] According to one embodiment of the present disclosure, the step of identifying the coordinates of the second pixel corresponding to the coordinates of the first pixel may include the step of identifying a vector corresponding to the coordinates of the first pixel using the target lookup table. According to one embodiment of the present disclosure, the step of identifying the coordinates of the second pixel corresponding to the coordinates of the first pixel may include the step of determining the coordinates of the second pixel based on an operation of subtracting the identified vector from the coordinates of the first pixel. According to one embodiment of the present disclosure, since a pre-distorted image can be generated by a subtraction operation without complex operations, fast image processing may be possible.

[0160] According to one embodiment of the present disclosure, the method may include a step of identifying coordinates of the second pixel corresponding to coordinates of the third pixel based on the target lookup table. According to one embodiment of the present disclosure, data of the second pixel in the pre-distorted image may be determined based on data of the first pixel and data of the third pixel in the target image. According to one embodiment of the present disclosure, a pre-distorted image having a lower resolution (or smaller size) than the target image may be generated.

[0161] According to one embodiment of the present disclosure, the method may include a step of identifying information about a user's viewing position. According to one embodiment of the present disclosure, the method may include a step of identifying a target lookup table from among a plurality of lookup tables based on the information about the user's viewing position. According to one embodiment of the present disclosure, the electronic device generates and outputs a pre-distorted image using a lookup table most suitable for the user's viewing position from among the plurality of lookup tables, so that the user can view an image with little or no distortion from any position.

[0162] According to one embodiment of the present disclosure, the user's viewing position information may be 6-DOF (Degrees of Freedom) position information. According to one embodiment of the present disclosure, an electronic device can generate and output a pre-distorted image by considering more specific viewing position information, and the user can be provided with an image that is free of distortion (or has less distortion) even when the viewing position, including the direction of gaze, changes.

[0163] According to one embodiment of the present disclosure, the method may include a step of identifying a resolution of an original image. According to one embodiment of the present disclosure, the method may include a step of generating a target image having a resolution equal to the resolution of the target lookup table from the original image based on whether the resolution of the original image is different from the resolution of the target lookup table. According to one embodiment of the present disclosure, even when the resolution of the original image is different from the resolution of the pre-stored lookup table, the electronic device may generate and output a pre-distorted image using the pre-stored lookup table.

[0164] According to one embodiment of the present disclosure, a method performed by an electronic device may include a step of identifying a value of a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, the method may include a step of calculating coordinates of a second pixel corresponding to coordinates of a first pixel based on the value of the scaling parameter. According to one embodiment of the present disclosure, the method may include a step of calculating a vector corresponding to the first pixel based on the coordinates of the first pixel and the coordinates of the second pixel. According to one embodiment of the present disclosure, the method may include a step of generating a lookup table for generating a pre-distorted image for a target image based on the calculated vector. According to one embodiment of the present disclosure, the electronic device may generate and provide a lookup table capable of omitting a scaling operation for the pre-distorted image. That is, a lookup table capable of reducing image / video processing time may be generated based on the scaling parameter.

[0165] According to one embodiment of the present disclosure, the step of calculating the vector corresponding to the first pixel may include the step of calculating the vector corresponding to the first pixel based on an operation of subtracting the coordinates of the second pixel from the coordinates of the first pixel. According to one embodiment of the present disclosure, the electronic device may generate a look-up table for pixel mapping by a subtraction operation. The electronic device that performs image / video processing using the look-up table generated by the subtraction operation may generate a pre-distorted image by the subtraction operation without complex operations, and thus the image / video processing speed may be improved.

[0166] According to one embodiment of the present disclosure, the value of the scaling parameter may be determined based on at least one of the distance between the image output device and the viewing position or the display resolution. According to one embodiment of the present disclosure, a look-up table may be generated for generating a pre-distorted image that fits the display resolution (or size). According to one embodiment of the present disclosure, when a pre-distorted image is generated and output using the look-up table generated based on the value of the scaling parameter according to the distance between the image output device and the viewing position, a user located at the corresponding viewing position can view an image without distortion or with less distortion.

[0167] According to one embodiment of the present disclosure, the method may include a step of identifying a value of a radial distortion parameter and a value of a tangential distortion parameter. According to one embodiment of the present disclosure, the step of calculating the coordinates of the second pixel may include a step of calculating the coordinates of the second pixel based on the value of the scaling parameter, the value of the radial distortion parameter, and the value of the tangential distortion parameter. According to one embodiment of the present disclosure, a look-up table reflecting linear distortion that can compensate for distortion due to a curvature difference between a display and a lens and distortion due to a tilt difference between the display and a lens may be provided.

[0168] According to one embodiment of the present disclosure, at least one of the values ​​of the scaling parameter, the radial distortion parameter, and the tangential distortion parameter may be determined based on a difference between an original reference image and an image input by a camera positioned at a target viewing position with respect to an image output device. According to one embodiment of the present disclosure, the lookup table may be associated with the target viewing position. According to one embodiment of the present disclosure, a lookup table capable of providing an image with minimized distortion to a viewer may be generated based on optimal parameter values.

[0169] According to one embodiment of the present disclosure, the method may include the step of generating multiple lookup tables for multiple viewing positions. According to one embodiment of the present disclosure, a user can view an image with no or little distortion at any of the multiple viewing positions.

[0170] According to one embodiment of the present disclosure, an electronic device may include a memory that stores one or more commands and at least one processor that executes the one or more commands stored in the memory. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify a target image including a plurality of pixels. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify coordinates of a second pixel corresponding to coordinates of a first pixel using a target lookup table generated based on a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may generate a pre-distorted image based on data of the first pixel of the target image and the coordinates of the second pixel. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may output the pre-distorted image. According to one embodiment of the present disclosure, data of the second pixel in the pre-distorted image can be determined based on data of the first pixel in the target image.

[0171] According to one embodiment of the present disclosure, the value of the scaling parameter may be determined based on at least one of a distance between an image output device and a viewing position or a display resolution.

[0172] According to one embodiment of the present disclosure, the target lookup table can be generated based on radial distortion parameters and tangential distortion parameters.

[0173] According to one embodiment of the present disclosure, at least one of the values ​​of the scaling parameter, the radial distortion parameter, or the tangential distortion parameter for the target lookup table may be determined based on a difference between an original reference image and an image input by a camera positioned at a target location with respect to an image output device. According to one embodiment of the present disclosure, the target lookup table may be associated with the target location.

[0174] According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device can identify a vector corresponding to the coordinates of the first pixel using the target lookup table. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device can determine the coordinates of the second pixel based on an operation of subtracting the identified vector from the coordinates of the first pixel.

[0175] According to one embodiment of the present disclosure, by the at least one processor executing the one or more commands, the electronic device can identify the coordinates of the second pixel corresponding to the coordinates of the third pixel based on the target lookup table. According to one embodiment of the present disclosure, the data of the second pixel in the pre-distorted image can be determined based on the data of the first pixel and the data of the third pixel in the target image.

[0176] According to one embodiment of the present disclosure, by having the at least one processor execute the one or more commands, the electronic device can identify the user's viewing location information. According to one embodiment of the present disclosure, by having the at least one processor execute the one or more commands, the electronic device can identify the target lookup table among a plurality of lookup tables based on the user's viewing location information.

[0177] According to one embodiment of the present disclosure, the viewing location information of the user may be 6-DOF (Degree Of Freedom) location information.

[0178] According to one embodiment of the present disclosure, by the at least one processor executing the one or more commands, the electronic device can identify the resolution of the original image. According to one embodiment of the present disclosure, by the at least one processor executing the one or more commands, the electronic device can generate the target image having the same resolution as the resolution of the target lookup table from the original image based on the fact that the resolution of the original image is different from the resolution of the target lookup table.

[0179] According to one embodiment of the present disclosure, an electronic device may include a memory that stores one or more commands and at least one processor that executes the one or more commands stored in the memory. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may identify a value of a scaling parameter associated with an image output size. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may calculate coordinates of a second pixel corresponding to coordinates of a first pixel based on the value of the scaling parameter. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may calculate a vector corresponding to the first pixel based on the coordinates of the first pixel and the coordinates of the second pixel by executing the one or more commands. According to one embodiment of the present disclosure, when the at least one processor executes the one or more commands, the electronic device may generate a look-up table for generating a pre-distorted image for a target image based on the calculated vector by executing the one or more commands.

[0180] According to one embodiment of the present disclosure, the electronic device may include a step of calculating a vector corresponding to the first pixel based on an operation of subtracting the coordinates of the second pixel from the coordinates of the first pixel by having the at least one processor execute the one or more instructions.

[0181] According to one embodiment of the present disclosure, the value of the scaling parameter may be determined based on at least one of a distance between an image output device and a viewing position or a display resolution.

[0182] According to one embodiment of the present disclosure, by the at least one processor executing the one or more commands, the electronic device can identify a value of a radial distortion parameter and a value of a tangential distortion parameter. According to one embodiment of the present disclosure, by the at least one processor executing the one or more commands, the electronic device can calculate the coordinates of the second pixel based on the value of the scaling parameter, the value of the radial distortion parameter, and the value of the tangential distortion parameter.

[0183] According to one embodiment of the present disclosure, at least one of the values ​​of the scaling parameter, the radial distortion parameter, or the tangential distortion parameter may be determined based on a difference between an original reference image and an image input by a camera positioned at a target viewing position with respect to an image output device. According to one embodiment of the present disclosure, the lookup table may be associated with the target viewing position.

[0184] According to one embodiment of the present disclosure, the electronic device can generate a plurality of lookup tables for a plurality of viewing locations by having the at least one processor execute the one or more instructions.

[0185] A program for performing a method for generating a pre-distorted image according to an embodiment of the present disclosure on a computer may be recorded on a computer-readable recording medium. A program for performing a method for generating a look-up table according to an embodiment of the present disclosure on a computer may be recorded on a computer-readable recording medium.

[0186] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0187] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In a method performed by an electronic device, A step of identifying a target image (440, 710) containing multiple pixels; A step of identifying the coordinates of a second pixel corresponding to the coordinates of a first pixel using a target look-up table (432, 610) generated based on a scaling parameter associated with the image output size; A step of generating a line distortion image (460, 720) based on data of the first pixel of the target image (440, 710) and coordinates of the second pixel; and Including a step of outputting the above-mentioned distortion image (460, 720), A method in which data of the second pixel in the above-described distortion image (460, 720) is determined based on data of the first pixel in the target image (440, 710).

2. In paragraph 1, A method wherein the value of the above scaling parameter is determined based on at least one of the distance between the image output device and the viewing position or the display resolution.

3. In paragraph 1 or 2, The above target lookup table (432, 610) is generated based on radial distortion parameters and tangential distortion parameters.

4. In any one of paragraphs 1 to 3, At least one of the values ​​of the scaling parameter, the radial distortion parameter, or the tangential distortion parameter for the target lookup table (432, 610) is determined based on the difference between the original reference image and the image input to the camera (550) located at the target position with respect to the image output device, The above target lookup table is associated with the above target location, method.

5. In any one of paragraphs 1 to 4, The step of identifying the coordinates of the second pixel corresponding to the coordinates of the first pixel comprises: A step of identifying a vector corresponding to the coordinates of the first pixel using the above target lookup table (432, 610); and A method comprising the step of determining coordinates of the second pixel based on an operation of subtracting the identified vector from the coordinates of the first pixel.

6. In any one of paragraphs 1 to 5, Further comprising a step of identifying the coordinates of the second pixel corresponding to the coordinates of the third pixel based on the target lookup table (432, 610). A method in which data of the second pixel in the above-described distortion image (460, 720) is determined based on data of the first pixel and data of the third pixel in the target image (440, 710).

7. In any one of paragraphs 1 to 6, Step for identifying the user's viewing location information; and A method further comprising the step of identifying the target lookup table among a plurality of lookup tables based on the viewing location information of the user.

8. In any one of paragraphs 1 to 7, A method in which the viewing location information of the above user is 6-DOF (Degree Of Freedom) location information.

9. In any one of paragraphs 1 to 8, Step of identifying the resolution of the original image; and A method further comprising the step of generating the target image (440, 710) having the same resolution as the resolution of the target lookup table (432, 610) from the original image based on the fact that the resolution of the original image is different from the resolution of the target lookup table (432, 610).

10. In electronic devices, A memory storing one or more instructions; and comprising at least one processor for executing one or more instructions stored in the memory; The electronic device, wherein the at least one processor executes the one or more instructions, Identify a target image (440, 710) containing multiple pixels, Using a target look-up table (432, 610) generated based on a scaling parameter associated with the image output size, the coordinates of the second pixel corresponding to the coordinates of the first pixel are identified, Generate a line distortion image (460, 720) based on the data of the first pixel of the target image (440, 710) and the coordinates of the second pixel, Output the above distortion image (460, 720), An electronic device, wherein data of the second pixel in the above-described distortion image (460, 720) is determined based on data of the first pixel in the target image (440, 710).

11. In paragraph 10, An electronic device, wherein the value of the above scaling parameter is determined based on at least one of the distance between the image output device and the viewing position or the display resolution.

12. In either of paragraphs 10 or 11, The above target lookup table (432, 610) is an electronic device generated based on radial distortion parameters and tangential distortion parameters.

13. In any one of paragraphs 10 to 12, The electronic device causes the at least one processor to execute the one or more instructions, Using the above target lookup table (432, 610), a vector corresponding to the coordinates of the first pixel is identified, An electronic device that determines the coordinates of the second pixel based on an operation of subtracting the identified vector from the coordinates of the first pixel.

14. In any one of paragraphs 10 to 13, The electronic device causes the at least one processor to execute the one or more instructions, Identify the user's viewing location information, An electronic device that identifies the target lookup table (432, 610) among a plurality of lookup tables based on the viewing location information of the user.

15. In any one of paragraphs 10 to 14, The electronic device causes the at least one processor to execute the one or more instructions, Based on the above target lookup table (432, 610), the coordinates of the second pixel corresponding to the coordinates of the third pixel are identified, An electronic device, wherein data of the second pixel in the above-described distortion image (460, 720) is determined based on data of the first pixel and data of the third pixel in the target image (440, 710).

Images