Display driving device and display driving method

Abstract

A display driving method according to embodiments of the present invention may include: a step of receiving a video and compensating for a chromatic aberration of a distorted image of the video on the basis of a weight for each position for the image; and an edge compensation step of compensating for the edge of the chromatic aberration-compensated video, including a step of calculating positional weights on the basis of the weight for each position of the image and weights related to the distances from the pixel center of the video, and a step of correcting pixels of the chromatic aberration-compensated video on the basis of the positional weights.

Description

Display driving device and display driving method

[0001] The embodiments relate to a display driving device and a display driving method.

[0002] During the process of driving a display, distortion may occur in the displayed image depending on the characteristics of the lens and / or the characteristics of the image. The input image is a digital image, and chromatic aberration correction can be applied during the process of processing the digital image. Since the input image is a digital image, performing chromatic aberration correction may result in a discrete resulting image. There is a problem that chromatic aberration correction results in an unnatural resulting image. Therefore, a display driving method and device that corrects chromatic aberration distortion while compensating for the unnatural resulting image resulting from the correction are required.

[0003] The embodiments provide a display method and device for obtaining a smooth image from a user's perception perspective by considering the degree of distortion for each lens by reflecting lens characteristics.

[0004] The embodiments provide a low-latency and low-complexity display driving method and device for resolving distortion of various display devices and lenses and obtaining natural images.

[0005] However, the scope of the embodiments is not limited to the technical tasks described above, and the scope of the embodiments may be expanded to other technical tasks that can be inferred by a person skilled in the art based on the entire described content.

[0006] A display driving method according to embodiments includes a step of compensating for chromatic aberration of an image; and a step of compensating for an edge of the image; wherein the step of compensating for the edge includes a step of analyzing an image, a step of analyzing a distortion parameter, and a step of calculating a compensation value, and based on the compensation value, an edge of the image is compensated for.

[0007] The display driving method and device according to the embodiments can process digital images with high efficiency.

[0008] The display driving method and device according to the embodiments can provide digital images with high quality and excellent recognition performance.

[0009] The display driving method device according to the embodiments can provide general services such as VR services.

[0010] The drawings are included to further understand the embodiments, and the drawings illustrate the embodiments together with the description related to the embodiments. For a better understanding of the various embodiments described below, reference should be made to the following description of the embodiments in conjunction with the following drawings, in which like reference numerals correspond to corresponding parts throughout the drawings.

[0011] FIG. 1 illustrates various appearances of XR devices to which an embodiment of the present invention can be applied.

[0012] Figure 2 is a perspective view of the HMD device illustrated in Figure 1 viewed from various angles.

[0013] FIG. 3 is a block diagram illustrating the main internal components included in the HMD device illustrated in FIG. 1.

[0014] FIG. 4 illustrates an example of a silicon semiconductor substrate including a driving circuit included in an HMD device.

[0015] Figure 5 shows the configuration of a display driving device according to embodiments.

[0016] Figure 6 shows an example of edge compensation according to embodiments.

[0017] Figure 7 shows an example of edge compensation according to embodiments.

[0018] Figure 8 shows weights (PosW) for edge compensation according to embodiments.

[0019] Figure 9 shows the compensation effect according to the display driving method according to the embodiments.

[0020] Figure 10 shows a compensation effect according to a display driving method according to embodiments.

[0021] Fig. 11 shows a display driving device according to embodiments.

[0022] Fig. 12 shows an example of the effect of a display driving device according to embodiments.

[0023] Fig. 13 shows a display driving device according to embodiments.

[0024] Figure 14 shows a display driving method according to embodiments.

[0025] Figure 15 shows a panel of a device according to embodiments.

[0026] FIG. 16 is a block diagram of a device that provides a VR (Virtual Reality) service according to one embodiment of the present invention.

[0027] Figure 17 shows the results of performing a distortion compensation algorithm of one embodiment of the present invention compared to the prior art.

[0028] Figure 18 is a drawing for explaining bilinear interpolation applied to one embodiment of the present invention.

[0029] FIG. 19 is a diagram for explaining a process of calculating a distortion correction weight differently by considering the distance between the center point and the current pixel, according to one embodiment of the present invention.

[0030] FIG. 20 is a diagram for explaining a process of calculating a distortion correction weight differently by considering the distance between the center point and the current pixel, according to one embodiment of the present invention.

[0031] FIG. 21 is a flow chart of a method for providing a VR (Virtual Reality) service according to another embodiment of the present invention.

[0032] Preferred embodiments of the embodiments are described in detail, examples of which are illustrated in the accompanying drawings. The following detailed description, with reference to the accompanying drawings, is intended to illustrate preferred embodiments of the embodiments, rather than merely show embodiments that can be implemented according to the embodiments. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments may be practiced without these details.

[0033] While most of the terms used in the examples are commonly used in the field, some terms were arbitrarily selected by the applicant, and their meanings are described in detail in the following descriptions as needed. Therefore, the examples should be understood based on the intended meaning of the terms, not simply their names or meanings.

[0034] FIG. 1 illustrates various appearances of XR devices to which an embodiment of the present invention can be applied.

[0035] Virtual Reality (VR) refers to a technology that uses virtual images, for example, for objects, backgrounds, and environments, rather than real ones. A representative product used in VR technology is the Head Mounted Display (HMD). When the HMD is worn on the head, small displays are positioned close to each eye, projecting three-dimensional (3D) images using parallax. Using a gyro sensor that tracks the user's movements and a rendering function that creates images based on those movements, the user can experience the feeling of being in a 3D space. Figure 1 (a) illustrates the exterior of an HMD device as an example.

[0036] Augmented Reality (AR) refers to a technology that superimposes a three-dimensional virtual image onto a real-world image or background, creating a single image. A representative product used in AR technology is AR glasses. These are eyeglass-shaped electronic devices that display AR content on transparent lenses. While worn like regular glasses, they display a large screen-like display in front of the user's eyes or allow them to access various AR content. Users can experience an expanded reality experience that combines AR content by utilizing all 360-degree spaces, centered on the user. Figure 1 (b) illustrates the exterior of AR glasses as an example.

[0037] Mixed Reality (MR) encompasses both Augmented Reality (AR), which adds virtual information to the real world, and Augmented Virtuality (AV), which adds real-world information to a virtual environment. In other words, it provides a smart environment where reality and virtuality are seamlessly connected, enabling users to experience a richer experience.

[0038] Extended Reality (XR) refers to ultra-realistic technologies and services that encompass mixed reality (MR) technologies that encompass virtual reality (VR) and augmented reality (AR).

[0039] In the following Figures 2 to 4, an HMD device to which VR technology is applied will be described as an example for convenience of explanation, but the present invention can be applied to any type of XR device.

[0040] Figure 2 is a perspective view of the HMD device illustrated in Figure 1 viewed from various angles.

[0041] FIG. 2 (a) is a perspective view from above of an HMD device according to an embodiment of the present invention, and reference numeral 210 denotes a lens, and there are two lenses for the left eye and the right eye, respectively.

[0042] No. 220 is a display, and it is possible to manufacture it in a form where there are two displays to display left-eye images and right-eye images respectively, or it is also within the scope of the present invention to manufacture it as a single display.

[0043] And, number 230 refers to an HMD device (other types of VR devices are also within the scope of the present invention).

[0044] FIG. 2 (b) is a perspective view of an HMD device according to an embodiment of the present invention viewed from the front, and FIG. 2 (c) is a perspective view of an HMD device according to an embodiment of the present invention viewed from the side.

[0045] FIG. 3 is a block diagram illustrating the main internal components included in the HMD device illustrated in FIG. 1.

[0046] As illustrated in FIG. 3, the image receiving unit (310) included in the left-eye device (300) and the image receiving unit (310) included in the right-eye device (350) receive the left-eye image and the right-eye image, respectively.

[0047] This is a characteristic of human vision, and since the viewpoint of an object must be different depending on the position of the two eyes, the images perceived by both eyes can be combined in the brain to be recognized as a single image, so the images for the left and right eyes are designed to be received respectively.

[0048] The first image correction unit (320) included in the left-eye device (300) and the second image correction unit (340) included in the right-eye device (350) independently process the received image for R, G, and B, and then transmit the image to the left-eye and right-eye panels included in the image output unit (330), respectively.

[0049] And, the images sent to the left eye device (300) and the right eye device (350) are generated by the AP (application processor).

[0050] Meanwhile, in Figure 3, the device is divided into a receiving unit, a correction unit, and an output unit, but this configuration exists independently for the left-eye and right-eye devices. Of course, implementing it as a single configuration also falls within the scope of the present invention.

[0051] And, FIG. 4 illustrates an example of a silicon semiconductor substrate including a driving circuit included in the HMD device illustrated in FIG. 1.

[0052] Unlike display panels in other products according to the prior art, the HMD device (including other types of VR devices) according to one embodiment of the present invention is designed so that the driving layer is raised for the silicon semiconductor.

[0053] Furthermore, when the image is corrected, the resolution for red pixels increases, while that for blue pixels decreases. Therefore, before outputting the video data, some data for red pixels is removed, and the blue pixels are filled with black and transmitted to the driving layer to output the video data.

[0054] Figure 5 shows the configuration of a display driving device according to embodiments.

[0055] The display driving device (100) of FIG. 5 may include a CAC processing unit (501) and / or an edge compensation unit (502). CAC stands for Chromatic Aberration Correction. The edge compensation unit (502) may include an input image analysis unit (503), a distortion parameter analysis unit (504), a compensation value calculation unit (505), and / or a compensation processing unit (506). The operation of each component is described below. Each component of the display driving device according to the embodiments of FIG. 5 may correspond to hardware, software, a processor, and / or a combination thereof.

[0056] The processing unit (501) may receive an input image and apply CAC processing. According to embodiments, CAC refers to an operation for mitigating chromatic aberration correction. The processing unit may be referred to as a CAC processing unit.

[0057] The compensation unit (502) can additionally compensate for edges generated during the CAC chromatic aberration compensation process. The edge compensation operation according to the embodiments is described in detail in FIGS. 6 to 9. The compensation unit may be referred to as an edge compensation unit.

[0058] The first analysis unit (503) can analyze an input image for edge compensation. The first analysis unit may be referred to as an input image analysis unit.

[0059] The second analysis unit (504) can analyze parameters related to distortion included in the input image. The distortion parameters may be parameter information according to the properties of a lens associated with a display driving device according to embodiments. The second analysis unit may be referred to as a distortion parameter analysis unit.

[0060] The calculation unit (505) can calculate a compensation value for edge compensation. The calculation unit may be referred to as a compensation value calculation unit.

[0061] The processing unit (506) can compensate for the edge based on the analyzed results, distortion parameters, and / or calculated compensation values. The processing unit may be referred to as a compensation processing unit.

[0062] The display driving device and method according to the embodiments of FIG. 5 can include and perform a boundary correction method in a process of compensating for chromatic aberration due to lens distortion.

[0063] The display driving device and method according to the embodiments of FIG. 5 can alleviate edge distortion by applying an edge reconstruction algorithm to an inappropriate edge from a perception perspective that occurs during a chromatic aberration compensation process for a discrete edge of an input digital image. In addition, for a distorted image generated by a lens distortion parameter after passing through a special lens, a chromatic aberration compensation algorithm is applied to correct chromatic aberration, and at the same time, an edge reconstruction algorithm is applied to correct the edge, thereby obtaining a natural edge for the edge of the image recognized after passing through the lens.

[0064] Figure 6 shows an example of edge compensation according to embodiments.

[0065] The display driving device according to the embodiments of FIG. 5 can effectively correct edge distortion as shown in FIG. 6.

[0066] In relation to prior art, since the input image is a digital image, the resulting image after chromatic aberration correction is a discrete image. The display driving method / device according to the embodiments applies an edge reconstruction algorithm (see FIG. 37, etc.) after chromatic aberration correction (501) to obtain a natural image with reduced edge distortion (see FIG. 9, etc.).

[0067] Edge reconstruction methods according to embodiments may utilize a filter. For example, the filter according to embodiments may be referred to as a modified shock filter. A user may tune discrete edges (506) according to lens distortion parameters (504).

[0068] The filter can receive an image and output a filtered image through a process similar to that in Fig. 6. The processing process in Fig. 6 is described as follows.

[0069] O stands for output. O r is the output red pixel, O g is the output green pixel, O b is the output blue pixel. i represents the input. i r is the input red pixel, i g is the input green pixel, i b is the input blue pixel.

[0070] x and y represent the position of the pixel in the orthogonal coordinate system. For example, if there are 3-by-3 pixels, (x, y) is the center pixel, (x-1, y) is the pixel located to the left on the x-axis, (x+1, y) is the pixel located to the right on the x-axis, and (x, y+1) is the pixel located above on the y-axis.

[0071] The del2(i) function and the gardient(i) function can be calculated from the pixels at each location of the input image, as shown in Fig. 2. That is, the pixels of the output image can be generated based on the sign of the del2(i) function and the gardient(i) function at each location for each color component of the input image. Distortion image effects such as blur can be alleviated through the shock filter in Fig. 6.

[0072] Figure 7 shows an example of edge compensation according to embodiments.

[0073] Figure 7 shows a specific algorithm of the edge restoration method described in Figure 6, i.e., the modified shock filter.

[0074] The position weight (PosW) in the modified shock filter of Fig. 7 is a weighted sum of the local weight of CAC and the local weight of edge reconstruction. Alternatively, only independent local weights may be used.

[0075] Since the degree of distortion varies depending on the pixel location due to lens characteristics, a distortion parameter is applied depending on the image location to obtain a smooth image from a perceptual perspective.

[0076] The modified shock filter of Fig. 7 can sharpen and blur edges by utilizing additional sign signals according to the characteristics of each channel in accordance with the characteristics of lens distortion parameters, and can adjust the tuning strength through a weight signal according to the settings of the user or the device according to the embodiments.

[0077] The filter can receive an image and output a filtered image through a process similar to that in Fig. 7. The processing process in Fig. 7 is described as follows.

[0078] In Fig. 7, the parameters that are the same as those in Fig. 6 have the same definitions. Looking at the definitions of the added parameters, weight (CFG_WEIGHT) represents a register for adjusting the intensity of edge tuning. Sign (CFG_SIGN) represents a register for selecting between sharpening and / or blurring. For example, if the sign is -1, sharpening is applied to the pixel, and if it is 1, blurring is applied to the pixel. Weight (PosW) represents a weight according to the distance from the center of the image.

[0079] Figure 7 has the following effects. For example, it further includes values ​​for adjusting the edge tuning intensity, and further includes values ​​for selecting simultaneous sharpening and / or blurring. These can be determined based on various lens characteristics, and filtering can be easily and quickly applied by reflecting various lens properties to obtain a smooth image from a perceptual perspective according to embodiments.

[0080] Figure 8 shows weights (PosW) for edge compensation according to embodiments.

[0081] Fig. 8 shows the configuration of weights applied to the edge compensation method of Fig. 7.

[0082] The display driving method / device according to the embodiments can generate weights as shown in FIG. 8 and compensate for the edges of an image as shown in FIG. 7 based on the weights. For example, the weights can be generated by two weights. The first weight (800) is a weight for each image position of CAC, and the second weight (801) is a weight for each pixel position of edge restoration. Variables α and β can be applied to each of the first and second weights.

[0083] The variables may vary depending on the characteristics of the lens and display driver. The weight value may be tuned depending on the variables. The first weight (800) may vary in a direction away from the center depending on the characteristics of the image. For example, the closer to the center of the image, the greater the weight for a pixel. The second weight (801) represents a weight according to the position of the pixel. That is, the weight (PosW) of the edge compensation of FIG. 7 may be calculated by applying a position-specific weight and a weight-specific tuning value by considering both the positions of the image and the pixels in each of the CAC and edge restoration. For example, the weight PoW may vary depending on the color. The weight PoW for red may be different from the weight PoW for blue. Different weights may be referred to as the first weight and the second weight. Both chromatic aberration compensation and edge compensation effects may be provided. With respect to the second weight (801), PosW_E can be used by referencing a weight in a curve interpolated with points of cfg according to the distance (dist) of the center position. Here, the distance can be calculated as in method (802) according to the distance on the x-axis and y-axis of the orthogonal coordinate system. The number of ptr is 9ea, so that a smooth curve can be formed with 9ea surfaces on the curvature. Local interpolation through the second weight (801) will be further described below.

[0084] Figure 9 shows the compensation effect according to the display driving method according to the embodiments.

[0085] The display driving device (500) of FIG. 5 can perform edge compensation described in FIGS. 6 to 8 to provide an effect similar to FIG. 9.

[0086] The display driving device (100) of FIG. 1 can receive an input image (900). The display driving device (500) can perform chromatic aberration compensation on the input image (900) by the CAC processing unit (501) and output an output image (901). Processing is applied to the output image (901) to alleviate chromatic aberration errors that may occur due to a lens. The display driving device (500) can perform edge compensation on the output image (901) by the edge compensation unit (502) and output an output image (902). Processing is applied to the output image (902) to alleviate edge distortion that may occur due to a lens. Specifically, the input image (900), the output image (901), and the output image (902) can include edges (903). The enlarged edges (904, 905, 906) are enlarged edges of each image. Edge (905) has chromatic aberration compensation, and edge (906) contains a result with more reduced distortion.

[0087] Figure 10 shows a compensation effect according to a display driving method according to embodiments.

[0088] Figure 10 illustrates in more detail an example of the result effect described in Figure 9.

[0089] Comparing before (1000) and after (1001) edge (boundary) compensation (correction), the results are as shown in Fig. 10.

[0090] When edge compensation such as in FIGS. 7 and 8 is not applied, when the input image (900) passes through the lens, edge distortion phenomenon occurring at the edge (903) included in the image can be seen.

[0091] When applying edge compensation as in FIGS. 7 and 8, when an edge-compensated input image (502) passes through a lens, it can be seen that the edge distortion is reduced in the edge (904) included in the image. As in FIG. 10, it can be seen that not only chromatic aberration errors occur due to the lens, but also edge distortion occurs due to the lens. Therefore, the display driving device according to the embodiments can compensate for all lens distortions through the CAC processing unit (501) and / or the edge compensation unit (502).

[0092] Fig. 11 shows a display driving device according to embodiments.

[0093] Fig. 11 may correspond to the display driving device of Fig. 5. Each component of Fig. 11 may correspond to hardware, software, a processor, and / or a combination thereof. Fig. 11 illustrates an example configuration of the edge compensation unit of Fig. 5. The parameters required for edge compensation methods such as those of Figs. 7 and 8 may be generated by the edge compensation unit of Fig. 11.

[0094] The gradient calculation unit (1100) can calculate the gradient of Fig. 7.

[0095] The del calculation unit (1101) can calculate del2 of Fig. 7.

[0096] The distortion information processing unit (1102) can store lens distortion parameters and provide them to the display driving device.

[0097] The position calculation unit (1103) can calculate the x-position and y-position of an image pixel in the orthogonal coordinate system.

[0098] The distortion weight calculation unit (1104) can calculate the weight applied to Fig. 7.

[0099] The offset value calculation unit (1105) can calculate the offset applied to FIG. 7. For example, offsets such as alpha and beta can be calculated. The offset can be referred to in various ways, such as a parameter or a value.

[0100] The edge compensation calculation unit (1106) can calculate the edge compensation value as shown in FIG. 7.

[0101] The device of Fig. 11 can compensate for the edge and output a compensated output image such as Fig. 9 and Fig. 10.

[0102] Fig. 12 shows an example of the effect of a display driving device according to embodiments.

[0103] As seen in Figures 9 and 10, Figure 12 shows an output image in which chromatic aberration and / or edge distortion of the output image has been compensated. The effect of alleviating chromatic aberration and / or edge distortion in the edge region can be observed. In other words, distortions such as color blurring can be seen to be reduced.

[0104] Fig. 13 shows a display driving device according to embodiments.

[0105] The display driving device of FIG. 5 and FIG. 11 can be configured as a device (1300) such as FIG. 13.

[0106] The display driving device (1300) may include an interface unit (1301), a processor (1302), and / or a memory (1303). Each component of FIG. 13 may correspond to hardware, software, a processor, and / or a combination thereof.

[0107] The interface unit (1301) can receive data required for the device and transmit it within the device. The interface unit (1301) can receive and transmit information related to image analysis, lens distortion parameter analysis, etc.

[0108] The processor (1302) is connected to the interface unit (1301) and the memory (1303) and can perform chromatic aberration compensation and / or edge compensation described in FIGS. 5, 6, 7, and 11. The processor can control and perform the operations of each component of FIGS. 5 and 11.

[0109] The memory (1303) is connected to the processor and can store and transmit information required for chromatic aberration compensation and / or edge compensation to the processor.

[0110] Figure 14 shows a display driving method according to embodiments.

[0111] A display driving method according to embodiments may include a step of compensating for chromatic aberration of an image (S1400) and / or a step of compensating for an edge of an image (S1401). The step of compensating for an edge of an image (S1401) may include a step of analyzing an image (S1401-1), a step of analyzing a distortion parameter (S1401-2), and / or a step of calculating a compensation value (S1401-3).

[0112] The step (S1400) of compensating for chromatic aberration of an image may correspond to the operation of the CAC processing unit (501) of FIG. 5. The step (S1401) of compensating for the edge of an image may correspond to the operation of the edge compensation unit (502) of FIG. 5. The step (S1401) of compensating for the edge of an image may include each operation of FIG. 11. For example, the step (S1401-1) of analyzing an image may obtain information from an image to obtain information for compensating for the edge of the image, such as the input image analysis unit (503). The step (S1401-2) of analyzing a distortion parameter may obtain parameters representing information related to influences such as distortion caused by a lens from a lens and / or a system, a display driving device. The step (S1401-3) of calculating a compensation value may calculate values ​​for compensating for an edge, as shown in FIGS. 7 to 11.

[0113] Meanwhile, the device of FIG. 3 represents a configuration of a device including the devices of FIG. 1 and FIG. 7. The device according to the embodiments of FIG. 3 is a VR (Virtual Reality) device and may be referred to as an HMD (Head Mount Display).

[0114] Referring to FIG. 3, the device may be composed of a left-eye device and a right-eye device. The left-eye device and the right-eye device may include an image receiving unit, an image correction unit, and / or an image output unit.

[0115] The image receiving unit can receive a left-eye image and / or a right-eye image. The input images of FIGS. 5 to 14 can each be configured as a left-eye image and / or a right-eye image.

[0116] The image correction unit can receive an input image from the image receiver and correct the image. The image correction unit can correct the image based on the color components of the image. For example, if the color system is red, green, and blue, the red, green, and blue components of the image can be corrected. The image correction unit can apply correction (compensation) operations such as those illustrated in FIGS. 5 to 14 to the image.

[0117] The image output unit can receive an image composed of corrected color components from the image correction unit and output the image through a first panel for the left eye and / or a second panel for the right eye. The user of the device can receive a high-quality corrected image.

[0118] The image receiver receives separate left-eye and right-eye images. This is due to the nature of human vision. Objects appear from different perspectives depending on the position of the two eyes. Therefore, the brain combines the images perceived by both eyes into a single image. Therefore, the images for the left and right eyes are received separately.

[0119] As described above, the received image is processed independently for R, G, and B, and then transmitted to the left-eye and right-eye panels, respectively. Meanwhile, although Fig. 3 shows a division into a receiving unit, a correction unit, and an output unit, this structure can be viewed as existing independently for the left-eye and right-eye devices. The images sent to the left-eye and right-eye devices, respectively, can be generated in the AP.

[0120] Figure 15 shows a panel of a device according to embodiments.

[0121] Unlike conventional panels, VR devices have a driving layer placed on top of a silicon substrate, and when the image is corrected as described above, the resolution of red increases and that of blue decreases. Before outputting this, the red data is removed (1500), and the blue data is filled with black (1501) and transmitted to the driving layer to output the data.

[0122] Meanwhile, referring to the configuration of the device of FIG. 3, for example, a VR device, the device according to the embodiments includes lenses, and images output from the display device can pass through the lenses and be transmitted to the user's eyes. In other words, the display device can mitigate or eliminate errors that may occur due to the characteristics of the lens as it passes through the lens.

[0123] FIG. 16 is a block diagram of a device that provides a VR (Virtual Reality) service according to one embodiment of the present invention.

[0124] Referring to the examples described above in FIGS. 1 to 15, supplementary interpretations can be made in FIG. 16 and the like.

[0125] In particular, since the product to which the present invention is applied allows the user to view image data through a lens (210) or the like as shown in FIG. 2, chromatic aberration occurs, and the present invention seeks to solve such chromatic aberration problem.

[0126] The device (500) of FIG. 16 can correspond to the device (500) of FIG. 5, and for example, can correspond to the CAC processing unit (501) of FIG. 5.

[0127] As described above, the present invention can provide a compensation algorithm for distortion caused by lens distortion parameters in a device (e.g., a VR device such as an HMD) that provides images to a user through a special lens. To implement this, the distortion parameter is designed to be reflected and compensated for based on the distance between the current pixel position and the center point position based on the center point of the lens distortion parameter.

[0128] For example, as illustrated in FIG. 16, a device (500) providing a VR (Virtual Reality) service includes a position calculation unit (510) between a current pixel and a center point, a lens distortion parameter storage unit (520), a distortion correction weight calculation unit (530) according to a distortion parameter and distance, an image distortion correction calculation unit (540), and an output image synthesis unit (550).

[0129] Meanwhile, it is also within the scope of the present invention to design the position calculation unit (510) between the current pixel and the center point, excluding the storage unit (520), the distortion correction weight calculation unit (530) according to the distortion parameter and distance, the image distortion correction calculation unit (540), and the output image synthesis unit (550) to be performed by one controller, processor, AP, etc.

[0130] Meanwhile, it is also possible to design the aforementioned 510, 530, 540 and 550 to operate through a display driver IC (DDI) instead of an AP, and if designed in this way, there is an additional effect of reducing power consumption by almost 50%, from about 100mW consumed by the AP to 50mW.

[0131] The lens distortion parameter storage unit (520) stores lens distortion parameters for chromatic aberration correction. The storage unit (520) can be implemented using various types of memory.

[0132] The position calculation unit (510) between the current pixel and the center point receives original image data and primarily calculates the distance between the center point and the current pixel.

[0133] The distortion correction weight calculation unit (530) according to the distortion parameter and distance is designed to secondarily calculate the distortion correction weight by taking into account the distance described above.

[0134] Unlike the conventional technology, the image distortion correction operation unit (540) is designed so that image distortion correction is performed according to the calculated distortion correction weights, rather than using the lens distortion parameters stored in the storage unit (520) as is.

[0135] In addition, the output image synthesis unit (550) synthesizes and outputs the corrected image daisy data. However, it is also a feature of the present invention that it is designed so that correction is performed on the red pixel and the blue pixel, while correction is not performed on the green pixel.

[0136] That is, according to one embodiment of the present invention, rather than directly using lens distortion parameters stored in memory, there is a technical effect of resolving the problem of chromatic aberration that still remains in the outer portion of the image by utilizing distortion correction weights that change depending on the distance between the center point and the current pixel. This function can be performed by a controller such as an AP.

[0137] In particular, the controller is designed such that the distortion correction weight increases as the distance between the center point and the current pixel increases, for example.

[0138] There is no particular limitation on the method for obtaining multiple distortion correction weights, and they can be obtained, for example, by using an interpolation technique or a look-up table (LUT).

[0139] Meanwhile, by designing only modules 510 and 530 to be temporarily and restrictively disabled when the current pixel is green, it is expected that there will be an additional technical effect of minimizing unnecessary power consumption.

[0140] On the other hand, if the current pixel corresponds to red or blue, modules 510 and 530 are designed to operate normally.

[0141] Therefore, the aforementioned center point corresponds to, for example, the center point of the distortion parameter for Red or the center point of the distortion parameter for Blue.

[0142] Figure 17 shows the results of performing a distortion compensation algorithm of one embodiment of the present invention compared to the prior art.

[0143] As described above, one of the features of the present invention is that the green pixel is designed to be bypassed without compensation, while compensation is performed only for the red and blue pixels.

[0144] As shown in (a) of Fig. 17, the center point of the distortion parameter for red is 610, and the center point of the distortion parameter for blue is 620.

[0145] However, when a distortion compensation algorithm is applied according to the conventional technology, as shown in (b) of Fig. 17, interpolation is performed uniformly according to the scale ratio without considering the distance between the center point and the current pixel (611, 621), so chromatic aberration frequently remains in the outer portion.

[0146] On the other hand, when a distortion compensation algorithm is applied according to an embodiment of the present invention, as shown in (c) of FIG. 17, interpolation is performed at a higher scale ratio the farther away from the center point, considering the distance between the center point and the current pixel (612, 622), so there is a technical effect of resolving the chromatic aberration phenomenon even in the outer part.

[0147] In this regard, bilinear interpolation can be modified and applied, which will be explained in more detail in Figure 18 below.

[0148] Figure 18 is a drawing for explaining bilinear interpolation applied to one embodiment of the present invention.

[0149] First, for implementing one embodiment of the present invention, for example, the following mathematical formula may be used, and in particular, a weight (PosW) according to the position difference (distance difference) between the center point and the current pixel is additionally used. The weight (PosW) according to the position difference (distance difference) will be described in more detail in FIG. 19 below. Meanwhile, the weight (PoW) described herein is used to adjust the scale ratio or compensation strength according to the distance from the center, and is different from the weight (PoW) described in FIGS. 5 and 7.

[0150] O val = (p1× w x1 + p2× w x2 ) × w y1+ (p3× w x1 + p4× w x2 ) × w y2

[0151] j : current position

[0152] I : current position

[0153]

[0154]

[0155] x1= rounddown(cal w )

[0156] y1= rounddown(cal h )

[0157] x2= x1+ 1

[0158] y2= y2+ 1

[0159] p1= input(x1, y1)

[0160] p2= input(x2, y1)

[0161] p3= input(x1, y2)

[0162] p4= input(x2, y2)

[0163] w x1 = 1 - (cal w - x1)

[0164] w x2 = cal w - x1

[0165] w y1 = 1 - (cal h - y1)

[0166] w y2 = cal h - y1

[0167] 여기서, O valFor example, as a pixel value of an output image, it means the pixel value of an image after interpolation is performed, and corresponds to number 700 shown in Fig. 18.

[0168] p1~p4 are pixel values ​​of the input image, and represent the pixel values ​​referenced when interpolation is performed. In this description, since it is bi-linear interpolation by way of example, an example of calculating with pixel values ​​of four points will be explained.

[0169] w x1 , w x2 , w y1 , w y2 refers to a weight value determined based on the pixel location in the output image compared to the pixel location in the input image.

[0170] cal h : h is an abbreviation for height, h input (height of input image), h output It refers to the position value of the output image in the y direction for j (current y position) according to the ratio of (height of the output image).

[0171] cal w : w is an abbreviation for width, w input (width of input image), w output It refers to the position value of the output image in the x direction for i (current x position), according to the ratio of (the width of the output image).

[0172] The interpolation process can be explained more specifically as follows:

[0173] First, let's assume a case of scaling from 100X80 to 160X100. In this case, when calculating the pixel value for (50,36) of the output image, if it is a process of finding the pixel locations x1, x2, y1, y2 to be referenced in the input image,

[0174] w input : 100, h input : 80, w output : 160, h output : 100, i : 50, j : 36

[0175]

[0176] And, cal h = 36 X 80 / 100 = 28.8

[0177] cal w = 50 X 100 / 160 = 31.25

[0178] x1 = 31, x2 = 32, y1 = 28, y2 = 29

[0179] p1 = pixel value at position (31,28) in the input image

[0180] p2 = pixel value at position (32,28) in the input image

[0181] p3 = pixel value at position (31,29) in the input image

[0182] p4 = pixel value at position (32,29) in the input image

[0183] w x1 = 1 - (31.25 - 31) = 0.75

[0184] w x2 = (31.25 - 31) = 0.25

[0185] w y1 = 1 - (28.8 - 28) = 0.2

[0186] w y2 = (28.8 - 28) = 0.8

[0187] -> w x1 , w x2 , w y1 , w y2 Since it is a bi-linear interpolation, it is a weight according to the 1's polinomial function according to the distance.

[0188] The result calculated as o_val = (p1 X 0.75 + p2 X 0.25) X 0.2 + (p3 X 0.75 + p4 X 0.25) X 0.8 becomes the pixel value corresponding to the location (50,36) of the output image.

[0189] However, when applying the present invention, as follows, cal h and cal w When calculating , PosW is designed to be calculated additionally.

[0190]

[0191]

[0192] FIG. 19 is a diagram for explaining a process of calculating a distortion correction weight differently by considering the distance between the center point and the current pixel, according to one embodiment of the present invention.

[0193] As shown in (a) of Fig. 19, the weight (PosW) is designed to increase as it gets farther from the center point.

[0194] An example of a formula for obtaining such a weight (PosW) is illustrated in the upper part of (b) of Fig. 19. As the current pixel position (i,j) moves away from the center point (x,y), PosW (weight) increases.

[0195] In addition, an embodiment of interpolating multiple weights by point is possible, and an embodiment of storing them in the form of a lookup table is also possible, and all of these are said to fall within the scope of the present invention.

[0196] FIG. 20 is a diagram for explaining a process of calculating a distortion correction weight differently by considering the distance between the center point and the current pixel, according to one embodiment of the present invention.

[0197] As shown in (a) of Figure 20, the weight (PosW) is designed to increase as it gets farther from the center point.

[0198] An example of a formula for obtaining such a weight (PosW) is illustrated in the upper part of (b) of Fig. 20. As the current pixel position (i,j) moves away from the center point (x,y), PosW (weight) increases.

[0199] In addition, an embodiment of interpolating multiple weights by point is possible, and an embodiment of storing them in the form of a lookup table is also possible, and all of these are said to fall within the scope of the present invention.

[0200] And, FIG. 21 is a flow chart of a method for providing a VR (Virtual Reality) service according to another embodiment of the present invention.

[0201] Receives image data input with distortion parameters for each channel (e.g., R / G / B, etc.) (S1000). However, as described above, it is designed to bypass green pixels (S1040).

[0202] On the other hand, for red pixels, the distance between the center point and the current pixel is calculated (S1010), weights are calculated according to the distance and distortion parameters (S1020), and distortion compensation is performed according to the calculated weights (S1030).

[0203] Meanwhile, for the blue pixel as well, the distance between the center point and the current pixel is calculated (S1050), a weight is calculated according to the distance and distortion parameters (S1060), and distortion compensation is performed according to the calculated weight (S1070).

[0204] And, it is designed to synthesize and output the data of each pixel for which compensation has been made (S1080).

[0205] To summarize, from the perspective of a device providing VR (Virtual Reality) services, lens distortion parameters for chromatic aberration correction are first stored.

[0206] The above device, after receiving original image data, first calculates the distance between the center point and the current pixel, and secondarily calculates a distortion correction weight considering the distance.

[0207] And, unlike the conventional technology, the device does not directly use the lens distortion parameters stored in memory or the like, but outputs the image data of the current pixel according to the calculated distortion correction weight.

[0208] In particular, the second calculation step (S1020, S1060) is designed so that the distortion correction weight increases as the distance increases.

[0209] The plurality of distortion correction weights are characterized in that they are obtained by an interpolation technique or by using a lookup table, as shown in (b) of Fig. 19.

[0210] To emphasize again, if the current pixel corresponds to Green, the aforementioned S1010, S1020, S1030, S1050, S1060, S1070 steps are designed to be disabled.

[0211] Therefore, the center points of steps S1010 and S1050 correspond to, for example, the center points of the distortion parameters for Red or the center points of the distortion parameters for Blue.

[0212] Additionally, at least one or more of the steps illustrated in FIG. 21 may be designed to be performed, for example, in a display driver IC (DDI).

[0213] For example, the image processing device according to the present disclosure may be implemented in the form of an IC by each component individually or by combining two or more components, and the functions of the image processing device may be implemented in the form of a program and installed in the IC. When the functions of the image processing device according to the present disclosure are implemented as a program, the functions of each component included in the image processing device may be implemented as a specific code, and the codes for implementing the specific functions may be implemented as a single program or may be implemented by dividing them into multiple programs.

[0214] Referring to FIG. 14, a display driving method may include a step of receiving an image and compensating for chromatic aberration of a distorted image of the image based on position-specific weights for the image; and a step of compensating for an edge of an image for which chromatic aberration has been compensated, the step of calculating positional weights based on position-specific weights for the image and weights related to a distance from a center of a pixel of the image, and a step of compensating for an edge including a step of correcting pixels of the image for which chromatic aberration has been compensated based on the positional weights.

[0215] Referring to FIG. 7, the value of an edge can be compensated by adding a value generated by multiplying a position weight (), a value for the intensity of tuning the edge (CFG_WEIGHT), and a sign value (CFG_SIGN) for selecting one of sharpening and blurring for the edge. That is, the pixel value of the edge is adjusted to make the edge sharper or softer.

[0216] Referring to FIGS. 7-8, in relation to the edge compensation operation, position weights are generated and used in embodiments. Specifically, the position weights can be generated by at least one of the positions of the image of the image related to the compensation of chromatic aberration or the positions of the pixels related to the compensation of the edge.

[0217] Referring to FIG. 7, with respect to the sharpening and / or blurring selection register, a positive sign value for selecting one of sharpening and blurring for an edge may indicate blurring, and a negative sign value may indicate sharpening.

[0218] Therefore, the edge of the image with chromatic aberration compensated includes distortion at each pixel location according to the lens characteristics, and the distortion of the edge is tuned by the input tuning strength value, and the distortion of the edge can be sharpened or blurred by a single sign value.

[0219] A display driving method according to embodiments may be performed by a display driving device, and the display driving device may include a processing unit that receives an image and compensates for chromatic aberration of a distorted image of the image based on position-specific weights for the image; and a compensation unit that compensates for an edge of an image for which chromatic aberration has been compensated, the compensation unit calculating positional weights based on position-specific weights for the image and weights related to a distance from a center of a pixel of the image, and correcting pixels of the image for which chromatic aberration has been compensated based on the positional weights. The display driving device according to embodiments may be configured with a memory and a processor.

[0220] In addition, a device including a display driving device can perform an operation according to embodiments. The device can correspond to a VR device, etc. For example, a VR (Virtual Reality) device includes a lens; and a display driving device; and the display driving device includes a processing unit that compensates for chromatic aberration of an image; and a compensation unit that compensates for an edge of the image; and the compensation unit analyzes an image, analyzes a distortion parameter, and calculates a compensation value, and based on the compensation value, an edge of the image is compensated, and an image output from the display driving device through the lens can be transmitted.

[0221] Accordingly, in order to resolve the discrete edges of the input digital image, the edge reconstruction operation can be applied to the inappropriate edges from the perception perspective that occur during the chromatic aberration compensation process to alleviate and resolve edge distortion. In addition, for the distorted image generated by the lens distortion parameter through a special lens, the chromatic aberration is corrected by applying the chromatic aberration compensation algorithm, and at the same time, the edge is corrected by applying the edge reconstruction operation, so that a natural edge can be obtained for the edge of the image recognized after passing through the lens.

[0222] The embodiments have been described in terms of methods and / or devices, and the descriptions of methods and devices may be applied complementarily.

[0223] For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium having a program recorded thereon for executing the previously described embodiments, as needed by a person skilled in the art, also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art to which the present invention pertains without departing from the gist of the embodiments claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the embodiments.

[0224] The various components of the devices of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented by a single chip, for example, a single hardware circuit. According to embodiments, the components according to the embodiments may be implemented by separate chips. According to embodiments, at least one of the components of the devices of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations / methods according to the embodiments. The executable instructions for performing the methods / operations of the devices of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may include implementations in the form of carrier waves, such as transmissions via the Internet. Furthermore, processor-readable recording media may be distributed across network-connected computer systems, allowing processor-readable code to be stored and executed in a distributed manner.

[0225] In this document, “ / ” and “,” are interpreted as “and / or”. For example, “A / B” is interpreted as “A and / or B”, and “A, B” is interpreted as “A and / or B”. Additionally, “A / B / C” means “at least one of A, B, and / or C”. Also, “A, B, C” means “at least one of A, B, and / or C”. Additionally, “or” in this document is interpreted as “and / or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

[0226] Terms such as "first," "second," etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be interpreted as limited by these terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a "second user input signal." Similarly, a second user input signal may be referred to as a "first user input signal." The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although "first user input signal" and "second user input signal" are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

[0227] The terminology used to describe the embodiments is for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and / or” are used to mean all possible combinations of terms. The expression “includes” describes the presence of features, numbers, steps, elements, and / or components, but does not mean that additional features, numbers, steps, elements, and / or components are not included. Conditional expressions such as “if” or “when” used to describe the embodiments are not intended to be limited to only optional cases. When a specific condition is satisfied, a related action is performed in response to a specific condition, or a related definition is intended to be interpreted.

[0228] Additionally, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in the memory.

[0229] Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device may include a transmitting / receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart, and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting / receiving device.

[0230] The processor may be referred to as a controller or the like, and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor. Furthermore, the processor may be implemented as an encoder / decoder or the like for the operations of the above-described embodiments.

[0231] As described above, the relevant contents have been described in the best form for carrying out the embodiments.

[0232] As described above, the embodiments can be applied in whole or in part to digital image transmission and reception devices and systems.

[0233] Those skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0234] Embodiments may include modifications / changes, which do not depart from the scope of the claims and their equivalents.

Claims

1. A step of receiving an image and compensating for chromatic aberration of a distorted image of the image based on a position-specific weight for the image; and As a step of compensating the edge of the image for which the above chromatic aberration has been compensated, A step of calculating a position weight based on a position-specific weight for the image and a weight related to a distance from the center of a pixel of the image, A display driving method, comprising: a step of compensating the edge, including a step of correcting pixels of the image for which chromatic aberration is compensated based on the positional weights; 2. In paragraph 1, A display driving method in which the edge is compensated by adding a value generated by multiplying the position weight, a value for the intensity of tuning the edge, and a sign value for selecting one of sharpening and blurring for the edge.

3. In paragraph 2, A display driving method, wherein the position weight is generated by at least one of the position of the image of the image associated with compensation for the chromatic aberration or the position of the pixel associated with compensation for the edge.

4. In paragraph 2, A display driving method, wherein a sign value for selecting one of sharpening and blurring for the edge is positive, indicating blurring, and a sign value is negative, indicating sharpening.

5. In paragraph 1, A display driving method, wherein the edge of the image for which the chromatic aberration is compensated includes distortion at each pixel location according to lens characteristics, the distortion of the edge is tuned by an input tuning strength value, and the distortion of the edge is sharpened or blurred by a single sign value.

6. A processing unit that receives an image and compensates for chromatic aberration of a distorted image of the image based on a position-specific weight for the image; and As a compensation unit that compensates for the edge of the image in which the above chromatic aberration has been compensated for. Calculate position weights based on position-specific weights for the image and weights related to the distance from the center of a pixel of the image, A display driving device, comprising: a compensation unit for correcting pixels of an image for which chromatic aberration is compensated based on the positional weights; 7. In paragraph 6, A display driving device in which the edge is compensated by adding a value generated by multiplying the position weight, a value for the intensity of tuning the edge, and a sign value for selecting one of sharpening and blurring for the edge.

8. In paragraph 7, A display driving device, wherein the position weight is generated by at least one of a position of an image of the image associated with compensation for the chromatic aberration or a position of a pixel associated with compensation for the edge.

9. In paragraph 8, A display driving device, wherein a sign value for selecting one of sharpening and blurring for the edge is positive, indicating blurring, and a sign value is negative, indicating sharpening.

10. In paragraph 6, A display driving device in which the edge of the image for which the chromatic aberration is compensated includes distortion at each pixel location according to lens characteristics, the distortion of the edge is tuned by an input tuning strength value, and the distortion of the edge is sharpened or blurred by a single sign value.

Images