Image distortion correction data storage method and image distortion correction method

Abstract

The application discloses an image distortion correction data storage method and an image distortion correction method; the method comprises the following steps: dividing pixel point coordinates on a display screen according to the center symmetry of an optical imaging system, and determining to-be-stored pixel coordinates; forming coordinate correction data based on distortion correction data corresponding to each to-be-stored pixel coordinate, and saving the coordinate correction data into a two-dimensional array, wherein the size of the two-dimensional array is determined according to half of the width and height of the display screen, thereby solving the problem of high consumption of calculation resources and storage resources in the image distortion correction process; the pixel point coordinates on the display screen are divided according to the center symmetry of the optical imaging system, to-be-stored pixel coordinates are obtained, and the to-be-stored pixel coordinates are saved into a two-dimensional array, so that all the pixel point coordinates in the display screen do not need to be stored, and the consumption of storage resources is reduced; the distortion correction data of the pixel point coordinates are prestored, the consumption of calculation resources is reduced, and the calculation speed of the image distortion correction is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image distortion correction data storage method and an image distortion correction method. BACKGROUND

[0002] After an image is processed by an optical imaging system, distortion will be generated. In order to eliminate the distortion, the image is usually corrected for distortion before being displayed. In the prior art, real-time calculation is usually used to correct the distortion, or a pre-calculated UV mapping table is used to accelerate the calculation of the distortion correction. However, the real-time calculation method consumes a large amount of computing resources, and the use of the pre-calculated UV mapping table consumes a large amount of storage resources. SUMMARY

[0003] The present application provides an image distortion correction data storage method and an image distortion correction method, which reduces the consumption of computing resources and storage resources in the image distortion correction process.

[0004] According to an aspect of the present application, an image distortion correction data storage method is provided, comprising:

[0005] The pixel coordinates on the display screen are divided according to the center symmetry of the optical imaging system, and the pixel coordinates to be stored are determined;

[0006] The distortion correction data corresponding to each of the pixel coordinates to be stored is formed into coordinate correction data, and the coordinate correction data is saved in a two-dimensional array, wherein the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0007] According to another aspect of the present application, an image distortion correction method is provided, comprising:

[0008] The first pixel coordinate to be corrected is determined based on the pixel coordinates in the display screen;

[0009] If the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture, the distortion correction data corresponding to the first pixel coordinate is determined according to the pre-stored coordinate correction data;

[0010] If the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of the correction data texture is determined according to the center symmetry of the optical imaging system, and the distortion correction data corresponding to the second pixel coordinate is determined according to the pre-stored coordinate correction data;

[0011] The first pixel coordinate is corrected for image distortion based on the distortion correction data.

[0012] The coordinate correction data is stored according to the image distortion correction data storage method in any of the embodiments of the present application.

[0013] According to another aspect of the present application, an image distortion correction data storage apparatus is provided, comprising:

[0014] A to-be-stored coordinate determination module is configured to divide pixel point coordinates on a display screen according to the center symmetry of an optical imaging system, and determine to-be-stored pixel coordinates.

[0015] A coordinate correction data saving module is configured to form coordinate correction data based on distortion correction data corresponding to each of the to-be-stored pixel coordinates, and save the coordinate correction data into a two-dimensional array, wherein the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0016] According to another aspect of the present application, an image distortion correction apparatus is provided, comprising:

[0017] A first pixel coordinate determination module is configured to determine a first pixel coordinate to be corrected based on pixel coordinates in a display screen.

[0018] A first distortion correction data determination module is configured to, if the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture, determine distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data.

[0019] A second distortion correction data determination module is configured to, if the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, determine a second pixel coordinate within the pixel coordinate range of the data area of the correction data texture corresponding to the first pixel coordinate according to the center symmetry of an optical imaging system, and determine distortion correction data corresponding to the second pixel coordinate according to pre-stored coordinate correction data.

[0020] A distortion correction determination module is configured to perform image distortion correction on the first pixel coordinate based on the distortion correction data.

[0021] The coordinate correction data is stored according to the image distortion correction data storage method in any of the embodiments of the present application.

[0022] According to another aspect of the present application, an electronic device is provided, comprising:

[0023] At least one processor, and a memory connected in communication with the at least one processor;

[0024] The memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the image distortion correction data storage method or the image distortion correction method according to any of the embodiments of the present application.

[0025] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the image distortion correction data storage method or the image distortion correction method according to any of the embodiments of the present application when executed by the processor.

[0026] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the image distortion correction data storage method or the image distortion correction method according to any of the embodiments of the present application when executed by a processor.

[0027] The technical solution of the embodiments of the present application determines the pixel coordinates to be stored by dividing the pixel coordinates on the display screen according to the center symmetry of the optical imaging system, forms coordinate correction data based on the distortion correction data corresponding to each of the pixel coordinates to be stored, and saves the coordinate correction data into a two-dimensional array, the size of the two-dimensional array being determined according to half of the width and height of the display screen, thereby solving the problem of high consumption of computing resources and storage resources in the image distortion correction process. By dividing the pixel coordinates on the display screen according to the center symmetry of the optical imaging system, the pixel coordinates to be stored are obtained, coordinate correction data is formed based on the distortion correction data corresponding to each of the pixel coordinates to be stored, and the coordinate correction data is saved into a two-dimensional array, the size of the two-dimensional array being determined according to half of the width and height of the display screen. Without storing all the pixel coordinates on the display screen, the consumption of storage resources is reduced. The distortion correction data of the pixel coordinates are pre-stored, the consumption of computing resources is reduced, and the calculation speed of image distortion correction is improved.

[0028] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1is a flow chart of an image distortion correction data storage method according to Embodiment One of the present application;

[0031] Figure 2 is an example diagram of a virtual image forming process according to Embodiment One of the present application;

[0032] Figure 3 is an example diagram of a display screen division according to Embodiment One of the present application;

[0033] Figure 4 is an example diagram of another display screen division according to Embodiment One of the present application;

[0034] Figure 5 is an example diagram of another display screen division according to Embodiment One of the present application;

[0035] Figure 6 is an example diagram of a coordinate correction image according to Embodiment One of the present application;

[0036] Figure 7 is a flow chart of an image distortion correction method according to Embodiment Two of the present application;

[0037] Figure 8 is a flow chart of an image distortion correction method according to Embodiment Three of the present application;

[0038] Figure 9 is an example diagram of an image display according to Embodiment Three of the present application;

[0039] Figure 10 is an example diagram of another image display according to Embodiment Three of the present application;

[0040] Figure 11 is a structural schematic diagram of an image distortion correction data storage device according to Embodiment Four of the present application;

[0041] Figure 12 is a structural schematic diagram of an image distortion correction device according to Embodiment Five of the present application;

[0042] Figure 13 is a structural schematic diagram of an electronic device according to Embodiment Six of the present application. DETAILED DESCRIPTION

[0043] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of the present application.

[0044] It should be noted that the terms "first", "second" and the like in the description and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units need not be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0045] Embodiment one

[0046] Figure 1 A flow chart of an image distortion correction data storage method provided by the first embodiment of the present application, the present embodiment can be applicable to the case of storing image distortion correction data, and the method can be executed by an image distortion correction data storage device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0047] S101, according to the center symmetry of the optical imaging system, the pixel point coordinates on the display screen are divided, and the pixel coordinates to be stored are determined.

[0048] In the present embodiment, the pixel coordinates to be stored can be understood as the pixel coordinates with storage requirements, and the number of the pixel coordinates to be stored is less than the total number of the pixel point coordinates in the display screen.

[0049] The optical imaging system has center symmetry, and the display screen is divided into a plurality of symmetrical regions based on the center symmetry of the optical imaging system, each region has corresponding pixel point coordinates in the region, so as to realize the division of the pixel point coordinates. A region is selected from the divided regions, and the pixel point coordinates in the selected region are taken as the pixel coordinates to be stored.

[0050] ​S102, form coordinate correction data based on the distortion correction data corresponding to each to-be-stored pixel coordinate and save into a two-dimensional array, the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0051] In the embodiment, the distortion correction data can be understood as data for image distortion correction. The original image will be distorted when forming a virtual image through an optical imaging system and displaying on a display screen, affecting the user's visual experience. In order to avoid image distortion, the original image is corrected before being processed by the optical imaging system. The distortion correction data in the embodiment is used to correct the original image. The image obtained after distortion correction will not be distorted when projected through the optical imaging system. The coordinate correction data can be understood as data for correcting the pixel coordinates.

[0052] Each to-be-stored pixel coordinate corresponds to distortion correction data. Each to-be-stored pixel coordinate and its corresponding distortion correction data form a corresponding data group. Based on this, coordinate correction data is formed, that is, the coordinate correction data includes the distortion correction data corresponding to each to-be-stored pixel coordinate. The coordinate correction data is saved in a two-dimensional array according to the corresponding position, that is, the position of the distortion correction data in the two-dimensional array is determined according to the to-be-stored pixel coordinate, and the distortion correction data is stored as an element in the two-dimensional array. The size of the two-dimensional array is determined according to half of the width W and height H of the display screen. For example, the size of the two-dimensional array is (W / 2) x (H / 2), that is, the number of rows and columns of the two-dimensional array is (W / 2) and (H / 2) respectively.

[0053] The mapping transformation relationship of the correction model is generally in the form of formula 1 below, which is determined by the design of the specific optical imaging system. Wherein x and y are the original sampling coordinates of the display screen, and x' and y' are the corrected sampling coordinates, both of which take the center of the image as the coordinate origin.

[0054] (x', y') = F xy (x,y) Formula 1;

[0055] For each sampling coordinate of the display screen, that is, the pixel coordinate, the above mapping transformation is performed to obtain the corrected sampling coordinate. Then the original image is sampled to render a corrected image on the display screen. Finally, through the optical imaging system, the user can see the image without distortion. For example, Figure 2 An example diagram of a virtual image forming process is provided.

[0056] For a center-symmetric optical imaging system, formula 1 can be further rewritten as formula 2, wherein W is the width of the display screen, H is the height of the display screen, and note that the pixel coordinates take the left corner of the display screen or image as the origin, and the radial r is relative to the center of the display screen or image The radial distance.

[0057] Formula 2;

[0058] R(r) represents the distortion correction data. R(r) can be considered as the ratio of r1 to r2, where r1 is the distance between the sampled coordinates before correction and the center of the display screen (or the image center), and r2 is the distance between the sampled coordinates after correction and the center of the display screen (or the image center). Therefore, by pre-calculating and storing the R(r) value for each pixel of the display screen as correction data using a correction data generation program, and then reading this set of values ​​during the distortion correction program to calculate the correction sampling coordinates and sample the original image, a corrected image can be rendered on the display screen. In this embodiment, the central symmetry of the optical imaging system is utilized to optimize the calculation process and storage space when calculating and storing the R(r) data.

[0059] This application provides an image distortion correction data storage method that solves the problem of high computational and storage resource consumption during image distortion correction. By utilizing the central symmetry of the optical imaging system, the pixel coordinates on the display screen are divided to obtain the pixel coordinates to be stored. Based on the distortion correction data corresponding to each pixel coordinate, coordinate correction data is formed and saved in a two-dimensional array. The size of the two-dimensional array is determined by half the width and height of the display screen, eliminating the need to store the coordinates of all pixels on the display screen and reducing storage resource consumption. Pre-storing the pixel coordinate distortion correction data further reduces computational resource consumption and improves the computational speed of image distortion correction.

[0060] Optionally, the pixel coordinates on the display screen are divided according to the central symmetry of the optical imaging system to determine the coordinates of the pixels to be stored, including steps A1 and A2:

[0061] A1. If the width and height of the display screen are equal, the display screen is divided into 8 regions according to the central symmetry of the optical imaging system. The coordinates of the first candidate pixel in one of the regions are determined as the coordinates of the pixel to be stored. The coordinates of the first candidate pixel are taken with the upper left corner of the display screen as the origin.

[0062] In the embodiment, the first candidate pixel point coordinate can be understood as a pixel point coordinate in a region of the display screen. If the width and the height of the display screen are equal, that is, the display screen is square, the display screen is divided into 8 regions according to the center symmetry of the optical imaging system, and the 8 regions are symmetric regions, which can be center symmetric, axis symmetric, etc. For example, the display screen is divided into four equilateral triangles by connecting the opposite corners of the display screen, and then each equilateral triangle is divided into two symmetric triangles along the height line, so that a total of 8 triangles are obtained, each of which is a region, and a total of 8 regions are obtained. One of the regions is selected, the first candidate pixel point coordinate in the region is determined as the to-be-stored pixel coordinate, and the first candidate pixel point coordinate takes the upper left corner of the display screen as the coordinate origin.

[0063] For example, Figure 3 An example diagram of display screen division is provided, as shown in the figure, the coordinates of the No. 1 pixel point are (x1, y1), and the coordinates of the No. 2 to No. 8 pixel points having a symmetric relationship with the No. 1 pixel point are (y1, x1), (-y1+W-1, x1), (-x1+W-1, y1), (-x1+W-1, -y1+W-1), (-y1+W-1, -x1+W-1), (y1, -x1+W-1), (x1, -y1+W-1). The No. 1 to No. 8 pixel points have the same radial length r relative to the center of the image, that is, they have the same R(r) value, so as long as the R(r) value data of the region where the No. 1 pixel point is located, that is, the R(r) value data satisfying the following condition, is calculated and stored.

[0064] Formula 3;

[0065] When the R(r) value data is saved by a two-dimensional array in the embodiment, only about half of the space of the two-dimensional array is used to store the R(r) value data, and the remaining space is kept as 0 value. In the subsequent compression of the coordinate correction data, the blank area of 0 value will be greatly compressed, and almost no additional storage space will be occupied.

[0066] A2, if the width and the height of the display screen are not equal, the display screen is expanded into a square based on the maximum value of the width and the height, the region division is performed based on the diagonal line of the expanded square, two regions are obtained, the region containing more pixel point coordinates of the display screen is taken as a to-be-stored region, and the second candidate pixel point coordinate in the to-be-stored region is determined as the to-be-stored pixel coordinate, and the second candidate pixel point coordinate takes the upper left corner of the display screen as the coordinate origin.

[0067] In the embodiment, the to-be-stored region can be understood as a region that needs to store pixel coordinates, and the second candidate pixel point coordinate can be understood as a pixel point coordinate in a region of the display screen.

[0068] If the width and height of the display screen are not equal, that is, the display screen is rectangular, the width and height are compared, and the maximum of the width and height is taken as the length of the positive direction, and the display screen is expanded to a square. For example, when the width is greater than the height, the expansion is performed upward along the height direction, and the length of the positive direction obtained is the width; when the width is less than the height, the expansion is performed leftward along the width direction, and the length of the positive direction obtained is the height. The diagonal of the expanded square is used for region division, the positive direction is divided into two regions, the number of pixel point coordinates contained in the two regions is compared, and the region containing more pixel point coordinates of the display screen is taken as the to-be-stored region. The second candidate pixel point coordinate in the to-be-stored region is determined as the to-be-stored pixel coordinate, and the second candidate pixel point coordinate takes the upper left corner of the display screen as the coordinate origin.

[0069] For example, Figure 4 Another example of display screen division is provided, taking the width W of the display screen greater than or equal to the height H as an example. The display screen is expanded to a square, then the square is divided along the diagonal, the square is divided into two regions, the region containing more pixel point coordinates is selected as the to-be-stored region, and the pixel point coordinates in the to-be-stored region are stored as the to-be-stored pixel coordinates; Figure 4 The data area shown in the figure is the to-be-stored region, and the pixel point coordinates in the blank area are not stored. Figure 5 Another example of display screen division is provided, taking the width W of the display screen less than the height H as an example. The display screen is expanded to a square, then the square is divided along the diagonal, the square is divided into two regions, the region containing more pixel point coordinates is selected as the to-be-stored region, and the pixel point coordinates in the to-be-stored region are stored as the to-be-stored pixel coordinates; Figure 5 The data area shown in the figure is the to-be-stored region, and the pixel point coordinates in the blank area are not stored. It can be seen that in the calculation and storage of the R(r) value data, the larger value of W and H can be taken as the reference width to expand the total region to a square in advance, and only the part corresponding to the actual W / 2 width and H / 2 height is taken in the subsequent actual calculation.

[0070] Optionally, when the width of the display screen is greater than or equal to the height, the to-be-stored pixel coordinates satisfy the following conditions:

[0071] Formula 4;

[0072] Wherein, x is the horizontal coordinate of the to-be-stored pixel coordinate, y is the vertical coordinate of the to-be-stored pixel coordinate, W is the width of the display screen, and H is the height of the display screen.

[0073] That is, when W≥H, taking the upper left corner of the display screen or the image as the coordinate origin, the R(r) value data satisfying the above condition is calculated and stored.

[0074] Optionally, when the width of the display screen is less than the height, the pixel coordinates to be stored satisfy the following condition:

[0075] Formula 5;

[0076] Wherein, x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

[0077] That is, when W<H, taking the upper left corner of the display screen or the image as the coordinate origin, the R(r) value data satisfying the above condition is calculated and stored.

[0078] It should be noted that the first candidate pixel point coordinate and the second candidate pixel point coordinate take the upper left corner of the display screen or the image as the coordinate origin. The upper left corner of the display screen and the image is the same point.

[0079] Optionally, the method further comprises: generating a coordinate correction image based on the two-dimensional array; wherein each pixel point in the coordinate correction image corresponds to the storage of distortion correction data, the distortion correction data is a four-byte floating point type, and each byte corresponds to the value of each pixel color channel.

[0080] In this embodiment, the coordinate correction image can be understood as an image for storing distortion correction data. An image of a corresponding size is generated according to the size of the two-dimensional array, each position in the two-dimensional array corresponds to a pixel point of the image, the distortion correction data in the corresponding position in the two-dimensional array is converted into the pixel value of the image, forming a coordinate correction image; each pixel point in the coordinate correction image corresponds to the storage of distortion correction data, the distortion correction data is a four-byte floating point type, and each byte corresponds to the value of each pixel color channel.

[0081] Each R(r) value is a four-byte floating point type, and the four bytes of data correspond to the values of the four channels RGBA of each pixel color in general picture data from low to high, so that an image storing complete R(r) value data is obtained, which is denoted as a coordinate correction image. Figure 6 An example of a coordinate correction image is provided, Figure 6 Taking the example where the width and height of the display screen are equal, the data area storing valid data only occupies about half of the space of the picture, and the remaining is a blank area maintaining 0 value data. The pixel coordinates to be stored and the corresponding distortion correction data are stored through the data area.

[0082] Optionally, after the coordinate-corrected image is generated, the coordinate-corrected image is compressed and saved. For example, the coordinate-corrected image is saved in a picture format compressed using a lossless compression algorithm, such as a PNG format, so that the storage space required for the R(r) value data can be minimized.

[0083] If no optimization method is used, about 29.5 MB of storage space is required to store the complete corrected sampling coordinate (x', y') data of a display screen with a resolution of 1920x1920. After the optimization method provided in the present application is used, only about 1 MB of storage space is required to achieve the same effect.

[0084] Moreover, for a scene that requires rendering of left and right 3D images, the method of directly storing the corrected sampling coordinate (x', y') data requires additional calculation and storage of the corrected data file in this scene, but the method provided in the present application does not require additional storage space, and only needs to add a calculation process supporting this scene in the subsequent distortion correction program.

[0085] The embodiment of the present application provides an image distortion correction data storage method, solves the problem of high consumption of computing resources and storage resources in the image distortion correction process, divides the display screen in different ways according to the width and height of the display screen, divides the display screen into 8 regions or 2 regions, then selects the pixel coordinates to be stored according to the divided regions, stores the pixel coordinates to be stored and the corresponding distortion correction data into a two-dimensional array, the size of the two-dimensional array is determined according to half of the width and height of the display screen, and all pixel point coordinates in the display screen do not need to be stored, thereby reducing the consumption of storage resources; the distortion correction data of the pixel point coordinates is pre-stored, thereby reducing the consumption of computing resources and improving the calculation speed of the image distortion correction. Moreover, the two-dimensional array can be converted into a coordinate-corrected image, the distortion correction data is stored in the form of an image, data storage is facilitated, and storage space can be further saved. The image distortion correction data storage method provided in the embodiment of the present application uses less storage space, can more conveniently store correction data of different requirements on one device, such as data corresponding to different diopters or different correction degrees, and can flexibly switch according to requirements during running.

[0086] Embodiment Two

[0087] Figure 7 A flowchart of an image distortion correction method provided in the embodiment two of the present application, the embodiment can be applicable to the case of correcting the distortion of an image, the method can be executed by an image distortion correction device, the image distortion correction device can be realized in the form of hardware and / or software, and the image distortion correction device can be configured in an electronic device. As shown in the figure, the method comprises the following steps. Figure 7 ​

[0088] S201, determine a first pixel coordinate to be corrected based on a pixel coordinate in the display screen.

[0089] In the embodiment, the first pixel coordinate can be understood as a pixel coordinate that needs to be corrected. The pixel coordinate in the display screen can be corrected by the method provided in the embodiment. By analyzing the pixel coordinate in the display screen, the coordinate in the original projection image to which the pixel coordinate in the display screen is mapped is determined, and the coordinate is taken as the first pixel coordinate to be corrected. For example, when the size of the display screen and the original projection image is the same, the coordinate in the original projection image to which the pixel coordinate in the display screen is mapped is the same as the coordinate of the pixel coordinate in the display screen, that is, the pixel coordinate in the display screen is directly taken as the first pixel coordinate to be corrected; when the size of the display screen and the original projection image is different, the pixel coordinate in the display screen needs to be scaled and then mapped to the original projection image. In this case, the coordinate in the original projection image to which the pixel coordinate in the display screen is mapped is different from the coordinate of the pixel coordinate in the display screen, and the pixel coordinate in the display screen needs to be scaled to obtain the corresponding first pixel coordinate.

[0090] S202, if the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture, determine the distortion correction data corresponding to the first pixel coordinate according to the pre-stored coordinate correction data.

[0091] In the embodiment, the data area of the correction data texture can be understood as a data area in which the coordinate correction data is stored; the pixel coordinate range can be understood as the range of the horizontal and vertical coordinates of the pixel coordinate. For example, the coordinate correction data is stored in a two-dimensional array, and the part of the two-dimensional array in which the coordinate correction data is stored can be called a data area, and the part of the two-dimensional array in which the coordinate correction data is not stored can be called a blank area. When storing the coordinate correction data, the pixel coordinate range can be determined according to the coordinates corresponding to the actually stored coordinate correction data. The coordinate correction data in the embodiment is stored according to the image distortion correction data storage method of any embodiment of the application, and therefore, the pixel coordinate range can be determined according to the pixel coordinate to be stored.

[0092] It is determined whether the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture. If yes, the pre-stored coordinate correction data is queried, the distortion correction data corresponding to the pixel coordinate to be stored matched with the first pixel coordinate is determined, and the distortion correction data is taken as the distortion correction data corresponding to the first pixel coordinate.

[0093] S203, if the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, determining a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of the correction data texture according to the central symmetry of the optical imaging system, and determining the distortion correction data corresponding to the second pixel coordinate according to the pre-stored coordinate correction data.

[0094] In the embodiment, the second pixel coordinate can be understood as a pixel coordinate equivalent to the first pixel coordinate obtained through coordinate transformation. If the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, coordinate transformation needs to be performed on the first pixel coordinate. The first pixel coordinate is transformed into the pixel coordinate range of the data area of the correction data texture according to the central symmetry of the optical imaging system, to obtain the second pixel coordinate corresponding to the first pixel coordinate, and the second pixel coordinate is within the pixel coordinate range of the data area of the correction data texture. The coordinate transformation of the first pixel coordinate according to the central symmetry of the optical imaging system can be based on the central symmetry of the optical imaging system to determine the formula or logic of coordinate transformation, and the first pixel coordinate is substituted into the formula or logic for coordinate transformation. The pre-stored coordinate correction data is queried to determine the distortion correction data corresponding to the to-be-stored pixel coordinate matched with the second pixel coordinate, and the distortion correction data is taken as the distortion correction data corresponding to the second pixel coordinate. The distortion correction data corresponding to the second pixel coordinate is the distortion correction data corresponding to the first pixel coordinate.

[0095] The coordinate correction data is stored according to the image distortion correction data storage method of any one of the embodiments of the application.

[0096] S204, performing image distortion correction on the first pixel coordinate based on the distortion correction data.

[0097] According to the distortion correction data, the first pixel coordinate is corrected for distortion to determine the pixel coordinate corresponding to the first pixel coordinate in the original image.

[0098] The embodiment of the present application provides a kind of image distortion correction method, solve the problem of high consumption of computing resources and storage resources in image distortion correction process, determine the first pixel coordinate to be corrected according to the pixel coordinate in display screen, pre-store coordinate correction data, determine the distortion correction data corresponding to the first pixel coordinate by coordinate correction data, to realize the image distortion correction of first pixel coordinate, without real-time calculation of the distortion correction data corresponding to each pixel coordinate, save computing resources, and the coordinate correction data in the embodiment of the present application is stored by the image distortion correction data storage method provided by any embodiment of the present application, can save storage resources, reduce storage resource consumption.Judge whether the first pixel coordinate is in the pixel coordinate range of the data area of correction data texture, if in, determine the distortion correction data corresponding to the first pixel coordinate by directly inquiring coordinate correction data, if not in, then according to the center symmetry of optical imaging system, the coordinate of first pixel coordinate is transformed, to determine its corresponding second pixel coordinate, then according to the second pixel coordinate, the corresponding distortion correction data of coordinate correction data is inquired, by the above-mentioned mode, it can be guaranteed that the distortion correction data corresponding to each pixel coordinate in display screen can be determined in the case that coordinate correction data only saves part of pixel coordinate, to realize the distortion correction of image quickly.

[0099] Embodiment three

[0100] Figure 8 A flow chart of the image distortion correction method provided by the third embodiment of the present application is provided, and the embodiment is refined on the basis of the above-mentioned embodiment. As shown in the figure, Figure 8 The method comprises:

[0101] S301, determine the first pixel coordinate to be corrected based on the pixel coordinate in display screen.

[0102] Optionally, the first pixel coordinate to be corrected is determined based on the pixel coordinate in display screen, comprising B1-B2:

[0103] B1, when the display screen displays two-dimensional image, the pixel coordinate in display screen is taken as the first pixel coordinate to be corrected.

[0104] When the display screen displays two-dimensional image, the size of display screen is same as the size of original image to be displayed, so the pixel coordinate in display screen can be taken as the first pixel coordinate to be corrected.

[0105] B2, when the display screen displays three-dimensional image, the 2 times of horizontal coordinate in pixel coordinate in display screen is taken as the horizontal coordinate of first pixel coordinate to be corrected, and the vertical coordinate in pixel coordinate in display screen is taken as the vertical coordinate of first pixel coordinate to be corrected.

[0106] When the display screen displays a three-dimensional image, the size of the display screen is different from the size of the original image to be displayed, and therefore the pixel coordinates in the display screen need to be scaled. When left and right 3D images need to be rendered, the distortion correction program needs to process the images in the pixel shader program, and scale the two images for the left and right eyes to half of the original size along the horizontal direction and combine them into a left and right 3D image. Therefore, when image distortion correction is performed, if the display screen displays a three-dimensional image, the horizontal coordinate of the pixel coordinate is multiplied by two to restore the pixel coordinate before scaling, that is, the horizontal coordinate of the pixel coordinate in the display screen is taken as twice the horizontal coordinate of the first pixel coordinate to be corrected, and the vertical coordinate of the pixel coordinate in the display screen is taken as the vertical coordinate of the first pixel coordinate to be corrected.

[0107] S302, if the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture, determining the distortion correction data corresponding to the first pixel coordinate according to the pre-stored coordinate correction data.

[0108] S303, if the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, determining a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of the correction data texture according to the center symmetry of the optical imaging system, and determining the distortion correction data corresponding to the second pixel coordinate according to the pre-stored coordinate correction data.

[0109] When the display screen displays a two-dimensional image, steps S304-S306 are performed for image distortion correction; when the display screen displays a three-dimensional image, steps S307-S309 are performed for image distortion correction.

[0110] S304, when the display screen displays a two-dimensional image, determining the texture coordinate corresponding to the first pixel coordinate.

[0111] When the display screen displays a two-dimensional image, the first pixel coordinate is converted into a texture coordinate according to the conversion relationship between the pixel coordinate and the texture coordinate, and the texture coordinate corresponding to the first pixel coordinate is determined, and the texture coordinate is the UV coordinate.

[0112] S305, multiplying the difference between the texture coordinate and the first center coordinate by the distortion correction data to obtain the first correction coordinate, and the first center coordinate is determined according to the center of the display screen.

[0113] In this embodiment, the first correction coordinate can be understood as the coordinate after correction by the distortion correction data; the first center coordinate can be understood as the center of the display screen or the image, and the first center coordinate is determined according to the center of the display screen, for example, taking the center of the display screen as the first center coordinate. The difference between the texture coordinate and the first center coordinate is calculated, and the difference is multiplied by the distortion correction data to obtain the first correction coordinate.

[0114] S306, taking the sum of the first corrected coordinate and the first center coordinate as the corrected sampling coordinate.

[0115] The first corrected coordinate is added to the first center coordinate to obtain a sum as the corrected sampling coordinate; the corrected sampling coordinate is a coordinate after final distortion correction, and pixel information can be obtained from a pixel point at a corresponding position in the original image according to the sampling coordinate and a virtual image is formed through the optical imaging system.

[0116] Exemplarily, the embodiment of the present application provides a calculation formula of the corrected sampling coordinate:

[0117] anti_uv = (screen_uv – (0.5, 0.5)) × R + (0.5, 0.5);

[0118] Wherein, anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e. the UV coordinate of the display screen); (0.5, 0.5) is the first center coordinate, and R is the distortion correction data.

[0119] S307, when the display screen displays the three-dimensional image, determining the texture coordinate corresponding to the first pixel coordinate.

[0120] When the display screen displays the three-dimensional image, the first pixel coordinate is converted into the texture coordinate according to the conversion relationship between the pixel coordinate and the texture coordinate, the texture coordinate corresponding to the first pixel coordinate is determined, and the texture coordinate is the UV coordinate.

[0121] S308, if the texture coordinate is greater than or equal to a set value, multiplying the difference between the texture coordinate and the second center coordinate by the distortion correction data to obtain a second corrected coordinate, and taking the sum of the second corrected coordinate and the second center coordinate as the corrected sampling coordinate.

[0122] In the embodiment, the second center coordinate can be understood as the center of the display screen or the image. For the three-dimensional image, it is actually an image composed of a left image and a right image, and therefore the centers of the left half image and the right half image are different. The second center coordinate in the embodiment of the present application is the center point coordinate of the right half image, and the second center coordinate is determined according to the 3 / 4 position in the horizontal direction of the display screen. The second corrected coordinate can be understood as a coordinate after correction by the distortion correction data.

[0123] A value is set as a setting value, and the setting value is used to distinguish whether the coordinate point is in the left half image or the right half image. If the texture coordinate is greater than or equal to the setting value, it can be considered that the texture coordinate is in the right half image, the difference between the texture coordinate and the second center coordinate is calculated, the difference is multiplied by the distortion correction data to obtain a second corrected coordinate, and the sum of the second corrected coordinate and the second center coordinate is calculated as the corrected sampling coordinate.

[0124] Exemplarily, the embodiment of the present application provides another formula for calculating the corrected sampling coordinate, the setting value is 0.5, and when screen_uv ≥ 0.5, the corrected sampling coordinate is calculated by the following formula.

[0125] anti_uv = (screen_uv – (0.75, 0.5)) × R + (0.75, 0.5);

[0126] Wherein, anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e. the UV coordinate of the display screen); (0.75, 0.5) is the second center coordinate, and R is the distortion correction data.

[0127] S309, if the texture coordinate is less than the setting value, the difference between the texture coordinate and the third center coordinate is multiplied by the distortion correction data to obtain a third corrected coordinate, and the sum of the third corrected coordinate and the third center coordinate is calculated as the corrected sampling coordinate.

[0128] In the embodiment, the third center coordinate can be understood as the center of the display screen or the image, and the third center coordinate is the center point coordinate of the left half image. The third center coordinate is determined according to the 1 / 4 position in the horizontal direction of the display screen. The third corrected coordinate can be understood as the coordinate after correction by the distortion correction data.

[0129] If the texture coordinate is less than the setting value, it can be considered that the texture coordinate is in the left half image, the difference between the texture coordinate and the third center coordinate is calculated, the difference is multiplied by the distortion correction data to obtain a third corrected coordinate, and the sum of the third corrected coordinate and the third center coordinate is calculated as the corrected sampling coordinate.

[0130] Exemplarily, the embodiment of the present application provides another formula for calculating the corrected sampling coordinate, the setting value is 0.5, and when screen_uv <0.5, the corrected sampling coordinate is calculated by the following formula.

[0131] anti_uv = (screen_uv – (0.25, 0.5)) × R + (0.25, 0.5);

[0132] wherein, anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e. the UV coordinate of the display screen); (0.25, 0.5) is the third center coordinate, and R is the distortion correction data.

[0133] The corrected sampling coordinate calculated in the embodiment of the application is a normalized sampling coordinate.

[0134] Optionally, the second pixel coordinate corresponding to the first pixel coordinate in the pixel coordinate range of the data area of the correction data texture is determined according to the center symmetry of the optical imaging system, including C1-C6:

[0135] C1, determining the initial horizontal coordinate and the initial vertical coordinate according to the first pixel coordinate and the width and height of the display screen.

[0136] In the embodiment, the initial horizontal coordinate and the initial vertical coordinate can be understood as initial coordinates for coordinate conversion. Based on a set formula or judgment mode, the first pixel coordinate is processed for coordinate conversion according to the width and height of the display screen, to obtain the initial horizontal coordinate and the initial vertical coordinate. For example, judging whether the first pixel coordinate is on the left or right side of the center line, and whether the first pixel coordinate is above or below the center line, the first pixel coordinate is mirrored to the upper left corner to obtain the initial horizontal coordinate and the initial vertical coordinate.

[0137] C2, calculating the difference between the width and the height of the display screen, and recording half of the difference as a first parameter.

[0138] In the embodiment, the first parameter can be understood as a parameter in the coordinate conversion process, which facilitates subsequent coordinate conversion. The difference between the width and the height of the display screen is recorded as the first parameter.

[0139] C3, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than the sum of the initial vertical coordinate and the first parameter, the sum of the initial vertical coordinate and the first parameter is taken as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is taken as the vertical coordinate of the second pixel coordinate.

[0140] The sum of the initial vertical coordinate and the first parameter is calculated as sum1, and if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than sum1, the sum of the initial vertical coordinate and the first parameter is taken as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is taken as the vertical coordinate of the second pixel coordinate.

[0141] C4, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, the initial horizontal coordinate is taken as the horizontal coordinate of the second pixel coordinate and the initial vertical coordinate is taken as the vertical coordinate of the second pixel coordinate.

[0142] if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to sum1 (sum1 is the sum of the initial vertical coordinate and the first parameter), the initial horizontal coordinate is taken as the horizontal coordinate of the second pixel coordinate and the initial vertical coordinate is taken as the vertical coordinate of the second pixel coordinate.

[0143] C5, if the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, the sum of the initial vertical coordinate and the first parameter is taken as the horizontal coordinate of the second pixel coordinate and the difference between the initial horizontal coordinate and the first parameter is taken as the vertical coordinate of the second pixel coordinate.

[0144] The difference d1 between the initial horizontal coordinate and the first parameter is calculated, if the first parameter is less than 0 and d1 is less than the initial vertical coordinate, the sum of the initial vertical coordinate and the first parameter is taken as the horizontal coordinate of the second pixel coordinate and the difference d1 between the initial horizontal coordinate and the first parameter is taken as the vertical coordinate of the second pixel coordinate.

[0145] C6, if the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is taken as the horizontal coordinate of the second pixel coordinate and the initial vertical coordinate is taken as the vertical coordinate of the second pixel coordinate.

[0146] if the first parameter is less than 0 and d1 (the difference between the initial horizontal coordinate and the first parameter) is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is taken as the horizontal coordinate of the second pixel coordinate and the initial vertical coordinate is taken as the vertical coordinate of the second pixel coordinate.

[0147] Optionally, the initial horizontal coordinate and the initial vertical coordinate are determined according to the first pixel coordinate and the width and height of the display screen, including D1-D2:

[0148] D1, the to-be-mapped pixel coordinate is determined according to the first pixel coordinate.

[0149] In the embodiment, the to-be-mapped pixel coordinate can be understood as the coordinate obtained by mapping the first pixel coordinate. When displaying an image of different dimensions, the first pixel coordinate is mapped based on the dimension of the image to obtain the to-be-mapped pixel coordinate. For example, when displaying a three-dimensional image, the first pixel coordinate is mapped to a side of the image to obtain the to-be-mapped pixel coordinate.

[0150] D2, the initial horizontal coordinate and the initial vertical coordinate are determined according to the to-be-mapped pixel coordinate and the width and height of the display screen.

[0151] The to-be-mapped pixel coordinate is subjected to coordinate conversion processing according to the width and height of the display screen, and initial horizontal coordinates and initial vertical coordinates are obtained. For example, it is determined whether the to-be-mapped pixel coordinate is on the left side or the right side of the middle line and whether the to-be-mapped pixel coordinate is above or below the middle line, the to-be-mapped pixel coordinate is subjected to mirror processing, and the to-be-mapped pixel coordinate is mirrored to the upper left corner to obtain the initial horizontal coordinates and the initial vertical coordinates.

[0152] Optionally, the to-be-mapped pixel coordinate is determined according to the first pixel coordinate, including E1-E2:

[0153] E1, when the display screen displays a two-dimensional image, the first pixel coordinate is taken as the to-be-mapped pixel coordinate.

[0154] When the display screen displays a two-dimensional image, the size of the two-dimensional image is the same as the size of the display screen, and scaling is not needed, so the first pixel coordinate can be directly taken as the to-be-mapped pixel coordinate.

[0155] E2, when the display screen displays a three-dimensional image, if the horizontal coordinate in the first pixel coordinate is greater than or equal to the width of the display screen, the horizontal coordinate in the first pixel coordinate is subtracted by the width of the display screen to obtain the horizontal coordinate of the to-be-mapped pixel coordinate, and the vertical coordinate in the first pixel coordinate is taken as the vertical coordinate of the to-be-mapped pixel coordinate; if the horizontal coordinate in the first pixel coordinate is less than the width of the display screen, the first pixel coordinate is taken as the to-be-mapped pixel coordinate.

[0156] When the display screen displays a three-dimensional image, it is necessary to determine the position of the first pixel coordinate in the screen and map it to one side. It is determined whether the horizontal coordinate in the first pixel coordinate is greater than or equal to the width of the display screen, if yes, the horizontal coordinate in the first pixel coordinate is subtracted by the width of the display screen to obtain the horizontal coordinate of the to-be-mapped pixel coordinate, and the vertical coordinate in the first pixel coordinate is taken as the vertical coordinate of the to-be-mapped pixel coordinate; if no, the first pixel coordinate is taken as the to-be-mapped pixel coordinate.

[0157] Optionally, the initial horizontal coordinates and the initial vertical coordinates are determined according to the to-be-mapped pixel coordinate and the width and height of the display screen, including F1-F2:

[0158] F1, if the ratio of 2 times of the horizontal coordinate X1 in the to-be-mapped pixel coordinate to the width W of the display screen is greater than or equal to 1, W-1-X1 is taken as the initial horizontal coordinate of the second pixel coordinate, otherwise, X1 is taken as the initial horizontal coordinate of the second pixel coordinate.

[0159] F2, if the ratio of 2 times of the vertical coordinate Y1 in the to-be-mapped pixel coordinate to the height H of the display screen is greater than or equal to 1, H-1-Y1 is taken as the initial vertical coordinate of the second pixel coordinate, otherwise, Y1 is taken as the initial vertical coordinate of the second pixel coordinate.

[0160] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate is determined according to the pre-stored coordinate correction data, including G1-G3:

[0161] G1, taking the first pixel coordinate or the second pixel coordinate as the to-be-corrected coordinate.

[0162] In the embodiment, the principle of determining the distortion correction data corresponding to the first pixel coordinate and the principle of determining the distortion correction data corresponding to the second pixel coordinate are the same. Therefore, when determining the distortion correction data, the first pixel coordinate or the second pixel coordinate can be taken as the to-be-corrected coordinate for subsequent processing to determine the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate.

[0163] G2, querying the pre-stored coordinate correction image based on the to-be-corrected coordinate to determine the values of each pixel color channel of the pixel coordinate corresponding to the to-be-corrected coordinate in the coordinate correction image.

[0164] When the coordinate correction data is a coordinate correction image, that is, the coordinate correction data is stored through the coordinate correction image, the coordinate correction image is queried based on the to-be-corrected coordinate to determine the pixel coordinate corresponding to the to-be-corrected coordinate in the coordinate correction image, and then the values of each pixel color channel of the pixel coordinate are determined. The pixel color channel is usually RGBA four channels.

[0165] G3, performing data recovery on the values of each pixel color channel to obtain the distortion correction data corresponding to the to-be-corrected coordinate.

[0166] The values of each pixel color channel are recombined from low to high and recovered into a 4-byte floating-point number to obtain the distortion correction data corresponding to the to-be-corrected coordinate, that is, the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate.

[0167] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the first pixel coordinate is determined according to the pre-stored coordinate correction data, including: querying the pre-stored coordinate correction image based on the first pixel coordinate to determine the values of each pixel color channel of the pixel coordinate corresponding to the first pixel coordinate in the coordinate correction image; and performing data recovery on the values of each pixel color channel to obtain the distortion correction data corresponding to the first pixel coordinate.

[0168] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the second pixel coordinate is determined according to the pre-stored coordinate correction data, including: querying the pre-stored coordinate correction image based on the second pixel coordinate to determine the values of each pixel color channel of the pixel coordinate corresponding to the second pixel coordinate in the coordinate correction image; and performing data recovery on the values of each pixel color channel to obtain the distortion correction data corresponding to the second pixel coordinate.

[0169] Exemplarily, the embodiment of the present application provides a way of distortion correction data recovery:

[0170] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24)).

[0171] Wherein, R is the distortion correction data, uintBitsToFloat is a floating point conversion function, data.r, data.g, data.b and data.a are the values of the four pixel color channels of RGBA. Through the above formula, the 4-byte data stored in the RGBA channel can be recombined from low to high and recovered into a 4-byte floating point number.

[0172] Exemplarily, the embodiment of the present application provides a flow of image distortion correction:

[0173] The distortion correction program reads the coordinate correction image storing the coordinate correction data when running. Exemplarily, the coordinate correction image can be a PNG image. The coordinate correction image is converted into a correction data texture and transmitted into the pixel shader program of the GPU. The pixel shader program reads and restores the R(r) value according to the special structure of the correction data, and then calculates the corrected UV coordinates according to the R(r) value. The UV coordinates are normalized sampling coordinates, which are used to sample the color value of the original image texture. This process can be divided into three steps.

[0174] First step: the special structure of the correction data texture is determined according to the center symmetry of the optical imaging system, and its width and height should be half of the width W and height H of the display screen. Therefore, the equivalent pixel coordinates (_x, _y) in the data area of the display screen are calculated when the pixel coordinates (x, y) of the display screen exceed the pixel coordinate range of the data area of the correction data texture. In the pixel shader program, (_x, _y) can be obtained by the logic calculation process described below.

[0175] Step 1: the display screen displays a two-dimensional image, and the pixel coordinates (x, y) of the display screen are directly used as the first pixel coordinates to be corrected, i.e. the first pixel coordinates are (x, y); the first pixel coordinates are directly used as the pixel coordinates to be mapped, i.e. the pixel coordinates to be mapped are (x, y), and the pixel coordinates to be mapped are marked as (X1, Y1), i.e. X1=x, Y1=y.

[0176] Step 2: determine whether 2X1 / W is greater than or equal to 1, if yes, the initial horizontal coordinate X2= W-1-X1, otherwise, the initial horizontal coordinate X2= X1.

[0177] Step 3: Determine if 2Y1 / H is greater than or equal to 1. If yes, the initial vertical coordinate Y2 = H-1-Y1. Otherwise, the initial vertical coordinate Y2 = Y1.

[0178] Step 4: Calculate the first parameter shift, shift = (W - H) / 2.

[0179] Step 5: If shift ≥ 0 and X2 > Y2 + shift, the second pixel coordinate (_x, _y) = (Y2 + shift, X2 - shift);

[0180] If shift ≥ 0 and X2 ≤ Y2 + shift, (_x, _y) = (X2, Y2);

[0181] If shift <0 and X2 - shift <Y2, (_x, _y) = (Y2 + shift, X2 - shift);

[0182] If shift <0 and X2 - shift ≥ Y2, (_x, _y) = (X2, Y2).

[0183] Second step: Read out the 4-byte data data at the pixel coordinate (_x, _y) of the corrected data texture, and get the R value according to the calculation process described below. It will store the 4-byte data of the RGBA channel in low to high order and restore it to a 4-byte floating-point number.

[0184] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24))

[0185] Third step: According to the logic of formula 2, the corrected UV coordinate can be calculated according to the calculation process described below, where screen_uv is the UV coordinate of the display screen, anti_uv is the corrected UV coordinate, and UV coordinate is the normalized sampling coordinate (i.e. the corrected sampling coordinate).

[0186] anti_uv = (screen_uv - (0.5, 0.5)) × R + (0.5, 0.5)

[0187] Using the corrected UV coordinate to sample the original image and render it on the display screen, finally through the optical imaging system, the image without distortion can be seen; for example, Figure 9An example diagram of image display is provided, an original image is obtained after distortion correction to obtain a corrected image, the corrected image is formed into an actual viewing image by an optical imaging system, and a user finally views the image without distortion on a display screen.

[0188] Exemplarily, the embodiment of the present application provides another flow of image distortion correction:

[0189] When left and right 3D images need to be rendered, the distortion correction program needs to use a slightly different process in the pixel shader program. For left and right 3D images that are scaled to half of the original along the horizontal direction in advance and combined together, the following process can be used to calculate the corrected UV coordinates.

[0190] First step: multiply x back to the pixel coordinate before scaling, and then calculate the equivalent pixel coordinate (_x, _y) in the data area of the correction data texture when the pixel coordinate (x, y) of the display screen exceeds the pixel coordinate range of the data area. In the pixel shader program, (_x, _y) can be obtained by the logical calculation process described below.

[0191] Step 1: the display screen displays a three-dimensional image, and the pixel coordinate (x, y) of the display screen is scaled to obtain a first pixel coordinate (2x, y) to be corrected; the first pixel coordinate is converted into a to-be-mapped pixel coordinate, if 2x is greater than or equal to W, the to-be-mapped pixel coordinate is (2x-W, y), if 2x is less than W, the to-be-mapped pixel coordinate is (2x, y), and the to-be-mapped pixel coordinate is marked as (X1, Y1). That is, when 2x is greater than or equal to W, X1=2x-W, Y1=y; when 2x is less than W, X1=2x, Y1=y.

[0192] Step 2: determine whether 2X1 / W is greater than or equal to 1, if yes, the initial horizontal coordinate X2= W-1-X1, otherwise, the initial horizontal coordinate X2= X1.

[0193] Step 3: determine whether 2Y1 / H is greater than or equal to 1, if yes, the initial vertical coordinate Y2= H-1-Y1, otherwise, the initial vertical coordinate Y2= Y1.

[0194] Step 4: calculate the first parameter shift, shift= (W – H) / 2.

[0195] Step 5: if shift ≥ 0 and X2> Y2+ shift, the second pixel coordinate (_x, _y) = (Y2+ shift, X2– shift);

[0196] If shift ≥ 0 and X2≤ Y2+ shift, (_x, _y) = (X2, Y2);

[0197] If shift <0 and X2 – shift<Y2, (_x, _y) = (Y2+ shift, X2– shift);

[0198] If shift <0 and X2 – shift≥ Y2, (_x, _y) = (X2, Y2).

[0199] Second step: read out 4 bytes of data data at the pixel coordinate (_x, _y) of the corrected data texture, and get the R value according to the calculation process described below, which recombines and restores the 4 bytes of data in the RGBA channel from low to high into a 4 bytes floating point number.

[0200] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24));

[0201] Third step: according to the logic of formula 2, the corrected UV coordinates can be calculated according to the calculation process described below, where screen_uv is the UV coordinate of the display screen, anti_uv is the corrected UV coordinate, and the UV coordinate is the normalized sampling coordinate. For the left half image, the third center coordinate is at 1 / 4 of the display screen or image in the horizontal direction; for the right half image, the second center coordinate is at 3 / 4 of the display screen or image in the horizontal direction.

[0202] If screen_uv≥ 0.5, anti_uv = (screen_uv – (0.75, 0.5)) × R + (0.75,0.5);

[0203] If screen_uv < 0.5, anti_uv = (screen_uv – (0.25, 0.5)) × R +(0.25, 0.5).

[0204] The original image is sampled using the corrected UV coordinates and rendered on the display screen, and finally the image without distortion can be seen through the binocular optical imaging system. An example is shown, Figure 10 Another example of image display is provided, the original image is distorted and corrected to obtain a corrected image, the corrected image is formed into an actual viewing image through an optical imaging system, and the user finally watches the image without distortion on the display screen.

[0205] The embodiment of the present application provides a kind of image distortion correction method, solve the problem of high consumption of computing resources and storage resources in image distortion correction process;If the first pixel coordinate is not in the pixel coordinate range of the data area of the correction data texture, for two-dimensional image and three-dimensional image, the corresponding second pixel coordinate can be determined according to the center symmetry of optical imaging system, so as to determine its corresponding coordinate correction data, and the corrected sampling coordinates are calculated quickly;The image distortion correction method provided by the embodiment of the present application supports the correction of two-dimensional image and three-dimensional image;Without real-time calculation of the distortion correction data corresponding to each pixel coordinate, the computing resources are saved;And the distortion correction data can be restored by reading the corresponding pixel in the coordinate correction image as the value of each color channel in the correction process, saving storage resources.

[0206] Embodiment four

[0207] Figure 11 The structure diagram of an image distortion correction data storage device provided by the fourth embodiment of the present application is shown in Figure 4. Figure 11 As shown in the figure, the device comprises a to-be-stored coordinate determination module 41 and a coordinate correction data saving module 42.

[0208] The to-be-stored coordinate determination module 41 is used to divide the pixel point coordinates on the display screen according to the center symmetry of the optical imaging system, and determine the to-be-stored pixel coordinates.

[0209] The coordinate correction data saving module 42 is used to form coordinate correction data based on the distortion correction data corresponding to each to-be-stored pixel coordinate and save it into a two-dimensional array, and the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0210] The embodiment of the present application provides an image distortion correction data storage device, which solves the problem of high consumption of computing resources and storage resources in image distortion correction process, divides the pixel point coordinates on the display screen according to the center symmetry of the optical imaging system to obtain the to-be-stored pixel coordinates, forms coordinate correction data based on the distortion correction data corresponding to each to-be-stored pixel coordinate and saves it into a two-dimensional array, and the size of the two-dimensional array is determined according to half of the width and height of the display screen, without storing all pixel point coordinates in the display screen, reducing the consumption of storage resources;The distortion correction data of pixel point coordinates is pre-stored, reducing the consumption of computing resources and improving the calculation speed of image distortion correction.

[0211] Optionally, the to-be-stored coordinate determination module 41 comprises:

[0212] The first dividing unit is configured to divide the display screen into 8 regions according to the central symmetry of the optical imaging system if the width and the height of the display screen are equal, and determine a first candidate pixel point coordinate in one of the 8 regions as a pixel coordinate to be stored, the first candidate pixel point coordinate taking the upper left corner of the display screen as a coordinate origin.

[0213] The second dividing unit is configured to extend the display screen into a square with the maximum value of the width and the height if the width and the height of the display screen are not equal, divide the extended square into two regions based on a diagonal line of the extended square, take a region containing more pixel point coordinates of the display screen as a region to be stored, and determine a second candidate pixel point coordinate in the region to be stored as a pixel coordinate to be stored, the second candidate pixel point coordinate taking the upper left corner of the display screen as a coordinate origin.

[0214] Optionally, when the width of the display screen is greater than or equal to the height, the pixel coordinate to be stored satisfies the following condition:

[0215]

[0216] When the width of the display screen is less than the height, the pixel coordinate to be stored satisfies the following condition:

[0217]

[0218] wherein x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

[0219] Optionally, the device further comprises:

[0220] The correction image generation module is configured to generate a coordinate correction image based on the two-dimensional array.

[0221] Each pixel point in the coordinate correction image stores the distortion correction data, the distortion correction data is a four-byte floating point number type, and each byte corresponds to the value of each pixel color channel.

[0222] The image distortion correction data storage device provided in the embodiments of the present application can execute the image distortion correction data storage method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0223] Embodiment Five

[0224] Figure 12 A structural schematic diagram of an image distortion correction device provided in Embodiment Five of the present application is shown in FIG. 5. Figure 12 ​​As shown, the device comprises a first pixel coordinate determination module 51, a first distortion correction data determination module 52, a second distortion correction data determination module 53, and a distortion correction determination module 54.

[0225] The first pixel coordinate determination module 51 is configured to determine a first pixel coordinate to be corrected based on a pixel coordinate in a display screen.

[0226] The first distortion correction data determination module 52 is configured to, if the first pixel coordinate is within a pixel coordinate range of a data area of a correction data texture, determine distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data.

[0227] The second distortion correction data determination module 53 is configured to, if the first pixel coordinate is not within the pixel coordinate range of the data area of the correction data texture, determine a second pixel coordinate within the pixel coordinate range of the data area of the correction data texture corresponding to the first pixel coordinate according to a center symmetry of an optical imaging system, and determine distortion correction data corresponding to the second pixel coordinate according to the pre-stored coordinate correction data.

[0228] The distortion correction determination module 54 is configured to perform image distortion correction on the first pixel coordinate based on the distortion correction data.

[0229] The coordinate correction data is stored according to the image distortion correction data storage method of any of the embodiments of the present application.

[0230] The embodiments of the present application provide an image distortion correction device, which solves the problem of high consumption of computing resources and storage resources in the image distortion correction process. The first pixel coordinate to be corrected is determined according to a pixel coordinate in a display screen, the coordinate correction data is pre-stored, the distortion correction data corresponding to the first pixel coordinate is determined through the coordinate correction data, so as to realize image distortion correction on the first pixel coordinate. The distortion correction data corresponding to each pixel coordinate does not need to be calculated in real time, which saves computing resources. The coordinate correction data in the embodiments of the present application is stored through the image distortion correction data storage method provided by any of the embodiments of the present application, which can save storage resources and reduce the consumption of storage resources. It is determined whether the first pixel coordinate is within the pixel coordinate range of the data area of the correction data texture. If yes, the distortion correction data corresponding to the first pixel coordinate is determined by directly querying the coordinate correction data. If not, the first pixel coordinate is coordinate-transformed according to the center symmetry of the optical imaging system, the second pixel coordinate corresponding to the first pixel coordinate is determined, and then the distortion correction data corresponding to the second pixel coordinate is determined by querying the coordinate correction data. Through the above-mentioned manner, the distortion correction data corresponding to each pixel coordinate in the display screen can be determined in the case that the coordinate correction data only saves the distortion correction data of part of the pixel coordinates, so as to realize fast distortion correction on the image.

[0231] Optionally, the first pixel coordinate determining module 51 comprises:

[0232] a first pixel coordinate determining unit, configured to determine, when the display screen displays a two-dimensional image, a pixel coordinate in the display screen as a first pixel coordinate to be corrected;

[0233] a second pixel coordinate determining unit, configured to determine, when the display screen displays a three-dimensional image, a horizontal coordinate of 2 times a horizontal coordinate in the pixel coordinate in the display screen as a horizontal coordinate of the first pixel coordinate to be corrected, and a vertical coordinate in the pixel coordinate in the display screen as a vertical coordinate of the first pixel coordinate to be corrected.

[0234] Optionally, the second distortion correction data determining module 53 comprises:

[0235] an initial coordinate determining unit, configured to determine an initial horizontal coordinate and an initial vertical coordinate according to the first pixel coordinate and a width and a height of the display screen;

[0236] a first parameter calculating unit, configured to calculate a difference between the width and the height of the display screen, and record half of the difference as a first parameter;

[0237] a third pixel coordinate determining unit, configured to, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than a sum of the initial vertical coordinate and the first parameter, determine the sum of the initial vertical coordinate and the first parameter as a horizontal coordinate of the second pixel coordinate, and determine a difference between the initial horizontal coordinate and the first parameter as a vertical coordinate of the second pixel coordinate;

[0238] a fourth pixel coordinate determining unit, configured to, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, determine the initial horizontal coordinate as a horizontal coordinate of the second pixel coordinate, and determine the initial vertical coordinate as a vertical coordinate of the second pixel coordinate;

[0239] a fifth pixel coordinate determining unit, configured to, if the first parameter is less than 0 and a difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, determine the sum of the initial vertical coordinate and the first parameter as a horizontal coordinate of the second pixel coordinate, and determine the difference between the initial horizontal coordinate and the first parameter as a vertical coordinate of the second pixel coordinate;

[0240] a sixth pixel coordinate determining unit, configured to, if the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate, determine the initial horizontal coordinate as a horizontal coordinate of the second pixel coordinate, and determine the initial vertical coordinate as a vertical coordinate of the second pixel coordinate.

[0241] Optionally, the initial coordinate determining unit is specifically configured to: determine a to-be-mapped pixel coordinate according to the first pixel coordinate; and determine an initial horizontal coordinate and an initial vertical coordinate according to the to-be-mapped pixel coordinate and the width and the height of the display screen.

[0242] Optionally, the method further includes:

[0243] when the display screen displays a two-dimensional image, taking the first pixel coordinate as the to-be-mapped pixel coordinate;

[0244] when the display screen displays a three-dimensional image, if the horizontal coordinate in the first pixel coordinate is greater than or equal to the width of the display screen, subtracting the width of the display screen from the horizontal coordinate in the first pixel coordinate to obtain the horizontal coordinate of the to-be-mapped pixel coordinate, and taking the vertical coordinate in the first pixel coordinate as the vertical coordinate of the to-be-mapped pixel coordinate; if the horizontal coordinate in the first pixel coordinate is less than the width of the display screen, taking the first pixel coordinate as the to-be-mapped pixel coordinate.

[0245] Optionally, the method further includes:

[0246] if the ratio of 2 times of the horizontal coordinate X1 in the to-be-mapped pixel coordinate to the width W of the display screen is greater than or equal to 1, taking W-1-X1 as the initial horizontal coordinate of the second pixel coordinate, otherwise, taking X1 as the initial horizontal coordinate of the second pixel coordinate;

[0247] if the ratio of 2 times of the vertical coordinate Y1 in the to-be-mapped pixel coordinate to the height H of the display screen is greater than or equal to 1, taking H-1-Y1 as the initial vertical coordinate of the second pixel coordinate, otherwise, taking Y1 as the initial vertical coordinate of the second pixel coordinate.

[0248] Optionally, the coordinate correction data is a coordinate correction image, and the method further includes:

[0249] taking the first pixel coordinate or the second pixel coordinate as a to-be-corrected coordinate;

[0250] querying a pre-stored coordinate correction image based on the to-be-corrected coordinate, and determining the values of each pixel color channel of a pixel coordinate corresponding to the to-be-corrected coordinate in the coordinate correction image;

[0251] performing data recovery on the values of each pixel color channel to obtain the distortion correction data corresponding to the to-be-corrected coordinate.

[0252] Optionally, when the display screen displays a two-dimensional image, the distortion correction determining module 54 is specifically configured to: determine a texture coordinate corresponding to the first pixel coordinate; multiply a difference between the texture coordinate and a first center coordinate by the distortion correction data to obtain a first corrected coordinate, wherein the first center coordinate is determined according to a center of the display screen; and take a sum of the first corrected coordinate and the first center coordinate as a corrected sampling coordinate.

[0253] Optionally, when the display screen displays a three-dimensional image, the distortion correction determining module 54 is specifically configured to: determine a texture coordinate corresponding to the first pixel coordinate; if the texture coordinate is greater than or equal to a set value, multiply a difference between the texture coordinate and a second center coordinate by the distortion correction data to obtain a second corrected coordinate, and take a sum of the second corrected coordinate and the second center coordinate as a corrected sampling coordinate; if the texture coordinate is less than the set value, multiply a difference between the texture coordinate and a third center coordinate by the distortion correction data to obtain a third corrected coordinate, and take a sum of the third corrected coordinate and the third center coordinate as a corrected sampling coordinate; wherein the second center coordinate is determined according to a 3 / 4 position in a horizontal direction of the display screen, and the third center coordinate is determined according to a 1 / 4 position in the horizontal direction of the display screen.

[0254] The image distortion correction apparatus provided by the embodiments of the present application can execute the image distortion correction method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0255] Embodiment six

[0256] Figure 13 A structural schematic diagram of an electronic device provided by Embodiment six of the present application. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (such as headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections, and their functions, as well as their relationships with the implementation of the present application described and / or claimed herein, are merely examples and are not intended to limit the present application.

[0257] As Figure 13As shown, the electronic device 60 includes at least one processor 61 and a memory, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, and the like, which is communicatively connected to the at least one processor 61. The memory stores a computer program that can be executed by the at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or loaded from the storage unit 68 into the random access memory (RAM) 63. Various programs and data required for the operation of the electronic device 60 can also be stored in the RAM 63. The processor 61, the ROM 62, and the RAM 63 are connected to each other through a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0258] Various components in the electronic device 60 are connected to the I / O interface 65, including an input unit 66, such as a keyboard, a mouse, and the like, an output unit 67, such as various types of displays, a speaker, and the like, a storage unit 68, such as a magnetic disk, an optical disk, and the like, and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0259] The processor 61 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The processor 61 performs various methods and processes described above, such as the image distortion correction data storage method or the image distortion correction method.

[0260] In some embodiments, the image distortion correction data storage method or the image distortion correction method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 68. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the image distortion correction data storage method or the image distortion correction method described above can be performed. Alternatively, in other embodiments, the processor 61 can be configured to perform the image distortion correction data storage method or the image distortion correction method by any other appropriate means, such as by means of firmware.

[0261] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0262] Computer programs implementing methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0263] Embodiments of the present application provide a computer program product, comprising a computer program which, when executed by a processor, implements the image distortion correction data storage method or the image distortion correction method according to any of the embodiments of the present application.

[0264] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0265] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0266] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0267] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0268] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0269] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the spirit and scope of the disclosure. Any further modifications, equivalents, and / or alternatives come within the scope of the present disclosure as described in the following claims.

Claims

1. A method for storing image distortion correction data, characterized in that, include: The coordinates of the pixels on the display screen are divided according to the central symmetry of the optical imaging system to determine the coordinates of the pixels to be stored. Based on the distortion correction data corresponding to the coordinates of each pixel to be stored, coordinate correction data is formed and saved in a two-dimensional array. The size of the two-dimensional array is determined according to half the width and height of the display screen. The step of dividing the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored includes: If the width and height of the display screen are equal, the display screen is divided into 8 regions according to the central symmetry of the optical imaging system. The coordinates of the first candidate pixel point in one of the regions are determined as the coordinates of the pixel to be stored. The coordinates of the first candidate pixel point take the upper left corner of the display screen as the origin. If the width and height of the display screen are not equal, the display screen is expanded into a square based on the maximum value of the width and height. The area is divided based on the diagonal of the expanded square to obtain two areas. The area with more pixel coordinates of the display screen is selected as the storage area. The second candidate pixel coordinates in the storage area are determined as the pixel coordinates to be stored. The origin of the second candidate pixel coordinates is the upper left corner of the display screen.

2. The method according to claim 1, characterized in that, When the width of the display screen is greater than or equal to its height, the pixel coordinates to be stored satisfy the following condition: ； When the width of the display screen is less than its height, the pixel coordinates to be stored satisfy the following condition: ； Where x is the horizontal coordinate of the pixel to be stored, y is the vertical coordinate of the pixel to be stored, W is the width of the display screen, and H is the height of the display screen.

3. The method according to any one of claims 1-2, characterized in that, Also includes: Generate a coordinate-corrected image based on the two-dimensional array; In this image, each pixel in the coordinate-corrected image stores the distortion correction data, which is a four-byte floating-point number, with each byte corresponding to the value of the color channel of each pixel.

4. An image distortion correction method, characterized in that, include: The coordinates of the first pixel to be corrected are determined based on the pixel coordinates on the display screen. If the first pixel coordinates are within the pixel coordinate range of the data area of ​​the corrected data texture, the distortion correction data corresponding to the first pixel coordinates is determined according to the pre-stored coordinate correction data; If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the corrected data texture, the second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the corrected data texture is determined according to the central symmetry of the optical imaging system, and the distortion correction data corresponding to the second pixel coordinate is determined according to the pre-stored coordinate correction data. Image distortion correction is performed on the first pixel coordinates based on the distortion correction data; The coordinate correction data is stored according to the image distortion correction data storage method according to any one of claims 1-3; Determining the coordinates of the first pixel to be corrected based on the pixel coordinates in the display screen includes: When the display screen displays a two-dimensional image, the pixel coordinates in the display screen are used as the first pixel coordinates to be corrected; When the display screen displays a three-dimensional image, twice the horizontal coordinate of the pixel coordinates on the display screen is used as the horizontal coordinate of the first pixel coordinate to be corrected, and the vertical coordinate of the pixel coordinates on the display screen is used as the vertical coordinate of the first pixel coordinate to be corrected.

5. The method according to claim 4, characterized in that, The step of determining the second pixel coordinates corresponding to the first pixel coordinates within the pixel coordinate range of the data area of ​​the corrected data texture based on the central symmetry of the optical imaging system includes: The initial horizontal and vertical coordinates are determined based on the first pixel coordinates and the width and height of the display screen. Calculate the difference between the width and height of the display screen, and record half of the difference as the first parameter; If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than the sum of the initial vertical coordinate and the first parameter, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate. If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate. If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate. If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

6. The method according to claim 5, characterized in that, Determining the initial horizontal and vertical coordinates based on the first pixel coordinates and the width and height of the display screen includes: The coordinates of the pixel to be mapped are determined based on the first pixel coordinates; The initial horizontal and vertical coordinates are determined based on the coordinates of the pixel to be mapped and the width and height of the display screen.

7. The method according to claim 6, characterized in that, Determining the coordinates of the pixel to be mapped based on the first pixel coordinates includes: When the display screen displays a two-dimensional image, the first pixel coordinates are used as the pixel coordinates to be mapped. When the display screen displays a three-dimensional image, if the horizontal coordinate of the first pixel coordinate is greater than or equal to the width of the display screen, the horizontal coordinate of the first pixel coordinate is subtracted from the width of the display screen to obtain the horizontal coordinate of the pixel coordinate to be mapped, and the vertical coordinate of the first pixel coordinate is obtained as the vertical coordinate of the pixel coordinate to be mapped; if the horizontal coordinate of the first pixel coordinate is less than the width of the display screen, the first pixel coordinate is obtained as the pixel coordinate to be mapped.

8. The method according to claim 6, characterized in that, The step of determining the initial horizontal and vertical coordinates based on the coordinates of the pixel to be mapped and the width and height of the display screen includes: If the ratio of twice the x-coordinate X1 in the pixel coordinates to be mapped to the width W of the display screen is greater than or equal to 1, then W-1-X1 is used as the initial x-coordinate of the second pixel coordinate; otherwise, X1 is used as the initial x-coordinate of the second pixel coordinate. If the ratio of twice the ordinate Y1 of the pixel to be mapped to the height H of the display screen is greater than or equal to 1, then H-1-Y1 is used as the initial ordinate of the second pixel coordinate; otherwise, Y1 is used as the initial ordinate of the second pixel coordinate.

9. The method according to claim 4, characterized in that, The coordinate correction data is a coordinate-corrected image. Determining the distortion correction data corresponding to the first pixel coordinates or the second pixel coordinates based on the pre-stored coordinate correction data includes: Use the first pixel coordinates or the second pixel coordinates as the coordinates to be corrected; Based on the coordinates to be corrected, query the pre-stored coordinate correction image to determine the value of each pixel color channel of the pixel coordinates corresponding to the coordinates to be corrected in the coordinate correction image; The values ​​of each pixel color channel are recovered to obtain the distortion correction data corresponding to the coordinates to be corrected.

10. The method according to claim 4, characterized in that, When the display screen displays a two-dimensional image, the step of performing image distortion correction on the first pixel coordinates based on the distortion correction data includes: Determine the texture coordinates corresponding to the first pixel coordinates; The difference between the texture coordinates and the first center coordinates is multiplied by the distortion correction data to obtain the first correction coordinates, and the first center coordinates are determined according to the center of the display screen. The sum of the first corrected coordinates and the first center coordinates is used as the corrected sampling coordinates.

11. The method according to claim 4, characterized in that, When the display screen displays a three-dimensional image, the step of performing image distortion correction on the first pixel coordinates based on the distortion correction data includes: Determine the texture coordinates corresponding to the first pixel coordinates; If the texture coordinates are greater than or equal to a set value, the difference between the texture coordinates and the second center coordinates is multiplied by the distortion correction data to obtain the second correction coordinates, and the sum of the second correction coordinates and the second center coordinates is used as the corrected sampling coordinates; If the texture coordinates are less than a set value, the difference between the texture coordinates and the third center coordinates is multiplied by the distortion correction data to obtain the third correction coordinates, and the sum of the third correction coordinates and the third center coordinates is used as the corrected sampling coordinates. The second center coordinate is determined based on the 3 / 4 position of the horizontal direction of the display screen, and the third center coordinate is determined based on the 1 / 4 position of the horizontal direction of the display screen.

12. An image distortion correction data storage device, characterized in that, include: The module for determining the coordinates to be stored is used to divide the coordinates of the pixels on the display screen according to the central symmetry of the optical imaging system, and to determine the coordinates of the pixels to be stored. The coordinate correction data storage module is used to form coordinate correction data based on the distortion correction data corresponding to the coordinates of each pixel to be stored and save it to a two-dimensional array. The size of the two-dimensional array is determined according to half the width and height of the display screen. The module for determining the coordinates to be stored includes: The first division unit is used to divide the display screen into 8 regions according to the central symmetry of the optical imaging system if the width and height of the display screen are equal, and to determine the coordinates of the first candidate pixel point in one of the regions as the coordinates of the pixel to be stored, wherein the coordinates of the first candidate pixel point take the upper left corner of the display screen as the origin. The second partitioning unit is used to expand the display screen into a square with the maximum value of the width and height if the width and height of the display screen are not equal, divide the area based on the diagonal of the expanded square to obtain two areas, select the area with more pixel coordinates of the display screen as the storage area, and determine the second candidate pixel coordinates in the storage area as the storage pixel coordinates, with the upper left corner of the display screen as the origin of the second candidate pixel coordinates.

13. An image distortion correction device, characterized in that, include: The first pixel coordinate determination module is used to determine the first pixel coordinates to be corrected based on the pixel coordinates in the display screen. The first distortion correction data determination module is used to determine the distortion correction data corresponding to the first pixel coordinates based on the pre-stored coordinate correction data if the first pixel coordinates are within the pixel coordinate range of the data area of ​​the correction data texture. The second distortion correction data determination module is used to determine the second pixel coordinates corresponding to the first pixel coordinates within the pixel coordinate range of the data area of ​​the correction data texture based on the central symmetry of the optical imaging system if the first pixel coordinates are not within the pixel coordinate range of the data area of ​​the correction data texture, and to determine the distortion correction data corresponding to the second pixel coordinates based on the pre-stored coordinate correction data. The distortion correction determination module is used to perform image distortion correction on the coordinates of the first pixel based on the distortion correction data; The coordinate correction data is stored according to the image distortion correction data storage method according to any one of claims 1-3; The first pixel coordinate determination module includes: The first pixel coordinate determination unit is used to determine the pixel coordinates in the display screen as the first pixel coordinates to be corrected when the display screen displays a two-dimensional image. The second pixel coordinate determination unit is used to, when the display screen displays a three-dimensional image, take twice the horizontal coordinate of the pixel coordinates in the display screen as the horizontal coordinate of the first pixel coordinate to be corrected, and take the vertical coordinate of the pixel coordinates in the display screen as the vertical coordinate of the first pixel coordinate to be corrected.

14. An electronic device, characterized in that, The electronic device includes: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the image distortion correction data storage method according to any one of claims 1-3 or the image distortion correction method according to any one of claims 4-11.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image distortion correction data storage method of any one of claims 1-3 or the image distortion correction method of any one of claims 4-11.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the image distortion correction data storage method according to any one of claims 1-3 or the image distortion correction method according to any one of claims 4-11.

Images