Image distortion correction data storage method and image distortion correction method

Abstract

The invention discloses an image distortion correction data storage method and an image distortion correction method. Comprising the following steps: dividing pixel point coordinates on a display screen according to the central symmetry of the optical imaging system, and determining to-be-stored pixel coordinates; coordinate correction data is formed based on distortion correction data corresponding to each to-be-stored pixel coordinate and is stored in a two-dimensional array, and the size of the two-dimensional array is determined according to half of the width and height of a display screen, so that the problem of relatively high consumption of computing resources and storage resources in an image distortion correction process is solved; the pixel point coordinates on the display screen are divided through the central symmetry of the optical imaging system, the to-be-stored pixel coordinates are obtained and stored in the two-dimensional array, all the pixel point coordinates in the display screen do not need to be stored, and consumption of storage resources is reduced; the distortion correction data of the pixel point coordinates are pre-stored, consumption of computing resources is reduced, and the computing speed of image distortion correction is increased.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image distortion correction data storage method and an image distortion correction method. Background Art

[0002] After an image is processed by an optical imaging system, it will be distorted. To eliminate this distortion, it is usually corrected before display. Existing technologies for image distortion correction typically use real-time calculations or pre-calculated UV mapping tables to accelerate the distortion correction calculations. However, real-time distortion correction consumes a lot of computing resources, while using pre-calculated UV mapping tables to accelerate distortion correction calculations consumes a lot of storage resources. Summary of the Invention

[0003] The present application provides an image distortion correction data storage method and an image distortion correction method, which reduce the consumption of computing resources and storage resources during the image distortion correction process.

[0004] According to one aspect of the present application, a method for storing image distortion correction data is provided, comprising:

[0005] Dividing the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored;

[0006] Coordinate correction data is formed based on the distortion correction data corresponding to each pixel coordinate to be stored and saved in a two-dimensional array, where the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0007] According to another aspect of the present application, a method for correcting image distortion is provided, comprising:

[0008] determining first pixel coordinates to be corrected based on pixel coordinates in the display screen;

[0009] If the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture, determining the distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data;

[0010] If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, determining a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the correction data texture according to the central symmetry of the optical imaging system, and determining distortion correction data corresponding to the second pixel coordinate according to pre-stored coordinate correction data;

[0011] performing image distortion correction on the first pixel coordinates based on the distortion correction data;

[0012] The coordinate correction data is stored according to the image distortion correction data storage method described in any embodiment of the present application.

[0013] According to another aspect of the present application, there is provided an image distortion correction data storage device, comprising:

[0014] A module for determining coordinates to be stored, configured to divide the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored;

[0015] A coordinate correction data storage module is used to form coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and save it into a two-dimensional array, where the size of the two-dimensional array is determined according to half the width and height of the display screen.

[0016] According to another aspect of the present application, there is provided an image distortion correction device, comprising:

[0017] A first pixel coordinate determining module, configured to determine first pixel coordinates to be corrected based on pixel coordinates on the display screen;

[0018] a first distortion correction data determining module, configured to determine, if the first pixel coordinate is within a pixel coordinate range of a data area of ​​a correction data texture, distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data;

[0019] a second distortion correction data determining module configured to, if the first pixel coordinate is not within a pixel coordinate range of a data region of the correction data texture, determine, based on the central symmetry of the optical imaging system, a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data region of the correction data texture, and determine, based on pre-stored coordinate correction data, distortion correction data corresponding to the second pixel coordinate;

[0020] a distortion correction determination module, configured to perform image distortion correction on the first pixel coordinates based on the distortion correction data;

[0021] The coordinate correction data is stored according to the image distortion correction data storage method described in any embodiment of the present application.

[0022] According to another aspect of the present application, an electronic device is provided, comprising:

[0023] at least one processor, and a memory communicatively coupled to the at least one processor;

[0024] In which, the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image distortion correction data storage method or the image distortion correction method described in any embodiment of the present application.

[0025] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image distortion correction data storage method or the image distortion correction method described in any embodiment of the present application when executed.

[0026] According to another aspect of the present application, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, it implements the image distortion correction data storage method or the image distortion correction method described in any embodiment of the present application.

[0027] The technical solution of the embodiment of the present application is to determine the pixel coordinates to be stored by dividing the pixel coordinates on the display screen according to the central symmetry of the optical imaging system; form coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and save it in a two-dimensional array, and the size of the two-dimensional array is determined according to half of the width and height of the display screen, which solves the problem of high consumption of computing resources and storage resources in the image distortion correction process. The pixel coordinates on the display screen are divided according to the central symmetry of the optical imaging system to obtain the pixel coordinates to be stored; form coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and save it in a two-dimensional array, and the size of the two-dimensional array is determined according to half of the width and height of the display screen. It is not necessary to store all pixel coordinates in the display screen, thereby reducing the consumption of storage resources; the distortion correction data of the pixel coordinates are pre-stored, thereby reducing the consumption of computing resources and improving the calculation speed of image distortion correction.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1This is a flowchart of a method for storing image distortion correction data according to the first embodiment of the present application;

[0031] Figure 2 This is an example diagram of a virtual image forming process provided in accordance with the first embodiment of the present application;

[0032] Figure 3 This is an example diagram of a display screen division provided according to the first embodiment of the present application;

[0033] Figure 4 This is another example diagram of display screen division provided according to the first embodiment of the present application;

[0034] Figure 5 This is another example diagram of display screen division provided according to the first embodiment of the present application;

[0035] Figure 6 This is an example diagram of a coordinate-corrected image provided according to the first embodiment of the present application;

[0036] Figure 7 This is a flow chart of an image distortion correction method provided according to the second embodiment of the present application;

[0037] Figure 8 This is a flowchart of an image distortion correction method provided according to the third embodiment of the present application;

[0038] Figure 9 This is an example diagram of an image display provided according to the third embodiment of the present application;

[0039] Figure 10 This is another example diagram of image display provided according to the third embodiment of the present application;

[0040] Figure 11 This is a structural diagram of an image distortion correction data storage device provided according to the fourth embodiment of the present application;

[0041] Figure 12 1 is a structural diagram of an image distortion correction device provided according to Embodiment 5 of the present application;

[0042] Figure 13 It is a structural diagram of an electronic device provided according to Example 6 of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] Example 1

[0046] Figure 1 This is a flowchart of a method for storing image distortion correction data provided in the first embodiment of the present application. This embodiment is applicable to the case of storing image distortion correction data. The method can be executed by an image distortion correction data storage device. The image distortion correction data storage device can be implemented in the form of hardware and / or software. The image distortion correction data storage device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0047] S101 : dividing pixel coordinates on a display screen according to the central symmetry of an optical imaging system to determine pixel coordinates to be stored.

[0048] In this embodiment, the pixel coordinates to be stored can be understood as pixel coordinates with storage requirements, and the number of pixel coordinates to be stored is less than the total number of pixel coordinates in the display screen.

[0049] The optical imaging system has central symmetry. Based on this central symmetry, the display screen is divided into multiple symmetrical regions. Each region has corresponding pixel coordinates, thus achieving pixel coordinate division. One region is selected from the divided regions, and the pixel coordinates in the selected region are used as the pixel coordinates to be stored.

[0050] S102 : generating coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and saving the data into a two-dimensional array, wherein the size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0051] In this embodiment, distortion correction data can be understood as data used to correct image distortion. When the original image is formed into a virtual image by the optical imaging system and displayed on the display screen, it will be distorted, affecting the user's visual experience. To avoid image distortion, the original image is first subjected to distortion correction before being processed by the optical imaging system. In the embodiment of this application, the distortion correction data is used to correct the distortion of the original image. The image obtained after distortion correction will not be distorted when it is projected by the optical imaging system. Coordinate correction data can be understood as data used to perform distortion correction on pixel coordinates.

[0052] Determine the distortion correction data corresponding to each pixel coordinate to be stored. Each pixel coordinate to be stored and its corresponding distortion correction data form a set of corresponding data. Based on this set of corresponding data, coordinate correction data is formed. That is, the coordinate correction data includes the distortion correction data corresponding to each pixel coordinate to be stored. The coordinate correction data is stored in a two-dimensional array according to the corresponding position. That is, the position of the distortion correction data in the two-dimensional array is determined based on the pixel coordinates to be stored, and the distortion correction data is stored as an element in the two-dimensional array. The size of the two-dimensional array is determined by half the width W and height H of the display screen. Exemplarily, the size of the two-dimensional array is (W / 2)×(H / 2), that is, the number of rows and columns of the two-dimensional array is (W / 2) and (H / 2), respectively.

[0053] The mapping transformation relationship of the correction model is generally in the form of Equation 1 below, which is determined by the specific optical imaging system design. Here, x and y are the original sampling coordinates of the display screen, and x' and y' are the corrected sampling coordinates. Both use the image center as the coordinate origin.

[0054] (x', y') = F xy (x,y) Formula 1;

[0055] For each sampling coordinate of the display screen, i.e. pixel coordinate, the above mapping transformation is performed to obtain the corrected sampling coordinate, and then the original image is sampled to render the corrected image on the display screen. Finally, after passing through the optical imaging system, the user can see the image without distortion. For example, Figure 2 An example diagram of the virtual image formation process is provided.

[0056] For a centrally symmetric optical imaging system, Formula 1 can be further rewritten as Formula 2, where W is the width of the display screen, H is the height of the display screen, and note that the pixel coordinates are based on the upper left corner of the display screen or image as the origin, and the radial r is relative to the center of the display screen or image. radial distance.

[0057] Formula 2;

[0058] R(r) is the distortion correction data. R(r) can be considered the ratio of r1 to r2, where r1 is the distance between the sampling coordinates and the center of the display screen (or image center) before the correction transformation, and r2 is the distance between the sampling coordinates and the center of the display screen (or image center) after the correction transformation. Therefore, by using a correction data generation program to pre-calculate the R(r) value for each pixel on the display screen as correction data and storing it, and then reading this set of values ​​when the distortion correction program is running to calculate the corrected sampling coordinates and sample the original image, the corrected image can be rendered on the display screen. When calculating and storing R(r) data, the embodiments of the present application utilize the central symmetry of the optical imaging system to optimize the calculation process and storage space.

[0059] An embodiment of the present application provides a method for storing image distortion correction data, which solves the problem of high consumption of computing resources and storage resources during the image distortion correction process. The pixel coordinates on the display screen are divided by the central symmetry of the optical imaging system to obtain the pixel coordinates to be stored. Coordinate correction data is formed based on the distortion correction data corresponding to each pixel coordinate to be stored and saved in a two-dimensional array. The size of the two-dimensional array is determined according to half the width and height of the display screen. There is no need to store the coordinates of all pixel points in the display screen, which reduces the consumption of storage resources. The distortion correction data of the pixel coordinates is pre-stored to reduce the consumption of computing resources and improve the calculation speed of image distortion correction.

[0060] Optionally, pixel coordinates on the display screen are divided according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored, including steps A1 and A2:

[0061] A1. If the width and height of the display screen are equal, the display screen is divided into 8 areas according to the central symmetry of the optical imaging system, and the coordinates of the first candidate pixel point in one of the areas are determined as the pixel coordinates to be stored. The coordinates of the first candidate pixel point use the upper left corner of the display screen as the coordinate origin.

[0062] In this embodiment, the coordinates of the first candidate pixel point can be understood as the coordinates of the pixel point in an area of ​​the display screen. If the width and height of the display screen are equal, that is, the display screen is a square, the display screen is divided into 8 areas according to the central symmetry of the optical imaging system. These 8 areas are all symmetrical areas, which can be centrally symmetrical, axially symmetrical, etc. For example, the diagonals of the display screen are first connected in pairs, and the display screen is divided into four equilateral triangles. Then, each equilateral triangle is divided into two symmetrical triangles along the height line, resulting in a total of 8 triangles, each triangle is an area, and a total of 8 areas. Select one of the areas, and determine the coordinates of the first candidate pixel point in this area as the pixel coordinates to be stored. The coordinates of the first candidate pixel point use the upper left corner of the display screen as the coordinate origin.

[0063] For example, Figure 3 An example diagram of screen division is provided. As shown in the figure, assume that the coordinates of pixel 1 are (x1, y1). The coordinates of pixels 2 to 8, which are symmetrical to it, are: (y1, x1), (-y1+W-1, x1), (-x1+W-1, y1), (-x1+W-1, -y1+W-1), (-y1+W-1, -x1+W-1), (y1, -x1+W-1), (x1, -y1+W-1). Pixels 1 to 8 have the same radial length r relative to the image center, meaning they have the same R(r) value. Therefore, we only need to calculate and store the R(r) value data for approximately 1 / 8 of the area where pixel 1 is located, that is, the data that satisfies the following conditions.

[0064] Formula 3;

[0065] In the embodiment of the present application, when storing R(r) value data in a two-dimensional array, only about half of the space in the two-dimensional array is used to store the R(r) value data, and the remaining space remains at 0. If the coordinate correction data is subsequently compressed, the blank area with a value of 0 will be significantly compressed, and almost no additional storage space will be occupied.

[0066] A2. If the width and height of the display screen are not equal, expand the display screen into a square with the maximum value of the width and the height, and divide the area based on the diagonal of the expanded square to obtain two areas. The area containing the larger number of pixel coordinates of the display screen of the two areas is used as the area to be stored, and the second candidate pixel coordinates in the area to be stored are determined as the pixel coordinates to be stored. The second candidate pixel coordinates use the upper left corner of the display screen as the coordinate origin.

[0067] In this embodiment, the area to be stored can be understood as an area where pixel coordinates need to be stored; the second candidate pixel point coordinates can be understood as pixel point coordinates in an area on the display screen.

[0068] If the width and height of the display screen are not equal, that is, the display screen is a rectangle, compare the width and height, use the maximum value of the width and height as the side length in the positive direction, and expand the display screen into a square. For example, when the width is greater than the height, expand upward along the height direction, and the obtained side length in the positive direction is the width; when the width is less than the height, expand leftward along the width direction, and the obtained side length in the positive direction is the height. Based on the diagonal of the expanded square, the area is divided, and the positive direction is divided into two areas. The number of pixel coordinates contained in the two areas is compared, and the area containing the larger number of pixel coordinates of the display screen is used as the area to be stored. The second candidate pixel coordinates in the area to be stored are determined as the pixel coordinates to be stored, and the second candidate pixel coordinates use the upper left corner of the display screen as the coordinate origin.

[0069] For example, Figure 4 Another example diagram of display screen division is provided. Taking the display screen width W as greater than or equal to the height H as an example, the display screen is expanded into a square, and then the square is divided along the diagonal to divide the square into two areas. The area containing the largest number of pixel coordinates is selected as the area to be stored, and the pixel coordinates in the area to be stored are stored as the pixel coordinates to be stored. Figure 4 The data area shown in is the area to be stored, and the pixel coordinates in the blank area are not stored. Figure 5 Another example diagram of display screen division is provided. Taking the display screen width W as an example and the display screen height H as an example, the display screen is expanded into a square, and then the square is divided along the diagonal to divide the square into two areas. The area containing the largest number of pixel coordinates is selected as the area to be stored, and the pixel coordinates in the area to be stored are stored as the pixel coordinates to be stored. Figure 5 The data area shown in is the area to be stored, and the pixel coordinates in the blank area are not stored. It can be seen that when calculating and storing R (r) value data, the embodiment of the present application can first expand the total area into a square using the larger value of W and H as the reference width, and then only intercept the portion corresponding to the actual W / 2 width and H / 2 height in the subsequent actual calculation.

[0070] Optionally, when the width of the display screen is greater than or equal to the height, the pixel coordinates to be stored meet the following conditions:

[0071] Formula 4;

[0072] Wherein, x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

[0073] That is, when W≥H, the upper left corner of the display screen or image is used as the coordinate origin, and the R (r) value data that meets the above conditions is calculated and stored.

[0074] Optionally, when the width of the display screen is smaller than the height, the pixel coordinates to be stored meet the following conditions:

[0075] Formula 5;

[0076] Wherein, x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

[0077] That is, when W<H, the upper left corner of the display screen or image is used as the coordinate origin, and the R (r) value data that meets the above conditions is calculated and stored.

[0078] It should be noted that the coordinates of the first candidate pixel point and the second candidate pixel point are based on the upper left corner of the display screen or the image as the coordinate origin. The upper left corner of the display screen and the image is the same point.

[0079] Optionally, the method also includes: generating a coordinate correction image based on a two-dimensional array; wherein each pixel point in the coordinate correction image corresponds to storing distortion correction data, the distortion correction data is a four-byte floating point type, and each byte corresponds to the value of each pixel color channel.

[0080] In this embodiment, the coordinate-corrected image can be understood as an image used to store distortion correction data. An image of corresponding size is generated based on the size of the two-dimensional array. Each position in the two-dimensional array corresponds to a pixel in the image. The distortion correction data at the corresponding position in the two-dimensional array is converted into an image pixel value to form the coordinate-corrected image. Each pixel in the coordinate-corrected image stores distortion correction data. The distortion correction data is a four-byte floating-point number, with each byte corresponding to the value of each pixel color channel.

[0081] Each R(r) value is a 4-byte floating-point number. By mapping these 4 bytes of data from low to high to correspond to the RGBA values ​​of the four channels of each pixel color in general image data, we can obtain an image that stores the complete R(r) value data. This image is recorded as the coordinate correction image. Figure 6 An example of a coordinate-rectified image is provided. Figure 6 Taking the display screen with equal width and height as an example, the data area for storing valid data only occupies about half of the image space, and the rest is a blank area for storing 0-value data. The data area is used to store the pixel coordinates to be stored and their corresponding distortion correction data.

[0082] Optionally, after the coordinate-corrected image is generated, the coordinate-corrected image is compressed and saved. For example, saving the coordinate-corrected image in an image format compressed using a lossless compression algorithm, such as PNG format, can minimize the storage space required for the R (r) value data.

[0083] Without any optimization, storing a set of completely rectified sampled coordinate (x', y') data for a 1920×1920 resolution display requires approximately 29.5MB of storage space. However, using the optimization method provided in this application, only approximately 1MB of storage space is required to achieve the same effect.

[0084] Moreover, for scenes that require rendering left and right 3D images, the method of directly storing the corrected sampling coordinate (x', y') data requires additional calculation and storage of the correction data file for this scene. However, the method provided in this application does not require additional storage space. It is only necessary to add a calculation process that supports this scene to the subsequent distortion correction program.

[0085] The present application provides an image distortion correction data storage method that solves the problem of high computing and storage resource consumption during image distortion correction. The display screen is divided into 8 or 2 areas according to the width and height of the display screen in different ways. The pixel coordinates to be stored are then selected based on the divided areas. The pixel coordinates to be stored and their corresponding distortion correction data are stored in a two-dimensional array. The size of the two-dimensional array is determined by half the width and height of the display screen. It is not necessary to store the coordinates of all pixels in the display screen, which reduces the consumption of storage resources. The distortion correction data of the pixel coordinates is pre-stored, which reduces the consumption of computing resources and improves the calculation speed of image distortion correction. The two-dimensional array can also be converted into a coordinate correction image, and the distortion correction data can be stored in the form of an image, which facilitates data storage and further saves storage space. The image distortion correction data storage method provided by the present application embodiment uses less storage space and can more conveniently store correction data of different requirements on a single device, such as data corresponding to different diopters or different correction levels, and can be flexibly switched according to needs during operation.

[0086] Example 2

[0087] Figure 7 This is a flow chart of an image distortion correction method provided in the second embodiment of the present application. This embodiment is applicable to the case of performing image distortion correction. The method can be performed by an image distortion correction device. The image distortion correction device can be implemented in the form of hardware and / or software. The image distortion correction device can be configured in an electronic device. Figure 7 As shown, the method includes:

[0088] S201: Determine first pixel coordinates to be corrected based on pixel coordinates on a display screen.

[0089] In this embodiment, the first pixel coordinates can be understood as the pixel coordinates that need to be corrected. Determine the pixel coordinates in the display screen, and each pixel coordinate in the display screen can be distorted by the method provided in the embodiment of the present application. Analyze the pixel coordinates in the display screen, determine the coordinates mapped to the original projected image, and use these coordinates as the first pixel coordinates to be corrected. For example, when the size of the display screen and the original projected image is the same, the coordinates of the pixel coordinates in the display screen mapped to the original projected image are the same as the coordinates of the pixel coordinates of the display screen, that is, the pixel coordinates in the display screen are directly used as the first pixel coordinates to be corrected; when the size of the display screen and the original projected image is different, the pixel coordinates in the display screen need to be scaled and then mapped to the original projected image. In this case, the coordinates mapped to the original projected image are different from the coordinates of the pixel coordinates of the display screen, and the pixel coordinates in the display screen need to be scaled to obtain their corresponding first pixel coordinates.

[0090] S202: If the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture, determine the distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data.

[0091] In this embodiment, the data area of ​​the correction data texture can be understood as the data area for storing coordinate correction data; the pixel coordinate range can be understood as the range of the horizontal and vertical coordinates of the pixel coordinates. Taking the storage of coordinate correction data in a two-dimensional array as an example, some positions in the two-dimensional array store coordinate correction data, which can be called the data area, and some positions in the two-dimensional array do not store coordinate correction data, which can be called blank areas. When storing coordinate correction data, the pixel coordinate range can be determined based on the coordinates corresponding to the actual stored coordinate correction data. The coordinate correction data in this embodiment is stored according to the image distortion correction data storage method of any embodiment of the present application. Therefore, the pixel coordinate range can be determined based on the pixel coordinates to be stored.

[0092] Determine whether the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture. If so, query the pre-stored coordinate correction data, determine the distortion correction data corresponding to the pixel coordinate to be stored that matches the first pixel coordinate, and use this distortion correction data as the distortion correction data corresponding to the first pixel coordinate.

[0093] S203. If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, determine the second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the correction data texture based on the central symmetry of the optical imaging system, and determine the distortion correction data corresponding to the second pixel coordinate based on the pre-stored coordinate correction data.

[0094] In this embodiment, the second pixel coordinate can be understood as a pixel coordinate equivalent to the first pixel coordinate obtained through coordinate transformation. If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, the first pixel coordinate needs to be coordinate transformed. The first pixel coordinate is coordinate transformed according to the central symmetry of the optical imaging system, and the first pixel coordinate is converted to the pixel coordinate range of the data area of ​​the correction data texture to obtain the second pixel coordinate corresponding to the first pixel coordinate, and the second pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture; the coordinate transformation of the first pixel coordinate according to the central symmetry of the optical imaging system can be a formula or logic for determining the coordinate transformation based on the central symmetry of the optical imaging system, and the first pixel coordinate is substituted into the formula or logic for coordinate transformation. Query the pre-stored coordinate correction data to determine the distortion correction data corresponding to the pixel coordinate to be stored that matches the second pixel coordinate, and use this distortion correction data as the distortion correction data corresponding to the second pixel coordinate. The distortion correction data corresponding to the second pixel coordinate is the distortion correction data corresponding to the first pixel coordinate.

[0095] The coordinate correction data is stored according to the image distortion correction data storage method of any embodiment of the present application.

[0096] S204: Perform image distortion correction on the first pixel coordinate based on the distortion correction data.

[0097] Distortion correction is performed on the first pixel coordinate according to the distortion correction data to determine the pixel coordinate corresponding to the first pixel coordinate in the original image.

[0098] An embodiment of the present application provides an image distortion correction method, which solves the problem of high consumption of computing resources and storage resources during the image distortion correction process. The first pixel coordinates to be corrected are determined according to the pixel coordinates in the display screen, and the coordinate correction data are pre-stored. The distortion correction data corresponding to the first pixel coordinates is determined through the coordinate correction data to achieve image distortion correction of the first pixel coordinates. There is no need to calculate the distortion correction data corresponding to each pixel coordinate in real time, which saves computing resources. In addition, the coordinate correction data in the embodiment of the present application is stored through the image distortion correction data storage method provided in any embodiment of the present application, which can save storage resources and reduce storage resource consumption. Determine whether the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture. If so, determine the distortion correction data corresponding to the first pixel coordinate by directly querying the coordinate correction data. If not, perform coordinate transformation on the first pixel coordinate according to the central symmetry of the optical imaging system to determine its corresponding second pixel coordinate. Then, query the coordinate correction data according to the second pixel coordinate to determine the corresponding distortion correction data. The above method can ensure that when the coordinate correction data only saves the distortion correction data of part of the pixel coordinates, the distortion correction data corresponding to each pixel coordinate in the display screen can be determined, so as to realize rapid distortion correction of the image.

[0099] Example 3

[0100] Figure 8 This is a flowchart of an image distortion correction method provided in Example 3 of this application. This example is refined based on the above examples. Figure 8 As shown, the method includes:

[0101] S301: Determine first pixel coordinates to be corrected based on pixel coordinates on a display screen.

[0102] Optionally, the first pixel coordinates to be corrected are determined based on the pixel coordinates in the display screen, including B1-B2:

[0103] B1. When the display screen displays a two-dimensional image, the pixel coordinates on the display screen are used as the first pixel coordinates to be corrected.

[0104] When the display screen displays a two-dimensional image, the size of the display screen is the same as the size of the original image to be displayed, so the pixel coordinates in the display screen can be used as the first pixel coordinates to be corrected.

[0105] B2. When the display screen displays a three-dimensional image, twice the horizontal coordinate of the pixel coordinates in the display screen is used as the horizontal coordinate of the first pixel coordinate to be corrected, and the vertical coordinate of the pixel coordinates in the display screen is used as the vertical coordinate of the first pixel coordinate to be corrected.

[0106] When the display screen displays a three-dimensional image, the size of the display screen is different from the size of the original image to be displayed, so the pixel coordinates in the display screen need to be scaled. When it is necessary to render left and right 3D images, the distortion correction program needs to process the image in the pixel shader program, scaling the left and right eye images horizontally to half of their original size and combining them into left and right 3D images. Therefore, when performing image distortion correction, if the display screen displays a three-dimensional image, first multiply the horizontal coordinate of the pixel coordinate to restore it to the pixel coordinate before scaling, that is, use twice the horizontal coordinate of the pixel coordinate in the display screen as the horizontal coordinate of the first pixel coordinate to be corrected, and use the vertical coordinate of the pixel coordinate in the display screen as the vertical coordinate of the first pixel coordinate to be corrected.

[0107] S302: If the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture, determine the distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data.

[0108] S303. If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, determine the second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the correction data texture based on the central symmetry of the optical imaging system, and determine the distortion correction data corresponding to the second pixel coordinate based on the pre-stored coordinate correction data.

[0109] When the display screen displays a two-dimensional image, steps S304-S306 are executed to perform image distortion correction; when the display screen displays a three-dimensional image, steps S307-S309 are executed to perform image distortion correction.

[0110] S304 : When the display screen displays a two-dimensional image, determine the texture coordinates corresponding to the first pixel coordinates.

[0111] When the display screen displays a two-dimensional image, the first pixel coordinate is converted into a texture coordinate according to the conversion relationship between pixel coordinates and texture coordinates, and the texture coordinate corresponding to the first pixel coordinate is determined. The texture coordinate is also the UV coordinate.

[0112] S305 : Multiply the difference between the texture coordinates and the first center coordinates by the distortion correction data to obtain first correction coordinates, where the first center coordinates are determined according to the center of the display screen.

[0113] In this embodiment, the first corrected coordinates can be understood as coordinates corrected using the distortion correction data; the first center coordinates can be understood as the center of the display screen or image. The first center coordinates are determined based on the center of the display screen. For example, the center of the display screen is used as the first center coordinate. The difference between the texture coordinates and the first center coordinates is calculated and multiplied by the distortion correction data to obtain the first corrected coordinates.

[0114] S306: Taking the sum of the first corrected coordinate and the first center coordinate as the corrected sampling coordinate.

[0115] The first corrected coordinates are added to the first center coordinates, and the sum obtained is used as the corrected sampling coordinates; the corrected sampling coordinates are the coordinates after distortion correction, and pixel information can be obtained from the pixel points at the corresponding position in the original image based on the sampling coordinates and a virtual image is formed through the optical imaging system.

[0116] For example, the present embodiment provides a calculation formula for the corrected sampling coordinates:

[0117] anti_uv = (screen_uv – (0.5, 0.5)) × R + (0.5, 0.5);

[0118] Where anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e., the UV coordinate of the display screen); (0.5, 0.5) is the first center coordinate, and R is the distortion correction data.

[0119] S307 : When the display screen displays a three-dimensional image, determine the texture coordinates corresponding to the first pixel coordinates.

[0120] When the display screen displays a three-dimensional image, the first pixel coordinate is converted into a texture coordinate according to the conversion relationship between pixel coordinates and texture coordinates, and the texture coordinate corresponding to the first pixel coordinate is determined. The texture coordinate is also called a UV coordinate.

[0121] S308: If the texture coordinate is greater than or equal to the set value, multiply the difference between the texture coordinate and the second center coordinate by the distortion correction data to obtain a second corrected coordinate, and use the sum of the second corrected coordinate and the second center coordinate as the corrected sampling coordinate.

[0122] In this embodiment, the second center coordinates can be understood as the center of the display screen or image. For a three-dimensional image, it is actually a single image composed of two images, the left and right halves of the image. Therefore, the centers of the left and right halves of the image are different. The second center coordinates in the embodiment of the present application are the coordinates of the center point of the right half of the image. The second center coordinates are determined based on the horizontal 3 / 4 position of the display screen. The second corrected coordinates can be understood as the coordinates after correction using the distortion correction data.

[0123] A preset value is used to distinguish whether a coordinate point is in the left or right half of the image. If the texture coordinate is greater than or equal to the set value, it is considered to be in the right half of the image. The difference between the texture coordinate and the second center coordinate is calculated and multiplied by the distortion correction data to obtain the second corrected coordinate. The sum of the second corrected coordinate and the second center coordinate is calculated and used as the corrected sampling coordinate.

[0124] Illustratively, an embodiment of the present application provides another calculation formula for the corrected sampling coordinates, with a set value of 0.5. When screen_uv ≥ 0.5, the corrected sampling coordinates are calculated using the following formula.

[0125] anti_uv = (screen_uv – (0.75, 0.5)) × R + (0.75, 0.5);

[0126] Where anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e., the UV coordinate of the display screen); (0.75, 0.5) is the second center coordinate, and R is the distortion correction data.

[0127] S309: If the texture coordinate is less than the set value, multiply the difference between the texture coordinate and the third center coordinate by the distortion correction data to obtain a third corrected coordinate, and use the sum of the third corrected coordinate and the third center coordinate as the corrected sampling coordinate.

[0128] In this embodiment, the third center coordinates can be understood as the center of the display screen or image. The third center coordinates are the coordinates of the center point of the left half of the image. The third center coordinates are determined based on the horizontal 1 / 4 position of the display screen. The third corrected coordinates can be understood as the coordinates corrected using the distortion correction data.

[0129] If the texture coordinate is less than the set value, it can be considered that the texture coordinate is in the left half of the image. The difference between the texture coordinate and the third center coordinate is calculated, and this difference is multiplied by the distortion correction data to obtain the third corrected coordinate; the sum of the third corrected coordinate and the third center coordinate is calculated, and the obtained sum is used as the corrected sampling coordinate.

[0130] Illustratively, the embodiment of the present application provides another calculation formula for the corrected sampling coordinates. The setting value is 0.5. When screen_uv < 0.5, the corrected sampling coordinates are calculated using the following formula.

[0131] anti_uv = (screen_uv – (0.25, 0.5)) × R + (0.25, 0.5);

[0132] Where anti_uv is the corrected sampling coordinate, anti_uv is the UV coordinate; screen_uv is the texture coordinate corresponding to the first pixel coordinate (i.e., the UV coordinate of the display screen); (0.25, 0.5) is the third center coordinate, and R is the distortion correction data.

[0133] The corrected sampling coordinates calculated in the embodiment of the present application are normalized sampling coordinates.

[0134] Optionally, the second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the corrected data texture is determined according to the central symmetry of the optical imaging system, including C1-C6:

[0135] C1. Determine an initial horizontal coordinate and an initial vertical coordinate according to the first pixel coordinate and the width and height of the display screen.

[0136] In this embodiment, the initial horizontal coordinate and the initial vertical coordinate can be understood as the initial coordinates for coordinate conversion. Based on a predetermined formula or determination method, the first pixel coordinate is subjected to coordinate conversion processing according to the width and height of the display screen to obtain the initial horizontal coordinate and the initial vertical coordinate. For example, it is determined whether the first pixel coordinate is on the left or right side of the midline, and whether the first pixel coordinate is above or below the midline, and the first pixel coordinate is mirrored to the upper left corner to obtain the initial horizontal coordinate and the initial vertical coordinate.

[0137] C2. Calculate the difference between the width and height of the display screen, and record half of the difference as the first parameter.

[0138] In this embodiment, the first parameter can be understood as a parameter in the coordinate conversion process to facilitate subsequent coordinate conversion. The width of the display screen is subtracted from the height, and half of the difference is recorded as the first parameter.

[0139] C3. If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than the sum of the initial vertical coordinate and the first parameter, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate.

[0140] Calculate the sum sum1 of the initial ordinate and the first parameter. If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than sum1, use the sum of the initial ordinate and the first parameter as the horizontal coordinate of the second pixel coordinate, and use the difference between the initial horizontal coordinate and the first parameter as the ordinate of the second pixel coordinate.

[0141] C4. If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

[0142] If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to sum1 (sum1 is the sum of the initial vertical coordinate and the first parameter), the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

[0143] C5. If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate.

[0144] Calculate the difference d1 between the initial horizontal coordinate and the first parameter. If the first parameter is less than 0 and d1 is less than the initial vertical coordinate, use the sum of the initial vertical coordinate and the first parameter as the horizontal coordinate of the second pixel coordinate, and use the difference d1 between the initial horizontal coordinate and the first parameter as the vertical coordinate of the second pixel coordinate.

[0145] C6. If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

[0146] If the first parameter is less than 0 and d1 (the difference between the initial horizontal coordinate and the first parameter) is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

[0147] Optionally, an initial horizontal coordinate and an initial vertical coordinate are determined according to the first pixel coordinate and the width and height of the display screen, including D1-D2:

[0148] D1. Determine the coordinates of the pixel to be mapped according to the first pixel coordinates.

[0149] In this embodiment, the pixel coordinates to be mapped can be understood as the coordinates obtained by mapping the first pixel coordinates. When displaying images of different dimensions, the first pixel coordinates are mapped based on the dimensions of the image to obtain the pixel coordinates to be mapped. For example, when displaying a three-dimensional image, the first pixel coordinates are mapped to one side of the image to obtain the pixel coordinates to be mapped.

[0150] D2. Determine the initial horizontal coordinate and the initial vertical coordinate according to the pixel coordinates to be mapped and the width and height of the display screen.

[0151] The coordinates of the pixel to be mapped are converted to initial horizontal and vertical coordinates according to the width and height of the display screen. For example, the pixel to be mapped is determined to be on the left or right side of the center line, or above or below the center line, and the pixel to be mapped is mirrored to the upper left corner to obtain the initial horizontal and vertical coordinates.

[0152] Optionally, the pixel coordinates to be mapped are determined according to the first pixel coordinates, including E1-E2:

[0153] E1. When the display screen displays a two-dimensional image, the first pixel coordinate is used as the pixel coordinate to be mapped.

[0154] When a two-dimensional image is displayed on a display screen, the size of the two-dimensional image is the same as the size of the display screen and does not need to be scaled. Therefore, the first pixel coordinates can be directly used as the pixel coordinates to be mapped.

[0155] E2. When the display screen displays a three-dimensional image, if the horizontal coordinate of the first pixel coordinate is greater than or equal to the width of the display screen, the width of the display screen is subtracted from the horizontal coordinate of the first pixel coordinate as the horizontal coordinate of the pixel coordinate to be mapped, and the vertical coordinate of the first pixel coordinate is used as the vertical coordinate of the pixel coordinate to be mapped; if the horizontal coordinate of the first pixel coordinate is less than the width of the display screen, the first pixel coordinate is used as the pixel coordinate to be mapped.

[0156] When a three-dimensional image is displayed on a display screen, it is necessary to determine the position of a first pixel coordinate on the screen and map it to one side. A determination is made as to whether the horizontal coordinate of the first pixel coordinate is greater than or equal to the width of the display screen. If so, the horizontal coordinate of the pixel coordinate to be mapped is subtracted from the width of the display screen, and the vertical coordinate of the first pixel coordinate is used as the vertical coordinate of the pixel coordinate to be mapped. If not, the first pixel coordinate is used as the pixel coordinate to be mapped.

[0157] Optionally, the initial horizontal coordinate and initial vertical coordinate are determined according to the pixel coordinates to be mapped and the width and height of the display screen, including F1-F2:

[0158] F1. If the ratio of twice the horizontal coordinate X1 of the pixel coordinate to be mapped to the width W of the display screen is greater than or equal to 1, then W-1-X1 is used as the initial horizontal coordinate of the second pixel coordinate; otherwise, X1 is used as the initial horizontal coordinate of the second pixel coordinate.

[0159] F2. If the ratio of twice the vertical coordinate Y1 in the pixel coordinate to be mapped to the height H of the display screen is greater than or equal to 1, then H-1-Y1 is used as the initial vertical coordinate of the second pixel coordinate; otherwise, Y1 is used as the initial vertical coordinate of the second pixel coordinate.

[0160] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate is determined according to the pre-stored coordinate correction data, including G1-G3:

[0161] G1. Use the first pixel coordinate or the second pixel coordinate as the coordinate to be corrected.

[0162] In this embodiment, the principles for determining the distortion correction data corresponding to the first pixel coordinate and the second pixel coordinate are the same. Therefore, when determining the distortion correction data, either the first pixel coordinate or the second pixel coordinate can be used as the coordinate to be corrected for subsequent processing to determine the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate.

[0163] G2. Query a pre-stored coordinate correction image based on the coordinates to be corrected, and determine the values ​​of each pixel color channel of the pixel coordinates corresponding to the coordinates to be corrected in the coordinate correction image.

[0164] When the coordinate correction data is a coordinate correction image, that is, when the coordinate correction data is stored in a coordinate correction image, the coordinate correction image is queried based on the coordinates to be corrected, the pixel coordinates corresponding to the coordinates to be corrected in the coordinate correction image are determined, and then the values ​​of each pixel color channel at these pixel coordinates are determined. The pixel color channels are typically the four RGBA channels.

[0165] G3. Perform data recovery on the color channel values ​​of each pixel to obtain distortion correction data corresponding to the coordinates to be corrected.

[0166] The values ​​of the color channels of each pixel are recombined from low to high and restored to a 4-byte floating point number to obtain the distortion correction data corresponding to the coordinates to be corrected, that is, the distortion correction data corresponding to the first pixel coordinates or the second pixel coordinates are obtained.

[0167] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the first pixel coordinate is determined based on the pre-stored coordinate correction data, including: querying the pre-stored coordinate correction image based on the first pixel coordinate, and determining the values ​​of each pixel color channel of the pixel coordinate corresponding to the first pixel coordinate in the coordinate correction image; performing data recovery on the values ​​of each pixel color channel to obtain the distortion correction data corresponding to the first pixel coordinate.

[0168] Optionally, the coordinate correction data is a coordinate correction image, and the distortion correction data corresponding to the second pixel coordinate is determined based on the pre-stored coordinate correction data, including: querying the pre-stored coordinate correction image based on the second pixel coordinate, and determining the values ​​of each pixel color channel of the pixel coordinate corresponding to the second pixel coordinate in the coordinate correction image; performing data recovery on the values ​​of each pixel color channel to obtain the distortion correction data corresponding to the second pixel coordinate.

[0169] For example, the present invention provides a method for recovering distortion-corrected data:

[0170] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24)).

[0171] Where R is the distortion correction data, uintBitsToFloat is the floating-point conversion function, and data.r, data.g, data.b, and data.a are the values ​​of the four RGBA pixel color channels. The above formula can be used to reassemble the 4-byte data stored in the RGBA channels from low to high bits and restore them to a 4-byte floating-point number.

[0172] For example, the present invention provides a process for image distortion correction:

[0173] During runtime, the distortion correction program reads a coordinate-corrected image, typically a PNG image, that stores the coordinate correction data. This image is then converted into a correction data texture and passed to the GPU's pixel shader. The pixel shader reads and restores the R (r) value based on the unique structure of the correction data. It then calculates the corrected UV coordinates based on the R (r) value. The UV coordinates are the normalized sampling coordinates used to sample color values ​​from the original image texture. This process can be divided into three steps.

[0174] Step 1: The unique structure of the correction data texture is determined by the central symmetry of the optical imaging system. Its width and height should be half the width W and height H of the display screen. Therefore, it is necessary to use the central symmetry relationship to calculate the equivalent pixel coordinates (_x, _y) within the data area of ​​the correction data texture when the display screen pixel coordinates (x, y) exceed the pixel coordinate range of the data area of ​​the correction data texture. In the pixel shader program, the logical calculation process described below can be used to obtain (_x, _y).

[0175] Step 1: Display a two-dimensional image on the screen. The pixel coordinates (x, y) of the screen are directly used as the first pixel coordinates to be corrected, that is, the first pixel coordinates are (x, y). The first pixel coordinates are directly used as the pixel coordinates to be mapped, that is, the pixel coordinates to be mapped are (x, y). The pixel coordinates to be mapped are labeled (X1, Y1), that is, X1=x, Y1=y.

[0176] Step 2: Determine whether 2X1 / W is greater than or equal to 1. If so, the initial horizontal coordinate X2 = W-1-X1; otherwise, the initial horizontal coordinate X2 = X1.

[0177] Step 3: Determine whether 2Y1 / H is greater than or equal to 1. If so, the initial vertical coordinate Y2 = H-1-Y1; otherwise, the initial vertical coordinate Y2 = Y1.

[0178] Step 4: Calculate the first parameter shift, shift = (W – H) / 2.

[0179] Step 5: If shift ≥ 0 and X2>Y2+shift, the second pixel coordinate (_x, _y) = (Y2+shift,X2–shift);

[0180] If shift ≥ 0 and X2 ≤ Y2 + shift, (_x, _y) = (X2, Y2);

[0181] If shift < 0 and X2 – shift < Y2, (_x, _y) = (Y2 + shift, X2 – shift);

[0182] If shift < 0 and X2 – shift ≥ Y2, (_x, _y) = (X2, Y2).

[0183] Step 2: Read the 4-byte data at the pixel coordinate (_x, _y) of the correction data texture and calculate the R value according to the calculation process described below. It reassembles the 4-byte data stored in the RGBA channel from low to high and restores it to a 4-byte floating point number.

[0184] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24))

[0185] Step 3: According to the logic of formula 2, the corrected UV coordinates can be calculated according to the calculation process described below, where screen_uv is the UV coordinate of the display screen, anti_uv is the corrected UV coordinate, and the UV coordinates are the normalized sampling coordinates (i.e., the corrected sampling coordinates).

[0186] anti_uv = (screen_uv – (0.5, 0.5)) × R + (0.5, 0.5)

[0187] The original image is sampled using the corrected UV coordinates and rendered on the display screen, and finally the undistorted image can be seen through the optical imaging system; for example, Figure 9An example diagram of image display is provided. The original image is subjected to distortion correction to obtain a corrected image. The corrected image is formed into an actual viewing image through an optical imaging system, and the user finally views an undistorted image on the display screen.

[0188] For example, the present embodiment provides another process for image distortion correction:

[0189] When rendering left and right 3D images, distortion correction requires a slightly different process in the pixel shader. For left and right 3D images that have been horizontally scaled in half and combined, the following process can be used to calculate the corrected UV coordinates.

[0190] Step 1: Multiply x to restore the original pixel coordinates. Then, using the central symmetry relationship, calculate the equivalent pixel coordinates (_x, _y) within the data area of ​​the correction data texture when the display screen pixel coordinates (x, y) exceed the pixel coordinate range of the correction data texture. In the pixel shader program, the following logical calculation process can be used to obtain (_x, _y).

[0191] Step 1: Display a 3D image on the screen. Scale the screen's pixel coordinates (x, y) to obtain the first pixel coordinates (2x, y) to be corrected. Convert the first pixel coordinates to the pixel coordinates to be mapped. If 2x is greater than or equal to W, the pixel coordinates to be mapped are (2x-W, y). If 2x is less than W, the pixel coordinates to be mapped are (2x, y). Label the pixel coordinates to be mapped as (X1, Y1). That is, when 2x is greater than or equal to W, X1 = 2x-W, and Y1 = y; when 2x is less than W, X1 = 2x, and Y1 = y.

[0192] Step 2: Determine whether 2X1 / W is greater than or equal to 1. If so, the initial horizontal coordinate X2 = W-1-X1; otherwise, the initial horizontal coordinate X2 = X1.

[0193] Step 3: Determine whether 2Y1 / H is greater than or equal to 1. If so, the initial vertical coordinate Y2 = H-1-Y1; otherwise, the initial vertical coordinate Y2 = Y1.

[0194] Step 4: Calculate the first parameter shift, shift = (W – H) / 2.

[0195] Step 5: If shift ≥ 0 and X2>Y2+shift, the second pixel coordinate (_x, _y) = (Y2+shift,X2–shift);

[0196] If shift ≥ 0 and X2 ≤ Y2 + shift, (_x, _y) = (X2, Y2);

[0197] If shift < 0 and X2 – shift < Y2, (_x, _y) = (Y2 + shift, X2 – shift);

[0198] If shift < 0 and X2 – shift ≥ Y2, (_x, _y) = (X2, Y2).

[0199] Step 2: Read the 4-byte data at the pixel coordinate (_x, _y) of the correction data texture and calculate the R value according to the calculation process described below. It reassembles the 4-byte data stored in the RGBA channel from low to high and restores it to a 4-byte floating point number.

[0200] R = uintBitsToFloat(data.r | (data.g << 8) | (data.b << 16) | (data.a<< 24));

[0201] Step 3: Based on the logic of Formula 2, the corrected UV coordinates can be calculated using the following calculation process, where screen_uv is the display screen's UV coordinates, anti_uv is the corrected UV coordinates, and the UV coordinates are the normalized sampling coordinates. For the left half of the image, the third center coordinate is located at the horizontal 1 / 4 of the screen or image; for the right half of the image, the second center coordinate is located at the horizontal 3 / 4 of the screen or image.

[0202] If screen_uv ≥ 0.5, anti_uv = (screen_uv – (0.75, 0.5)) × R + (0.75, 0.5);

[0203] If screen_uv < 0.5, anti_uv = (screen_uv – (0.25, 0.5)) × R + (0.25, 0.5).

[0204] The original image is sampled using the corrected UV coordinates and rendered on the display screen, and finally the undistorted image can be seen through the binocular optical imaging system. For example, Figure 10 Another example of image display is provided. The original image is subjected to distortion correction to obtain a corrected image. The corrected image is then passed through an optical imaging system to form an actual viewing image. The user ultimately views an undistorted image on the display screen.

[0205] An embodiment of the present application provides an image distortion correction method, which solves the problem of high consumption of computing resources and storage resources during the image distortion correction process; if the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, both the two-dimensional image and the three-dimensional image can be subjected to coordinate transformation according to the central symmetry of the optical imaging system to determine the corresponding second pixel coordinate, so as to determine the corresponding coordinate correction data and quickly calculate the corrected sampling coordinates; the image distortion correction method provided by the embodiment of the present application supports the correction of two-dimensional images and three-dimensional images; there is no need to calculate the distortion correction data corresponding to each pixel coordinate in real time, saving computing resources; and the coordinate correction data can be stored through the coordinate correction image, and the corresponding pixels in the coordinate correction image can be read as the values ​​of each color channel during the correction process to restore the distortion correction data, saving storage resources.

[0206] Example 4

[0207] Figure 11 This is a structural diagram of an image distortion correction data storage device provided in the fourth embodiment of the present application. Figure 11 As shown, the device includes: a coordinate determination module 41 to be stored and a coordinate correction data storage module 42.

[0208] A module 41 for determining coordinates to be stored, configured to divide the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored;

[0209] The coordinate correction data storage module 42 is used to form coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and save it into a two-dimensional array. The size of the two-dimensional array is determined according to half of the width and height of the display screen.

[0210] An embodiment of the present application provides an image distortion correction data storage device, which solves the problem of high consumption of computing resources and storage resources in the image distortion correction process. The pixel coordinates on the display screen are divided by the central symmetry of the optical imaging system to obtain the pixel coordinates to be stored. Coordinate correction data is formed based on the distortion correction data corresponding to each pixel coordinate to be stored and saved in a two-dimensional array. The size of the two-dimensional array is determined according to half of the width and height of the display screen. There is no need to store the coordinates of all pixel points in the display screen, which reduces the consumption of storage resources. The distortion correction data of the pixel coordinates are pre-stored to reduce the consumption of computing resources and improve the calculation speed of image distortion correction.

[0211] Optionally, the module 41 for determining coordinates to be stored includes:

[0212] a first dividing unit, configured to, if the width and height of the display screen are equal, divide the display screen into eight regions according to the central symmetry of the optical imaging system, and determine the coordinates of a first candidate pixel point in one of the regions as the pixel coordinates to be stored, wherein the coordinates of the first candidate pixel point have an upper left corner of the display screen as a coordinate origin;

[0213] The second division unit is used to expand the display screen into a square with the maximum value of the width and the height if the width and the height of the display screen are not equal, perform region division based on the diagonal of the expanded square to obtain two regions, and use the region containing the larger number of pixel coordinates of the display screen of the two regions as the region to be stored, and determine the second candidate pixel coordinates in the region to be stored as the pixel coordinates to be stored, and the second candidate pixel coordinates use the upper left corner of the display screen as the coordinate origin.

[0214] Optionally, when the width of the display screen is greater than or equal to the height, the pixel coordinates to be stored meet the following conditions:

[0215] ;

[0216] When the width of the display screen is smaller than the height, the pixel coordinates to be stored meet the following conditions:

[0217] ;

[0218] Wherein, x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

[0219] Optionally, the device further includes:

[0220] A correction image generation module, configured to generate a coordinate correction image based on the two-dimensional array;

[0221] Wherein, each pixel point in the coordinate-corrected image stores the distortion correction data correspondingly, and the distortion correction data is a four-byte floating-point number type, and each byte corresponds to the value of each pixel color channel.

[0222] The image distortion correction data storage device provided in the embodiment of the present application can execute the image distortion correction data storage method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0223] Example 5

[0224] Figure 12 This is a structural diagram of an image distortion correction device provided in Example 5 of this application. Figure 12As shown, the device includes: a first pixel coordinate determination module 51, a first distortion correction data determination module 52, a second distortion correction data determination module 53 and a distortion correction determination module 54.

[0225] A first pixel coordinate determining module 51 is configured to determine first pixel coordinates to be corrected based on pixel coordinates on the display screen;

[0226] a first distortion correction data determining module 52 for determining, if the first pixel coordinates are within a pixel coordinate range of a data area of ​​the correction data texture, distortion correction data corresponding to the first pixel coordinates based on pre-stored coordinate correction data;

[0227] a second distortion correction data determining module 53 configured to, if the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, determine, based on the central symmetry of the optical imaging system, a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the correction data texture, and determine, based on pre-stored coordinate correction data, distortion correction data corresponding to the second pixel coordinate;

[0228] a distortion correction determination module 54, configured to perform image distortion correction on the first pixel coordinates based on the distortion correction data;

[0229] The coordinate correction data is stored according to the image distortion correction data storage method described in any embodiment of the present application.

[0230] An embodiment of the present application provides an image distortion correction device, which solves the problem of high consumption of computing resources and storage resources during the image distortion correction process. The first pixel coordinates to be corrected are determined according to the pixel coordinates in the display screen, and the coordinate correction data are pre-stored. The distortion correction data corresponding to the first pixel coordinates is determined through the coordinate correction data to achieve image distortion correction of the first pixel coordinates. There is no need to calculate the distortion correction data corresponding to each pixel coordinate in real time, which saves computing resources. In addition, the coordinate correction data in the embodiment of the present application is stored through the image distortion correction data storage method provided in any embodiment of the present application, which can save storage resources and reduce storage resource consumption. Determine whether the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture. If so, determine the distortion correction data corresponding to the first pixel coordinate by directly querying the coordinate correction data. If not, perform coordinate transformation on the first pixel coordinate according to the central symmetry of the optical imaging system to determine its corresponding second pixel coordinate. Then, query the coordinate correction data according to the second pixel coordinate to determine the corresponding distortion correction data. The above method can ensure that when the coordinate correction data only saves the distortion correction data of part of the pixel coordinates, the distortion correction data corresponding to each pixel coordinate in the display screen can be determined, so as to realize rapid distortion correction of the image.

[0231] Optionally, the first pixel coordinate determining module 51 includes:

[0232] a first pixel coordinate determining unit, configured to use the pixel coordinates on the display screen as first pixel coordinates to be corrected when the display screen displays a two-dimensional image;

[0233] The second pixel coordinate determining unit is configured to, when the display screen displays a three-dimensional image, use twice the horizontal coordinate of the pixel coordinates in the display screen as the horizontal coordinate of the first pixel coordinate to be corrected, and use the vertical coordinate of the pixel coordinates in the display screen as the vertical coordinate of the first pixel coordinate to be corrected.

[0234] Optionally, the second distortion correction data determination module 53 includes:

[0235] an initial coordinate determining unit, configured to determine an initial horizontal coordinate and an initial vertical coordinate according to the first pixel coordinate and the width and height of the display screen;

[0236] a first parameter calculation unit, configured to calculate a difference between a width and a height of the display screen, and record half of the difference as a first parameter;

[0237] a third pixel coordinate determining unit, configured to, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than the sum of the initial vertical coordinate and the first parameter, use the sum of the initial vertical coordinate and the first parameter as the horizontal coordinate of the second pixel coordinate, and use the difference between the initial horizontal coordinate and the first parameter as the vertical coordinate of the second pixel coordinate;

[0238] a fourth pixel coordinate determining unit, configured to, if the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, use the initial horizontal coordinate as the horizontal coordinate of the second pixel coordinate and use the initial vertical coordinate as the vertical coordinate of the second pixel coordinate;

[0239] a fifth pixel coordinate determining unit, configured to, if the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, use the sum of the initial vertical coordinate and the first parameter as the horizontal coordinate of the second pixel coordinate, and use the difference between the initial horizontal coordinate and the first parameter as the vertical coordinate of the second pixel coordinate;

[0240] The sixth pixel coordinate determination unit is used to use the initial horizontal coordinate as the horizontal coordinate of the second pixel coordinate and the initial vertical coordinate as the vertical coordinate of the second pixel coordinate if the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate.

[0241] Optionally, the initial coordinate determining unit is specifically configured to: determine the pixel coordinates to be mapped according to the first pixel coordinates; and determine the initial horizontal coordinate and the initial vertical coordinate according to the pixel coordinates to be mapped and the width and height of the display screen.

[0242] Optionally, determining the pixel coordinates to be mapped according to the first pixel coordinates includes:

[0243] When the display screen displays a two-dimensional image, the first pixel coordinates are used as pixel coordinates to be mapped;

[0244] When the display screen displays a three-dimensional image, if the horizontal coordinate in the first pixel coordinate is greater than or equal to the width of the display screen, the width of the display screen is subtracted from the horizontal coordinate in the first pixel coordinate as the horizontal coordinate of the pixel coordinate to be mapped, and the vertical coordinate in the first pixel coordinate is used as the vertical coordinate of the pixel coordinate to be mapped; if the horizontal coordinate in the first pixel coordinate is less than the width of the display screen, the first pixel coordinate is used as the pixel coordinate to be mapped.

[0245] Optionally, determining the initial horizontal coordinate and the initial vertical coordinate according to the pixel coordinates to be mapped and the width and height of the display screen includes:

[0246] If the ratio of twice the horizontal coordinate X1 in the pixel coordinate to be mapped to the width W of the display screen is greater than or equal to 1, W-1-X1 is used as the initial horizontal coordinate of the second pixel coordinate; otherwise, X1 is used as the initial horizontal coordinate of the second pixel coordinate;

[0247] If the ratio of twice the vertical coordinate Y1 in the pixel coordinate to be mapped to the height H of the display screen is greater than or equal to 1, H-1-Y1 is used as the initial vertical coordinate of the second pixel coordinate; otherwise, Y1 is used as the initial vertical coordinate of the second pixel coordinate.

[0248] Optionally, the coordinate correction data is a coordinate correction image, and determining the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate according to the pre-stored coordinate correction data includes:

[0249] Using the first pixel coordinate or the second pixel coordinate as the coordinate to be corrected;

[0250] Querying a pre-stored coordinate correction image based on the coordinates to be corrected, and determining the values ​​of each pixel color channel of the pixel coordinates corresponding to the coordinates to be corrected in the coordinate correction image;

[0251] The values ​​of the color channels of each pixel are restored to obtain the distortion correction data corresponding to the coordinates to be corrected.

[0252] Optionally, when the display screen displays a two-dimensional image, the distortion correction determination module 54 is specifically used to: determine the texture coordinates corresponding to the first pixel coordinates; multiply the difference between the texture coordinates and the first center coordinates by the distortion correction data to obtain first correction coordinates, where the first center coordinates are determined based on the center of the display screen; and use the sum of the first correction coordinates and the first center coordinates as the corrected sampling coordinates.

[0253] Optionally, when the display screen displays a three-dimensional image, the distortion correction determination module 54 is specifically used to: determine the texture coordinates corresponding to the first pixel coordinates; if the texture coordinates are greater than or equal to a set value, multiply the difference between the texture coordinates and the second center coordinates by the distortion correction data to obtain a second corrected coordinate, and use the sum of the second corrected coordinates and the second center coordinates as the corrected sampling coordinates; if the texture coordinates are less than the set value, multiply the difference between the texture coordinates and the third center coordinates by the distortion correction data to obtain a third corrected coordinate, and use the sum of the third corrected coordinates and the third center coordinates as the corrected sampling coordinates; wherein, the second center coordinates are determined based on the 3 / 4 position in the horizontal direction of the display screen, and the third center coordinates are determined based on the 1 / 4 position in the horizontal direction of the display screen.

[0254] The image distortion correction device provided in the embodiments of the present application can execute the image distortion correction method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0255] Example 6

[0256] Figure 13 A schematic diagram of the structure of an electronic device provided for Example 6 of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0257] like Figure 13As shown, electronic device 60 includes at least one processor 61 and memory, such as read-only memory (ROM) 62 and random access memory (RAM) 63, communicatively connected to at least one processor 61. The memory stores computer programs executable by the at least one processor. Processor 61 can perform various appropriate actions and processes based on the computer programs stored in ROM 62 or loaded from storage unit 68 into RAM 63. RAM 63 can also store various programs and data required for the operation of electronic device 60. Processor 61, ROM 62, and RAM 63 are interconnected via bus 64. An input / output (I / O) interface 65 is also connected to bus 64.

[0258] Multiple components in the electronic device 60 are connected to the I / O interface 65, including an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a magnetic disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0259] Processor 61 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 61 executes the various methods and processes described above, such as the image distortion correction data storage method or the image distortion correction method.

[0260] In some embodiments, the image distortion correction data storage method or the image distortion correction method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the image distortion correction data storage method or the image distortion correction method described above may be performed. Alternatively, in other embodiments, processor 61 may be configured to execute the image distortion correction data storage method or the image distortion correction method via any other suitable means (e.g., via firmware).

[0261] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0262] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0263] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the image distortion correction data storage method or the image distortion correction method described in any embodiment of the present application.

[0264] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0265] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0266] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0267] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0268] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0269] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for storing image distortion correction data, characterized in that: include: Dividing the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored; Coordinate correction data is formed based on the distortion correction data corresponding to each pixel coordinate to be stored and saved in a two-dimensional array, where the size of the two-dimensional array is determined according to half of the width and height of the display screen.

2. The method according to claim 1, characterized in that The dividing of pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored includes: If the width and height of the display screen are equal, the display screen is divided into 8 areas according to the central symmetry of the optical imaging system, and the coordinates of a first candidate pixel point in one of the areas are determined as the pixel coordinates to be stored, where the coordinates of the first candidate pixel point have the upper left corner of the display screen as the coordinate origin; If the width and height of the display screen are not equal, the display screen is expanded into a square with the maximum value of the width and the height, and the region is divided based on the diagonal of the expanded square to obtain two regions. The region containing the larger number of pixel coordinates of the display screen of the two regions is used as the region to be stored, and the second candidate pixel coordinates in the region to be stored are determined as the pixel coordinates to be stored, and the second candidate pixel coordinates use the upper left corner of the display screen as the coordinate origin.

3. The method according to claim 2, characterized in that When the width of the display screen is greater than or equal to the height, the pixel coordinates to be stored meet the following conditions: ； When the width of the display screen is smaller than the height, the pixel coordinates to be stored meet the following conditions: ； Wherein, x is the horizontal coordinate of the pixel coordinate to be stored, y is the vertical coordinate of the pixel coordinate to be stored, W is the width of the display screen, and H is the height of the display screen.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: generating a coordinate-rectified image based on the two-dimensional array; Wherein, each pixel point in the coordinate-corrected image stores the distortion correction data correspondingly, and the distortion correction data is a four-byte floating-point number type, and each byte corresponds to the value of each pixel color channel.

5. A method for correcting image distortion, characterized in that: include: determining first pixel coordinates to be corrected based on pixel coordinates in the display screen; If the first pixel coordinate is within the pixel coordinate range of the data area of ​​the correction data texture, determining the distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data; If the first pixel coordinate is not within the pixel coordinate range of the data area of ​​the correction data texture, determining a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data area of ​​the correction data texture according to the central symmetry of the optical imaging system, and determining distortion correction data corresponding to the second pixel coordinate according to pre-stored coordinate correction data; performing image distortion correction on the first pixel coordinates based on the distortion correction data; Wherein, the coordinate correction data is stored according to the image distortion correction data storage method according to any one of claims 1-4.

6. The method according to claim 5, characterized in that The determining of the first pixel coordinates to be corrected based on the pixel coordinates in the display screen includes: When the display screen displays a two-dimensional image, taking pixel coordinates on the display screen as first pixel coordinates to be corrected; When the display screen displays a three-dimensional image, twice the horizontal coordinate of the pixel coordinates in the display screen is used as the horizontal coordinate of the first pixel coordinate to be corrected, and the vertical coordinate of the pixel coordinates in the display screen is used as the vertical coordinate of the first pixel coordinate to be corrected.

7. The method according to claim 5, characterized in that The determining, based on the central symmetry of the optical imaging system, a second pixel coordinate corresponding to the first pixel coordinate within a pixel coordinate range of a data area of ​​the corrected data texture includes: determining an initial horizontal coordinate and an initial vertical coordinate according to the first pixel coordinate and the width and height of the display screen; Calculating the difference between the width and height of the display screen, and recording half of the difference as a first parameter; If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is greater than the sum of the initial vertical coordinate and the first parameter, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate; If the first parameter is greater than or equal to 0 and the initial horizontal coordinate is less than or equal to the sum of the initial vertical coordinate and the first parameter, use the initial horizontal coordinate as the horizontal coordinate of the second pixel coordinate, and use the initial vertical coordinate as the vertical coordinate of the second pixel coordinate; If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is less than the initial vertical coordinate, the sum of the initial vertical coordinate and the first parameter is used as the horizontal coordinate of the second pixel coordinate, and the difference between the initial horizontal coordinate and the first parameter is used as the vertical coordinate of the second pixel coordinate; If the first parameter is less than 0 and the difference between the initial horizontal coordinate and the first parameter is greater than or equal to the initial vertical coordinate, the initial horizontal coordinate is used as the horizontal coordinate of the second pixel coordinate, and the initial vertical coordinate is used as the vertical coordinate of the second pixel coordinate.

8. The method according to claim 7, characterized in that The determining of the initial horizontal coordinate and the initial vertical coordinate according to the first pixel coordinate and the width and height of the display screen includes: determining the coordinates of the pixel to be mapped according to the first pixel coordinates; An initial horizontal coordinate and an initial vertical coordinate are determined according to the pixel coordinates to be mapped and the width and height of the display screen.

9. The method according to claim 8, characterized in that The determining the pixel coordinates to be mapped according to the first pixel coordinates includes: When the display screen displays a two-dimensional image, the first pixel coordinates are used as pixel coordinates to be mapped; When the display screen displays a three-dimensional image, if the horizontal coordinate in the first pixel coordinate is greater than or equal to the width of the display screen, the width of the display screen is subtracted from the horizontal coordinate in the first pixel coordinate as the horizontal coordinate of the pixel coordinate to be mapped, and the vertical coordinate in the first pixel coordinate is used as the vertical coordinate of the pixel coordinate to be mapped; if the horizontal coordinate in the first pixel coordinate is less than the width of the display screen, the first pixel coordinate is used as the pixel coordinate to be mapped.

10. The method according to claim 8, characterized in that The determining of the initial horizontal coordinate and the initial vertical coordinate according to the pixel coordinates to be mapped and the width and height of the display screen includes: If the ratio of twice the horizontal coordinate X1 in the pixel coordinate to be mapped to the width W of the display screen is greater than or equal to 1, W-1-X1 is used as the initial horizontal coordinate of the second pixel coordinate; otherwise, X1 is used as the initial horizontal coordinate of the second pixel coordinate; If the ratio of twice the vertical coordinate Y1 in the pixel coordinate to be mapped to the height H of the display screen is greater than or equal to 1, H-1-Y1 is used as the initial vertical coordinate of the second pixel coordinate; otherwise, Y1 is used as the initial vertical coordinate of the second pixel coordinate.

11. The method according to claim 5, characterized in that The coordinate correction data is a coordinate correction image, and determining the distortion correction data corresponding to the first pixel coordinate or the second pixel coordinate according to the pre-stored coordinate correction data includes: Using the first pixel coordinate or the second pixel coordinate as the coordinate to be corrected; Querying a pre-stored coordinate correction image based on the coordinates to be corrected, and determining the values ​​of each pixel color channel of the pixel coordinates corresponding to the coordinates to be corrected in the coordinate correction image; The values ​​of the color channels of each pixel are restored to obtain the distortion correction data corresponding to the coordinates to be corrected.

12. The method according to claim 5, characterized in that When the display screen displays a two-dimensional image, performing image distortion correction on the first pixel coordinates based on the distortion correction data includes: determining texture coordinates corresponding to the first pixel coordinates; multiplying the difference between the texture coordinates and the first center coordinates by the distortion correction data to obtain first correction coordinates, where the first center coordinates are determined based on the center of the display screen; The sum of the first corrected coordinate and the first center coordinate is used as the corrected sampling coordinate.

13. The method according to claim 5, characterized in that When the display screen displays a three-dimensional image, performing image distortion correction on the first pixel coordinates based on the distortion correction data includes: determining texture coordinates corresponding to the first pixel coordinates; If the texture coordinate is greater than or equal to a set value, multiplying the difference between the texture coordinate and the second center coordinate by the distortion correction data to obtain a second corrected coordinate, and using the sum of the second corrected coordinate and the second center coordinate as the corrected sampling coordinate; If the texture coordinate is less than a set value, multiplying the difference between the texture coordinate and the third center coordinate by the distortion correction data to obtain a third corrected coordinate, and using the sum of the third corrected coordinate and the third center coordinate as the corrected sampling coordinate; The second center coordinates are determined based on a ¾ position in the horizontal direction of the display screen, and the third center coordinates are determined based on a ¼ position in the horizontal direction of the display screen.

14. An image distortion correction data storage device, characterized in that: include: A module for determining coordinates to be stored, configured to divide the pixel coordinates on the display screen according to the central symmetry of the optical imaging system to determine the pixel coordinates to be stored; A coordinate correction data storage module is used to form coordinate correction data based on the distortion correction data corresponding to each pixel coordinate to be stored and save it into a two-dimensional array, where the size of the two-dimensional array is determined according to half the width and height of the display screen.

15. An image distortion correction device, characterized in that: include: A first pixel coordinate determining module, configured to determine first pixel coordinates to be corrected based on pixel coordinates on the display screen; a first distortion correction data determining module, configured to determine, if the first pixel coordinate is within a pixel coordinate range of a data area of ​​a correction data texture, distortion correction data corresponding to the first pixel coordinate according to pre-stored coordinate correction data; a second distortion correction data determining module configured to, if the first pixel coordinate is not within a pixel coordinate range of a data region of the correction data texture, determine, based on the central symmetry of the optical imaging system, a second pixel coordinate corresponding to the first pixel coordinate within the pixel coordinate range of the data region of the correction data texture, and determine, based on pre-stored coordinate correction data, distortion correction data corresponding to the second pixel coordinate; a distortion correction determination module, configured to perform image distortion correction on the first pixel coordinates based on the distortion correction data; Wherein, the coordinate correction data is stored according to the image distortion correction data storage method according to any one of claims 1-4.

16. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively coupled to the at least one processor; In which, the memory stores a computer program executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image distortion correction data storage method described in any one of claims 1-4 or the image distortion correction method described in any one of claims 5-13.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image distortion correction data storage method according to any one of claims 1 to 4 or the image distortion correction method according to any one of claims 5 to 13 when executed.

18. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the image distortion correction data storage method according to any one of claims 1 to 4 or the image distortion correction method according to any one of claims 5 to 13.

Images