Image distortion correction method and device
By dividing pixel rows and filling the images captured by the roller shutter shutter camera, and correcting and computing them in combination with target correction parameters, the time-consuming problem of image correction process in the prior art is solved and the correction efficiency is improved.
Patent Information
- Application Number
- CN202311397426.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-25
- Publication Date
- 2025-04-29
AI Technical Summary
When correcting the original image taken by a roller shutter shutter camera, the prior art needs to wait for all pixel lines of the entire image to be read into memory, resulting in a time-consuming and inefficient correction process.
By dividing the pixel rows of the image to be corrected, multiple original pixel blocks are acquired, and pixel-filled on these blocks based on the maximum offset distance to obtain the initial pixel block. Then, the initial pixel block is corrected using the target correction parameters to output the target pixel block.
Corrections are started without waiting for all pixel lines to be read into memory, which improves the efficiency of image correction.
Smart Images
Figure CN119904392A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image distortion correction method and device. Background Art
[0002] In the prior art, when the original image is subjected to distortion correction, the original pixels adjacent to each output pixel position are read from the memory according to the pre-calibrated distortion correction parameters, and the corrected pixel points are obtained after interpolation, and then the corrected image is obtained by splicing.
[0003] However, when the above correction method is used to correct the original image taken by a rolling shutter camera, since the rolling shutter camera writes the pixel rows of the original image into the memory one by one, then when correcting each pixel, it is necessary to wait for all the pixel rows of the entire original image to be read into the memory before the original pixels adjacent to each pixel can be read from the memory for correction. Therefore, when correcting the original image taken by the rolling shutter camera, it is necessary to ensure that each pixel row of the original image is written into the memory before the correction can be started, which makes the image correction process time-consuming and inefficient. Summary of the invention
[0004] In view of this, an embodiment of the present application provides an image distortion correction method and device for improving the efficiency of correcting distorted images.
[0005] In a first aspect, an embodiment of the present application provides an image distortion correction method, comprising:
[0006] Dividing at least one pixel row of the image to be corrected to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along a pixel column direction;
[0007] Filling pixels around the multiple original pixel blocks based on a maximum offset distance to obtain multiple initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected;
[0008] Obtain target correction parameters respectively corresponding to the multiple initial pixel blocks, and for each of the multiple initial pixel blocks, use the target correction parameters to perform correction calculation on the initial pixel block to obtain and output target pixel blocks respectively corresponding to the multiple initial pixel blocks.
[0009] As an optional implementation of the embodiment of the present application, the step of performing pixel filling around the multiple original pixel blocks based on the maximum offset distance to obtain multiple initial pixel blocks includes:
[0010] Obtaining the maximum offset distance of pixels in the image to be corrected;
[0011] Acquire the number of target pixels based on the maximum offset distance;
[0012] Add pixel layers having the target pixel number around the multiple original pixel blocks respectively to obtain the multiple initial pixel blocks.
[0013] As an optional implementation of the embodiment of the present application, the obtaining of correction parameters corresponding to the plurality of target pixel blocks respectively includes:
[0014] Acquire position information of the plurality of initial pixel blocks;
[0015] Acquire the original correction parameter corresponding to the position information based on a preset corresponding relationship; the preset corresponding relationship includes the corresponding relationship between each position information and the original correction parameter corresponding to each position information;
[0016] The original correction parameters are calculated to obtain the target correction parameters.
[0017] As an optional implementation of the embodiment of the present application, before obtaining the position information of the multiple initial pixel blocks, the method further includes:
[0018] Performing correction analysis on multiple original pixel blocks of the test image to obtain original correction parameters corresponding to the multiple original pixel blocks;
[0019] The initial correction parameters of the plurality of original pixel blocks and the position information of the plurality of original pixel blocks are correspondingly saved to generate the preset corresponding relationship.
[0020] As an optional implementation of the embodiment of the present application, the target correction parameters corresponding to the multiple initial pixel blocks are obtained, based on the target correction parameters corresponding to the multiple initial pixel blocks.
[0021] Correction parameters, correcting the multiple initial pixel blocks, include:
[0022] With respect to the original correction parameters respectively corresponding to the multiple initial pixel blocks, target correction parameters respectively corresponding to the multiple initial pixel blocks are obtained by using a preset interpolation algorithm.
[0023] As an optional implementation of the embodiment of the present application, after obtaining and outputting target pixel blocks respectively corresponding to the multiple initial pixel blocks, the method further includes:
[0024] splicing the multiple target pixel blocks to obtain a complete pixel block composed of the multiple target pixel blocks;
[0025] Each pixel row of the complete pixel block is obtained and outputted.
[0026] In a second aspect, an embodiment of the present application provides an image distortion correction device, comprising:
[0027] A dividing unit, used for dividing at least one pixel row of the image to be corrected to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along a pixel column direction;
[0028] A filling unit, configured to fill pixels around the plurality of original pixel blocks based on a maximum offset distance to obtain a plurality of initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected;
[0029] An acquisition unit, used for acquiring target correction parameters respectively corresponding to the plurality of initial pixel blocks;
[0030] The correction unit is used to perform correction calculation on each of the multiple initial pixel blocks using the target correction parameter to obtain and output target pixel blocks corresponding to the multiple initial pixel blocks respectively.
[0031] As an optional implementation of the embodiment of the present application, the filling unit is specifically used to obtain the maximum offset distance of the pixel points in the image to be corrected; obtain the target number of pixels based on the maximum offset distance; and add pixel layers with the target number of pixels around the multiple original pixel blocks to obtain the multiple initial pixel blocks.
[0032] As an optional implementation of the embodiment of the present application, the correction parameters are specifically used to obtain the position information of the multiple initial pixel blocks; based on a preset correspondence relationship, the original correction parameters corresponding to the position information are obtained; the preset correspondence relationship includes the correspondence between each position information and the original correction parameters corresponding to each position information; the original correction parameters are calculated to obtain the target correction parameters.
[0033] As an optional implementation of the embodiment of the present application, the acquisition unit is specifically used to perform correction analysis on multiple original pixel blocks of the test image to obtain original correction parameters corresponding to the multiple original pixel blocks; and save the initial correction parameters of the multiple original pixel blocks and the position information of the multiple original pixel blocks in correspondence to generate the preset corresponding relationship.
[0034] As an optional implementation of the embodiment of the present application, the correction unit is specifically used to obtain target correction parameters corresponding to the multiple initial pixel blocks respectively through a preset interpolation algorithm for the original correction parameters corresponding to the multiple initial pixel blocks respectively.
[0035] As an optional implementation of the embodiment of the present application, the correction unit is also used to splice the multiple target pixel blocks to obtain a complete pixel block composed of the multiple target pixel blocks; obtain each pixel row of the complete pixel block and output it.
[0036] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the image distortion correction method described in any one of the above embodiments when executing the computer program.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computing device, the computing device implements the image distortion correction method described in any of the above embodiments.
[0038] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising: the image distortion correction device described in the second aspect or the electronic device described in the third aspect.
[0039] The image distortion correction method provided in the embodiment of the present application is specifically as follows: at least one pixel row of the image to be corrected is divided to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along the pixel column direction; pixels are filled around the plurality of original pixel blocks based on a maximum offset distance to obtain a plurality of initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected; target correction parameters corresponding to the plurality of initial pixel blocks are obtained, and for each of the plurality of initial pixel blocks, the target correction parameters are used to perform correction calculations on the initial pixel blocks to obtain target pixel blocks corresponding to the plurality of initial pixel blocks and output them. The embodiment of the present application obtains at least one pixel row in the image to be corrected, divides the pixel row, obtains a plurality of original pixel blocks, and then performs pixel filling around the standard pixel block to obtain the initial pixel block, and then uses the target correction parameter to start correcting the initial pixel point, that is, the image to be corrected is corrected step by step according to the pixel blocks, and there is no need to wait until all the pixel rows of the image to be corrected are read before starting to correct the image to be corrected. Therefore, the embodiment of the present application can be used to improve the efficiency of correcting the image. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0042] Figure 1 One of the step flow charts of the image distortion correction method provided in the embodiment of the present application;
[0043] Figure 2 One of the schematic diagrams of the image distortion correction method provided in the embodiment of the present application;
[0044] Figure 3 A second schematic diagram of the image distortion correction method provided in an embodiment of the present application;
[0045] Figure 4 The third schematic diagram of the image distortion correction method provided in the embodiment of the present application;
[0046] Figure 5 The second step flow chart of the image distortion correction method provided in the embodiment of the present application;
[0047] Figure 6 A schematic diagram of the structure of an image distortion correction device provided in an embodiment of the present application;
[0048] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0050] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0051] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In addition, in the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more.
[0052] It should be noted that, in this article, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0053] In the prior art, when correcting the original image taken by a rolling shutter camera, since the rolling shutter camera writes the pixel rows of the original image into the memory one by one, then when correcting each pixel, it is necessary to wait for all the pixel rows of the entire original image to be read into the memory before reading the original pixels adjacent to each pixel from the memory for correction. Therefore, when correcting the original image taken by the rolling shutter camera, it is necessary to ensure that each pixel row of the original image is written into the memory before correction can begin, making the image correction process time-consuming and inefficient. In order to overcome the above problems, the present application proposes an image distortion correction method, and the specific embodiments are as follows:
[0054] The present application embodiment provides an image distortion correction method, referring to Figure 1 As shown, the image distortion correction method includes the following steps S101-S104:
[0055] S101. Divide at least one pixel row of an image to be corrected to obtain a plurality of original pixel blocks.
[0056] The multiple original pixel blocks are obtained by dividing the at least one pixel row along the pixel column direction.
[0057] In some embodiments, the image to be corrected is acquired by photographing with a rolling shutter camera; specifically, different cameras adopt different exposure methods (which can be understood as the way in which the photosensitive element senses light), which can be divided into rolling shutter and global shutter. Since most image sensors on the market use CMOS (a semiconductor field effect transistor) as a material, and the cost of a global shutter is too high, manufacturers use mostly rolling shutters for process and cost considerations.
[0058] In some embodiments, the rolling shutter camera adopts a rolling shutter mode. In the rolling shutter mode, as the readout "wave" sweeps across the sensor, different rows of the array are exposed at different times, the first row is exposed first, and after the readout time, the second row begins to be exposed, and so on. Therefore, after each row is read out, the next row can be read. Each pixel unit of the rolling shutter sensor only needs two transistors to transmit electrons, so less heat is generated and the noise is low. Compared with the global shutter sensor, the rolling shutter sensor has a simpler structure and lower cost, but because each row is not exposed at the same time, distortion will occur when capturing high-speed moving objects. Electronic products on the market basically adopt the rolling shutter mode for cost and imaging effect considerations.
[0059] Specifically, refer to Figure 2 As shown in the figure, it is a schematic diagram of a rolling shutter camera reading pixel rows. The rolling shutter camera can only read one row of pixels at a time, that is, Figure 2 After the first pixel row is read, the next pixel row is read. Figure 2 The second pixel row shown in FIG. B is read in this way, and so on, until the last pixel row is read. In the prior art, all pixel rows need to be written into a memory, and all pixel rows are combined into a complete image in the memory. Then, during correction, the pixels are read one by one to correct and repair the pixels. In the present application, after reading at least one pixel row, the pixel row is divided into a plurality of pixel blocks, and correction is performed on the plurality of pixel blocks. For example, Figure 2 The first pixel row shown in Figure A can be divided into pixel blocks and then image correction is performed, without having to start image correction after all pixel rows are acquired.
[0060] In some embodiments, the image taken by the rolling shutter camera can be analyzed in advance to observe the distortion of the image, and the image to be corrected can be divided into a plurality of sub-pixel blocks according to the distortion. Since the rolling shutter camera acquires pixel rows one by one, when a preset number of pixel rows need to be read, a plurality of small pixel blocks, i.e., the plurality of standard pixel blocks, can be acquired by dividing the pixel columns after acquiring four pixel rows. For example, a plurality of standard pixel blocks can be acquired by dividing the pixel columns.
[0061] In some embodiments, a row counter can be used to count the pixel rows written into the cache. The counter increases by one each time a pixel row is stored in the cache space. A threshold can be set for the counter. When the pixel rows stored in the cache space reaches the threshold, the counter can be divided into a threshold number of pixel rows.
[0062] S102: Fill pixels around the multiple original pixel blocks based on the maximum offset distance to obtain multiple initial pixel blocks.
[0063] The maximum offset distance is obtained by analyzing a test image; and the test image is obtained by the same target shooting device before correcting the image to be corrected.
[0064] In some embodiments, the test image is usually an image obtained by photographing a chessboard by the target photographing device, so as to better highlight the image distortion.
[0065] In some embodiments, when the size of the image to be corrected is 100*100 (unit: pixel), assuming that the image to be corrected does not produce any distortion, the image to be corrected can be evenly divided into 100 small pixel blocks of 10*10, but in fact distortion will occur. Since the camera parameters are calibrated, the distortion of the image is also fixed. Figure 3 As shown, the outermost rectangle 31 is a standard undistorted image, and the curved area 32 is a distorted image, i.e., the image to be corrected; it can be seen that the square pixel block 33 that was originally supposed to be found has a pixel shift due to the distortion, resulting in a pixel block area shown by the strip shadow. For example, the original pixel point should be at the position of 341, but due to the distortion, it has shifted to the position of 342; the present application utilizes the maximum offset distance among the pixel points to fill in the four sides of the original 10*10 pixel block, so that during the subsequent correction, all the pixel points in the distorted pixel block generated by the pixel shift can be completely obtained by the distortion of the pixel points in the original standard undistorted pixel block.
[0066] Specifically, when filling, the pixel block is filled according to the maximum offset distance, and the pixel range of the original pixel block is adjusted. For example, when the maximum offset distance corresponds to the size of 2 pixels, the original 10*10 pixel block can be filled to 12*12. The filled pixel block can be obtained by obtaining adjacent pixel points that meet the maximum offset distance. If there are no adjacent pixel points around the original pixel block, it can be filled with blank pixel points. After filling, the initial pixel block corresponding to the original pixel block can be obtained. Exemplary, refer to Figure 4 As shown, it is a schematic diagram of the multiple initial pixel blocks obtained after the multiple original pixel blocks are filled. After the pixels are filled around the original pixel blocks, the initial pixel blocks include all the pixel points offset and distorted by the original pixel blocks 41.
[0067] In some embodiments, the pixels of the test image are analyzed to obtain a maximum offset distance of the pixels in the test image, where the maximum offset distance is the pixel with the largest offset distance among the pixels of the test image obtained relative to the calibration image.
[0068] Optionally, the specific method for obtaining the maximum offset distance may be: using the calibrated internal parameters of the rolling camera to correct the test image to obtain a calibration image containing the calibration, and obtaining the offset value of the coordinates of each pixel point in the calibration image relative to the coordinates of the pixel point in the original image; and then comparing the offset values of each pixel to obtain the pixel point with the largest offset value among the pixels, and using the corresponding offset value as the maximum offset distance.
[0069] It should be noted that, when filling the multiple original pixel blocks, the embodiment of the present application may use the maximum offset distance as the standard for filling pixels, and may also be greater than the maximum offset distance, and the present application does not impose any limitation on this.
[0070] S103: Obtain target correction parameters corresponding to the multiple initial pixel blocks respectively.
[0071] In some embodiments, the target correction parameters corresponding to the multiple initial pixel blocks are calculated based on the original correction parameters obtained in advance when the test image is corrected and analyzed. The original correction parameters need to be obtained first according to the test image. Specifically, different shooting devices will cause different degrees of distortion to the image, mainly radial deformation and tangential deformation, which causes the straight line to become curved. The farther the pixel is from the center of the image, the more serious the distortion. In order to obtain a standard and correct image, these distortions need to be corrected first. To correct the distortion, in order to find the original correction parameters, the camera needs to be calibrated according to the pre-defined test image to obtain the corresponding correction parameters.
[0072] Specifically, the test image can be calibrated and acquired using Zhang's calibration method; the Zhang's calibration method is a practical method for camera calibration using a plane chessboard. This method is between the photographic calibration method and the self-calibration method. It not only overcomes the shortcomings of the high-precision three-dimensional calibration object required by the photographic calibration method, but also solves the problem of poor robustness of the self-calibration method. The calibration process only requires the use of a printed chessboard and taking several groups of pictures from different directions (theoretically, 3 groups can solve all unknowns, but usually 15-20 groups of pictures are taken, and an optimized method is used to ensure a high solution accuracy). Anyone can make a calibration pattern by themselves, which is not only practical, flexible and convenient, but also has high accuracy and good robustness. Compared with other calibration methods, Zhang's calibration method is simple to operate and low cost while ensuring a certain accuracy and robustness.
[0073] Specifically, the overall idea of Zhang's calibration method is to first find the homography matrix from the world coordinate system to the pixel coordinate system (the homography matrix is actually the transformation matrix from the pixel points in one image to the pixel points in another image. There is also a homography matrix in the binocular camera system, which is the transformation matrix between the left and right camera images). Then, the intrinsic parameter matrix is obtained according to the homography matrix, and finally the extrinsic parameter matrix is obtained. Zhang Zhengyou's calibration method uses a checkerboard calibration plate. After obtaining an image of the calibration plate, the corresponding image detection algorithm can be used to obtain the pixel coordinates of each corner point (u ,v), and then fix the world coordinate system on the chessboard, then the physical coordinates of any point on the chessboard (u,v,w=0). Since the world coordinate system of the calibration plate is pre-defined by humans, the size of each grid on the calibration plate is known, and the physical coordinates (u,v) of each corner point in the world coordinate system can be calculated. Using this information: the pixel coordinates (u,v) of each corner point and the physical coordinates (u,v,w=0) of each corner point in the world coordinate system, the camera can be calibrated to obtain the camera's internal and external parameter matrix and correction parameters.
[0074] S104: For each of the multiple initial pixel blocks, use the target correction parameter to perform correction calculation on the initial pixel block to obtain and output target pixel blocks respectively corresponding to the multiple initial pixel blocks.
[0075] In the embodiment of the present application, correction parameters of corresponding areas are obtained according to the range of multiple original pixel blocks, and corresponding position information can be generated based on the pixel block position information, so as to obtain corresponding correction parameters according to the position information of the original pixel blocks. After obtaining the target correction parameters corresponding to the multiple initial pixel blocks respectively, correction is performed based on the correction parameters combined with an interpolation algorithm, so that the corrected pixel blocks corresponding to the multiple initial pixel blocks can be obtained.
[0076] The image distortion correction method provided in the embodiment of the present application is specifically as follows: at least one pixel row of the image to be corrected is divided to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along the pixel column direction; pixels are filled around the plurality of original pixel blocks based on a maximum offset distance to obtain a plurality of initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected; target correction parameters corresponding to the plurality of initial pixel blocks are obtained, and for each of the plurality of initial pixel blocks, the target correction parameters are used to perform correction calculations on the initial pixel blocks to obtain target pixel blocks corresponding to the plurality of initial pixel blocks and output them. The embodiment of the present application obtains at least one pixel row in the image to be repaired, divides the pixel row to obtain a plurality of original pixel blocks, and then fills pixels around the standard pixel block to obtain the initial pixel block. There is no need to wait for all pixel rows of the image to be read before correcting each pixel point of the image. It is only necessary to divide the read pixel rows into pixel blocks and correct the pixel blocks. Therefore, the embodiment of the present application can be used to improve the efficiency of correcting distorted images.
[0077] As an extension and refinement of the above embodiment, the present application embodiment provides an image distortion correction method, referring to Figure 5 As shown, the image distortion correction method includes the following steps S501-S507:
[0078] S501 , dividing at least one pixel row of an image to be corrected to obtain a plurality of original pixel blocks.
[0079] The multiple original pixel blocks are obtained by dividing the at least one pixel row along the pixel column direction.
[0080] S502: Acquire the target pixel quantity based on the maximum offset distance.
[0081] In some embodiments, the maximum offset distance is measured, and the minimum number of pixels corresponding to the maximum offset distance is obtained and used as the target number of pixels.
[0082] S503 , adding pixel layers of the target number of pixels around the multiple original pixel blocks respectively to obtain the multiple initial pixel blocks.
[0083] In some embodiments, the above embodiments may be referred to. Figure 4 As shown, it is a schematic diagram corresponding to the initial pixel blocks obtained by adding pixel layers of the target pixel number around the multiple original pixel blocks.
[0084] S504: Obtain position information of the multiple initial pixel blocks.
[0085] In some embodiments, correction parameters of corresponding areas are obtained based on pixel areas where multiple original pixel blocks are located, and corresponding position information is generated based on the positions of the multiple original pixel blocks, so as to obtain corresponding original correction parameters based on the position information of the multiple original pixel blocks, wherein the position information is the coordinates of the four corners of each original pixel block on the test image.
[0086] S505: Acquire original correction parameters corresponding to the position information based on a preset corresponding relationship.
[0087] The preset corresponding relationship includes the corresponding relationship between each position information in the image and the original correction parameter corresponding to each position information.
[0088] After obtaining the original correction parameters respectively corresponding to the multiple initial pixel blocks, processing is performed based on the original correction parameters in combination with an interpolation algorithm to obtain the corrected pixel blocks corresponding to the multiple initial pixel blocks.
[0089] In some embodiments, the method for obtaining the preset corresponding relationship may refer to the following steps a and b:
[0090] Step a: performing correction analysis on a plurality of original pixel blocks of a test image to obtain original correction parameters corresponding to the plurality of original pixel blocks.
[0091] In some embodiments, the test image can be divided according to a preset division strategy to obtain multiple original pixel blocks; wherein the preset division strategy can be to divide the image according to a preset division direction and division size. Exemplarily, the test image can be divided into 20 original pixel blocks of 4 rows and 5 columns by the preset division strategy; then the correction analysis is performed on the multiple original pixel blocks to obtain the initial correction parameters corresponding to each original pixel block, and then the corresponding position information is generated according to the corresponding positions of the multiple original pixel blocks, and the position information of each original pixel block and its corresponding initial correction parameter are saved to generate the preset corresponding relationship.
[0092] It should be noted that the test image and the image to be corrected are acquired by the same camera, and the size of the test image and the image to be corrected are the same. Therefore, when the image correction method is used to correct the image to be corrected, it is only necessary to divide the image to be corrected using the same preset division strategy as the test image to obtain a plurality of initial pixel blocks corresponding to the image to be corrected; illustratively, the preset division strategy can also be used to divide the image to be corrected into 20 initial pixel blocks of 4 rows and 5 columns, and then the corresponding initial correction parameters can be obtained according to the position information corresponding to the positions of the plurality of initial pixel blocks.
[0093] Step b: saving the initial correction parameters of the plurality of original pixel blocks and the position information of the plurality of original pixel blocks in correspondence with each other to generate the preset corresponding relationship.
[0094] S506: Calculate the original correction parameters to obtain the target correction parameters.
[0095] In some embodiments, the above method for correcting the multiple initial pixel blocks may refer to the following steps:
[0096] With respect to the original correction parameters respectively corresponding to the multiple initial pixel blocks, target correction parameters respectively corresponding to the multiple initial pixel blocks are obtained by using a preset interpolation algorithm.
[0097] Specifically, the original correction parameters are generally sparse correction parameters, so the target correction parameters can be densely formed through the preset interpolation algorithm. The preset interpolation algorithm can be a bilinear interpolation algorithm or a bicubic interpolation algorithm, and the embodiment of the present application does not impose any limitation on this.
[0098] S507 . For each of the multiple initial pixel blocks, use the target correction parameter to perform correction calculation on the initial pixel block to obtain and output target pixel blocks respectively corresponding to the multiple initial pixel blocks.
[0099] In some embodiments, after correcting the pixel blocks, the present application can not only output them in the form of pixel blocks, but also obtain pixel rows according to the target pixel blocks corresponding to the multiple initial pixel blocks, and output them in the form of pixel rows. The specific implementation method includes the following steps:
[0100] Step 1: splice the multiple target pixel blocks to obtain a complete pixel block composed of the multiple target pixel blocks.
[0101] Step 2: Obtain and output each pixel row of the complete pixel block.
[0102] In some embodiments, the pixel rows of the corrected image can be input into a downstream image processing module, such as an image signal processor; this type of processing module mainly performs operations on pixel rows, and through the conversion of pixel blocks to pixel rows, it is convenient for various image processors to perform corresponding processing, so that the rolling shutter camera, distortion correction, and image signal processing can achieve streaming execution performance, which is a complete set of processing solutions.
[0103] Since the embodiment of the present application does not need to store the image in the memory space, but stores the image in the cache space and then reads the pixel rows of the image from the cache space, the present application can also reduce the memory space occupied, greatly reduce the memory read and write bandwidth and delay, and thus improve the real-time performance of image distortion correction.
[0104] Based on the same inventive concept, as an implementation of the above method, an embodiment of the present application also provides an image distortion correction device, which corresponds to the above method embodiment. For ease of reading, this embodiment will no longer repeat the details of the above method embodiment one by one, but it should be clear that the image distortion correction device in this embodiment can correspond to all the contents in the above method embodiment.
[0105] The present application embodiment provides an image distortion correction device, Figure 6 is a structural schematic diagram of the image distortion correction device, as shown in Figure 6 As shown, the image distortion correction device 600 includes:
[0106] A dividing unit 601 is used to divide at least one pixel row of the image to be corrected to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along a pixel column direction;
[0107] A filling unit 602 is used to fill pixels around the multiple original pixel blocks based on a maximum offset distance to obtain multiple initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected;
[0108] An acquisition unit 603 is used to acquire target correction parameters corresponding to the multiple initial pixel blocks respectively;
[0109] The correction unit 604 is configured to perform correction calculation on each of the multiple initial pixel blocks using the target correction parameter to obtain and output target pixel blocks corresponding to the multiple initial pixel blocks.
[0110] As an optional implementation of the embodiment of the present application, the filling unit 602 is specifically used to obtain the maximum offset distance of the pixel points in the image to be corrected; obtain the target number of pixels based on the maximum offset distance; and add pixel layers with the target number of pixels around the multiple original pixel blocks to obtain the multiple initial pixel blocks.
[0111] As an optional implementation of the embodiment of the present application, the correction parameter is specifically used to obtain the position information corresponding to the multiple pixel blocks; based on the position information, the original correction parameters corresponding to the multiple pixel blocks are obtained from a preset correspondence relationship; the preset correspondence relationship includes the correspondence between the position information and the correction parameter.
[0112] As an optional implementation of the embodiment of the present application, the acquisition unit 603 is specifically used to perform correction analysis on multiple original pixel blocks of a test image to obtain original correction parameters corresponding to the multiple original pixel blocks; the test image is acquired by the same acquisition device before correcting the image to be corrected; the initial correction parameters of the multiple original pixel blocks and the position information of the multiple original pixel blocks are saved in correspondence to generate the preset corresponding relationship.
[0113] As an optional implementation of the embodiment of the present application, the correction unit 604 is specifically used to obtain target correction parameters corresponding to the multiple initial pixel blocks respectively through a preset interpolation algorithm for the original correction parameters corresponding to the multiple initial pixel blocks respectively.
[0114] As an optional implementation of the embodiment of the present application, the correction unit 604 is also used to splice the multiple target pixel blocks to obtain a complete pixel block composed of the multiple target pixel blocks; obtain each pixel row of the complete pixel block and output it.
[0115] Based on the same inventive concept, an embodiment of the present disclosure also provides an electronic device. Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as Figure 7 As shown, the electronic device provided in this embodiment includes: a memory 701 and a processor 702, wherein the memory 701 is used to store a computer program; and the processor 702 is used to execute the audio data processing method provided in the above embodiment when executing the computer program.
[0116] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computing device implements the image distortion correction method provided in the above embodiment.
[0117] Based on the same inventive concept, an embodiment of the present application further provides a vehicle, which includes the image distortion correction device provided by the above embodiment or the electronic device provided by the above embodiment.
[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media that include computer-usable program code.
[0119] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0120] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0121] Computer readable media include permanent and non-permanent, removable and non-removable storage media. Storage media can be implemented by any method or technology to store information, and the information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for correcting image distortion, characterized in that: include: Dividing at least one pixel row of the image to be corrected to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along a pixel column direction; Filling pixels around the multiple original pixel blocks based on a maximum offset distance to obtain multiple initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected; Obtaining target correction parameters corresponding to the multiple initial pixel blocks respectively; For each of the multiple initial pixel blocks, correction calculation is performed on the initial pixel block using the target correction parameter to obtain and output target pixel blocks respectively corresponding to the multiple initial pixel blocks.
2. The method according to claim 1, characterized in that The step of filling pixels around the multiple original pixel blocks based on the maximum offset distance to obtain multiple initial pixel blocks includes: Acquire the number of target pixels based on the maximum offset distance; Add pixel layers having the target pixel number around the multiple original pixel blocks respectively to obtain the multiple initial pixel blocks.
3. The method according to claim 1, characterized in that The obtaining of target correction parameters respectively corresponding to the plurality of initial pixel blocks comprises: Acquire position information of the plurality of initial pixel blocks; Acquire the original correction parameter corresponding to the position information based on a preset corresponding relationship; the preset corresponding relationship includes the corresponding relationship between each position information and the original correction parameter corresponding to each position information; The original correction parameters are calculated to obtain the target correction parameters.
4. The method according to claim 3, characterized in that Before acquiring the position information of the plurality of initial pixel blocks, the method further includes: Performing correction analysis on multiple original pixel blocks of the test image to obtain original correction parameters corresponding to the multiple original pixel blocks; The original correction parameters of the plurality of original pixel blocks and the position information of the plurality of original pixel blocks are correspondingly saved to generate the preset corresponding relationship.
5. The method according to claim 1, characterized in that The obtaining of target correction parameters respectively corresponding to the plurality of initial pixel blocks comprises: With respect to the original correction parameters respectively corresponding to the multiple initial pixel blocks, target correction parameters respectively corresponding to the multiple initial pixel blocks are obtained through a preset interpolation algorithm.
6. The method according to claim 1, characterized in that After obtaining and outputting target pixel blocks respectively corresponding to the plurality of initial pixel blocks, the method further includes: splicing the multiple target pixel blocks to obtain a complete pixel block composed of the multiple target pixel blocks; Each pixel row of the complete pixel block is obtained and outputted.
7. An image distortion correction device, characterized in that: include: A dividing unit, used for dividing at least one pixel row of the image to be corrected to obtain a plurality of original pixel blocks; the plurality of original pixel blocks are obtained by dividing the at least one pixel row along a pixel column direction; A filling unit, configured to fill pixels around the plurality of original pixel blocks based on a maximum offset distance to obtain a plurality of initial pixel blocks; the maximum offset distance is obtained by analyzing a test image; the test image is obtained by the same target shooting device before correcting the image to be corrected; An acquisition unit, used for acquiring target correction parameters respectively corresponding to the plurality of initial pixel blocks; The correction unit is used to perform correction calculation on each of the multiple initial pixel blocks using the target correction parameter to obtain and output target pixel blocks corresponding to the multiple initial pixel blocks respectively.
8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to enable the electronic device to implement the image distortion correction method described in any one of claims 1 to 7 when executing the computer program.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the image distortion correction method according to any one of claims 1 to 7.
10. A vehicle, characterized in that: include: The image distortion correction device according to claim 7 or the electronic device according to claim 8.