A method and device for correcting lens shading in image stitching

Abstract

The application discloses a lens shadow correction method and device in image stitching, which comprises the following steps: obtaining the image to be stitched and performing mirror transformation and rotation processing; calculating the brightness distribution of the image and performing gain compensation to eliminate the shadow; stitching the image after gain compensation; calculating the row and column mean value of each pixel to correct the pixel brightness and saturation, further eliminate the shadow and reduce the color difference, and finally obtaining the corrected image. By combining the overall dynamic correction and single-point dynamic correction, the application can effectively eliminate the shadow and improve the image stitching effect. Compared with the prior art, the application has the advantages of high efficiency and accuracy, can perform real-time image correction, avoids the complex preprocessing process, can reduce the color difference and color distortion, and ensures the color consistency of the stitched image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a lens shading correction method and device in image stitching. BACKGROUND

[0002] Due to the optical structure of the lens, the light received by the center of the sensor is more than that around the center, resulting in a situation that the middle region of the digital image is bright and the surrounding region is dark. The LSC (Lens Shading Correction) lens shading correction algorithm expects that the brightness of the corrected image is consistent around the center.

[0003] At present, the commonly used correction methods include storage gain method and polynomial fitting method. The storage gain is divided into concentric circle method and grid method. As shown in the formula (1), the concentric circle method is to find the brightness center, and each pixel on the same circle from the center to the corner is multiplied by a same coefficient, so as to eliminate the shading. As shown in the formula (2), the grid method is to divide the whole image into m*n grids, and then the correction gain of the grid vertex is calculated and stored, and the gain of other points is calculated by interpolation. Figure 1 Figure 2 The polynomial fitting method is to use the sampling points with a radius, fit the sampling points into a high-order curve by high-order fitting, and then store the parameters of the high-order curve. When used, the actual radius is brought into the formula to calculate the corresponding gain for correction.

[0004] In actual application, due to the differences of the lens itself and the exposure time and other factors, the above-mentioned storage gain method and polynomial fitting method and other correction methods may cause excessive or insufficient gain, and cannot completely eliminate the shading, especially when the continuous collected images are stitched, the situation is more prominent. In addition, these correction methods need to create compensation data in advance, and the whole algorithm consumes a long time and has low correction efficiency.

[0005] In actual application, due to the differences of the lens itself and the exposure time and other factors, the above-mentioned storage gain method and polynomial fitting method and other correction methods may cause excessive or insufficient gain, and cannot completely eliminate the shading, especially when the continuous collected images are stitched, the situation is more prominent. In addition, these correction methods need to create compensation data in advance, and the whole algorithm consumes a long time and has low correction efficiency. SUMMARY

[0006] Technical purpose: in view of the shortcomings of the existing shading correction method, the present application discloses a lens shading correction method and device in image stitching, which can effectively eliminate the shading while reducing the color difference by combining the whole dynamic correction and single-point dynamic correction, so as to improve the quality and efficiency of image stitching.

[0007] Technical scheme: in order to realize the above technical purpose, the present application adopts the following technical scheme:

[0008] A lens shading correction method in image stitching, comprising the following steps:

[0009] ​Obtain N images to be spliced, and rotate and mirror each image to obtain a plurality of transformed images;

[0010] Add and average the RGB three-channel values of the pixels at the same position in the plurality of transformed images and the original N images to obtain a luminance distribution map;

[0011] Calculate the difference between the average of the original image and the luminance distribution map to obtain gain data, and perform gain compensation on the N images based on the gain data to obtain an image with preliminary shadow elimination;

[0012] Arrange the N gain-compensated images in splicing order to form a large spliced image;

[0013] For the large spliced image, calculate the row and column averages of the position of each pixel;

[0014] Based on the row and column averages of each pixel, correct the brightness and saturation of each pixel to ensure that the color difference is reduced while the brightness is adjusted, thereby obtaining a final corrected image.

[0015] The application also provides a lens shadow correction device in image splicing, which is used to implement the lens shadow correction method in image splicing as described above, and comprises:

[0016] An image acquisition module is configured to acquire images to be spliced;

[0017] An image transformation module is configured to rotate and mirror N images to be spliced to obtain a plurality of transformed images;

[0018] A first calculation module is configured to add and average the RGB three-channel values of the pixels at the same position in the plurality of transformed images and the original N images, respectively, to obtain a luminance distribution map;

[0019] A gain compensation module is configured to calculate the difference between the average of the original image and the luminance distribution map, and perform gain compensation on the N images based on the gain data;

[0020] An image splicing module is configured to arrange the N images in splicing order to form a large spliced image;

[0021] A second calculation module is configured to calculate the row and column averages of the position of each pixel in the large spliced image;

[0022] A correction module is configured to correct the brightness and saturation of each pixel based on the row and column averages of each pixel.

[0023] The application further provides an electronic device comprising a processor and a memory, wherein the memory stores computer instructions capable of running on the processor, and the computer instructions are capable of executing the lens shading correction method in image stitching as described above when running on the processor.

[0024] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is capable of implementing the lens shading correction method in image stitching as described above when running.

[0025] The application further provides a computer program product, wherein the program instructions in the computer program product are capable of implementing the lens shading correction method in image stitching as described above when running.

[0026] Beneficial effects: the lens shading correction method and device in image stitching provided by the application have the following beneficial effects:

[0027] 1. The lens shading correction method in image stitching provided by the application can effectively eliminate the lens shading problem in image stitching by combining the overall dynamic correction and the single-point dynamic correction. The overall dynamic correction step firstly increases the diversity of the data set by performing the up-down, left-right mirroring and 180-degree rotation on the image, and obtains the uniform brightness distribution map through the brightness distribution calculation. The gain compensation step eliminates the problem of uneven brightness between the image center and the surrounding area by performing the gain compensation according to the difference between the original image and the brightness distribution map. In addition, the single-point dynamic correction further corrects the local color difference in the large image after stitching, thereby realizing the global and local shadow elimination and significantly improving the image quality.

[0028] 2. Compared with the traditional method in the prior art which needs to create complex compensation data in advance, the application avoids the complex preprocessing process by calculating the brightness distribution map and the gain compensation data in real time. Therefore, the entire correction process is more concise and has higher calculation efficiency, and the processing speed is significantly improved. In the traditional lens shading correction method, the brightness adjustment is often accompanied by color distortion or color difference problem. The application introduces the color difference compensation coefficient and the color adjustment coefficient, which can effectively compensate the color difference caused by the shadow correction while adjusting the brightness. Through the pixel correction based on the row and column mean values, the color consistency of the image is ensured, the color distortion that may occur in the traditional method is avoided, and the color of the stitched image is natural and coordinated. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below.

[0030] Figure 1 It is a schematic diagram of the concentric circle correction method in the prior art.

[0031] Figure 2 A schematic diagram of the grid correction method in the prior art;

[0032] Figure 3 A flow chart of the shadow correction method in the embodiment of the present application;

[0033] Figure 4 A schematic diagram of one of the N images in the embodiment of the present application;

[0034] Figure 5 A schematic diagram of the luminance distribution map in the embodiment of the present application;

[0035] Figure 6 A schematic diagram of the image after preliminary elimination of shadows after gain compensation in the embodiment of the present application;

[0036] Figure 7 A schematic diagram of the large image after splicing in the embodiment of the present application;

[0037] Figure 8 A schematic diagram of the corrected image in the embodiment of the present application. DETAILED DESCRIPTION

[0038] The present application will be described in more detail below by way of a preferred embodiment and with reference to the accompanying drawings, but the present application is not limited to the embodiment described.

[0039] As shown in Figure 3 , a lens shadow correction method in image splicing comprises the following steps:

[0040] S1, obtaining N images to be spliced, and performing up-down, left-right mirroring and 180-degree rotation on each image to obtain a total of N*4 images, the N*4 images including the original N images and N*3 transformed images, as shown in Figure 4 , which is one of the N images to be spliced selected by the present application;

[0041] S2, adding and averaging the RGB three-channel values of pixels at the same position in the N*4 images to obtain a luminance distribution map, as shown in Figure 5 , which is a schematic diagram of the obtained luminance distribution map, the average value of the luminance distribution map being calculated by the following formula:

[0042]

[0043] wherein, G mean ∈[R mean ,G mean ,B mean ] is the average value of the luminance distribution map, R mean , G mean and Bmean respectively, are the mean values of the pixels in the R channel, the G channel and the B channel at each same position in the N images, i is the index of the image to be spliced, N is the number of the images to be spliced, w is the width of the image to be spliced, h is the length of the image to be spliced, x and y are the position coordinates of the pixel, and I(x, y) is the pixel value at the position (x, y) in the image;

[0044] S3, a difference between the original image mean value and the brightness distribution map is calculated to obtain gain data, and the N images are gain compensated based on the gain data to obtain an image with shadows preliminarily eliminated, as shown in FIG. 3; Figure 6 After the brightness distribution map is obtained, the brightness distribution map can be updated according to the images to be spliced subsequently, so that the compensation effect of the brightness distribution map is better and better;

[0045] S4, the N gain-compensated images are arranged in the splicing order to form a large image after splicing, as shown in FIG. 4; Figure 7

[0046] S5, for the large image after splicing, the row and column mean values of each pixel at the position are calculated, and the specific calculation formula is as follows:

[0047]

[0048] wherein, x and y are the position coordinates of the pixel, is the row and column mean value of the pixel at the position (x, y) in the image, and respectively, are the row and column mean values of the pixels in the R channel, the G channel and the B channel at the position (x, y) in the image, W is the width of the large image after splicing, H is the length of the large image after splicing, and I(x, y) is the pixel value at the position (x, y) in the image;

[0049] S6, based on the row and column mean values of each pixel, the brightness and saturation of each pixel are corrected to ensure that the color difference is reduced while the brightness is adjusted, so as to obtain a final corrected image, as shown in FIG. 5; Figure 8 As shown in FIG. 5, the shadows have been basically eliminated and the color difference is not obvious, and the correction formula is as follows:

[0050]

[0051] wherein, x and y are the position coordinates of the pixel, C' xy is the pixel value at the position (x, y) in the corrected image, C xy ∈ [R xy , G xy , B xy ​R(x,y) is a pixel value at position (x,y) in the original image, B(x,y) is a pixel value at position (x,y) in the original image, and xy G(x,y) is a pixel value at position (x,y) in the original image, xy B(x,y) is a pixel value at position (x,y) in the original image, xy R(x,y) is a pixel value at position (x,y) in the original image, G(x,y) is a pixel value at position (x,y) in the original image, and B(x,y) is a pixel value at position (x,y) in the original image, is an image mean value of the large image after splicing, is an average value of the large image after splicing in the R channel, G channel and B channel, is a row-column mean value of a pixel at position (x,y) in the image, and is a row-column mean value of a pixel at position (x,y) in the image in the R channel, G channel and B channel, and xy is a color difference compensation coefficient, and the calculation formula is f xy is a color adjustment coefficient, and the calculation formula is

[0052] The lens shadow correction method of the application mainly includes two basic steps of overall dynamic correction and single-point dynamic correction, wherein steps S1 to S3 are overall dynamic correction, and steps S4 to S6 are single-point dynamic correction. The application can comprehensively optimize the shadow area and color difference problems in the image by combining the two-step method of overall dynamic correction and single-point dynamic correction. Through this double correction strategy, the application can realize high-quality image splicing effect in most cases, not only eliminates the shadow caused by the lens, but also reduces the color difference, ensures the brightness and color consistency of the spliced image, and makes the splicing result more natural and meets the visual requirements.

[0053] The application also provides a lens shadow correction device in image splicing, which is used for realizing the lens shadow correction method in image splicing as described above, and includes:

[0054] An image acquisition module is used for acquiring images to be spliced;

[0055] An image transformation module is used for performing up-down, left-right mirroring and 180-degree rotation on the N images to be spliced to obtain N*3 transformed images;

[0056] A first calculation module is used for adding and averaging the RGB three-channel values of the same position in the N*3 transformed images and the original N images respectively to obtain a brightness distribution graph;

[0057] A gain compensation module is used for calculating the difference between the original image mean value and the brightness distribution graph, and performing gain compensation on the N images based on gain data;

[0058] An image splicing module is arranged to arrange N images in a splicing order to form a large image after splicing;

[0059] A second calculation module is arranged to calculate the average of the row and column of the position of each pixel in the large image after splicing;

[0060] A correction module is arranged to correct the brightness and saturation of each pixel based on the average of the row and column of the position of each pixel, so as to ensure that the color difference is reduced while the brightness is changed.

[0061] Those skilled in the art can understand that the present application can relate to a device for performing one or more of the operations described in the present application. The device can be specially designed and manufactured for the required purpose, or can also include known devices in a general-purpose computer, which has a program selectively activated or restructured stored therein. Such a computer program can be stored in a device (for example, a computer) readable medium or in any type of medium suitable for storing electronic instructions and respectively coupled to the bus, including but not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), random access memory (RAM), read-only memory (ROM), electrically programmable ROM, electrically erasable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic card or optical card. The readable medium includes any mechanism for storing or transmitting information in the form readable by the device (for example, a computer). For example, the readable medium includes random access memory (RAM), read-only memory (ROM), magnetic disk storage medium, optical storage medium, flash memory device, signals (such as carrier wave, infrared signal, digital signal) propagated in electrical, optical, acoustic or other form, etc.

[0062] Those skilled in the art can understand that each block in these structural diagrams and / or block diagrams and / or flow diagrams and the combination of blocks in these structural diagrams and / or block diagrams and / or flow diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a professional computer or a processor of other programmable data processing method to generate a machine, so that the instructions executed by the computer or the processor of other programmable data processing method create a method specified in the block or blocks of the structural diagram and / or block diagram and / or flow diagram.

[0063] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be regarded as the protection scope of the present application.

Claims

1. A method for lens shadow correction in image stitching, characterized in that, The method comprises the following steps: Obtaining N images to be spliced, and performing rotation and mirror image processing on each image to obtain a plurality of transformed images; Adding and averaging the RGB three-channel values of the pixels at the same position in the plurality of transformed images and the original N images to obtain a luminance distribution map; Calculating the difference between the average value of the original image and the luminance distribution map to obtain gain data, and performing gain compensation on the N images based on the gain data to obtain an image with preliminary shadow elimination; Arranging the N gain-compensated images in splicing order to form a large image after splicing; For the large image after splicing, calculating the row and column average values of the position of each pixel; Based on the row and column average values of each pixel, performing luminance and saturation correction on each pixel to ensure that the luminance is adjusted while reducing the generation of color difference, thereby obtaining a final corrected image. 2.The method of claim 1, wherein, The rotation and mirror image processing on each image to obtain a plurality of transformed images specifically includes performing up-down, left-right mirror image and 180-degree rotation on each image to obtain N*3 transformed images. 3.The method of claim 1, wherein, The average value of the luminance distribution map is calculated by the following formula: wherein C mean ∈ [R mean , G mean , B mean ] is the average of the luminance distribution map, R mean , G mean , and B mean are the average values of the pixels in the R channel, the G channel, and the B channel of each same position in the N transformed images and the original N images, i is the index of the image to be spliced, N is the number of images to be spliced, w is the width of the image to be spliced, h is the length of the image to be spliced, x and y are the position coordinates of the pixels, and I(x, y) is the pixel value of the image at position (x, y).

4. The method of claim 1, wherein, The calculation formula of the row and column average values of the position of each pixel is as follows: where x and y are the position coordinates of the pixel, is the row-column mean of the pixel at position (x, y) in the image, and are the row-column means of the pixel at position (x, y) in the image in the R channel, G channel, and B channel, respectively, W is the width of the large image after stitching, H is the length of the large image after stitching, and I(x, y) is the pixel value at position (x, y) in the image. 5.The method of claim 1, wherein, The correction formula is: where x and y are the position coordinates of the pixel, C xy is the pixel value at the position (x, y) of the corrected image, C xy ∈ [R xy , G xy , B xy ] is the pixel value at the position (x, y) in the original image, R xy , G xy , and B xy represent the pixel values at the position (x, y) in the original image on the R channel, G channel, and B channel, respectively, is the image mean value of the large image after stitching, are the average values of the large image after stitching on the R channel, G channel, and B channel, respectively, is the row-column mean of the pixel at the position (x, y) in the image, and are the row-column means of the pixel at the position (x, y) in the image on the R channel, G channel, and B channel, respectively, α xy is the color difference compensation coefficient, and its calculation formula is f xy is the color adjustment coefficient, and its calculation formula is 6. An apparatus for correcting lens shading in image stitching, for implementing the method for correcting lens shading in image stitching according to any one of claims 1-5, characterized in that, It comprises: An image acquisition module for acquiring images to be spliced; An image transformation module for performing rotation and mirror image processing on the N images to be spliced to obtain a plurality of transformed images; A first calculation module for adding and averaging the RGB three-channel values of the pixels at the same position in the plurality of transformed images and the original N images, respectively, to obtain a luminance distribution map; A gain compensation module for calculating the difference between the average value of the original image and the luminance distribution map, and performing gain compensation on the N images based on the gain data; An image splicing module for arranging the N images in splicing order to form a large image after splicing; A second calculation module for calculating the row and column average values of the position of each pixel in the large image after splicing; A correction module for performing luminance and saturation correction on the pixels based on the row and column average values of each pixel.

7. The apparatus according to claim 6, wherein The rotation and mirror image processing on each image to obtain a plurality of transformed images specifically includes performing up-down, left-right mirror image and 180-degree rotation on each image to obtain N*3 transformed images.

8. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program can implement the lens shadow correction method in image splicing according to any one of claims 1-5 when running.

9. A computer-readable storage medium, characterized in that, The computer program product in the computer readable storage medium can implement the lens shadow correction method in image splicing according to any one of claims 1-5 when running.

10. A computer program product, characterised in that, ​

Images