Lens shadow correction method and device in image stitching

Abstract

The invention discloses a lens shadow correction method and device in image stitching. The method comprises the following steps: acquiring a to-be-stitched image, and performing mirror image transformation and rotation processing on the to-be-stitched image; brightness distribution of the image is calculated, gain compensation is carried out, and shadow is eliminated; splicing the images after gain compensation; by calculating the row and column mean value of each pixel, the pixel brightness and saturation are corrected, the shadow is further eliminated, meanwhile, the chromatic aberration is reduced, and finally the corrected image is obtained. By combining the overall dynamic correction and the single-point dynamic correction, the shadow can be effectively eliminated, and the image splicing effect can be improved. Compared with the prior art, the method has the advantages of high efficiency and accuracy, image correction can be carried out in real time, a complex preprocessing process is avoided, chromatic aberration and color distortion can be reduced, and the color consistency of the spliced image is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and device for lens shadow correction in image stitching. Background Art

[0002] Due to the optical structure of the lens, more light is received at the center of the sensor than at the periphery, resulting in a situation where the middle region of the digital image is bright while the peripheral region is dark. Through the LSC (Lens Shading Correction) lens shadow correction algorithm, it is expected that the brightness of the periphery and the center of the corrected image is consistent.

[0003] Currently, the correction methods used more frequently include the storage gain method and the polynomial fitting method. And the storage gain is further divided into the concentric circle method and the grid method. As Figure 1 shown, the concentric circle method is to find the brightness center. From the center to the four corners, each pixel on the same circumference is multiplied by the same coefficient, so as to achieve the purpose of eliminating the shadow. As Figure 2 shown, the grid method is to divide the entire image into m*n grids, and then find and store the correction gain for the grid vertices. The gain of other points is obtained by interpolation.

[0004] The polynomial fitting method is to use the radius as the sampling point, fit these sampling points into a high-order curve by high-order fitting, and then store the parameters of the high-order curve. When using, substitute the actual radius into the formula to find the corresponding gain for correction.

[0005] In practical applications, due to factors such as the differences in the lenses themselves and the exposure time, the above correction methods such as the storage gain method and the polynomial fitting method will cause the situation of excessive or insufficient gain, and cannot completely eliminate the shadow. Especially when stitching continuously acquired images, this situation is more prominent. In addition, these correction methods need to create compensation data in advance, and the entire algorithm consumes a long time and has low correction efficiency. Summary of the Invention

[0006] Technical Objective: Aiming at the deficiencies of the existing shadow correction methods, the present invention discloses a method and device for lens shadow correction in image stitching. By combining the overall dynamic correction and the single-point dynamic correction, it can effectively eliminate the shadow while reducing the color difference, thereby improving the quality and efficiency of image stitching.

[0007] Technical Solution: To achieve the above technical objective, the present invention adopts the following technical solution:

[0008] A method for lens shadow correction in image stitching, comprising the following steps:

[0009] Obtain N images to be stitched, and perform rotation and mirroring processing on each image to obtain a number of transformed images;

[0010] Add the RGB three-channel values of the pixels at the same position in the number of transformed images and the original N images and calculate the average value to obtain a brightness distribution map;

[0011] Calculate the difference between the average value of the original images and the brightness distribution map to obtain gain data, and perform gain compensation on the N images based on this gain data to obtain images with shadows preliminarily eliminated;

[0012] Arrange the N gain-compensated images in the stitching order to form a large stitched image;

[0013] For the large stitched image, calculate the row and column average values of the position where each pixel is located;

[0014] Based on the row and column average values of each pixel, perform brightness and saturation correction on each pixel to ensure that while adjusting the brightness, the generation of color difference is reduced, so as to obtain the finally corrected image.

[0015] The present invention also provides a lens shadow correction device in image stitching, which is used to implement a lens shadow correction method in image stitching as described above, including:

[0016] An image acquisition module, which is used to acquire images to be stitched;

[0017] An image transformation module, which is used to perform rotation and mirroring processing on N images to be stitched to obtain a number of transformed images;

[0018] A first calculation module, which is used to add the RGB three-channel values of the pixels at the same position in the number of transformed images and the original N images respectively and calculate the average value to obtain a brightness distribution map;

[0019] A gain compensation module, which is used to calculate the difference between the average value of the original images and the brightness distribution map, and perform gain compensation on the N images based on the gain data;

[0020] An image stitching module, which is used to arrange N images in the stitching order to form a large stitched image;

[0021] A second calculation module, which is used to calculate the row and column average values of the position where each pixel is located in the large stitched image;

[0022] A correction module, which is used to perform brightness and saturation correction on pixels based on the row and column average values of each pixel.

[0023] The present invention also provides an electronic device, including a processor and a memory. A computing-based instruction capable of running on the processor is stored on the memory. When the computer instruction runs on the processor, it can execute a lens shadow correction method in an image stitching as described above.

[0024] The present invention also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs, it can implement a lens shadow correction method in an image stitching as described above.

[0025] The present invention also provides a computer program product. When the program instruction in the computer program product runs, it can implement a lens shadow correction method in an image stitching as described above.

[0026] Advantageous effects: A lens shadow correction method and device in an image stitching provided by the present invention have the following advantageous effects:

[0027] 1. By combining overall dynamic correction and single-point dynamic correction, the present invention can effectively eliminate the lens shadow problem in image stitching. In the overall dynamic correction step, by mirroring the image vertically and horizontally and rotating it by 180 degrees, the diversity of the data set is increased, and a uniform brightness distribution map is obtained through brightness distribution calculation. The gain compensation step then performs gain compensation according to the difference between the original image and the brightness distribution map, eliminating the problem of uneven brightness between the center and the surrounding areas of the image. In addition, single-point dynamic correction further corrects the local color difference in the large stitched image, thereby achieving global and local shadow elimination and significantly improving the image quality.

[0028] 2. Compared with the traditional method in the prior art that requires creating complex compensation data in advance, the present invention avoids the complex preprocessing process by calculating the brightness distribution map and gain compensation data in real time. Therefore, the entire correction process is more concise and has a higher calculation efficiency, significantly improving the processing speed. Moreover, in the traditional lens shadow correction method, brightness adjustment is often accompanied by color distortion or color difference problems. The present invention introduces a color difference compensation coefficient and a color adjustment coefficient, which can effectively compensate for the color difference caused by shadow correction while adjusting the brightness. Through pixel correction based on row and column means, the color consistency of the image is ensured, avoiding color distortion that may occur in the traditional method and ensuring that the color of the stitched image is natural and coordinated. Description of the Drawings

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art.

[0030] Figure 1 It is a schematic diagram of the concentric circle correction method in the prior art;

[0031] Figure 2 It is a schematic diagram of the grid correction method in the prior art;

[0032] Figure 3 It is a flowchart of the shadow correction method in the embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of one of the N images in the embodiment of the present invention;

[0034] Figure 5 It is a schematic diagram of the brightness distribution map in the embodiment of the present invention;

[0035] Figure 6 It is a schematic diagram of the image after initially eliminating the shadow after gain compensation in the embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the large image after stitching in the embodiment of the present invention;

[0037] Figure 8 It is a schematic diagram of the corrected image in the embodiment of the present invention. Specific embodiments

[0038] The present invention will be more clearly and completely described below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the described embodiments.

[0039] As Figure 3 shown, a lens shadow correction method in image stitching includes the following steps:

[0040] S1. Obtain N images to be stitched, and perform up-down, left-right mirroring and 180-degree rotation on each image to obtain a total of N*4 images. The N*4 images include the original N images and N*3 transformed images. As Figure 4 shown, it is one of the N images to be stitched selected by the present invention;

[0041] S2. Add the RGB three-channel values of the pixels at the same position in the N*4 images and calculate the average value to obtain a brightness distribution map. As Figure 5 shown, it is a schematic diagram of the obtained brightness distribution map. The average value of the brightness distribution map is calculated by the following formula:

[0042]

[0043] Among them, G mean ∈[R mean , G mean , B mean is the average value of the brightness distribution map, R mean , G mean and Bmean They are the means of pixels at the same position in each of the N * 4 images in the R channel, G channel, and B channel respectively. i is the index of the image to be stitched, N is the number of images to be stitched, w is the width of the image to be stitched, h is the length of the image to be stitched, x and y are the position coordinates of the pixel, and I(x, y) is the pixel value at position (x, y) in the image;

[0044] S3. Calculate the difference between the mean of the original image and the brightness distribution map to obtain gain data, and perform gain compensation on the N images based on this gain data to obtain an image with shadows preliminarily removed, as Figure 6 shown in the schematic diagram of the image with shadows preliminarily removed. After having the brightness distribution map, the brightness distribution map can be updated according to the images to be stitched collected later, so that the compensation effect of the brightness distribution map will be better and better;

[0045] S4. Arrange the N gain-compensated images in the stitching order to form a large stitched image, as Figure 7 shown in the schematic diagram of the large stitched image;

[0046] S5. For the large stitched image, calculate the row and column means of the position where each pixel is located. The specific calculation formula is as follows:

[0047]

[0048] where x and y are the position coordinates of the pixel, is the row and column mean of the pixel at position (x, y) in the image, and are the row and column means of the pixel at position (x, y) in the R channel, G channel, and B channel of the image respectively. W is the width of the large stitched image, H is the length of the large stitched image, and I(x, y) is the pixel value at position (x, y) in the image;

[0049] S6. Based on the row and column means of each pixel, perform brightness and saturation correction on each pixel to ensure that while adjusting the brightness, the generation of color difference is reduced, so as to obtain the finally corrected image, as Figure 8 shown in the schematic diagram of the corrected image. It can be seen from the figure that the shadows have been basically removed and the color difference is not obvious. The correction formula is:

[0050]

[0051] where x and y are the position coordinates of the pixel, C’ xy is the pixel value at position (x, y) in the corrected image, C xy ∈[R xy , G xy , B xyis the pixel value at position (x, y) in the original image, B xy , G xy , B xy respectively represent the pixel values on the R channel, G channel, and B channel at position (x, y) in the original image, is the image mean value of the stitched large image, are the average values of the stitched large image on the R channel, G channel, and B channel respectively, is the row-column mean of the pixel located at position (x, y) in the image, and are the row-column means of the pixel located at position (x, y) in the image on the R channel, G channel, and B channel respectively, α xy is the chromatic aberration compensation coefficient, and its calculation formula is f xy is the color adjustment coefficient, and its calculation formula is

[0052] The lens shadow correction method of the present invention mainly includes two basic steps: overall dynamic correction and single-point dynamic correction. Among them, steps S1 to S3 are overall dynamic correction, and steps S4 to S6 are single-point dynamic correction. By combining the two-step method of overall dynamic correction and single-point dynamic correction, the present invention can comprehensively optimize the shadow area and chromatic aberration problems in the image. Through this dual correction strategy, the present invention can achieve high-quality image stitching effects in most cases, not only eliminating the shadows caused by the lens, but also reducing chromatic aberration, ensuring the brightness and color consistency of the stitched image, and making the stitching result more natural and meeting the visual requirements.

[0053] The present invention also provides a lens shadow correction device in image stitching for implementing a lens shadow correction method in image stitching as described above, including:

[0054] An image acquisition module for acquiring the images to be stitched;

[0055] An image transformation module for performing vertical, horizontal mirroring, and 180-degree rotation on the N images to be stitched to obtain N * 3 transformed images;

[0056] A first calculation module for adding and averaging the RGB three-channel values of the pixels at the same position in the N * 3 transformed images and the original N images respectively to obtain a brightness distribution map;

[0057] A gain compensation module for calculating the difference between the original image mean and the brightness distribution map, and performing gain compensation on the N images based on the gain data;

[0058] An image stitching module, configured to arrange N images in a stitching order to form a large stitched image;

[0059] A second calculation module, configured to calculate the row-column mean of the position of each pixel in the large stitched image;

[0060] A correction module, configured to perform brightness and saturation correction on pixels based on the row-column mean of each pixel, ensuring that while changing the brightness, the generation of color difference is reduced.

[0061] Those skilled in the art of the present technology can understand that the present invention may relate to a device for performing one or more of the operations described in the present application. The device may be specifically designed and manufactured for the required purpose, or may also include known devices in a general-purpose computer, and the general-purpose computer has a program stored therein that is selectively activated or reconstructed. Such a computer program may be stored in a device (e.g., a computer) readable medium or in any type of medium suitable for storing electronic instructions and coupled to a bus respectively. The computer readable medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), random access memory (RAM), read-only memory (ROM), electrically programmable ROM, electrically erasable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic cards, or optical cards. The readable medium includes any mechanism for storing or transmitting information in a form readable by a device (e.g., a computer). For example, the readable medium includes random access memory (RAM), read-only memory (ROM), disk storage media, optical storage media, flash memory devices, signals propagated in electrical, optical, acoustic, or other forms (e.g., carrier waves, infrared signals, digital signals), etc.

[0062] Those skilled in the art of the present technology can understand that each block in these structural diagrams and / or block diagrams and / or flowcharts, as well as combinations of blocks in these structural diagrams and / or block diagrams and / or flowcharts, can be implemented with computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing methods to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing methods create a method for implementing the functions specified in the block or blocks of the structural diagrams and / or block diagrams and / or flowcharts.

[0063] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art of the present technology, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A lens shading correction method in image stitching, characterized in that: The following steps are involved: Obtain N images to be stitched, and perform rotation and mirror processing on each image to obtain several transformed images; The RGB three-channel values ​​of each pixel at the same position in the several transformed images and the original N images are added and averaged to obtain a brightness distribution map; The difference between the original image mean and the brightness distribution map is calculated to obtain gain data, and gain compensation is performed on N images based on the gain data to obtain an image with preliminary shadow elimination; Arrange N gain-compensated images in a splicing order to form a spliced ​​large image; For the stitched large image, calculate the row and column mean of each pixel; Based on the row and column mean of each pixel, the brightness and saturation of each pixel are corrected to ensure that the color difference is reduced while adjusting the brightness, so as to obtain the final corrected image.

2. The lens shading correction method in image stitching according to claim 1, characterized in that: Each image is rotated and mirrored to obtain a number of transformed images. Specifically, each image is mirrored up and down, left and right, and rotated 180 degrees to obtain N*3 transformed images.

3. The lens shading correction method in image stitching according to claim 1, characterized in that: The mean of the brightness distribution map is calculated using the following formula: Among them, C mean ∈[R mean ,G mean ,B mean ] is the mean value of the brightness distribution diagram, R mean , G mean and B mean are the mean values ​​of the pixels at the same position in the R, G and B channels of several transformed images and the original N images, respectively; i is the index of the image to be stitched; N is the number of images to be stitched; w is the width of the image to be stitched; h is the length of the image to be stitched; x and y are the position coordinates of the pixel; I(x,y) is the pixel value at position (x,y) in the image.

4. The lens shading correction method in image stitching according to claim 1, characterized in that: The calculation formula for the row and column means of each pixel location is as follows: Among them, x and y are the position coordinates of the pixel, is the row and column mean of the pixel at position (x, y) in the image, and are the row and column means of the pixel at position (x, y) in the image in the R channel, G channel, and B channel, respectively. W is the width of the stitched large image, H is the length of the stitched large image, and I(x, y) is the pixel value at position (x, y) in the image.

5. The lens shading correction method in image stitching according to claim 1, characterized in that: The correction formula is: Where x and y are the pixel position coordinates, C' xy is the pixel value at the corrected image position (x, y), C xy ∈[R xy ,G xy ,B xy ] is the pixel value at position (x, y) in the original image, R xy ,G xy ,B xy Respectively represent the pixel values ​​on the R channel, G channel, and B channel at the position (x, y) in the original image. is the image mean value of the spliced ​​large image, They are the average values ​​of the spliced ​​large image on the R channel, G channel, and B channel, respectively. is the row and column mean of the pixel at position (x, y) in the image, and are the row and column means of the pixel at position (x, y) in the image in the R channel, G channel, and B channel, respectively, α xy is the chromatic aberration compensation coefficient, and its calculation formula is: f xy is the color adjustment coefficient, and its calculation formula is 6. A lens shading correction device in image stitching, used to implement a lens shading correction method in image stitching as claimed in any one of claims 1 to 5, characterized in that: include: An image acquisition module, used for acquiring images to be stitched; An image transformation module is used to rotate and mirror the N images to be stitched to obtain a number of transformed images; The first calculation module is used to add and average the RGB three-channel values ​​of each pixel at the same position in the plurality of transformed images and the original N images to obtain a brightness distribution map; A gain compensation module is used to calculate the difference between the original image mean and the brightness distribution map, and perform gain compensation on N images based on the gain data; An image stitching module is used to arrange N images in stitching order to form a stitched large image; The second calculation module is used to calculate the row and column means of each pixel in the spliced ​​large image; The correction module is used to correct the brightness and saturation of pixels based on the row and column mean of each pixel.

7. The lens shading correction device in image stitching according to claim 6, characterized in that: The N images to be spliced ​​are rotated and mirrored to obtain a number of transformed images, specifically including mirroring each image up and down, left and right, and rotating it 180 degrees to obtain N*3 transformed images.

8. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computing base instructions that can be run on the processor, and when the computer instructions are run on the processor, the lens shading correction method in image stitching as described in any one of claims 1 to 5 can be executed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is run, the method for correcting lens shading in image stitching as described in any one of claims 1 to 5 can be implemented.

10. A computer program product, characterized in that When the program instructions in the computer program product are executed, the lens shading correction method in image stitching as described in any one of claims 1 to 5 can be implemented.

Images