Method for eliminating latticed shadow of spliced image based on overlapping region consistency

Abstract

The invention discloses a spliced image latticed shadow elimination method based on overlapping region consistency. The method comprises the following specific steps: acquiring adjacent image sequences with partial overlapping regions; selecting a central region as an ideal gray value reference by using a calibration image shot under a uniform light source, and calculating a deviation image; establishing a basic vignetting correction model through spherical surface fitting based on the deviation image; determining an overlapping region through an image registration algorithm, and calculating a gray value error sum in the overlapping region; an optimal correction coefficient is determined by minimizing the sum of total gray value errors of all overlapped areas, and the correction model is dynamically adjusted; and finally, correcting the image sequence by using the updated correction model, and eliminating the latticed shadow. According to the method, the brightness uniformity and the visual consistency of the spliced image are improved, and the latticed shadow caused by the vignetting effect can be effectively eliminated.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for eliminating grid-shaped shadows of spliced ​​images based on consistency of overlapping areas. Background Art

[0002] In optical imaging instruments, due to the limitation of the camera's field of view, when observing larger samples, a single field of view cannot capture the entire sample. At this time, serialized image stitching is usually required, that is, the imaging unit cooperates with the mobile platform to capture images of different positions of the sample and stitch them into a panoramic image.

[0003] In an optical imaging system, the farther the light is from the optical axis, the smaller the effective aperture of the optical system is, and the smaller the light intensity is on the imaging plane, resulting in a vignetting phenomenon where the image is bright in the middle and dark at the edges. If the vignetting phenomenon cannot be effectively suppressed, a grid effect with distinct light and dark will appear during the stitching of serialized images. Therefore, it is necessary to perform vignetting correction on individual images before image stitching.

[0004] Vignetting image correction is usually based on two methods: hardware-based calibration method and image-based feature method. The hardware-based calibration method calculates the grayscale attenuation model by photographing a uniform calibration plate in a scene with uniform illumination. The model obtained by this method has high accuracy and is widely used in industrial sites with stable scenes. However, this method is only applicable to the optical environment during calibration. When the environment changes, recalibration is required. The image feature-based method uses the features of a single or multiple images to correct vignetting and minimizes logarithmic entropy to perform vignetting correction. However, a single logarithmic entropy is not applicable to all scenes, especially in serialized spliced ​​images, where it is difficult to maintain the consistency of multi-frame image processing results. Summary of the invention

[0005] Technical purpose: In view of the shortcomings of the existing vignetting correction technology, the present invention discloses a method for eliminating grid shadows in spliced ​​images based on consistency of overlapping areas, which can better suppress the grid shadow phenomenon in serialized spliced ​​images.

[0006] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A method for removing grid-like shadows from a spliced ​​image based on consistency of overlapping areas specifically comprises the following steps:

[0008] Acquire an image sequence to be stitched, where two adjacent images in the image sequence have a partial overlapping area;

[0009] Obtain a calibration image taken under a uniform light source, select the central area of ​​the calibration image as the reference area of ​​the ideal grayscale value, calculate the deviation of the grayscale value of each pixel in the calibration image relative to the reference area, and generate a deviation image;

[0010] Based on the generated deviation image, a basic vignetting correction model is established through spherical fitting;

[0011] Determine the overlapping area between each adjacent image through the image registration algorithm, and calculate the gray value error sum of the corresponding pixel points in each overlapping area;

[0012] Based on the gray value error sum of the corresponding pixel points in each overlapping area, a total gray value error sum of the corresponding pixel points in all overlapping areas is constructed;

[0013] The optimal correction coefficient is determined by minimizing the sum of gray value errors of corresponding pixels in all overlapping areas, and the basic vignetting correction model is dynamically adjusted according to the optimal correction coefficient to obtain an updated correction model;

[0014] The updated correction model is used to correct the image sequence to be stitched, generate a corrected image sequence, complete vignetting correction, and eliminate grid-like shadows.

[0015] Preferably, the deviation image is of the form:

[0016] I dif =I i -I c

[0017] Among them, I dif represents the deviation image, I i Represents the ideal grayscale value, that is, all grayscale values ​​are equal to I c The mean value of the central area, I c is the gray value of the calibration image.

[0018] Preferably, based on the generated deviation image, the basic vignetting correction model is established by spherical fitting. The specific steps are as follows:

[0019] Each pixel in the deviation image is regarded as a point in three-dimensional space. Based on the characteristics of the deviation image, a spherical fitting model is established. The spherical equation used is:

[0020] (x) 2 +(yb) 2 +(zc) 2 =R 2

[0021] Among them, x and y represent the two-dimensional coordinates of the pixel point, z represents the gray value of the pixel point, a, b and c represent the coordinates of the sphere center, and R represents the radius of the sphere;

[0022] An error function is constructed to represent the fitting error, and the best fitting spherical parameters are solved by minimizing the error function. The formula of the error function is as follows:

[0023] E=∑(x 2 +y 2 +z 2 -Ax-By-Cz+D) 2

[0024] Among them, parameter A = 2a, parameter B = 2b, parameter C = 2c, parameter D = a 2 +b 2 +c 2 -R 2 ;

[0025] The calculation formulas for parameters A, B, C and D are:

[0026]

[0027] Where i represents the index of the pixel point involved in the fitting, i = 0, 1, 2…, n, and n represents the total number of pixel points involved in the fitting;

[0028] The spherical radius of the spherical center coordinates of the spherical equation is determined according to the solved parameters, and the basic vignetting correction model is obtained. The formula of the basic vignetting correction model is as follows:

[0029]

[0030] Among them, M b represents the basic vignetting correction model, and F(x,y) represents the spherical equation deformation.

[0031] Preferably, the calculation formula for the sum of grayscale value errors of corresponding pixels in each overlapping area is as follows:

[0032]

[0033] Among them, Dif(R) represents the gray value error sum of corresponding pixels in the overlapping area, R represents the overlapping area, and A jl and B jl Respectively represent the grayscale values ​​of the two images in the overlapping area at the pixel coordinates (j, l), w and h represent the width and height of the overlapping area respectively;

[0034] The calculation formula for the total gray value error of corresponding pixels in all overlapping areas is as follows:

[0035]

[0036] Among them, F represents the total gray value error sum of corresponding pixels in all overlapping areas, k represents the overlapping area index, and m represents the total number of overlapping areas.

[0037] Preferably, the specific steps of determining the optimal correction coefficient by minimizing the sum of grayscale value errors of corresponding pixels in all overlapping areas, and dynamically adjusting the basic vignetting correction model according to the optimal correction coefficient to obtain the updated correction model are as follows:

[0038] Set the value range of the correction coefficient and divide the value range into several parts;

[0039] Several groups of correction coefficients within the value range are combined with the basic vignetting correction model to perform vignetting correction on the image sequence to be spliced, and the sum of the total gray value errors of the corresponding pixels in all overlapping areas of the image sequence after vignetting correction is calculated;

[0040] The total gray value error of the corresponding pixels in all overlapping areas and the minimum correction coefficient are selected as the optimal correction coefficient, and the basic vignetting correction model is updated to obtain an updated correction model.

[0041] Preferably, the calculation formula for performing vignetting correction on an image by combining several groups of correction coefficients within the value range with a basic vignetting correction model is as follows:

[0042] I ds =I+δ s M b

[0043] Among them, I ds represents the image sequence after vignetting correction by applying the sth group of correction coefficients, I represents the image sequence before vignetting correction, δ s represents the sth group of correction coefficients, where s represents the correction coefficient index.

[0044] Preferably, the calculation formula for correcting the image sequence to be spliced ​​using the updated correction model is as follows:

[0045] I d =I+M

[0046] Among them, I d represents the corrected image sequence, I represents the image sequence before correction, M represents the updated correction model and M = δM b , δ represents the optimal correction coefficient, M b Represents the basic vignetting correction model.

[0047] Beneficial effects: The method for removing grid-like shadows from a spliced ​​image based on consistency of overlapping areas provided by the present invention has the following beneficial effects:

[0048] 1. The present invention can accurately correct the vignetting effect in the image and eliminate the grid shadow by establishing a vignetting correction model based on the calibration image and the spherical fitting method, so as to make the brightness distribution of the image more uniform; by dynamically calculating the optimal correction coefficient and adjusting the correction model according to the gray value error of the overlapping area, the correction process can dynamically adapt to the changes in the vignetting effect of different images, thereby improving the adaptability of the correction model, eliminating the need for manual adjustment for each group of images, greatly improving the calibration accuracy and automation, and reducing the error caused by the vignetting effect in the image.

[0049] 2. The present invention can ensure the consistency of grayscale values ​​in overlapping areas of serialized images during the stitching process by minimizing the grayscale value error in overlapping areas, avoiding problems such as inconsistent brightness and grid-like shadows, thereby eliminating inconsistent brightness and stitching marks and ensuring smooth transition and uniformity of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0051] Figure 1 It is the overall flow chart of the method of the present invention;

[0052] Figure 2 This is a schematic diagram of the deviation image of the present invention;

[0053] Figure 3 is a schematic diagram of a basic vignetting correction model of the present invention;

[0054] Figure 4 is a schematic diagram of the overlapping area between the sequence images in the present invention;

[0055] Figure 5 It is an image data acquisition path diagram in an embodiment of the present invention;

[0056] Figure 6 It is a comparison diagram before and after vignetting calibration is performed on an image in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described more clearly and completely below by way of a preferred embodiment in conjunction with the accompanying drawings, but the present invention is not limited to the scope of the embodiment.

[0058] like Figure 1 As shown, the present invention provides a method for removing grid-like shadows from a spliced ​​image based on consistency of overlapping areas, which specifically includes the following steps:

[0059] S101, obtaining an image sequence to be stitched, wherein two adjacent images in the image sequence have a partial overlapping area.

[0060] S102, obtaining a calibration image shot under a uniform light source, selecting a central area of ​​the calibration image as a reference area for an ideal grayscale value, calculating a deviation of the grayscale value of each pixel in the calibration image relative to the reference area, and generating a deviation image.

[0061] Ideally, when the camera images a uniform target, the grayscale values ​​of all pixels in the image should be the same in theory. However, due to the vignetting phenomenon, the grayscale value of each pixel will gradually decrease as the distance from the imaging center increases. In one embodiment, the grayscale value of the image in the central area of ​​the calibration image is selected as the ideal grayscale value, and the difference between it and the grayscale value of all pixels in the image is made to obtain a deviation image. The deviation image is in the form of:

[0062] I dif =I i -I c

[0063] Among them, I dif represents the deviation image, I i Represents the ideal grayscale value, that is, all grayscale values ​​are equal to I c The mean value of the central area, I c is the gray value of the calibration image.

[0064] like Figure 2 As shown in the figure, a spatial coordinate system is constructed with the image width as the X-axis, the image height as the Y-axis, and the grayscale deviation value as the Z-axis to visualize the morphology of the deviation image.

[0065] S103 . Based on the generated deviation image, establish a basic vignetting correction model through spherical fitting.

[0066] Affected by external conditions such as the camera response curve, light source uniformity, and calibration plate uniformity, the deviation image will have abnormal points. Based on the grayscale distribution characteristics of the vignetting image, the present invention uses a spherical fitting vignetting model to filter out the abnormal points. In one embodiment, the specific steps are as follows:

[0067] Each pixel in the deviation image is regarded as a point in three-dimensional space. Based on the characteristics of the deviation image, a spherical fitting model is established. The spherical equation used is:

[0068] (x) 2 +(yb) 2 +(zc) 2 =R 2

[0069] Among them, x and y represent the two-dimensional coordinates of the pixel point, z represents the gray value of the pixel point, a, b and c represent the coordinates of the sphere center, and R represents the radius of the sphere;

[0070] Expand the above spherical equation to get:

[0071] x 2 +y 2 +z 2 -2ax-2by-2cz+a 2 +b 2 +c 2 =R 2

[0072] An error function is constructed to represent the fitting error, and the best fitting spherical parameters are solved by minimizing the error function. The formula of the error function is as follows:

[0073] E=Σ(x 2 +y 2 +z 2 -Ax-By-Cz+D) 2

[0074] Among them, parameter A = 2a, parameter B = 2b, parameter C = 2c, parameter D = a 2 +b 2 +c 2 -R 2 ;

[0075] When the error function E is the smallest, the circle formed by the parameters A, B, C, and D is closest to the fitting data. The following matrix is ​​constructed to solve the parameters A, B, C, and D:

[0076]

[0077] Let the matrix T be:

[0078]

[0079] Multiply both sides of the solution matrix of parameters A, B, C, and D by matrix T on the left to obtain:

[0080]

[0081] The simplified calculation formulas for parameters A, B, C and D are:

[0082]

[0083] Where i represents the index of the pixel point involved in the fitting, i = 0, 1, 2…, n, and n represents the total number of pixel points involved in the fitting;

[0084] According to the parameters A, B, C, and D, the coordinates a, b, and c of the sphere center and the radius R of the sphere can be calculated. The formula is as follows:

[0085]

[0086] The spherical radius of the spherical center coordinates of the spherical equation is determined according to the solved parameters, and the basic correction model is obtained. The formula of the basic vignetting correction model is as follows:

[0087]

[0088] Among them, M b represents the basic vignetting correction model, and F(x,y) represents the spherical equation deformation.

[0089] like Figure 3 As shown, the columns of the deviation image are taken as x and the rows as y, and the above basic vignetting correction model formula is introduced to generate an image to obtain the basic vignetting correction model M. b .

[0090] S104, determining the overlapping area between each adjacent image by an image registration algorithm, and calculating the sum of grayscale value errors of corresponding pixels in each overlapping area.

[0091] In the process of serialized image stitching, the positional relationship between each image to be stitched and the adjacent image to be stitched can be solved through the image registration algorithm, thereby obtaining the overlapping area between each image to be stitched and the adjacent image, such as Figure 4 , which is a schematic diagram of the overlapping area between adjacent images to be stitched, wherein rectangular frames A, B, and C are three images to be stitched, and regions R1 and R2 are the overlapping areas of images A and B and images B and C, respectively.

[0092] In an ideal state, since the objects are the same, the difference in the grayscale values ​​of the corresponding pixels of the two images A and B within the range of region R1 is 0, but due to the vignetting phenomenon, the difference in the corresponding pixels of the two images in the overlapping area is large. In one embodiment, in the left half of region R1, the grayscale value of image A is greater than that of image B, and in the right half of region R1, the grayscale value of image B is greater than that of image A, and the same is true for region R2. Therefore, the objective function is constructed so that the sum of the corresponding pixel differences of all images in all overlapping areas of the panoramic image after vignetting elimination is minimized.

[0093] In one embodiment, the calculation formula for the sum of grayscale value errors of corresponding pixels in each overlapping area is as follows:

[0094]

[0095] Among them, Dif(R) represents the gray value error sum of corresponding pixels in the overlapping area, R represents the overlapping area, and A jl and B jl Respectively represent the grayscale values ​​of the two images in the overlapping area at the pixel coordinates (j, l), w and h represent the width and height of the overlapping area respectively;

[0096] S105 , constructing a total grayscale value error sum of corresponding pixels in all overlapping regions based on the grayscale value error sum of corresponding pixels in each overlapping region.

[0097] The calculation formula for the total gray value error of corresponding pixels in all overlapping areas is as follows:

[0098]

[0099] Among them, F represents the total gray value error sum of corresponding pixels in all overlapping areas, k represents the overlapping area index, and m represents the total number of overlapping areas.

[0100] S106, determining an optimal correction coefficient by minimizing the sum of grayscale value errors of corresponding pixels in all overlapping areas, and dynamically adjusting the basic vignetting correction model according to the optimal correction coefficient to obtain an updated correction model.

[0101] In one embodiment, the specific steps of determining the optimal correction coefficient by minimizing the sum of grayscale value errors of corresponding pixels in all overlapping areas, and dynamically adjusting the basic vignetting correction model according to the optimal correction coefficient to obtain the updated correction model are as follows:

[0102] S601. Set a value range of the correction coefficient, and divide the value range into several parts.

[0103] The correction coefficient range is set to m~n, and the range of m and n is equally divided into s parts. According to experimental results, the correction coefficient range is set to 0~2, and s is set to 20, which is more ideal.

[0104] S602, performing vignetting correction on the image sequence to be spliced ​​by combining several groups of correction coefficients within the value range with a basic vignetting correction model, and calculating the sum of total grayscale value errors of corresponding pixels in all overlapping areas of the image sequence after vignetting correction.

[0105] The calculation formula for vignetting correction of an image by combining several groups of correction coefficients within the value range with the basic vignetting correction model is as follows:

[0106] I ds =I+δ s M b

[0107] Among them, I ds represents the image sequence after vignetting correction by applying the sth group of correction coefficients, I represents the image sequence before vignetting correction, δ s represents the sth group of correction coefficients, where s represents the correction coefficient index.

[0108] Substitute s groups of data in the range of m to n into the above formula for vignetting correction, and use the calculation formula of the total gray value error and the total gray value error of the corresponding pixels in all overlapping areas of the vignetting corrected image to calculate the error F, and select the correction coefficient value with the smallest error F as the optimal correction coefficient.

[0109] S603 , selecting the total gray value error of corresponding pixels in all overlapping areas and the minimum correction coefficient as the optimal correction coefficient, updating the basic vignetting correction model, and obtaining an updated correction model.

[0110] S107 , using the updated correction model to correct the image sequence to be spliced, generating a corrected image sequence, completing vignetting correction, and eliminating grid-like shadows.

[0111] The calculation formula for correcting the image sequence to be stitched using the updated correction model is as follows:

[0112] I d =I+M

[0113] Among them, I d represents the corrected image sequence, I represents the image sequence before correction, M represents the updated correction model and M = δM b , δ represents the optimal correction coefficient, M b Represents the basic vignetting correction model.

[0114] In a practical application of the present invention, a super-depth digital microscope is used to collect a serialized image data set, which is a vignetted image generated by optical characteristics. The image collection method is: the vignetted image sequence is collected along an S-shaped trajectory with a step length of 2 / 3 of the image width in the X direction and a step length of 2 / 3 of the image height in the Y direction. Figure 5 The image acquisition path diagram is shown. The camera moves along the path 1 to 6 and acquires images. The position of each image to be stitched on the panoramic image is determined by local matching and global optimization of the image sequence. The overlapping area R is calculated according to the position of each image to be stitched on the panoramic image, the correction coefficient is solved, the vignetting correction is performed using the correction model, and the images are fused.

[0115] When the vignetting image generated by optical characteristics is used as a data set and the method provided by the present invention is applied to perform vignetting calibration to eliminate grid-like shadows, the comparison images before and after calibration are as follows: Figure 6 As shown, the left side of the figure is the image before vignetting calibration, and the right side is the image after vignetting calibration. Using the method of the present invention, all the images to be stitched are calibrated and then stitched and fused. The obtained panoramic image has better uniformity and weakens the grid-like shadows in the panoramic image.

[0116] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas, characterized in that: The specific steps include: Acquire an image sequence to be stitched, where two adjacent images in the image sequence have a partial overlapping area; Obtain a calibration image taken under a uniform light source, select the central area of ​​the calibration image as the reference area of ​​the ideal grayscale value, calculate the deviation of the grayscale value of each pixel in the calibration image relative to the reference area, and generate a deviation image; Based on the generated deviation image, a basic vignetting correction model is established through spherical fitting; Determine the overlapping area between each adjacent image through the image registration algorithm, and calculate the gray value error sum of the corresponding pixel points in each overlapping area; Based on the gray value error sum of the corresponding pixel points in each overlapping area, a total gray value error sum of the corresponding pixel points in all overlapping areas is constructed; The optimal correction coefficient is determined by minimizing the sum of gray value errors of corresponding pixels in all overlapping areas, and the basic vignetting correction model is dynamically adjusted according to the optimal correction coefficient to obtain an updated correction model; The updated correction model is used to correct the image sequence to be stitched, generate a corrected image sequence, complete vignetting correction, and eliminate grid-like shadows.

2. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 1, characterized in that: The deviation image is of the form: I dif =I i -I c Among them, I dif represents the deviation image, I i Represents the ideal grayscale value, that is, all grayscale values ​​are equal to I c The mean of the central area, I c is the gray value of the calibration image.

3. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 1, characterized in that: Based on the generated deviation image, the basic vignetting correction model is established through spherical fitting. The specific steps are as follows: Each pixel in the deviation image is regarded as a point in three-dimensional space. Based on the characteristics of the deviation image, a spherical fitting model is established. The spherical equation used is: (x-a) 2 +(y-b) 2 +(z-c) 2 =R 2 Among them, x and y represent the two-dimensional coordinates of the pixel point, z represents the gray value of the pixel point, a, b and c represent the coordinates of the sphere center, and R represents the radius of the sphere; An error function is constructed to represent the fitting error, and the best fitting spherical parameters are solved by minimizing the error function. The formula of the error function is as follows: Among them, parameter A = 2a, parameter B = 2b, parameter C = 2c, parameter D = a 2 +b 2 +c 2 -R 2 ; The calculation formulas for parameters A, B, C and D are: Where i represents the index of the pixel point involved in the fitting, i = 0, 1, 2…, n, and n represents the total number of pixel points involved in the fitting; The spherical radius of the spherical center coordinates of the spherical equation is determined according to the solved parameters, and the basic vignetting correction model is obtained. The formula of the basic vignetting correction model is as follows: Among them, M b represents the basic vignetting correction model, and F(x,y) represents the spherical equation deformation.

4. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 1, characterized in that: The calculation formula for the sum of the gray value errors of corresponding pixels in each overlapping area is as follows: Among them, Dif(R) represents the gray value error sum of corresponding pixels in the overlapping area, R represents the overlapping area, and A jl and B jl Respectively represent the grayscale values ​​of the two images in the overlapping area at the pixel coordinates (j, l), w and h represent the width and height of the overlapping area respectively; The calculation formula for the total gray value error of corresponding pixels in all overlapping areas is as follows: Among them, F represents the total gray value error sum of corresponding pixels in all overlapping areas, k represents the overlapping area index, and m represents the total number of overlapping areas.

5. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 1, characterized in that: The optimal correction coefficient is determined by minimizing the sum of the grayscale value errors of the corresponding pixels in all overlapping areas, and the basic vignetting correction model is dynamically adjusted according to the optimal correction coefficient. The specific steps to obtain the updated correction model are as follows: Set the value range of the correction coefficient and divide the value range into several parts; Several groups of correction coefficients within the value range are combined with the basic vignetting correction model to perform vignetting correction on the image sequence to be spliced, and the sum of the total gray value errors of the corresponding pixels in all overlapping areas of the image sequence after vignetting correction is calculated; The total gray value error of the corresponding pixels in all overlapping areas and the minimum correction coefficient are selected as the optimal correction coefficient, and the basic vignetting correction model is updated to obtain an updated correction model.

6. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 5, characterized in that: The calculation formula for vignetting correction of an image by combining several groups of correction coefficients within the value range with the basic vignetting correction model is as follows: I ds =I+δ s M b Among them, I ds represents the image sequence after vignetting correction by applying the sth group of correction coefficients, I represents the image sequence before vignetting correction, δ s represents the sth group of correction coefficients, where s represents the correction coefficient index.

7. The method for removing grid-like shadows from spliced ​​images based on consistency of overlapping areas according to claim 1, characterized in that: The calculation formula for correcting the image sequence to be stitched using the updated correction model is as follows: I d =I+M Among them, I d represents the corrected image sequence, I represents the image sequence before correction, M represents the updated correction model and M = δM b , δ represents the optimal correction coefficient, M b Represents the basic vignetting correction model.

Images