A method and system for color uniformity of photos of a dual-configuration photogrammetric camera

Abstract

The present invention discloses a method and system for color grading of stitched photos using a double-stitched photogrammetry camera. When two original photos are stitched into one photo along the column direction, the photo color grading method includes the following steps: S1, sliding a first sliding window from one end to the other in the transition zone between the two photos to obtain N uniform color blocks; S2, selecting one side of the seam between the stitched photos as a reference side, and calculating the brightness average deviation ΔVi for each uniform color block obtained. i , i = 0, 1, 2, ..., N-1; S3, optimize the average brightness deviation of the uniform color block; S4, adjust the brightness value of the non-reference side; S5, feather the seam. This method can even out the colors of stitched photos without obtaining the overlapping and non-overlapping areas of the original photos. It can effectively eliminate color differences at the seam, making the color transition at the seam more natural.

Description

Technical Field

[0001] The invention belongs to the field of aerial surveying and mapping, and in particular relates to a method and a system for color-leveling stitched photos taken by a large-format double-stitched photogrammetry camera. Background Art

[0002] Digital aerial photogrammetric cameras are one of the most widely used sensors in aerial surveying and mapping, primarily used to collect ground image data. To improve aerial photography efficiency, surveying units generally prefer large-format photogrammetric cameras. However, if the camera's CCD / CMOS sensor is made very large, the camera's production cost will be extremely high. To balance cost and efficiency, large-format photogrammetric cameras currently utilize stitching and synthesis to achieve a larger format and improve data collection efficiency. Available methods include single-camera pan-scan stitching, virtual synthesis of multiple oblique cameras, intra-field stitching of multiple vertical cameras, and spectroscopic stitching. By combining techniques such as lens geometric and radiometric calibration, photo feature matching, and POS data and photographic parameters, multiple images captured by multiple or a single camera are stitched together into a single "synthetic" image with a virtual projection center and a fixed virtual focal length.

[0003] Due to limitations in optical technology and machining processes, even cameras of the same model manufactured from different batches of modules may not have identical optical lenses. In addition to lenses, differences in shutter speed, CCD gain, and other factors can also lead to variations in brightness and color, even when using the same camera model under the same imaging conditions. For this reason, before large-format stitched cameras are shipped, manufacturers may use methods such as whiteboard calibration to determine the exposure differences between the component cameras under the same imaging conditions. Based on this information, they adjust the camera exposure parameters and write them into the camera firmware to ensure that photos produced by different cameras under the same imaging conditions are essentially consistent. This ensures that the brightness and color transitions at the seams of the stitched photos are natural, with no noticeable color difference.

[0004] To facilitate use and maintenance and enhance camera reliability, large-format stitching cameras typically allow for replacement of component cameras. If a component camera malfunctions, it can be promptly replaced with a camera of the same model. However, this presents a significant problem: exposure calibration and parameter adjustments are not performed between the replaced and remaining cameras, resulting in color differences within the stitched large-format images. This color difference is further amplified in conditions of poor visibility and illumination, as well as in areas such as water and dense vegetation. Failure to eliminate this color difference can significantly negatively impact the quality of the resulting image. Therefore, color difference in the stitched images must be eliminated during the preprocessing stage.

[0005] Currently, workers in production typically rely on image processing software to manually perform color grading on stitched photos. This process can take several weeks when the number of photos is large, consuming considerable time and effort.

[0006] Current research on color balancing in stitched photos typically uses feature-based matching algorithms to extract and match key points of interest when stitching and balancing raw photos taken by multiple cameras. This algorithm then estimates overlapping and non-overlapping areas between photos. Color balancing utilizes color distribution information from overlapping areas or the entire image. For example, the technical solutions disclosed in literature 1: Lü Nan, Gou Yonggang, Long Chuan, et al. Multi-camera image stitching color equalization algorithm [J]. Bulletin of Surveying and Mapping, 2016, (7): 44-47; Literature 2: Li Yong. Research on image seamless stitching and color equalization technology [D]. Xidian University, 2013; Literature 3: Chongqing Surveying Institute. A multi-camera image stitching color equalization method: ZL 20151 0375104.5 [P]. 2018-6-1; Literature 4: Wang Xiaoli, Dai Huayang, Yu Tao, et al. Research on UAV image stitching color equalization based on multi-resolution fusion [J]. Bulletin of Surveying and Mapping, 2013, (6): 27-30; Literature 5: Lin Jingliang, Chen Yuelin. Multi-camera image stitching automatic color equalization algorithm [J]. Computer Applications, 2010, 30 (12): 3236-3237, 3251. The above methods are difficult to eliminate the brightness and color differences caused by land type changes. In addition, for the stitched photos, the overlapping and non-overlapping areas of the original photos cannot be obtained, and the above method cannot be performed. Summary of the Invention

[0007] Purpose of the invention: In response to the problems existing in the prior art, the present invention provides a method for color-leveling stitched photos using a dual-stitched photogrammetry camera. This method can level the colors of stitched photos without obtaining the overlapping and non-overlapping areas of the original photos. It can effectively eliminate color differences at the seams and make the color transition at the seams more natural.

[0008] Technical solution: On one hand, the present invention discloses a method for color-matching stitched photos using a double-stitched photogrammetry camera. When two original photos are stitched into one photo along the column direction, the method comprises the following steps:

[0009] S1, the first sliding window slides from one end to the other end in the transition zone between the two photos to obtain N uniform color blocks; the transition zone is centered on the seam and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2 ΔR , the covered image block is a uniform color block; the sliding step of the first sliding window is N hgt； ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0;

[0010] S2. Select one side of the seam between the two sides of the spliced photos as the reference side, and calculate the average value deviation of brightness ΔV for each uniform color block obtained. i , i=0, 1, 2, ..., N-1; the lightness average value deviation is the difference between the lightness average values of the non-reference side and the reference side on both sides of the seam;

[0011] S3. Optimize the brightness average deviation of the uniform color block;

[0012] S4, adjusting the brightness value of the non-reference side;

[0013] S5. Feather the seams.

[0014] Furthermore, in S2, for the i-th uniform color block B i Calculate the lightness average deviation ΔV i The specific steps are:

[0015] S21, against B i Left block B on the left side of the seam i，left Calculate the average brightness of the pixels within Including steps:

[0016] S211, let B i，left All pixels in the set S pxl ;

[0017] S212, to S pxl Calculate the average brightness of all pixels within and standard deviation σ V ;

[0018] S213, in S pxl Inner filter brightness is located at The pixels within constitute a new pixel set S′ pxl , where z is the brightness mean value shift coefficient, z>0;

[0019] S214, if S pxl and S′ pxl For different sets, let S pxl =S′ pxl , enter step S212 iterative processing, otherwise stop the iteration, let

[0020] S22. Calculate B i Right block B on the right side of the seam i，right The average brightness

[0021] S23, if the left side of the joint is taken as the reference side, B iThe lightness mean deviation is:

[0022] If the right side of the joint is taken as the reference side,

[0023] Furthermore, the specific steps of optimizing the brightness average value deviation of the uniform color block in step S3 are:

[0024] S31. Calculate the mean value of the brightness average deviation of N uniform color blocks and standard deviation σ ΔV ;

[0025] S32, in N uniform color blocks, the average value of brightness deviation is in interval I ΔV The uniform color blocks in the set S ΔV , S ΔV The elements in (l, ΔV l ), where l is the uniform color block number, ΔV l is the average value deviation of the brightness of the uniform color block;

[0026] The lightness average value deviation is not in interval I ΔV The uniform color blocks in the set S invalidΔV , S invalidΔV The elements in are the uniform color block numbers, I ΔV is defined as:

[0027]

[0028] Where u is the deviation coefficient of the average value of brightness, u>0; ΔV max >0, the maximum value of the preset brightness average deviation;

[0029] S33, to S ΔV The uniform color blocks within the image are fitted with a cubic spline curve, the independent variable is the uniform color block number, and the dependent variable is the average value deviation of the uniform color block brightness;

[0030] S34, the cubic spline curve fitted by S33 is S invalidΔV The uniform color block in the inner part interpolates the new brightness average deviation ΔV′ with the serial number as the independent variable. If ΔV′>ΔV max , then let ΔV′=ΔV max , if ΔV′<-ΔV max , then let ΔV′=-ΔV max , add the newly interpolated uniform color block to the set S ΔV ;

[0031] S35, S ΔV The uniform color blocks are sorted by serial number;

[0032] S36, using sliding window mean filtering to filter S ΔV The deviation from the average brightness value of the internal uniform color block is smoothed and rounded to an integer value.

[0033] Furthermore, the step S4 is specifically as follows:

[0034] S41, set i=0;

[0035] S42, for the non-reference side, the line number is in the interval [iN hgt , (i+1)N hgt -1], using S ΔV The average value deviation of brightness of the uniform color block with serial number i is ΔV i Adjust the lightness value:

[0036] V modify =V orig -ΔV i

[0037] Where V orig is the original brightness value of the pixel, V modify is the brightness value of the pixel after adjustment;

[0038] S43. If i=N-1, end; otherwise, increment the value of i by one and execute S42 again.

[0039] Furthermore, the step S5 is specifically as follows:

[0040] S51, the second sliding window slides from one end to the other end of the feathering zone of the two photos to obtain M feathering blocks; the feathering zone is centered at the seam and extends N blocks to both sides. blk / 2 pixels to obtain the area; the width and height of the second sliding window are both N blk , the covered image block is the feathered block, N blk is an even number; the sliding step of the second sliding window is N blk ;

[0041] S52, extract the j-th feathering block Y j Background color The steps include:

[0042] S521, Y j Perform K-means clustering based on the brightness value, assuming that K clusters are obtained;

[0043] S522: Among the K clusters, if the distance between the centers of two clusters is less than the threshold ΔV cluster , then merge the two clusters, and finally get K′ clusters;

[0044] S523, find the cluster C with the most pixels among the K′ clustersmax , calculate C max The minimum value V of the pixel brightness min and maximum value V max ;

[0045] S524, traverse F j If the pixel has a brightness value in the interval [V min , V max ], then add the pixel to the background pixel set S bg ;

[0046] S525, calculate Y j Average background color in RGB color space The value of the nth channel For S bg The average value of the nth channel that makes up the pixel; n = 1, 2, 3;

[0047] S53, unifying the background color of the feathered block, the steps specifically include:

[0048] S531, Y j Left block Y located on the left side of the seam j，left and the right block Y located on the right side of the seam j，right The kernel size is N kernel , the standard deviation is σ kernel The Gaussian filter is used for filtering, and the filtering results are G j，left and G j，right ;

[0049] S532. Calculate the image F of the left block and the right block respectively by fusing the high-frequency information and the average background. j，left and F j，right :

[0050]

[0051]

[0052] S533, F j，left and F j，right Splice along the column direction to form F j ;

[0053] S54, to S bg The pixels in are feathered, and the specific steps include:

[0054] S541, set S bg A pixel in Y j Calculate the feathering weight parameter p in the rth row and cth column of

[0055]

[0056] If p>1, then let p=1; where R a The calculation method is:

[0057] R a =(rN blk / 2) 2 +(cN blk / 2) 2

[0058] The calculation method of d is:

[0059]

[0060] where f feather is the feathering level coefficient, which is in the interval [0, 1];

[0061] S542, calculate the final feather value DN of channel b j，r，c，b :

[0062] DN j，r，c，b =pY j，r，c，b +(1-p)F j，r，c，b

[0063] Among them, Y j，r，c，b It's Y j The value of the pixel in the rth row and cth column in the bth channel, F j，r，c，b It's F j The value of the pixel in the rth row and the cth column in the bth channel; b = 1, 2, 3;

[0064] S543, DN j，r，c，b Restricted to the legal value range of the data type: If DN j，r，c，b Greater than the upper limit of the DN value data type DN max , then let DN j，r，c，b =DN max If DN j，r，c，b Less than the lower limit of the DN value data type DN min , then let DN j，r，c，b =DN min .

[0065] Furthermore, the value range of the brightness average value shift coefficient z is [1.282, 1.960].

[0066] Furthermore, after step S532 and before step S533, the following steps are further included:

[0067] S532B, let Y j，left =F j，left , Y j，right =F j，right ;

[0068] Jump to step S531 for Y j，left and Y j，right Again, the kernel size is N kernnel , the standard deviation is σ kernel The Gaussian filter is used to filter and update the filtering result G j，left and G j，right , recalculate F according to formula (1) and formula (2) j，left and F j，right ; and count the number of Gaussian filters. If the count value reaches the preset number of filters T, execute step S533.

[0069] Furthermore, it also includes:

[0070] When two photos are stitched in row direction, rotate the stitched image 90° or -90°;

[0071] Process the rotated image according to steps S1-S5;

[0072] Rotate the processed image by -90° or 90°.

[0073] On the other hand, the present invention also discloses a system for implementing the above-mentioned method for color-leveling photos of a double-stitched photogrammetry camera, comprising:

[0074] The uniform color block acquisition module 1 is used to slide the first sliding window from one end to the other end of the transition zone between the two photos to obtain N uniform color blocks; the transition zone is centered on the seam and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2ΔR; the sliding step of the first sliding window is N hgt； ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0;

[0075] Luminance average deviation calculation module 2, used to calculate the luminance average deviation ΔV of the uniform color block i , i=0, 1, 2, ..., N-1; the brightness average deviation is the difference between the brightness averages of the non-reference side and the reference side on both sides of the seam; the reference side is the side of the seam of the spliced photos that is selected and has a constant brightness value;

[0076] The brightness average deviation optimization module 3 is used to optimize the brightness average deviation of the uniform color block;

[0077] A brightness value adjustment module 4, configured to adjust the brightness value of the non-reference side;

[0078] The feathering module 5 is used to feather the seams.

[0079] Furthermore, it also includes: an image rotation module 6, which is used to rotate the image by 90° or -90°.

[0080] Beneficial effects: The color-uniform method for stitching photos of a double-stitched photogrammetry camera disclosed in the present invention has good adaptability by processing the transition zones and feathering zones on both sides of the seam, and can adapt to different types of land objects at the seam, and is particularly suitable for water surfaces and vegetation. By estimating the local bilateral brightness difference and background color based on small areas such as uniform color blocks and feathering blocks, it is possible to better avoid the influence of land type changes on the estimation, effectively eliminate the color difference at the seam, and make the color transition smooth and natural. For the water surface, the present invention can retain the characteristic information of the water surface without changing the color and brightness of the water surface flare. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Schematic diagram of the division of the transition zone in the embodiment;

[0082] Figure 2 Schematic diagram of the division of the feathering zone in the embodiment;

[0083] Figure 3 This is a flow chart of the color uniformity method for stitching photos using a double-stitched photogrammetry camera disclosed in the present invention;

[0084] Figure 4 The figure is a schematic diagram of the composition of the stitching and color grading system of the double-stitching photogrammetry camera disclosed in the present invention. DETAILED DESCRIPTION

[0085] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0086] The present invention discloses a method for color-uniform stitching of photos using a double-stitched photogrammetry camera. When two original photos are stitched into one photo along a column direction, the method comprises the following steps:

[0087] S1, the first sliding window slides from one end to the other end in the transition zone between the two photos to obtain N uniform color blocks; the transition zone is centered on the seam and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2ΔR, the covered image block is a uniform color block; the sliding step of the first sliding window is N hgt ; ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0;

[0088] like Figure 1 As shown, two photos pic1 and pic2 are spliced into one image along the column direction, and the seam is expanded N on both sides. widthThe area of pixels is the transition zone 100, such as Figure 1 The middle oblique line fills the area. The transition zone is the color transition area on both sides of the seam of the spliced photos. The first sliding window 200 slides from top to bottom in the transition zone, dividing the transition zone area into multiple uniform color blocks. In this embodiment, in order to avoid independent calculation of adjacent uniform color blocks, which leads to excessive changes in brightness difference, the height of the first sliding window is set to a value greater than the sliding step size, that is, ΔR>0, so that there is partial overlap between adjacent uniform color blocks, and the row height of the overlapping area is 2ΔR. In this embodiment, N hgt Value and N width Same, ΔR≤N hgt / 2.

[0089] S2. Select one side of the joint between the two sides of the spliced photos as the reference side, and calculate the average value deviation ΔV of the brightness for each uniform color block obtained. i , i=0, 1, 2, ..., N-1; the lightness average value deviation is the difference between the lightness average values of the non-reference side and the reference side on both sides of the seam;

[0090] The brightness on both sides of the seam differs. In the present invention, color grading is achieved by maintaining the brightness of one side constant while adjusting the brightness of the other side to bring the two sides closer together. The side with the constant brightness is the reference side. This preserves the original characteristics of the reference side as much as possible.

[0091] For the i-th uniform color block B i Calculate the lightness average deviation ΔV i The specific steps are:

[0092] S21, against B i Left block B on the left side of the seam i，left Calculate the average brightness of the pixels within Including steps:

[0093] S211, let B i，left All pixels in the set S pxl ;

[0094] S212, to S pxl Calculate the average brightness of all pixels within and standard deviation σ V ;

[0095] S213, in S pxl Inner filter brightness is located at The pixels within constitute a new pixel set S′ pxl , where z is the brightness average shift coefficient, z>0;

[0096] By the above screening, the influence of larger or smaller brightness outliers on the statistics can be eliminated. In this embodiment, the value range of z is [1.282, 1.960].

[0097] S214、If S pxl and S′ pxl For different sets, let S pxl =S′ pxl , enter step S212 iterative processing, otherwise stop the iteration, let

[0098] S22. Calculate B i Right block B on the right side of the seam i，right The average brightness

[0099] The calculation method and same.

[0100] S23, if the left side of the joint is taken as the reference side, B i The lightness mean deviation is:

[0101] If the right side of the joint is taken as the reference side,

[0102] S3. Optimize the average brightness deviation of the uniform color block. The specific steps are as follows:

[0103] S31. Calculate the mean value of the brightness average deviation of N uniform color blocks and standard deviation σ ΔV ;

[0104] S32, in N uniform color blocks, the average value of brightness deviation is in interval I ΔV The uniform color blocks in the set S ΔV , S ΔV The elements in (l, ΔV l ), where l is the uniform color block number, ΔV l is the average value deviation of the brightness of the uniform color block;

[0105] The lightness average value deviation is not in interval I ΔV The uniform color blocks in the set S invalidΔV , S invalidΔV The elements in are the uniform color block numbers, I ΔV is defined as:

[0106]

[0107] Wherein, u is the brightness mean value deviation offset coefficient, u>0; in this embodiment, the value range of u is [1.282, 1.960]; ΔVmax >0, which is the maximum value of the preset brightness average deviation; in this embodiment, the maximum brightness difference on both sides of the seam is measured as ΔV max .

[0108] S33, to S ΔV The uniform color blocks within the image are fitted with a cubic spline curve, the independent variable is the uniform color block number, and the dependent variable is the average value deviation of the uniform color block brightness;

[0109] S34, the cubic spline curve fitted by S33 is S invalidΔv The uniform color block in the inner part interpolates the new brightness average deviation ΔV′ with the serial number as the independent variable. If ΔV′>ΔV max , then let ΔV′=ΔV max , if ΔV′<-ΔV max , then let ΔV′=-ΔV max , that is, the value of ΔV′ is limited to [-ΔV max , ΔV max ] range; add the newly interpolated uniform color block (including uniform color block sequence number and ΔV′) to the set S ΔV ;

[0110] S35, S ΔV The uniform color blocks are sorted by serial number;

[0111] S36, using sliding window mean filtering to filter S ΔV The deviation of the brightness average value of the internal uniform color block is smoothed and rounded to an integer value;

[0112] S4. Adjust the brightness value of the non-reference side; specifically:

[0113] S41, set i=0;

[0114] S42, for the non-reference side, the line number is in the interval [iN hgt , (i+1)N hgt -1], using S ΔV The average value deviation of brightness of the uniform color block with serial number i is ΔV i Adjust the lightness value:

[0115] V modify =V orig -ΔV i

[0116] Where V orig is the original brightness value of the pixel, V modify is the brightness value of the pixel after adjustment;

[0117] S43. If i=N-1, end; otherwise, increment the value of i by one and execute S42 again.

[0118] S5. Feather the seams to make the seam transition natural. The specific steps are:

[0119] S51, the second sliding window slides from one end to the other end of the feathering zone of the two photos to obtain M feathering blocks; the feathering zone is centered at the seam and extends N blocks to both sides. blk / 2 pixels to obtain the area; the width and height of the second sliding window are both N blk , the covered image block is the feathering block, N blk is an even number; the sliding step of the second sliding window is N blk ;

[0120] like Figure 2 As shown, the feather zone 300 is inside the transition zone 100, N blk <2N width In this embodiment, N blk =N width , that is, the width of the feathering zone is half the width of the transition zone; the second sliding window 400 slides from top to bottom in the feathering zone, and obtains multiple N blk ×N blk Feather block.

[0121] S52, extract the j-th feathering block Y j Background color The steps include:

[0122] S521, Y j Perform K-means clustering based on the brightness value, assuming that K clusters are obtained;

[0123] S522: Among the K clusters, if the distance between the centers of two clusters is less than the threshold ΔV cluster , then merge the two clusters, and finally get K′ clusters;

[0124] In the present invention, clustering is performed based on the brightness value of the feathered block pixels, so the distance between the centers of two clusters is the absolute value of the brightness difference between the two cluster centers; the threshold ΔV cluster is greater than or equal to 0 and less than ΔV max The value of .

[0125] S523, find the cluster C with the most pixels among the K′ clusters max , calculate C max The minimum value V of the pixel brightness min and maximum value V max ;

[0126] S524, traverse Y j If the pixel has a brightness value in the interval [V min , Vmax ], then add the pixel to the background pixel set S bg ;

[0127] S525, calculate Y j Average background color in RGB color space The value of the nth channel For S bg The average value of the nth channel that makes up the pixel; n = 1, 2, 3;

[0128] S53, unifying the background color of the feathered block, the steps specifically include:

[0129] S531, Y j Left block Y located on the left side of the seam j，left and the right block Y located on the right side of the seam j，right The kernel size is N kernel , the standard deviation is σ kernel The Gaussian filter is used for filtering, and the filtering results are G j，left and G j，right ;

[0130] N kernel >0, in this embodiment, take N kernel =N blk / 2+1,σ kernel The value can be in the interval [1, 3];

[0131] S532. Calculate the image F of the left block and the right block respectively by fusing the high-frequency information and the average background. j，left and F j，right :

[0132]

[0133]

[0134] To obtain a more uniform background, the calculation can be performed multiple times, that is, after S532, the following steps are also included:

[0135] S532B, let Y j，left =F j，left , Y j，right =F j，right ; Jump to step S531 for Y j，left and Y j，right Again, the kernel size is N kernel , the standard deviation is σ kernel The Gaussian filter is used to filter and update the filtering result G j，left and G j，right , recalculate F according to formula (1) and formula (2) j，left and Fj，right ; and count the number of Gaussian filters. If the count value reaches the preset number of filters T, execute step S533;

[0136] S533, F j，left and F j，right Splice along the column direction to form F j ;

[0137] S54, to S bg The pixels in are feathered, and the specific steps include:

[0138] S541, set S bg A pixel in Y j Calculate the feathering weight parameter p in the rth row and cth column of

[0139]

[0140] If p>1, let p=1; where R a The calculation method is:

[0141] R a =(rN blk / 2) 2 +(cN blk / 2) 2

[0142] The calculation method of d is:

[0143]

[0144] where f feather is the feathering level coefficient, which is in the interval [0, 1]. In this embodiment, f feather Take a value greater than or equal to 0.3;

[0145] S542, calculate the final feather value DN of channel b j，r，c，b :

[0146] DN j，r，c，b =pY j，r，c，b +(1-p)F j，r，c，b

[0147] Among them, Y j，r，c， b is Y j The value of the pixel in the rth row and cth column in the bth channel, F j，r，c，b It's F j The value of the pixel in the rth row and the cth column in the bth channel; b = 1, 2, 3;

[0148] S543, DN j，r，c，b Restricted to the legal value range of the data type: If DN j，r，c，bGreater than the upper limit of the DN value data type DN max , then let DN j，r，c，b =DN max If DN j，r，c，b Less than the lower limit of the DN value data type DN min , then let DN j，r，c，b =DN min .

[0149] If DN j，r，c，b It is represented by a 1-byte unsigned integer, and its upper limit is DN max 255, lower limit DN min =0; that is, its legal value range is 0 to 255. Values outside this range cannot be represented by a 1-byte unsigned integer. This ensures that all channel values after the feathering band is feathered are valid values.

[0150] The above steps S1-S5 realize the uniform color of the photos spliced along the column. The process is as follows: Figure 3 When two photos are spliced in the row direction, the spliced image is first rotated 90° or -90° so that the seam is in the column direction; then the rotated image is processed according to steps S1-S5; finally, the processed image is rotated -90° or 90°.

[0151] The present invention also discloses a system for realizing the color uniformity method of stitching photos of the double-stitched photogrammetry camera, such as Figure 4 As shown, including:

[0152] The uniform color block acquisition module 1 is used to slide the first sliding window from one end to the other end of the transition zone between the two photos to obtain N uniform color blocks; the transition zone is centered on the seam and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2ΔR; the sliding step of the first sliding window is N hgt ; ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0;

[0153] Luminance average deviation calculation module 2, used to calculate the luminance average deviation ΔV of the uniform color block i , i=0, 1, 2, ..., N-1; the brightness average deviation is the difference between the brightness averages of the non-reference side and the reference side on both sides of the seam; the reference side is the side of the seam of the spliced photos that is selected and has a constant brightness value;

[0154] The brightness average deviation optimization module 3 is used to optimize the brightness average deviation of the uniform color block;

[0155] A brightness value adjustment module 4, configured to adjust the brightness value of the non-reference side;

[0156] Feathering module 5, used for feathering the seams;

[0157] The image rotation module 6 is used to rotate the image by 90° or -90°.

Claims

1. A method for color grading of photos using a double-stitched photogrammetry camera, characterized in that: When two original photos are spliced into one photo along the column direction, the steps include: S1, the first sliding window slides from one end to the other end in the transition zone between the two photos to obtain N uniform color blocks; the transition zone is centered on the seam and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2ΔR, the covered image block is a uniform color block; the sliding step of the first sliding window is N hgt ; ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0; S2. Select one side of the seam between the two sides of the spliced photos as the reference side, and calculate the average value deviation of brightness ΔV for each uniform color block obtained. i , i = 0, 1, 2, ..., N-1; the brightness average deviation is the difference between the brightness averages of the non-reference side and the reference side on both sides of the seam; the brightness average deviation ΔV i The specific calculation steps are: S21, against B i Left block B on the left side of the seam i,left Calculate the average brightness of the pixels within Including steps: S211, let B i,left All pixels in the set S pxl ; S212, to S pxl Calculate the average brightness of all pixels within and standard deviation σ V ; S213, in S pxl Inner filter brightness is located at The pixels within constitute a new pixel set S′ pxl , where z is the brightness mean value shift coefficient, z>0; S214, if S pxl and S′ pxl For different sets, let S pxl =S′ pxl , enter step S212 iterative processing, otherwise stop the iteration, let S22. Calculate B i Right block B on the right side of the seam i,right The average brightness S23, if the left side of the joint is taken as the reference side, B i The lightness mean deviation is: If the right side of the joint is taken as the reference side, S3. Optimize the brightness average deviation of the uniform color block; S4, adjusting the brightness value of the non-reference side; S5. Feather the seams.

2. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 1, wherein: The specific steps of optimizing the brightness average value deviation of the uniform color block in step S3 are: S31. Calculate the mean value of the brightness average deviation of N uniform color blocks and standard deviation σ ΔV ; S32, in N uniform color blocks, the average value of brightness deviation is in interval I ΔV The uniform color blocks in the set S ΔV , S ΔV The elements in (l,ΔV l ), where l is the uniform color block number, ΔV l is the average value deviation of the brightness of the uniform color block; The lightness average value deviation is not in interval I ΔV The uniform color blocks in the set S invalidΔV , S invalidΔV The elements in are the uniform color block numbers, I ΔV is defined as: Where u is the brightness mean deviation coefficient, u>0; ΔV max >0, which is the maximum value of the preset brightness average deviation; S33, to S ΔV The uniform color blocks within the image are fitted with a cubic spline curve, the independent variable is the uniform color block number, and the dependent variable is the average value deviation of the uniform color block brightness; S34, the cubic spline curve fitted by S33 is S invalidΔV The uniform color block in the inner part interpolates the new brightness average deviation ΔV′ with the serial number as the independent variable. If ΔV′>ΔV max , then let ΔV′=ΔV max , if ΔV′<-ΔV max , then let ΔV′=-ΔV max , add the newly interpolated uniform color block to the set S ΔV ; S35, S ΔV The uniform color blocks are sorted by serial number; S36, using sliding window mean filtering to filter S ΔV The deviation from the average brightness value of the internal uniform color block is smoothed and rounded to an integer value.

3. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 1, wherein: The step S4 is specifically as follows: S41, set i=0; S42, for the non-reference side, the line number is in the interval [iN hgt ,(i+1)N hgt -1], using S ΔV The average value deviation of brightness of the uniform color block with serial number i is ΔV i Adjust the lightness value: V modify =V orig -ΔV i Where V orig is the original brightness value of the pixel, V modify is the brightness value of the pixel after adjustment; S43. If i=N-1, end; otherwise, increment the value of i by one and execute S42 again.

4. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 1, wherein: The step S5 is specifically as follows: S51, the second sliding window slides from one end to the other end of the feathering zone of the two photos to obtain M feathering blocks; the feathering zone is centered at the seam and extends N blocks to both sides. blk / 2 pixels to obtain the area; the width and height of the second sliding window are both N blk , the covered image block is the feathered block, N blk is an even number; the sliding step of the second sliding window is N blk ; S52, extract the j-th feathering block Y j Background color The steps include: S521, Y j Perform K-means clustering based on the brightness value, assuming that K clusters are obtained; S522: Among the K clusters, if the distance between the centers of two clusters is less than the threshold ΔV cluster , then merge the two clusters, and finally get K′ clusters; S523, find the cluster C with the most pixels among the K′ clusters max , calculate C max The minimum value V of the pixel brightness min and maximum value V max ; S524, traverse Y j If the pixel has a brightness value in the interval [V min ,V max ], then add the pixel to the background pixel set S bg ; S525, calculate Y j Average background color in RGB color space The value of the nth channel For S bg The average value of the nth channel that makes up the pixel; n = 1, 2, 3; S53, unifying the background color of the feathered block, the steps specifically include: S531, Y j Left block Y located on the left side of the seam j,left and the right block Y located on the right side of the seam j,right The kernel size is N kernel , the standard deviation is σ kernel The Gaussian filter is used for filtering, and the filtering results are G j,left and G j,right ; S532. Calculate the image F of the left block and the right block respectively by fusing the high-frequency information and the average background. j,left and F j,right : S533, F j,left and F j,right Splice along the column direction to form F j ; S54, to S bg The pixels in are feathered, and the specific steps include: S541, set S bg A pixel in Y j Calculate the feathering weight parameter p in the rth row and cth column of If p>1, then let p=1; where R a The calculation method is: R a =(r-N blk / 2) 2 +(c-N blk / 2) 2 The calculation method of d is: where f feather is the feathering level coefficient, which is in the interval [0,1]; S542, calculate the final feather value DN of channel b j,r,c,b : DN j,r,c,b =pY j,r,c,b +(1-p)F j,r,c,b Among them, Y j,r,c,b It's Y j The value of the pixel in the rth row and cth column in the bth channel, F j,r,c,b It's F j The value of the pixel in the rth row and the cth column in the bth channel; b = 1, 2, 3; S543, DN j,r,c,b Restricted to the legal value range of the data type: If DN j,r,c,b Greater than the upper limit of the DN value data type DN max , then let DN j,r,c,b =DN max If DN j,r,c,b Less than the lower limit of the DN value data type DN min , then let DN j,r,c,b =DN min .

5. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 1, wherein: The value range of the brightness average value shift coefficient z is [1.282, 1.960].

6. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 4, wherein: After step S532 and before step S533, the following steps are further included: S532B, let Y j,left = F j,left , Y j,right = F j,right ; Jump to step S531 for Y j,left and Y j,right Again, the kernel size is N kernel , the standard deviation is σ kernel The Gaussian filter is used to filter and update the filtering result G j,left and G j,right , recalculate F according to formula (1) and formula (2) j,left and F j,right ; and count the number of Gaussian filters. If the count value reaches the preset number of filters T, execute step S533.

7. The method for color grading of a photo using a double-jointed photogrammetry camera according to claim 1, wherein: Also includes: When two photos are stitched in row direction, rotate the stitched image 90° or -90°; Process the rotated image according to steps S1-S5; Rotate the processed image by -90° or 90°.

8. A photo color grading system for a double-stitched photogrammetry camera, characterized in that: include: The uniform color block acquisition module (1) is used to slide the transition zone near the joint of the spliced photos from one end to the other end using a first sliding window to obtain N uniform color blocks; the transition zone is centered at the joint and extends N blocks to both sides. width The width of the first sliding window is 2N width , high is N hgt +2ΔR; the sliding step of the first sliding window is N hgt ; ΔR is the uniform color block overlap parameter, and ΔR is an integer greater than or equal to 0; The lightness average deviation calculation module (2) is used to calculate the lightness average deviation ΔV of the uniform color block. i , i = 0, 1, 2, ..., N-1; the brightness average deviation is the difference between the brightness averages of the non-reference side and the reference side on both sides of the seam; the reference side is the side of the seam of the spliced photo that is selected and has a constant brightness value; the brightness average deviation ΔV i The specific calculation steps are: S21, against B i Left block B on the left side of the seam i,left Calculate the average brightness of the pixels within Including steps: S211, let B i,left All pixels in the set S pxl ; S212, to S pxl Calculate the average brightness of all pixels within and standard deviation σ V ; S213, in S pxl Inner filter brightness is located at The pixels within constitute a new pixel set S′ pxl , where z is the brightness mean value shift coefficient, z>0; S214、If S pxl and S′ pxl For different sets, let S pxl =S′ pxl , enter step S212 iterative processing, otherwise stop the iteration, let S22. Calculate B i Right block B on the right side of the seam i,right The average brightness S23, if the left side of the joint is taken as the reference side, B i The lightness mean deviation is: If the right side of the joint is taken as the reference side, A brightness average deviation optimization module (3) is used to optimize the brightness average deviation of the uniform color block; A brightness value adjustment module (4) is used to adjust the brightness value of the non-reference side; The feathering module (5) is used for feathering the seam.

9. The photo color balancing system for a double-jointed photogrammetry camera according to claim 8, characterized in that: Also includes: The image rotation module (6) is used to rotate the image by 90° or -90°.

Images