Large scene UAV image uniform light and color method based on Bernstein polynomials

Abstract

The invention proposes a large-scene unmanned aerial vehicle image uniform light and color method based on Bernstein polynomials, comprising obtaining an object point through aerial triangulation processing, obtaining feature same-name point coordinate information in the image according to the relationship between the object point and the image, and extracting color values ​​corresponding to the same-name point coordinate information from different images; averaging the same-name point color values ​​of multiple images and assigning them to the object point as a reference benchmark value for uniform light and color; constructing a mapping function between the same-name image point pixel value of each image and the reference benchmark value, the mapping function uses multiple Bernstein polynomials as basis functions, combines them into a Bezier curve, and solves each coefficient of the combined polynomial by using the mapping relationship between the same-name image point pixel value and the reference benchmark value; applying the constructed Bezier function to the entire image, adjusting the pixel value to the reference benchmark value, achieving approximation of each image to the reference benchmark value, and completing the uniform light and color processing of multiple images.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision image processing, and in particular relates to a technical solution for uniform light and color of large-scene UAV images based on Bernstein polynomials, which is particularly suitable for large-scene UAV image processing at the city level. Background Art

[0002] In the process of using drones to obtain earth observation image data, due to factors such as shooting time, lighting, and the camera itself, the image data obtained often have differences in brightness and color, which brings difficulties to subsequent products such as using drone image processing to generate digital orthophoto maps (DOM). Therefore, it is necessary to perform light and color processing on drone images. The existing light and color processing methods can be roughly divided into four main methods: single image adaptive color method, reference image color method, adjacent area stitching method, and multiple image color method.

[0003] The single image adaptive color uniformity method is mostly used to solve the problem of inconsistent brightness and contrast in a single image caused by factors such as sunlight, atmospheric attenuation, and fog. The processing methods include using polynomial fitting of grayscale distribution to correct the grayscale unevenness of the image; using multi-resolution reconstruction of wavelet transform to eliminate uneven illumination, etc. However, this type of method only solves specific problems and it is difficult to achieve theoretical results without considering the constraints of adjacent images.

[0004] The reference image color uniformity method requires specifying a specific image in the scene as the tone matching benchmark. Representative methods include the histogram matching algorithm and the Wallis method. The image to be processed achieves the unification of radiation brightness and tone by establishing a color statistical information mapping function with the reference image. This type of method is directly related to statistical information and cannot reflect the consistency of content information with the same name. The effect is related to the window size corresponding to the statistic.

[0005] The adjacent area stitching method is often used in stitching images. By extracting the color difference of the closed area of ​​the stitching line buffer as the observation value, a regional correction model is constructed, such as the Poisson image color uniformity method. This type of method is a local correction method with a large degree of freedom and conforms to the laws of nature. However, its overall transition constraint is weak, it is easy to produce spots, and the amount of calculation is large. It is mostly used for manual image editing.

[0006] The multiple image color balancing method uses overlapping information and information of regions with the same name to correct the color distribution of all images in the scene. The method used is similar to the reference image color balancing method, such as histogram matching algorithm, variance mean method, and information entropy matching method, but it is still necessary to select a specific reference image.

[0007] Therefore, there is an urgent need for better technical means in this field to achieve uniform light and color processing for drone images acquired in complex environments. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a technical solution for uniform light and color of urban-level drone images based on Bernstein polynomials. Through this solution, it is possible to efficiently and high-quality uniform light and color processing of drone images acquired in complex environments to obtain multiple drone images with color consistency.

[0009] In order to achieve the above object, the present invention proposes a method for uniform light and color of large-scene UAV images based on Bernstein polynomials, which includes the following processing:

[0010] The object point is obtained through aerial triangulation, and the coordinate information of the characteristic same-name points in the image is obtained according to the relationship between the object point and the image, and the color values ​​corresponding to the coordinate information of these same-name points are extracted from different images;

[0011] The color values ​​of the same-name points in multiple images are averaged and assigned to the object point as a reference value for uniform light and color;

[0012] A mapping function is constructed between the pixel values ​​of the same-named image points of each image and the reference benchmark value, wherein the mapping function uses multiple Bernstein polynomials as basis functions, which are combined into a Bezier curve, and the coefficients of the combined polynomial are solved by using the mapping relationship between the pixel values ​​of the same-named image points and the reference benchmark value;

[0013] The constructed Bessel function is applied to the entire image, and the pixel value is adjusted to the reference benchmark value, so that each image is close to the reference benchmark value, and the processing of uniform light and color of multiple images is completed.

[0014] Furthermore, the method of obtaining the coordinate information of characteristic same-name points based on the relationship between the object point and the image is to obtain the same-name points on multiple images by performing aerial triangulation on the image and utilizing the relationship that the same object point has corresponding pixels on multiple images.

[0015] Moreover, when the Bernstein polynomial is used to construct the functional relationship F(x) between the image homonymous point x and the reference datum value y, the Bernstein polynomial is expressed as follows:

[0016]

[0017] Where n is the highest term of the Bernstein polynomial, k is the degree of the Bernstein polynomial, and the value of k ranges from 0 to n.

[0018] are the Bernstein polynomial coefficients, is the Bernstein polynomial.

[0019] Moreover, using the combination of 6 Bernstein basis functions with the highest order being 5, a non-negative, monotonically increasing high-order polynomial color transfer function is finally constructed as F(x), which is implemented as follows:

[0020] set up They are the basis functions of Bernstein polynomials of order 0 to 5, and the six Bernstein basis functions are combined to form the Bessel function:

[0021]

[0022] Among them, a, b, c, d, e, and f are the coefficients of the function polynomial to be solved.

[0023] Furthermore, when the same pixel color value matches multiple reference base values, the pixel color value is selected to correspond to the average value of the matched reference base values.

[0024] On the other hand, the present invention also provides a large-scene UAV image uniform light and color system based on Bernstein polynomials, which is used to implement the large-scene UAV image uniform light and color method based on Bernstein polynomials as described above.

[0025] Furthermore, the following modules are included,

[0026] The first module is used to obtain object points through aerial triangulation, obtain the coordinate information of characteristic points of the same name in the image according to the relationship between the object points and the image, and extract the color values ​​corresponding to the coordinate information of these points of the same name from different images;

[0027] The second module is used to average the color values ​​of the same-name points in multiple images and assign them to the object point as a reference value for uniform light and color;

[0028] The third module is used to construct a mapping function between the pixel values ​​of the same-named image points of each image and the reference benchmark value, wherein the mapping function uses multiple Bernstein polynomials as basis functions, which are combined into a Bezier curve, and the coefficients of the combined polynomial are solved by using the mapping relationship between the pixel values ​​of the same-named image points and the reference benchmark value;

[0029] The fourth module is used to apply the constructed Bessel function to the entire image, adjust the pixel value to the reference benchmark value, make each image close to the reference benchmark value, and complete the processing of uniform light and color for multiple images.

[0030] Alternatively, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned large-scene drone image uniform light and color method based on Bernstein polynomials.

[0031] Alternatively, it includes a readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for uniform light and color of large-scene drone images based on Bernstein polynomials as described above is implemented.

[0032] Compared with the prior art, the present invention has the following differences and advantages:

[0033] 1. The present invention is based on the object-side connection point to uniformly even the light and color of the global image, so that the global color tends to be consistent. However, most of the existing technologies are incremental uniform light and color, which will lead to error accumulation and local color deviation.

[0034] 2. The present invention can naturally adjust the image curve obtained by the camera and can more accurately simulate the propagation of light, including direct and indirect illumination, thereby producing a more realistic and natural image effect. The existing technology is through iterative calculation, and Poisson uniform illumination also requires the prior knowledge of the illumination radiance.

[0035] 3. The computational complexity of the implementation process of the present invention is low, the system runs fast, and it is easier to apply to large-scale 3D modeling scenes. However, the traditional method has high complexity, sensitive parameter selection, and boundary effects will appear in the image, making it difficult to apply to large-scale scenes.

[0036] The solution of the present invention is simple and convenient to implement, has strong practicality, solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0038] Figure 2 4 is a graph of a Bernstein basis function according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0040] The present invention is based on a uniform light and color method combining a reference image and multiple images. The technical process is as follows: Figure 1 As shown. The average value of the color information of the same-name image points corresponding to different images is used as the reference benchmark value, and then a mapping function relationship between the same-name image points in different images and the reference benchmark value is constructed. In order to achieve this goal, the present invention introduces Bernstein polynomials to construct a mapping function, so that each image can be adjusted to the reference benchmark value to achieve the goal of uniform light and color.

[0041] Specifically, the present invention is based on the method of uniform light and color of multiple overlapping images, introduces Bernstein polynomials to construct grayscale transfer functions, and forms Bezier curves after mapping and settlement to achieve color and illumination adjustment of drone images. First, after aerial triangulation processing of drone images, encrypted object points and feature points in the image are obtained, and the coordinate information of feature points with the same name in the overlapping area of ​​the image is extracted according to the relationship between the object points and the feature points of the image. The same-name color information in different images is extracted using the coordinate information of the same-name feature points, and the pixel values ​​of these same-name image points are averaged and assigned to the sparse point cloud color as the reference reference value for uniform light and color. Next, a mapping function between the pixel values ​​of the same-name image points of each image and the reference reference value is constructed, and this function will be represented by a combination of Bernstein polynomials. The coefficients of the function are solved according to the corresponding matching relationship between the pixel values ​​of the same-name image points and the reference reference value, and a functional formula for mobilizing the image to the reference reference value is established, so that each image has its corresponding mapping function to adjust the image to the reference reference value, thereby achieving uniform light and color processing between images.

[0042] A method for uniform light and color of large-scene city-level UAV images based on Bernstein polynomials is proposed in an embodiment of the present invention, comprising the following steps:

[0043] (1) First, encrypted object points are obtained through aerial triangulation processing. The coordinate information of characteristic points with the same name in the image is obtained based on the relationship between the encrypted points and the image. The color values ​​corresponding to the coordinate information of these points with the same name are extracted from different images.

[0044] (2) The color values ​​of the same-name points in multiple images are averaged and assigned to the object point as a reference value for uniform light and color;

[0045] The present invention uses the color average value as the reference value when performing uniform light and color processing on multiple images. After the image is processed by aerotriangulation, the same object point has corresponding pixels on multiple images, and the same-name image points on multiple images are obtained. The color information of these same-name points is averaged and assigned to the object sparse points as the reference image value.

[0046] The specific implementation method of obtaining the reference benchmark value further preferably adopted in the embodiment is as follows:

[0047] 1) Feature extraction and matching. First, use the SIFT operator to extract features from the input drone image, and then match the extracted feature point descriptors. The matching algorithm preferably uses ASIFT matching, which is an improvement on the SIFT matching algorithm and solves the problem of limited viewing angle changes of the SIFT algorithm. A rotation space is introduced under the image Gaussian scale space to generate copies of the image at different rotation angles. This is usually done by rotating the original image to generate multiple rotated versions. SIFT feature point matching is performed for each scale and rotation angle. After the matching is completed, the epipolar geometry relationship needs to be used to optimize the matching point pairs in each matching image pair and delete the mismatched point pairs.

[0048] 2) Incremental reconstruction. Select the initial matching image pair as the seed image, use the epipolar geometry decomposition to obtain the pose of the image pair, and use the forward intersection method to generate the 3D corresponding points of the matching feature points. Next, add a new image, estimate the pose of the newly added image through PnP and triangulate it to generate 3D points, and use BA to optimize the error. Repeat the above process until all images are added to the system.

[0049] 3) Calculate the reference value. After aerial triangulation, the image pose, feature points and 3D sparse points on the image are obtained. The coordinate values ​​of the same-name image points on multiple images are obtained by using the relationship that the object point has corresponding feature points on different images. The corresponding image color information is obtained according to the image point coordinate value, and the color information of the same-name image points in multiple images is averaged as the reference value for uniform light and color.

[0050] (3) A mapping function can be constructed between the pixel values ​​of the same-named image points in each image and the reference benchmark value. This function uses multiple Bernstein polynomials as basis functions and is combined into a Bezier curve. The coefficients of the combined polynomial are solved using the mapping relationship between the pixel values ​​of the same-named image points and the reference benchmark value.

[0051] The method for constructing the image color transfer function of the present invention is: after calculating the reference benchmark value, the relationship between the object point and the image point is used to obtain the same-name image point information of multiple images, and the color value of the same-name point in each image is adjusted to be close to the corresponding reference benchmark value. The present invention uses Bernstein polynomials to construct the functional relationship between the same-name points of the image and the reference benchmark value.

[0052] The specific implementation method of constructing the functional relationship between the image points with the same name and the reference benchmark value in the embodiment of the present invention is as follows:

[0053] First, observe the correspondence between the color values ​​of the same-name image points and the reference benchmark values. Normally, one image point value of the same name on an image should correspond to one reference benchmark value. However, due to errors and image noise interference, there will be a many-to-one situation, that is, multiple color values ​​of the same-name image points correspond to one reference benchmark value, and the functional relationship cannot be constructed. At this time, we take the average of the color values ​​of multiple image points of the same name corresponding to a reference benchmark value to solve this problem. At the same time, the values ​​are not evenly distributed, and the same-name image points and their corresponding reference benchmark values ​​are all concentrated in a certain interval. For example, if the color value interval is [0, 255], the image point color values ​​and the reference benchmark values ​​are concentrated in [100, 200], and [0, 100] and (200, 255) lack the information relationship for constructing a functional formula. The present invention solves this problem by using a one-to-one mapping relationship within the interval range where the relationship between the image point color value and the reference benchmark value is missing.

[0054] Then, construct a function between the pixel color value and the reference value

[0055] A functional relationship can be constructed between the color value of the same-named image point on each image and the reference value:

[0056] y=F(x)

[0057] Among them, x is the color value of the same-name image point on the image, y is the reference value, and F() is the mapping relationship.

[0058] The present invention introduces Bernstein polynomials as basis functions to form Bezier curves to describe this function. Bernstein polynomials are as follows:

[0059]

[0060] Where n is the highest term of the Bernstein polynomial, k is the degree of the Bernstein polynomial, and the value of k ranges from 0 to n. are the Bernstein polynomial coefficients, is the Bernstein polynomial.

[0061] The Bezier curve is as follows:

[0062]

[0063] Where l is the Bessel polynomial coefficient, B n (l,x) is a Bezier curve.

[0064] In specific implementation, the highest order of the Bernstein basis function can be selected to be greater than 3. Through experiments, it is found that the fitting result is optimal when the highest order of the Bernstein basis function is 5.

[0065] See also Figure 2, a curve is generated by interpolating a set of control points in parameter space. B_05-B_55 correspond to six different order curves. The Bernstein curve is a curve generated by a set of control points and the corresponding Bernstein basis functions. For the fifth order Bernstein curve, it consists of six control points and six basis functions. The parameters vary in the interval [0,1], controlling the position of the points on the curve. By changing the value of the parameters, different points on the curve can be obtained, thereby depicting the entire curve.

[0066] The embodiment of the present invention uses Bernstein polynomials as basis functions, preferably using a combination of 6 Bernstein basis functions with the highest order being 5, and finally constructs a non-negative, monotonically increasing high-order polynomial color transfer function. Bernstein 0 to 5 orders (n=0, 1, 2, 3, 4, 5) is expressed as:

[0067]

[0068] in, are the basis functions of Bernstein polynomials of order 0 to 5, respectively.

[0069] The embodiment of the present invention uses a fifth-order Bernstein polynomial combination form to combine six basis functions with n varying from 0 to 5 to construct a color transfer function relationship F(x), that is, six Bernstein basis functions are combined to form a Bessel function:

[0070]

[0071] Among them, a, b, c, d, e, and f are the coefficients of the function polynomial to be solved.

[0072] The specific implementation method of the function solution in the embodiment of the present invention is as follows:

[0073] For the function polynomial coefficients a, b, c, d, e, and f that need to be solved in the above formula, the embodiment of the present invention preferably uses the conjugate gradient descent method to solve the function coefficients. The function formula is Ax=B, where A is the color value of the same-name image point, B is the corresponding reference benchmark value, and the function polynomial coefficient to be solved is X.

[0074] a. Initialize X, where X(k) represents the solution vector of the kth iteration. p(k) represents the direction vector of the kth iteration, and r(k) represents the residual vector of the kth iteration.

[0075] b. Calculate the residual vector r(k)=AX(k-1)-B.

[0076] c. Calculate the direction vector Where T stands for transpose.

[0077] d. Calculate the update step size

[0078] e. Update the solution vector X(k) = X(k-1) + α(k)p(k)

[0079] f. Stop condition: the process is terminated when the predetermined number of iterations is reached or the residual is less than a set threshold. Preferably, the number of iterations is 10,000 and the threshold is set to 1e-8.

[0080] Since the color value is composed of three channels: red R, green G, and blue B, 18 function polynomial coefficients need to be solved in the end.

[0081] (4) The constructed Bessel function is applied to the entire image, and the pixel value can be adjusted to the reference value, that is, each image is finally close to the reference value, so as to achieve the purpose of uniform light and color for multiple images.

[0082] After constructing the functional relationship between the color value of the image point with the same name and the reference value, the functional relationship of the three channels is applied to the entire image, and the change values ​​of R, G, and B of the entire image are calculated respectively, so that the color value of each pixel coordinate is adjusted to be close to the reference value, thereby achieving the effect of uniform light and color.

[0083] The present invention further solves the problem that may occur when the same-name points are mapped to reference benchmark values. When the same pixel color value matches multiple reference benchmark values, the present invention selects the pixel color value to correspond to the average value of the matched reference benchmark values.

[0084] In addition, when the color values ​​of the same-named pixels and the reference values ​​are unevenly distributed, for example, they are concentrated in a certain interval, and not all values ​​in the color interval have a mapping relationship, the intervals lacking the mapping relationship between the color values ​​of the same-named pixels and the reference values ​​are processed using a one-to-one correspondence. For example, the 100-255 interval has a value and a mapping relationship, and the 0-100 interval has no corresponding value, then x=1, y=1, x=2, y=2, ... x=100, y=100.

[0085] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0086] In some possible embodiments, a large-scene drone image uniform light and color system based on Bernstein polynomials is provided, including the following modules:

[0087] The first module is used to obtain object points through aerial triangulation, obtain the coordinate information of characteristic points of the same name in the image according to the relationship between the object points and the image, and extract the color values ​​corresponding to the coordinate information of these points of the same name from different images;

[0088] The second module is used to average the color values ​​of the same-name points in multiple images and assign them to the object point as a reference value for uniform light and color;

[0089] The third module is used to construct a mapping function between the pixel values ​​of the same-named image points of each image and the reference benchmark value, wherein the mapping function uses multiple Bernstein polynomials as basis functions, which are combined into a Bezier curve, and the coefficients of the combined polynomial are solved by using the mapping relationship between the pixel values ​​of the same-named image points and the reference benchmark value;

[0090] The fourth module is used to apply the constructed Bessel function to the entire image, adjust the pixel value to the reference benchmark value, make each image close to the reference benchmark value, and complete the processing of uniform light and color for multiple images.

[0091] In some possible embodiments, a large-scene UAV image uniform light and color system based on Bernstein polynomials is provided, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a large-scene UAV image uniform light and color method based on Bernstein polynomials as described above.

[0092] In some possible embodiments, a large-scene UAV image uniform light and color system based on Bernstein polynomials is provided, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a large-scene UAV image uniform light and color method based on Bernstein polynomials as described above is implemented.

[0093] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for uniform light and color of large-scene UAV images based on Bernstein polynomials, characterized by: The method includes the following steps: obtaining object points through aerial triangulation, obtaining coordinate information of characteristic points of the same name in the image based on the relationship between the object points and the image, and extracting color values ​​corresponding to the coordinate information of these points of the same name from different images; The color values ​​of the same-name points in multiple images are averaged and assigned to the object point as a reference value for uniform light and color; A mapping function is constructed between the pixel values ​​of the same-named image points of each image and the reference benchmark value, wherein the mapping function uses a plurality of Bernstein polynomials as basis functions to form a Bezier curve, and the Bezier curve is formed by combining the corresponding n+1 basis functions using a combination form of an n-th order Bernstein polynomial with a highest order of 3 or more, and constructing a color transfer function relationship F(x) by combining the corresponding n+1 basis functions, and solving the coefficients of the combination polynomial using the mapping relationship between the pixel values ​​of the same-named image points and the reference benchmark value; The constructed Bessel function is applied to the entire image, and the pixel value is adjusted to be close to the reference benchmark value, so that each image is close to the reference benchmark value, and the processing of uniform light and color of multiple images is completed.

2. According to the Bernstein polynomial-based large-scene UAV image uniform light and color method of claim 1, it is characterized by: The method for obtaining the coordinate information of the characteristic same-name points based on the relationship between the object point and the image is to obtain the same-name points on multiple images by performing aerial triangulation on the image and utilizing the relationship that the same object point has corresponding pixels on multiple images.

3. According to the Bernstein polynomial-based large-scene UAV image uniform light and color method of claim 1, it is characterized by: When the Bernstein polynomial is used to construct the functional relationship F(x) between the image homonymous point x and the reference datum value y, the Bernstein polynomial is expressed as follows: Where n is the highest term of the Bernstein polynomial, k is the degree of the Bernstein polynomial, and the value of k ranges from 0 to n. are the Bernstein polynomial coefficients, is the Bernstein polynomial.

4. According to claim 1, the method for uniform light and color of large-scene UAV images based on Bernstein polynomials is characterized by: Using the combination of 6 Bernstein basis functions with the highest order being 5, a non-negative, monotonically increasing high-order polynomial color transfer function is finally constructed as F(x), which is implemented as follows: set up They are the basis functions of Bernstein polynomials of order 0 to 5, and the six Bernstein basis functions are combined to form the Bessel function: Among them, a, b, c, d, e, and f are the coefficients of the function polynomial to be solved.

5. The method for uniform light and color of large-scene UAV images based on Bernstein polynomials according to claim 1, 2, 3 or 4, characterized in that: When the same pixel color value matches multiple reference base values, the pixel color value is selected to correspond to the average value of the matched reference base values.

6. A large-scene UAV image uniform light and color system based on Bernstein polynomials, characterized by: Used to implement a large-scene drone image uniform light and color method based on Bernstein polynomials as described in any one of claims 1-5.

7. According to claim 6, the large-scene UAV image uniform light and color system based on Bernstein polynomials is characterized by: The invention comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a large-scene drone image uniforming method based on Bernstein polynomials as described in any one of claims 1 to 5.

8. The large-scene UAV image uniform light and color system based on Bernstein polynomials according to claim 6, characterized in that: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, a large-scene drone image uniforming method based on Bernstein polynomials is implemented as described in any one of claims 1 to 5.

Images