A method and device for multi-target color correction of SAR images

Abstract

This invention provides a multi-target color correction method and device for SAR images, including preprocessing the original image; constructing a multi-target optimization model based on linear stretching transformation of the image grayscale values; solving for the linear stretching parameters of all images based on the multi-target optimization model; and using the obtained linear stretching parameters to stretch the original image and then mosaicking it. This invention solves for the linear stretching coefficients of SAR images based on the multi-target optimization concept, effectively utilizing the grayscale information in the SAR images to ensure color consistency in the mosaicked SAR images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multi-target color correction method and device for SAR images. Background Technology

[0002] Seasonal changes, radar signal attenuation, and noise can reduce the radiometric quality of SAR images, leading to significant brightness differences even after calibration. If a SAR image exhibits brightness anomalies, it will appear generally dark. Most radiometric correction methods for SAR images are based on optical images. Depending on the application, radiometric correction methods for multiple images can be categorized into global, local, or combined models. Global models are a mainstream approach. Global models assume that the radiometric relationship between image pairs can be fitted by linear or nonlinear functions. Linear models assume that the intensity differences between different images are determined by a linear relationship. To describe the linear relationship between images, some researchers have constructed linear models based on pseudo-invariant features (PIFs). Histogram matching is a nonlinear model that matches the histograms of two images to make the explicit distribution of pixel values ​​in the two images as close as possible. However, histogram matching cannot identify differences when ground features in the image change or when the same ground feature shows significant differences due to seasonal factors. Summary of the Invention

[0003] The purpose of this invention is to solve the technical problems in the prior art and provide a multi-target color correction method for SAR images, which simultaneously optimizes the two targets of the image after stretching, which have the highest color consistency and the least information loss.

[0004] To achieve the above objectives, this invention proposes a multi-target color correction method for SAR images, comprising the following processing steps.

[0005] Preprocess the original image;

[0006] A multi-objective optimization model is constructed based on linear stretching transformation of image gray values;

[0007] The linear stretching parameters of all images are solved based on a multi-objective optimization model;

[0008] The original image is stretched using the obtained linear stretching parameters and then mosaicked.

[0009] Furthermore, the preprocessing of the original image includes obtaining the total number of all images, the maximum pixel gray value of each image, the minimum pixel gray value of each image, the mean gray value of each image, the standard deviation of each image, the number of pixels in each image, the mean gray value of image pairs with overlapping regions, the standard deviation of image pairs with overlapping regions, and the number of effective pixels in image pairs with overlapping regions.

[0010] Furthermore, the multi-objective optimization model constructed based on linear stretching transformation of image grayscale values ​​includes setting two objective functions: maximizing the color consistency of the stretched image and minimizing information loss, with the linear stretching parameter as the decision variable; setting equality constraints based on maintaining the overall brightness and contrast of the image before and after correction; and setting inequality constraints based on the pixel grayscale values ​​after correction meeting the quantization bit limit.

[0011] Furthermore, when setting the objective function that maximizes color consistency, the color consistency optimization objective is constructed using the mean and standard deviation of the index in the overlapping area as observed values.

[0012] Moreover, when setting the objective function to minimize information loss, the goal of optimizing information loss is to minimize the total number of pixels whose gray values ​​exceed the limit after stretching the original image.

[0013] Moreover, the equation constraint based on keeping the overall brightness and contrast of the image unchanged before and after correction is implemented by setting conditions that require the sum of the mean gray values ​​and the sum of the standard deviations of all overlapping areas of a single image to remain unchanged before and after stretching.

[0014] Furthermore, the inequality constraint based on the quantization bit limit of the corrected pixel gray value is implemented by setting conditions that require the maximum and minimum values ​​of the gray values ​​after image stretching to satisfy the range of values ​​before transformation.

[0015] Furthermore, the linear stretching parameters of all images are solved based on the multi-objective optimization model, including the fusion metaheuristic algorithm NSAG-II and the barrier function interior point method. The barrier function interior point method is used to calculate the objective function value of the color consistency optimization objective.

[0016] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the multi-target color correction method for SAR images as described above.

[0017] On the other hand, the present invention also provides a computer program product, including a computer program, characterized in that: when the computer program is executed by a processor, it implements the multi-target color correction method for SAR images as described above.

[0018] This invention performs color correction processing on SAR images based on a multi-objective optimization method, effectively utilizing the pixel information in the SAR images to make the corrected SAR images present a more consistent tone.

[0019] The present invention is simple and convenient to implement, highly practical, and solves the problems of low practicality and inconvenience in actual application of related technologies. It can improve user experience and has significant market value. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the multi-target color correction method for SAR images according to an embodiment of the present invention.

[0021] Figure 2 This is a mosaic effect diagram of the original SAR image according to an embodiment of the present invention;

[0022] Figure 3 This is an illustration of the effect of the multi-target color correction processing method for SAR images according to an embodiment of the present invention;

[0023] Figure 4 This is a diagram showing the effect of the comparison method in an embodiment of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0025] See Figure 1 This invention provides a multi-target color correction method for SAR images, including the following processing: S1) Preprocessing the original image;

[0026] In this embodiment, the preprocessing of the original image is implemented as follows:

[0027] Obtain the following data: total number of images n, maximum pixel grayscale value of each image Minimum pixel grayscale value of each image The average gray level of each image is μ. i Standard deviation σ for each image i Number of pixels per image (s) i The gray-scale mean μ of image pairs with overlapping regions ij μ ji The standard deviation σ of image pairs with overlapping areas ij σ ji The number of effective pixels S in overlapping image pairs ij Where i and j are image labels, and i ≠ j.

[0028] S2) Based on the image grayscale values, a linear stretching transformation is performed to construct a multi-objective optimization model;

[0029] In this embodiment, the construction of a multi-objective optimization model based on linear stretching transformation of image grayscale values ​​is implemented as follows:

[0030] With linear stretching parameter a i b iFor the decision variables, the optimization simultaneously targets two objectives: maximizing color consistency and minimizing information loss in the stretched image. Maintaining the overall brightness and contrast of the image before and after correction serves as an equality constraint, while ensuring the pixel grayscale values ​​meet the quantization bit limit after correction serves as an inequality constraint. The optimization model is expressed as:

[0031]

[0032] A = {Y i ∈N * |a i Y i +b i >2 m -1,a i Y i +b i <0}

[0033]

[0034] stmin(y′)≥min(Y1);

[0035] max(y′)≤max(Y1);

[0036]

[0037]

[0038] Where n is the total number of images, m is the number of image quantization bits, and μ ij μ ji The mean gray values ​​of the overlapping regions i and j are respectively, and σ is the mean gray value of the region i and j. ij σ ji S represents the standard deviation of the overlapping regions of i and j. ij Let a be the number of pixels in the overlapping region of images i and j. i b i Let y be the linear stretching coefficient to be determined, y′ be the grayscale value of the image after linear stretching, min(Y1) and max(Y1) be the minimum and maximum allowable grayscale values ​​for this type of image, respectively, and s be the grayscale value of the image after linear stretching. i μ is the number of pixels in image i. i Let σ be the grayscale mean of image i. i Y is the standard deviation of image i; i Let be the set of grayscale values ​​of the original image of the i-th scene.

[0039] The specific construction instructions for the optimization model are as follows:

[0040] Using the mean and standard deviation of the indicators in the overlapping areas as observations, a color consistency optimization objective is constructed, namely objective function 1, which can be expressed as:

[0041]

[0042] Where n is the total number of images, μ ij μ ji The mean gray values ​​of the overlapping regions i and j are respectively, and σ is the mean gray value of the region i and j. ij σ ji S represents the standard deviation of the overlapping regions of i and j. ij The number of pixels in the overlapping region of images i and j.

[0043] The objective of optimizing information loss is to minimize the total number of pixels whose grayscale values ​​exceed the limits after stretching the original image, i.e., objective function 2, which can be expressed as:

[0044] A = {Y i ∈N * |a i Y i +b i >2 m -1, a i Y i +b i <0}

[0045]

[0046] Among them, a i b i Let Y be the linear stretching coefficient to be determined. i Let m be the set of grayscale values ​​of the original image of the i-th scene, m be the number of bits for image quantization, and N be the number of bits for image quantization. * is a positive integer, num(A) is the number of pixels that exceed the image quantization range, and minimize is the objective function to be minimized.

[0047] To ensure that the corrected pixel grayscale values ​​meet the quantization bit limit as an inequality constraint, the embodiment further proposes that the maximum and minimum values ​​of the grayscale values ​​after image stretching must satisfy the allowed range of values ​​for the image category, which can be expressed as:

[0048] min(y′)≥min(Y1)

[0049] max(y′)≤max(Y1)

[0050] Where y′ is the gray value of the image after linear stretching, min(y′) and max(y′) are the minimum and maximum gray values ​​of the image after linear stretching, and min(Y1) and max(Y1) are the minimum and maximum gray values ​​allowed for this type of image, respectively. The solution of this invention is mainly designed for SAR images, and the value range of SAR images is 0 to 65535 (16 bits).

[0051] In order to set equality constraints based on keeping the overall brightness and contrast of the images unchanged before and after correction, the embodiment further proposes equality constraint conditions for the model, requiring that the sum of the mean and standard deviation of grayscale values ​​in all overlapping areas of a single image remain unchanged before and after stretching.

[0052]

[0053] Where n is the total number of images, s i Let a be the number of pixels in the overlapping region of image i. i b i Let μ be the decision variable to be determined. i The average gray level of the overlapping region of image i;

[0054]

[0055] Where n is the total number of images, s i Let a be the number of pixels in the overlapping region of image i. i Let σ be the decision variable to be determined. i denoted as the standard deviation of the overlapping region of image i.

[0056] S3) Solve the multi-objective optimization model established in S2) to obtain the linear stretching parameters of all images;

[0057] In this embodiment, for cases where the model has multiple constraints, a continuous objective function, and a discrete objective function, a preferred fusion algorithm is proposed that fully combines the global search capability of the metaheuristic algorithm NSAG-II with the interior-point method of obstacle functions for accurately solving quadratic programming models. The implementation process includes the following steps:

[0058] Step 1: Generate the initial population, i.e., the linear stretching coefficients of the decision variables;

[0059] In specific implementation, it is preferred to use the NSGA-II algorithm to generate the initial population. Detailed implementation instructions for the NSGA-II algorithm can be found in existing technical literature, which will not be repeated here: W. Zheng and B. Doerr, “Approximation Guarantees for the Non-Dominated Sorting Genetic Algorithm II (NSGA-II),” IEEE Trans. Evol. Computat., pp. 1–1, 2024, doi:10.1109 / TEVC.2024.3402996.

[0060] Step 2: Calculate the fitness value of each individual in the initial population.

[0061] (1) Calculate the objective function value of 1 for each individual in the initial population;

[0062] In specific implementation, it is preferred to use the interior point method of the obstacle function to calculate the objective function value of the color consistency optimization target. For a detailed explanation of the interior point method of the obstacle function, please refer to the existing technical literature, which will not be repeated here: P. Malisani, "Interior Point Methods in Optimal Control", doi:10.1051 / cocv / 2024049.

[0063] (2) Calculate the objective function value 2 for each individual in the initial population; in practice, it is preferred to use the statistical method (i.e., count accumulation) to calculate the objective function value of the information loss optimization objective.

[0064] Step 3: Perform fast non-dominated sorting on the two objective function values ​​(objective function values ​​for color consistency optimization and information loss optimization), and generate a new offspring population using selection, crossover, and mutation operations;

[0065] Step 4: Merge the parent and offspring populations;

[0066] Step 5: Perform fast non-dominated sorting and crowding distance calculation on the merged population, and select superior individuals to form a new parent population;

[0067] Step 6: Use operations such as selection, crossover, and mutation to generate a new offspring population, and merge the new parent population with the new offspring population to form a newer population;

[0068] Step 7: Repeat steps 5 and 6. Stop iterating when the number of iterations exceeds the maximum limit;

[0069] Step 8: Output the final Pareto front solution and terminate the algorithm. Each solution on the front surface represents a set of stretching parameters for all images.

[0070] S4) The original image is stretched using the obtained linear stretching parameters and then mosaicked.

[0071] In practice, if adjacent SAR images have overlapping latitude and longitude, there will generally be a certain overlapping area. In this embodiment, the stretching process of the original image with overlapping area using the obtained linear stretching parameter is implemented by using the obtained decision variable a. i b i The original image's grayscale values ​​are linearly stretched using the following formula to obtain a color-corrected image.

[0072] y′=a i Y i +b i

[0073] To facilitate the illustration of the technical effects of the present invention, Table 1 provides a comparison of the results obtained by applying the present invention and existing technical methods. Figure 2 This is a mosaic effect diagram of the original SAR image according to an embodiment of the present invention; Figure 3 This is an illustration of the effect of the multi-target color correction processing method for SAR images according to an embodiment of the present invention; Figure 4 The image shows the effect of the comparison algorithm method in this embodiment of the invention. It is evident that the results of this invention are superior in many aspects.

[0074] Table 1

[0075]

[0076] For the implementation of the comparison method, please refer to the following literature: Cresson R, Saint-Geours N. Natural Color Satellite Image Mosaicking Using Quadratic Programming in Decorrelated Color Space[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 8(8): 4151-4162. DOI: 10.1109 / JSTARS.2015.2449233.

[0077] In practice, the above process can be automated using computer software technology.

[0078] In addition, this application also relates to an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described multi-target color correction method for SAR images.

[0079] This application also relates to a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the above-described multi-target color correction method for SAR images.

[0080] This application also relates to a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the above-described multi-target color correction method for SAR images.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-target color correction method for SAR images, characterized in that: Includes the following processing, Preprocess the raw SAR images; A multi-objective optimization model is constructed based on linear stretching transformation of image gray values; The multi-objective optimization model constructed based on linear stretching transformation of image grayscale values ​​is implemented as follows. With linear stretching parameters For the decision variables, the optimization simultaneously targets two objectives: maximizing color consistency and minimizing information loss in the stretched image. Maintaining the overall brightness and contrast of the image before and after correction serves as an equality constraint, while ensuring the pixel grayscale values ​​meet the quantization bit limit after correction serves as an inequality constraint. The optimization model is expressed as: in, This represents the total number of images. To quantize the image to bits, They are respectively The average gray value of the overlapping region. They are respectively Standard deviation of overlapping regions For images The number of pixels in the overlapping region Let be the linear stretching coefficient to be determined. The image grayscale value is the result of linear stretching. These are the minimum and maximum grayscale values ​​allowed for this type of image, respectively. For images The number of pixels, For images The average gray level, For images Standard deviation; For the first i The set of grayscale values ​​of the original image of the scene; It is a positive integer. The number of pixels that exceed the image quantization range; The linear stretching parameters of all images are solved based on a multi-objective optimization model, including the fusion metaheuristic algorithm NSGA-II and the barrier function interior point method. The barrier function interior point method is used to calculate the objective function value for color consistency optimization. The implementation process is as follows. Step 1: Generate an initial population using the NSGA-II algorithm, which consists of linear stretching coefficients; Step 2: Calculate the fitness value of each individual in the initial population, including: (1) Calculate the objective function value of the color consistency optimization objective using the barrier function interior point method; (2) Calculate the objective function value of the information loss optimization objective using statistical methods; Step 3: Perform fast non-dominated sorting on the two objective function values, and generate offspring populations using selection, crossover, and mutation operations; Step 4: Merge the parent population with the offspring population; Step 5: Perform fast non-dominated sorting and crowding distance calculation on the merged population, and select excellent individuals to form a new parent population; Step 6: Repeat the iterative steps until the termination condition is met; Step 7: Output the final Pareto front solution, which is represented as a set of stretching parameters for all images; The original image is stretched using the obtained linear stretching parameters and then mosaicked.

2. The multi-target color correction method for SAR images according to claim 1, characterized in that: The preprocessing of the original image includes obtaining the total number of all images, the maximum pixel gray value of each image, the minimum pixel gray value of each image, the mean gray value of each image, the standard deviation of each image, the number of pixels in each image, the mean gray value of image pairs with overlapping areas, the standard deviation of image pairs with overlapping areas, and the number of valid pixels in image pairs with overlapping areas.

3. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the multi-target color correction method for SAR images as described in any one of claims 1 to 2.

4. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the multi-target color correction method for SAR images as described in any one of claims 1 to 2.

Images