Multi-scale polarization data fusion method based on 4K polarization imaging detector

Abstract

The invention provides a multi-scale polarization data fusion method based on a 4K polarization imaging detector, and relates to the technical field of image processing, and the method comprises the steps: carrying out the adaptive edge detection of four original polarization images, generating a comprehensive edge image, and carrying out the definition scoring; performing color calibration on each original polarization image based on the ambient light intensity and the color temperature, and outputting a polarization image after color calibration and a color deviation score; combining definition scores, color deviation scores and global brightness information entropies of the four polarization images after color calibration to construct a multi-objective optimization function, and solving the multi-objective optimization function in a multi-exposure parameter space to obtain an optimal exposure parameter combination; applying the optimal exposure parameter combination to image capture of the next period, and calculating global quality reward values of four newly collected polarization images; when the global quality reward value meets a preset condition, wavelet decomposition and frequency division multi-scale fusion are carried out on the polarization images after color correction, a fused image is generated, and the quality of the polarization images is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a multi-scale polarization data fusion method based on a 4K polarization imaging detector. Background Art

[0002] As a new imaging method, polarization imaging technology can obtain the polarization information of target objects, which is difficult to obtain in traditional intensity imaging and spectral imaging. Polarization information contains important clues such as the surface characteristics, material composition, and geometric shape of the target object, and has wide application value in many fields. For example, in military reconnaissance, polarization imaging can enhance the contrast between the target and the background and improve the recognition rate of the target; in the field of biomedicine, polarization imaging can be used to detect the microstructure and pathological conditions of biological tissues; in remote sensing detection, polarization imaging helps to identify the types and states of ground objects and provides strong support for environmental monitoring and resource exploration.

[0003] Currently, when using a polarization imaging detector to collect polarization images, due to the changes in ambient light intensity and color temperature, the collected original polarization images often have problems such as color deviation and uneven brightness. At the same time, the image clarity at different polarization angles may also vary, which seriously affects the subsequent image processing and analysis effects. For example, in a complex lighting environment, the difference in the transmittance of polarizers will cause spectral shift, resulting in color distortion of the collected images and making it difficult to accurately reflect the true color information of the target object.

[0004] Therefore, it is necessary to provide a multi-scale polarization data fusion method based on a 4K polarization imaging detector to solve the above technical problems. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a multi-scale polarization data fusion method based on a 4K polarization imaging detector, which effectively improves the quality of polarization images, with richer details and more accurate colors, and generates high-quality fused images. The present invention provides a multi-scale polarization data fusion method based on a 4K polarization imaging detector, and the method includes the following steps: Performing adaptive edge detection on four original polarization images captured synchronously according to a preset period to generate a comprehensive edge map and calculating a clarity score; Based on the ambient light intensity and color temperature collected in real time, performing color calibration on each original polarization image, and outputting the polarization image after color calibration and a color deviation score; Combining the clarity score, color deviation score, and the global brightness information entropy of the four polarization images after color calibration to construct a multi-objective optimization function, and solving in the multi-exposure parameter space to obtain an optimal exposure parameter combination; Apply the optimal exposure parameter combination to the next-cycle image capture, calculate the global quality reward values of the newly acquired four polarization images, and dynamically update the exposure parameter selection strategy table through the Q-learning algorithm; When the global quality reward value meets the preset condition, perform wavelet decomposition on each color-corrected polarization image in combination with the comprehensive edge map, and perform frequency-division multi-scale fusion on the low-frequency sub-band and high-frequency sub-band after wavelet decomposition to generate a fused image.

[0006] Preferably, the adaptive edge detection is performed on the four original polarization images captured synchronously at a preset cycle to generate a comprehensive edge map and calculate the sharpness score, specifically including: Dynamically select a combination of edge detection algorithms based on the local contrast of each polarization image, where: When the local contrast is higher than the first threshold, use the weighted fusion of the Canny algorithm and the Sobel gradient operator; When the local contrast is between the first threshold and the second threshold, use the LoG operator for multi-scale edge extraction; When the local contrast is lower than the second threshold, use the phase congruency edge detection algorithm; Perform confidence weighting on the initial edge map generated by the selected algorithm to generate a comprehensive edge map containing pixel-level weight values, where the pixel-level weight values are positively correlated with the edge response intensity; Calculate the sharpness score of each polarization image according to the average gradient magnitude of the comprehensive edge map.

[0007] Preferably, the color calibration is performed on each original polarization image based on the real-time acquired ambient light intensity and color temperature, and the color-calibrated polarization image and the color deviation score are output, specifically including: Based on the ambient light intensity and color temperature data, perform white balance correction on the four original polarization images through the von Kries color adaptation transformation to eliminate the spectral shift caused by the difference in the transmittance of the polarizer; For each polarization image after white balance correction, use the Retinex theory for multi-scale illumination estimation to generate a color-calibrated polarization image; Calculate the color deviation score of each polarization image from the reference value of the standard color card through the CIEDE2000 color difference formula.

[0008] Preferably, the multi-objective optimization function is constructed by combining the sharpness score, the color deviation score, and the global brightness information entropy of the four color-calibrated polarization images, and solved in the multi-exposure parameter space to obtain the optimal exposure parameter combination, specifically including: Normalize the sharpness score and the color deviation score respectively; Merge the luminance channels of the four polarized images after color calibration, and calculate the global luminance information entropy of their joint histogram; Construct a linearly weighted multi-objective optimization function based on the sharpness score, color deviation score, and global luminance information entropy after normalization; In the multi-exposure parameter space composed of exposure time, aperture value, and gain, iteratively search for the maximum value of the multi-objective optimization function through the Bayesian optimization algorithm, and terminate the search when the preset termination condition is met, and output the optimal exposure parameter combination.

[0009] Preferably, applying the optimal exposure parameter combination to the next cycle of image capture, calculating the global quality reward value of the newly acquired four polarized images, and dynamically updating the exposure parameter selection strategy table through the Q-learning algorithm, specifically including: Adjust the parameters of the detector based on the optimal exposure parameter combination, collect four polarized images in the next cycle and perform color calibration to generate corrected polarized images; Calculate the sharpness score, color deviation score, and global luminance information entropy in the next cycle, and calculate the global quality reward value through the linear weighting formula; Construct a Q-learning state-action pair, which includes the state jointly encoded by the current ambient light intensity and the optimal exposure parameter combination, and the actions composed of the adjustment amounts of the exposure time, aperture value, and gain respectively; According to the combination of the current state and action, combined with the currently obtained global quality reward value and the maximum expected return of the next state, update the corresponding Q value in the selection strategy table according to the preset rules; When the Q value of the set number of iterations is less than the preset value, perform the generation of the fused image.

[0010] Preferably, the wavelet decomposition specifically includes: Perform five-layer discrete wavelet transform on each polarized image after color calibration, and use the db4 wavelet basis function to decompose to generate a low-frequency subband and a high-frequency subband; Based on the comprehensive edge map, calculate the initial weights of the high-frequency subbands of the polarized images at each angle, and the calculation formula is: Among them, represents the initial weight of the high-frequency subband at the pixel position for the th polarization angle, represents the gradient magnitude at the pixel position of the comprehensive edge map for the th polarization angle, represents the Euclidean norm, represents the polarization angle index.

[0011] Preferably, the frequency division multi-scale fusion includes: Based on the Stokes parameters, calculate the fusion weights of each angular low-frequency sub-band through the principal component analysis method, and perform low-frequency fusion; Based on the initial weights of the high-frequency sub-bands, adjust the weights in combination with the time-domain stability factor, and fuse each high-frequency sub-band according to the adjusted weights; Perform inverse wavelet transform on the fused low-frequency sub-band and high-frequency sub-band to generate the final fused image.

[0012] Compared with the related technologies, a multi-scale polarization data fusion method based on a 4K polarization imaging detector provided by the present invention has the following beneficial effects: The present invention improves the image quality through preprocessing operations such as adaptive edge detection, color calibration, and exposure parameter optimization on the original polarization image; then decides whether to perform frequency division multi-scale fusion according to the global quality reward value, fully considers the different characteristics of the low-frequency and high-frequency sub-bands during the fusion process, adopts targeted fusion strategies, effectively retains and fuses the important information of each polarization channel, thereby generating a high-quality fused image to meet the requirements of polarization images in different fields. Description of the Drawings

[0013] Figure 1 It is a flowchart of a multi-scale polarization data fusion method based on a 4K polarization imaging detector provided by the present invention. Detailed Embodiments

[0014] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings rather than all structures. Furthermore, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0015] It should also be noted that for the sake of description, only parts related to the present invention are shown in the drawings rather than all content. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but there can also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0016] In this application, a polarization imaging detector is an advanced optical device that enhances imaging capabilities by capturing and analyzing the polarization state information of light waves. Its core principle is to utilize the polarization modulation characteristics of the object surface to incident light (such as reflection, transmission, scattering, etc.) to obtain detailed information that cannot be revealed by traditional intensity imaging (such as surface roughness, material properties, stress distribution, etc.).

[0017] It includes a multi-angle polarizer array (0°, 45°, 90°, 135°), a high-sensitivity image sensor (supporting 4K resolution), and a filter module to achieve time-division or synchronous acquisition of polarized light.

[0018] A signal processing unit is used to calculate Stokes parameters ( , , ) in real time to characterize the light intensity and polarization state distribution, providing a data basis for multi-scale fusion.

[0019] A control module is used to dynamically adjust the exposure time, aperture value, and gain to adapt to complex lighting environments.

[0020] The present invention provides a multi-scale polarization data fusion method based on a 4K polarization imaging detector. As shown in Figure 1 , the method includes the following steps: S1: Perform adaptive edge detection on four original polarization images captured synchronously at a preset period to generate a comprehensive edge map and calculate a sharpness score.

[0021] Specifically, step S1 includes the following steps: S11: Dynamically select a combination of edge detection algorithms based on the local contrast of each polarization image, where: When the local contrast is higher than the first threshold, a weighted fusion of the Canny algorithm and the Sobel gradient operator is used; When the local contrast is between the first threshold and the second threshold, the LoG operator is used for multi-scale edge extraction; When the local contrast is lower than the second threshold, the phase consistency edge detection algorithm is used.

[0022] In this embodiment, for the texture complexity and lighting conditions of different scenes, the optimal combination of edge detection algorithms is dynamically selected to reduce the computational complexity while ensuring the edge positioning accuracy and adapting to the high-resolution characteristics of 4K polarization images.

[0023] During implementation, first, for the original image at each polarization angle, the local contrast is calculated using a 7×7 sliding window. The local contrast is determined by calculating the ratio of the difference between the maximum and minimum gray values within the image window to the sum of the two values. The scene is divided into three categories: high, medium, and low according to the contrast range. When the local contrast is higher than 0.6, a weighted fusion of the Canny algorithm and the Sobel gradient operator is used, where the Canny edge detection result accounts for 70% of the weight and the Sobel gradient response accounts for 30% of the weight. When the local contrast is between 0.3 and 0.6, the Laplacian of Gaussian (LoG) operator is used for multi-scale edge extraction. The edge responses are calculated separately using Gaussian kernels of two different scales (σ = 1.0 and 2.0), and the maximum value is taken as the final result. When the local contrast is lower than 0.3, the phase congruency algorithm is used to calculate the local phase congruency in four directions (0°, 45°, 90°, 135°), and a binary edge map is generated through thresholding.

[0024] S12: Confidence weighting is performed on the initial edge map generated by the selected algorithm to generate a comprehensive edge map containing pixel-level weight values, where the pixel-level weight values are positively correlated with the edge response intensity.

[0025] In this embodiment, the edge detection results of different algorithms are integrated, and noise is suppressed and real edges are enhanced through pixel-level confidence weighting, providing robust edge features for subsequent sharpness scoring.

[0026] During implementation, for the initial edge map generated by each algorithm, its pixel-level confidence weight is calculated. The weight value is determined by the ratio of the edge gradient magnitude at this pixel position to the maximum gradient magnitude of the entire image. The larger the gradient magnitude, the higher the weight. Subsequently, all the initial edge maps at the same polarization angle are weighted and fused according to the weights. For example, in a high-contrast scene, the results of Canny and Sobel are superimposed with weights of 0.7 and 0.3 respectively. After fusion, redundant edges are eliminated through non-maximum suppression, and morphological closing operations are performed on the broken edges using a 3×3 rectangular kernel to connect them, finally generating a comprehensive edge map.

[0027] S13: Calculate the sharpness score of each polarization image according to the average gradient magnitude of the comprehensive edge map.

[0028] In this embodiment, the gradient intensity of the edge map is quantified into a standardized sharpness score, providing a comparable quality index for subsequent exposure parameter optimization.

[0029] During implementation, first calculate the average gradient magnitude of each polarization angle integrated edge map, that is, the sum of the gradient magnitudes of all pixels in the entire image divided by the total number of pixels. Then, map the average gradient magnitude to the range of 0 to 1 through normalization: if the average gradient magnitude is lower than the calibrated minimum value (e.g., 5), the score is forced to 0; if it is higher than the calibrated maximum value (e.g., 50), the score is forced to 1; in other cases, it is calculated according to a linear ratio. Finally, take the lowest sharpness score among the four polarization angles as the final index to ensure that the image quality of all angles meets the requirements. For example, if the scores of 0°, 45°, 90°, and 135° are 0.92, 0.85, 0.78, and 0.88 respectively, the final sharpness score is 0.78.

[0030] S2: Based on the ambient light intensity and color temperature collected in real time, perform color calibration on each original polarization image, and output the polarization image after color calibration and the color deviation score.

[0031] Specifically, step S2 includes the following steps: S21: Based on the ambient light intensity and color temperature data, perform white balance correction on the four original polarization images through von Kries color adaptation transformation to eliminate the spectral shift caused by the difference in the transmittance of the polarizer.

[0032] In this embodiment, to eliminate the spectral shift caused by the difference in the transmittance of different polarizers and unify the color reference of the four-angle images, first obtain the color temperature value of the current scene (e.g., about 5500K in a daylight environment) in real time through an ambient light sensor, and then use von Kries color adaptation transformation to convert the image from the RGB space to the LMS cone response space to simulate the adaptability of the human eye to the light source. Adjust the gain of each color channel through matrix operations to map the white point of the image to the color gamut range of the standard light source D65 (6504K), and apply compensation coefficients respectively for the transmittance differences of each polarization angle (0°, 45°, 90°, 135°) to ensure the consistent color response of the four-angle images. For example, when the blue light transmittance of the 90° polarizer is low, compensate by increasing the gain of the blue channel.

[0033] S22: For each polarization image after white balance correction, use the Retinex theory to perform multi-scale illumination estimation to generate the polarization image after color calibration.

[0034] In this embodiment, to address the local brightness distortion caused by polarization modulation (such as overexposure of highlights on the metal surface or loss of shadow details), the Retinex theory is used to restore the true color of the scene. In the specific implementation, first filter the image after white balance correction using three Gaussian kernels with different scales ( = 15, 80, 200): the small scale ( = 15) captures local brightness changes, and the medium scale ( = 80) Extract the regional illumination features, large scale ( = 200) Estimate the global illumination distribution. Subsequently, linearly superimpose the illumination components of the three by weights 0.4, 0.3, and 0.3 to generate a comprehensive illumination estimation map. Finally, in the HSV color space, divide the value (V) channel of the original image by the illumination estimation map to separate the reflection component (i.e., the texture details after removing the illumination influence), and combine it with the original chromaticity (H, S channels) to output the polarization image after color calibration.

[0035] S23: Calculate the color deviation scores of each polarization image and the reference values of the standard color card through the CIEDE2000 color difference formula.

[0036] In this embodiment, to quantify the deviation degree between the calibrated image and the true color, deploy a 24-color standard color card (such as X-Rite ColorChecker) in the scene, and collect its four-angle reflection spectra as the reference benchmark. Convert the calibrated image to the CIELAB color space, extract the value (L*), red-green axis (a*), and yellow-blue axis (b*) values of each color block of the color card, and calculate the color difference from the reference value block by block through the CIEDE2000 formula: First, calculate the lightness difference ΔL*, chromaticity difference ΔC, and hue difference ΔH, and then combine the human eye perception weights (kL = 1.0, kC = 1.0, kH = 1.0) to synthesize the total color difference ΔE00. Finally, take the average ΔE00 of the 24 color blocks as the color deviation score.

[0037] S3: Combine the sharpness score, color deviation score, and the global brightness information entropy of the four polarization images after color calibration to construct a multi-objective optimization function, and solve it in the multi-exposure parameter space to obtain the optimal exposure parameter combination.

[0038] Specifically, step S3 includes the following steps: S31: Normalize the sharpness score and the color deviation score respectively.

[0039] In this embodiment, the normalization process eliminates the dimensional difference between the sharpness score (based on the gradient magnitude) and the color deviation score (based on the color difference ΔE00), making the two comparable in the optimization function and ensuring the fairness and effectiveness of the multi-objective optimization.

[0040] The specific implementation content is as follows: The dynamic range of the clarity score is preset according to the hardware characteristics of the detector. The minimum gradient amplitude threshold is set to 5 (corresponding to a blurred image), and the maximum threshold is set to 50 (corresponding to a high sharpness image). The actual clarity score is linearly scaled to the range of 0 to 1: if the score is lower than the minimum value, it is forced to 0, and if it is higher than the maximum value, it is forced to 1. Exemplarily, when the actual score is 35, subtracting the minimum value of 5 gives 30, and then dividing by the difference between the maximum and minimum values of 45, the normalized result is 0.67.

[0041] Based on the maximum acceptable color difference threshold of 5.0 in industrial inspection standards, the actual color difference is converted into a score from 0 to 1. The smaller the color difference, the higher the score. When the color difference is equal to or exceeds 5.0, the score is 0. Exemplarily, if the actual color difference between an image and a standard color card is 2.3, the normalized score is 1 minus 2.3 divided by 5.0, and the result is 0.54.

[0042] S32: Merge the luminance channels of the four polarized images after color calibration, and calculate the global luminance information entropy of their joint histogram.

[0043] In this embodiment, it specifically includes converting the calibrated images of the four polarization angles into grayscale images, extracting the luminance components and stitching them into a joint luminance matrix. Exemplarily, when the single-angle image resolution is 1920×1080, the size of the merged matrix expands to 3840×2160.

[0044] Count the number of pixels of each gray level (0 - 255) in the merged matrix, and calculate its proportion of the total number of pixels. Exemplarily, if the number of pixels of gray level 128 is 82944 and the total number of pixels is 8.29 million, then its proportion is 1%.

[0045] Calculate the entropy value according to the probability distribution of all gray levels to measure the uniformity of the luminance distribution. When the probabilities of all gray levels are equal, the entropy value reaches the maximum of 8.0; if the image is all black or all white, the entropy value is 0. Exemplarily, the luminance entropy of a certain scene is 6.2, indicating that the luminance distribution is relatively balanced.

[0046] S33: Construct a linearly weighted multi-objective optimization function based on the normalized clarity score, color deviation score, and global luminance information entropy.

[0047] In this embodiment, based on historical data analysis, the contribution weights of each index are determined. The weight of the clarity score is 0.5 (the most sensitive to details), the weight of the luminance entropy is 0.3 (to prevent overexposure / underexposure), and the weight of the color deviation score is 0.2. The sum of the three is 1.

[0048] Multiply the normalized sharpness score by 0.5, divide the brightness entropy by the maximum value of 8.0 and then multiply by 0.3, and subtract the normalized color difference score multiplied by 0.2. The final optimization objective is to maximize this weighted sum. For example, if the sharpness score of a set of parameters is 0.67, the brightness entropy is 0.775 (6.2 / 8.0), and the color difference score is 0.54, then the comprehensive score is 0.5×0.67 + 0.3×0.775 - 0.2×0.54 = 0.59.

[0049] Limit the exposure time to be between 1 and 100 milliseconds, the aperture value is a discrete value (such as f / 1.8 to f / 16), and the gain range is 0 to 30 dB to ensure the feasibility of the parameters.

[0050] S34: In the multi-exposure parameter space composed of exposure time, aperture value, and gain, iteratively search for the maximum value of the multi-objective optimization function through the Bayesian optimization algorithm, and terminate the search when the preset termination condition is met, and output the optimal exposure parameter combination.

[0051] In this embodiment, the Gaussian process model is used to simulate the parameter space, the Matérn5 / 2 kernel function is selected to describe the relationship between parameters, and 20 groups of initial parameters are randomly generated (such as exposure time 20 milliseconds, aperture f / 4.0, gain 12 dB).

[0052] Then perform iterative optimization, specifically including: selecting the next sampling point through the expected improvement strategy to balance the exploration of the unknown region and the utilization of the known high-quality region.

[0053] Update the model after each iteration and record the current optimal score. Exemplarily, the best score found in the first 10 iterations is 0.72, and subsequent iterations gradually approach a better solution.

[0054] If the score improvement in 5 consecutive iterations is less than 0.01, then terminate the search. Exemplarily, the score stabilizes at 0.81 after the 35th iteration and is determined to converge.

[0055] Select the historical optimal parameter combination, such as exposure time 80 milliseconds, aperture f / 2.0, gain 18 dB. At this time, the comprehensive score is 0.81, which meets the requirements of the dynamic scene.

[0056] S4: Apply the optimal exposure parameter combination to the next cycle of image capture, calculate the global quality reward value of the four newly acquired polarization images, and dynamically update the exposure parameter selection strategy table through the Q-learning algorithm.

[0057] Specifically, step S4 includes the following steps: S41: Adjust the parameters of the detector based on the optimal exposure parameter combination, collect the four polarization images of the next cycle and perform color calibration to generate the corrected polarization images.

[0058] In this embodiment, the optimal exposure parameter combination (such as exposure time 80 ms, aperture f / 2.0, gain 18 dB) optimized in step S3 is accurately configured to the detector hardware to ensure that the image quality collected in the next cycle meets the optimization goal.

[0059] The exposure parameters are dynamically updated through the detector control interface, and the image acquisition modules for four polarization angles (0°, 45°, 90°, 135°) are synchronously triggered to ensure strict alignment of timestamps. After the acquisition is completed, the white balance correction and Retinex illumination compensation algorithms in step S2 are called to perform color calibration on the original image. For example, in a low-light scene, according to the blue light attenuation characteristics of the 90° polarizer, the blue channel gain is automatically increased by 15% to compensate for spectral shift.

[0060] S42: Calculate the sharpness score, color deviation score, and global brightness information entropy in the next cycle, and calculate the global quality reward value through a linear weighting formula.

[0061] In this embodiment, by quantifying the quality metrics of the newly acquired images, an interpretable feedback signal is provided for Q-learning strategy optimization.

[0062] Call step S1 to generate a comprehensive edge map, calculate the sharpness scores of the four angles and take the minimum value (such as 0.78); call the CIEDE2000 color difference calculation process in step S2 to obtain the average color difference ΔE = 2.1 between the current frame and the standard color card; merge the brightness channels of the four angles and calculate the global brightness information entropy H = 6.5. Linearly weight according to the preset weights (sharpness 0.6, brightness entropy 0.3, color difference 0.1): 0.6×0.78 + 0.3×(6.5 / 8.0) - 0.1×(2.1 / 5.0) = 0.654 to generate the global quality reward value.

[0063] S43: Construct a Q-learning state-action pair, which includes the state jointly encoded by the current ambient light intensity and the optimal exposure parameter combination, and the action composed of the adjustment amounts of the exposure time, aperture value, and gain respectively.

[0064] In this embodiment, the ambient light intensity (0 - 800 lux) is discretized into 8 levels (such as 200 - 300 lux is level 3), and the current exposure parameters (exposure time, aperture value, gain) are normalized to the 0 - 1 interval values and spliced into a 9-dimensional state vector (light intensity level + 3 parameters). The action space is defined as the parameter adjustment amount combination: exposure time ±5 ms, aperture ±0.5 stops, gain ±3 dB, a total of 27 actions (3×3×3).

[0065] S44: According to the combination of the current state and action, combined with the globally obtained quality reward value and the maximum expected return of the next state, update the corresponding Q value in the selection strategy table according to the preset rules.

[0066] In this embodiment, the Q value is updated using the temporal difference learning rule, and the learning rate = 0.1, and the discount factor = 0.9. Assume the current state of , and execute the action . After that, the obtained reward is R = 0.654, and the maximum expected Q value of the next state is 0.72. Then the update formula is: = 0.5 + 0.1×(0.654 + 0.9×0.72 - 0.5) = 0.568. The ε-greedy strategy (ε = 0.1) is adopted, and the current optimal action is selected with a 90% probability, and a new action is randomly explored with a 10% probability to jump out of the local optimum.

[0067] S45: When the Q value of the set number of iterations is less than the preset value, perform the generation of the fused image.

[0068] In this embodiment, continuously monitor the change amplitude of the Q value in the last 5 iterations. If the maximum change amount is less than the threshold 0.01 (such as 0.008, 0.005, 0.003, etc.), it is determined to converge. Immediately freeze the Q table, stop parameter adjustment, and call the wavelet decomposition and fusion process in step S5 to generate the final image. For example, after the 30th iteration, the Q value stabilizes at 0.81 ± 0.005, triggering fusion and outputting a high-dynamic-range fused image, and at the same time resetting the Q table to adapt to the optimization requirements of the next scene.

[0069] S5: When the globally obtained quality reward value meets the preset conditions, perform wavelet decomposition on each polarized image after color correction in combination with the comprehensive edge map, and perform frequency-division multi-scale fusion on the low-frequency subbands and high-frequency subbands after wavelet decomposition to generate a fused image.

[0070] Specifically, in step S5, the wavelet decomposition includes: First, perform five-layer discrete wavelet transform on each polarized image after color calibration, and use the db4 wavelet basis function to decompose to generate low-frequency subbands and high-frequency subbands.

[0071] In this embodiment, the images of four polarization angles (0°, 45°, 90°, 135°) after color calibration are decomposed into low-frequency subbands and high-frequency subbands, and multi-scale features are extracted to support subsequent fusion. Specifically: Perform five - layer discrete wavelet transform on the images at each polarization angle using the db4 wavelet basis function. The first layer decomposes the original image into a low - frequency sub - band (retaining the overall brightness and polarization information) and three high - frequency sub - bands (horizontal, vertical, and diagonal details). Iteratively decompose the low - frequency part layer by layer, finally generating five low - frequency sub - bands (with gradually decreasing resolution) and corresponding high - frequency sub - bands. The sub - bands of the same scale at the four polarization angles are strictly aligned to ensure consistent scale during fusion. For example, the resolution of the fifth - layer low - frequency sub - band is 120×68, which centrally represents the global polarization characteristics of the scene.

[0072] Secondly, based on the comprehensive edge map, calculate the initial weights of the high - frequency sub - bands of the polarization images at each angle, where the calculation formula is: Among them, represents the initial weight of the high - frequency sub - band at the th polarization angle at the pixel position , represents the gradient magnitude at the pixel position of the comprehensive edge map of the th polarization angle, represents the Euclidean norm, represents the polarization - angle index.

[0073] In this embodiment, based on the gradient response intensity of the comprehensive edge map, assign fusion weights to the high - frequency sub - bands at each angle to strengthen important details and suppress noise, specifically including: Extract the pixel - level gradient magnitude (measuring the edge strength) of each polarization angle from the comprehensive edge map generated in step S1. For a specific pixel position, calculate the sum of the gradient magnitudes of the four angles at this position, and use the ratio of the gradient magnitude of each angle to the sum as the initial weight. The larger the weight value, the more significant the edge details of this angle. For example, if the gradient magnitude ratio of 0° polarization at a certain pixel is 40%, then the fusion weight of its high - frequency sub - band at this position is 0.4. The weight calculation covers all high - frequency sub - bands (LH, HL, HH) to ensure balanced enhancement of multi - direction details.

[0074] Specifically, in step S5, the frequency - division multi - scale fusion includes: Low - frequency fusion: Calculate the fusion weights of the low - frequency sub - bands at each angle based on the principal - component analysis (PCA) of the Stokes parameters ( , , ). Extract the first three principal components (contribution rate ≥95%) through PCA, and assign weights according to the variance contribution rate, and perform weighted fusion on the low - frequency sub - bands of the four angles. For example, the weights of the 0° and 90° sub - bands with larger variances are 0.4 and 0.3 respectively, and the weights of the 45° and 135° sub - bands are 0.2 and 0.1 respectively, generating a low - frequency fusion result that retains polarization consistency.

[0075] High-frequency fusion: Based on the initial weights calculated (calculated from the edge gradient magnitudes), introduce a temporal stability factor β to dynamically adjust the weights. β is calculated from the sliding average and standard deviation of the historical frame quality reward values. When the reward value fluctuations are small, β ≈ 1.0 (stable state), and when the fluctuations are large, β is less than 1.0 (suppressing noise). The adjusted weights are weighted and fused with the high-frequency subbands (LH, HL, HH) pixel by pixel to strengthen the detail contribution of strong edge angles.

[0076] Integrate the fused low-frequency and high-frequency subbands to generate a final fused image that conforms to optical laws and has enhanced details, specifically including: Reconstruct the image layer by layer in the reverse order of the five-layer decomposition. Starting from the fifth-layer low-frequency subband (resolution 120×68), successively stack the high-frequency subbands of the same scale, and iterate to the first layer to restore the full resolution (3840×2160).

[0077] Verify that the Stokes parameters of the fused image satisfy greater than or equal to + , interpolate and correct the polarization angle for the non-conforming regions; then enhance the contrast through adaptive histogram equalization and apply non-local means denoising to eliminate high-frequency noise, and output a fused result with optimized visual quality.

[0078] This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0079] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0080] It should also be noted that the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent in such a process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the element.

Claims

1. A multi-scale polarization data fusion method based on a 4K polarization imaging detector, characterized in that The method includes the following steps: Perform adaptive edge detection on four original polarization images synchronously captured at a preset period, generate a comprehensive edge map, and calculate a sharpness score; Based on the ambient light intensity and color temperature collected in real time, perform color calibration on each original polarization image, and output the polarization image after color calibration and a color deviation score; Combine the sharpness score, color deviation score, and the global luminance information entropy of the four polarization images after color calibration to construct a multi-objective optimization function, and solve it in the multi-exposure parameter space to obtain an optimal exposure parameter combination; Apply the optimal exposure parameter combination to the next-cycle image capture, calculate the global quality reward value of the newly captured four polarization images, and dynamically update the exposure parameter selection strategy table through the Q-learning algorithm; When the global quality reward value meets the preset condition, perform wavelet decomposition on each polarization image after color correction in combination with the comprehensive edge map, and perform frequency-division multi-scale fusion on the low-frequency sub-band and high-frequency sub-band after wavelet decomposition to generate a fused image.

2. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 1, characterized in that, The performing adaptive edge detection on four original polarization images synchronously captured at a preset period, generating a comprehensive edge map, and calculating a sharpness score specifically includes: Dynamically select a combination of edge detection algorithms based on the local contrast of each polarization image, where: When the local contrast is higher than the first threshold, use a weighted fusion of the Canny algorithm and the Sobel gradient operator; When the local contrast is between the first threshold and the second threshold, use the LoG operator for multi-scale edge extraction; When the local contrast is lower than the second threshold, use the phase congruency edge detection algorithm; Perform confidence weighting on the initial edge map generated by the selected algorithm to generate a comprehensive edge map containing pixel-level weight values, where the pixel-level weight values are positively correlated with the edge response intensity; Calculate the sharpness score of each polarization image according to the average gradient magnitude of the comprehensive edge map.

3. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 2, characterized in that, The performing color calibration on each original polarization image based on the ambient light intensity and color temperature collected in real time, and outputting the polarization image after color calibration and a color deviation score specifically includes: Based on the ambient light intensity and color temperature data, perform white balance correction on the four original polarization images through the von Kries color adaptation transformation to eliminate the spectral shift caused by the difference in the transmittance of the polarizer; For each polarization image after white balance correction, use the Retinex theory for multi-scale illumination estimation to generate the polarization image after color calibration; Calculate the color deviation score of each polarization image from the reference value of the standard color card through the CIEDE2000 color difference formula.

4. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 3, characterized in that, The combining the sharpness score, color deviation score, and the global luminance information entropy of the four polarization images after color calibration to construct a multi-objective optimization function, and solving it in the multi-exposure parameter space to obtain an optimal exposure parameter combination specifically includes: Normalize the sharpness score and color deviation score respectively; Merge the luminance channels of the four polarization images after color calibration, and calculate the global luminance information entropy of their joint histogram; Construct a linearly weighted multi-objective optimization function based on the clarity score, color deviation score, and global brightness information entropy after normalization processing; In the multi-exposure parameter space composed of exposure time, aperture value, and gain, iteratively search for the maximum value of the multi-objective optimization function through the Bayesian optimization algorithm, and terminate the search when the preset termination condition is met, and output the optimal exposure parameter combination.

5. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 4, wherein Applying the optimal exposure parameter combination to the next cycle of image capture, calculating the global quality reward value of the newly acquired four polarization images, and dynamically updating the exposure parameter selection strategy table through the Q-learning algorithm, specifically including: Adjust the parameters of the detector based on the optimal exposure parameter combination, collect four polarization images in the next cycle and perform color calibration to generate corrected polarization images; Calculate the clarity score, color deviation score, and global brightness information entropy in the next cycle, and calculate the global quality reward value through the linear weighting formula; Construct a Q-learning state-action pair, which includes the state jointly encoded by the current ambient light intensity and the optimal exposure parameter combination, and the actions composed of the adjustment amounts of exposure time, aperture value, and gain respectively; According to the combination of the current state and action, combined with the currently obtained global quality reward value and the maximum expected return of the next state, update the corresponding Q value in the selection strategy table according to the preset rules; When the Q value of the set number of iterations is less than the preset value, execute the generation of the fused image.

6. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 5, characterized in that, Wavelet decomposition specifically includes: Perform five-layer discrete wavelet transform on each polarization image after color calibration, and use the db4 wavelet basis function to decompose to generate low-frequency subbands and high-frequency subbands; Based on the comprehensive edge map, calculate the initial weights of the high-frequency subbands of the polarization images at each angle, and the calculation formula is: Among them, represents the initial weight of the high-frequency subband at the -th polarization angle at the pixel position . represents the gradient magnitude of the comprehensive edge map of the -th polarization angle at the pixel position . represents the Euclidean norm, represents the polarization angle index.

7. A multi-scale polarization data fusion method based on a 4K polarization imaging detector according to claim 6, characterized in that, Frequency division multi-scale fusion, including: Based on the Stokes parameters, calculate the fusion weights of the low-frequency subbands at each angle through the principal component analysis method, and perform low-frequency fusion; Based on the initial weights of the high-frequency subbands, adjust the weights in combination with the time-domain stability factor, and fuse the high-frequency subbands according to the adjusted weights; Perform wavelet inverse transform on the fused low-frequency subbands and high-frequency subbands to generate the final fused image.

Images