An integrated system for image anti-interference processing and quality evaluation

Abstract

This application relates to an integrated system for image anti-interference processing and quality evaluation. The method includes: an image interference mode determination module for acquiring an image to be processed; extracting image quality features from the image to be processed and automatically determining the image interference mode based on the image quality features; an image anti-interference module for calling the corresponding processing module from a dehazing module, a ghosting elimination module, or a dual-light fusion module according to the determined image interference mode to perform anti-interference enhancement processing on the image to be processed and outputting the processed result image; an image quality evaluation and visualization module for extracting and evaluating image quality features of the processed result image and outputting image quality evaluation indicators; and visualizing the image to be processed, the processed result image, and the image quality evaluation indicators. This method can automatically select the corresponding anti-interference processing flow according to the imaging scene and interference type.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an integrated system for image anti-interference processing and quality evaluation. Background Technology

[0002] With the widespread application of imaging technology in remote sensing monitoring, security surveillance, and industrial inspection, the image quality acquired in complex imaging environments has a significant impact on subsequent visual analysis and interpretation tasks. However, in practical applications, the imaging process is often affected by a variety of interference factors, such as cloud and fog obstruction, motion blur caused by target or platform movement, and differences in multimodal imaging. These factors lead to reduced image contrast, blurred edges, and loss of texture details, severely restricting the effective utilization of image information.

[0003] To address the aforementioned issues, existing technologies typically employ single image enhancement methods such as dehazing algorithms, de-ghosting algorithms, or multimodal image fusion algorithms for anti-interference processing. However, existing solutions still suffer from the following shortcomings: In real-world applications, when multiple types of interference exist simultaneously, existing technologies struggle to collaboratively handle different types of interference within the same system. Users often need to manually select the corresponding processing algorithm based on the image degradation type, a cumbersome process that is difficult to adapt to complex and changing imaging environments. There is a lack of automatic interference type determination mechanisms. Existing image anti-interference processing systems typically require users to pre-determine the type of interference affecting the image and manually select the appropriate processing module accordingly. However, in practical engineering applications, users often struggle to accurately determine the type of image interference, especially in batch image processing scenarios where different images may be subject to different types of interference. Manual judgment is inefficient and prone to errors. Existing technologies lack a solution that can automatically identify image interference types and call upon the corresponding processing module. Furthermore, most existing image enhancement systems only provide processing result display functionality, lacking an objective quantitative evaluation mechanism for the processing effect. Users typically rely on subjective visual judgment to assess the processing effect, making quantitative analysis and comparative verification of the results difficult. Especially in engineering applications, the lack of objective evaluation metrics makes it difficult to optimize algorithm parameters, evaluate system performance, and verify the consistency of batch processing results. Summary of the Invention

[0004] Therefore, it is necessary to provide an integrated image anti-interference processing and quality evaluation system that can automatically select the corresponding anti-interference processing flow according to the imaging scene and interference type, and adapt to the needs of diverse application scenarios such as complex sea surfaces and mountainous areas, in order to address the above-mentioned technical problems.

[0005] An integrated system for image anti-interference processing and quality evaluation, the system comprising an image interference mode determination module, an image anti-interference module, and an image quality evaluation and visualization module: the image anti-interference module includes a dehazing module, a ghosting elimination module, and a dual-light fusion module; The image interference mode determination module is used to acquire the image to be processed; extract image quality features from the image to be processed; and automatically determine the image interference mode based on the image quality features. The image interference modes include cloud and fog mode, ghosting mode, or dual-light mode. The image anti-interference module is used to call the corresponding processing module from the dehazing processing module, the ghosting elimination processing module or the dual-light fusion processing module according to the determined image interference mode, to perform anti-interference enhancement processing on the image to be processed, and output the processed result image. The image quality assessment and visualization module is used to extract and evaluate image quality features from the processed images, output image quality assessment indicators, and visualize the images to be processed, the processed images, and the image quality assessment indicators.

[0006] The aforementioned integrated image anti-interference processing and quality assessment system first integrates three major processing functions—dehazing, motion blur removal, and dual-light fusion—through an image anti-interference module. A modular design ensures that each processing module is independent and flexibly accessible. Once the image interference mode determination module automatically identifies the interference type of the current image, the image anti-interference module can precisely select the corresponding processing module from the three modules for targeted enhancement processing based on the determination result. This design enables the system to handle various complex interference scenarios, such as cloud and fog interference, motion blur interference, and multimodal imaging differences, within a unified framework, avoiding the shortcomings of existing technologies that require separate algorithms and cannot coordinate processing. Furthermore, this system extracts contrast features, edge sharpness features, and high-frequency energy features from the input image through the image interference mode determination module, constructing an image degradation feature vector. Based on a classification function, it automatically determines the image interference type as cloud / fog mode, motion blur mode, or dual-light mode. For dual-light mode, the system automatically identifies the alignment relationship between infrared and visible light images by analyzing the image file path and folder structure; for single-modal images, the system automatically distinguishes between cloud / fog mode and motion blur mode using a classification function. This automatic judgment mechanism completely changes the cumbersome process of requiring users to manually select processing modes in existing technologies. Especially in batch image processing scenarios, it can automatically match the optimal processing flow for each image, significantly improving the system's automation level and processing efficiency. Finally, by setting up an image quality evaluation and visualization module, after completing the anti-interference processing, image quality feature extraction and evaluation analysis are performed on the processed image, outputting multiple objective quantitative evaluation indicators, including full-reference indicators and no-reference indicators, and visually displaying the original image, the processed image, and the evaluation indicators. This design allows the image enhancement effect to no longer rely solely on subjective visual judgment, but can be expressed and analyzed in the form of objective quantitative indicators, providing a scientific basis for algorithm parameter tuning, system performance evaluation, and consistency verification of batch processing effects. This application achieves precise processing of different interferences through an image anti-interference module that integrates three major processing modules, and achieves objective quantification and visualization of processing effects through the image quality evaluation and visualization module. This enables the system to automatically select the corresponding anti-interference processing flow according to the imaging scene and interference type, effectively adapting to the needs of complex sea surfaces and mountainous areas and other variable application scenarios, solving the technical problems of scattered processing flows, lack of a unified framework, and difficulty in objectively evaluating processing effects in existing technologies. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of an integrated system for image anti-interference processing and quality evaluation in one embodiment. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0009] In one embodiment, such as Figure 1 As shown, an integrated system for image anti-interference processing and quality evaluation is provided. The system includes an image interference mode determination module, an image anti-interference module, and an image quality evaluation and visualization module. The image anti-interference module includes a dehazing module, a ghosting elimination module, and a dual-light fusion module. The image interference mode determination module is used to acquire the image to be processed; extract image quality features from the image to be processed; and automatically determine the image interference mode based on the image quality features. The image interference mode includes cloud and fog mode, ghosting mode, or dual-light mode.

[0010] Image acquisition primarily employs offline acquisition methods. The image to be processed can be a single image or a continuously acquired image sequence. Image sources include visible light imaging devices and infrared imaging devices, and image reading formats include common image formats such as JPG, PNG, and TIFF. Through these methods, the system can adapt to image input requirements in different application scenarios. Different types of image degradation typically possess different statistical characteristics. For example, haze reduces overall image contrast, and motion blur causes edge blurring and loss of high-frequency information. Therefore, determining the image degradation type before performing image anti-interference processing helps select a more suitable image enhancement method, thereby improving the image anti-interference processing effect. The image interference mode is used to indicate the specific execution path of subsequent image anti-interference processing steps.

[0011] The image anti-interference module is used to call the corresponding processing module from the dehazing processing module, the ghosting elimination processing module, or the dual-light fusion processing module according to the determined image interference mode, to perform anti-interference enhancement processing on the image to be processed, and output the processed result image.

[0012] The anti-interference processing module is built in a modular manner, with different processing modules operating independently and being flexibly invoked. To enable image anti-interference processing to adapt to interference conditions under remote sensing or aerial photography perspectives, this system has made targeted innovations to existing image dehazing and de-ghosting algorithms, guiding existing algorithms according to different interference modes to achieve better algorithm recovery results.

[0013] The image quality evaluation and visualization module is used to extract and evaluate image quality features from the processed image, output image quality evaluation indicators, and visualize the image to be processed, the processed image, and the image quality evaluation indicators.

[0014] This system visualizes the original image, the processed image, and the corresponding image quality evaluation indicators for comparative analysis. The implementation supports one or more of the following display methods: displaying the original image and the processed image side-by-side; displaying the image quality evaluation indicators numerically or graphically. Furthermore, the system can record relevant information during the processing for log display and subsequent analysis.

[0015] In the aforementioned system, the image interference pattern determination module automatically identifies the interference type of the image to be processed without manual specification; the image anti-interference module automatically calls the corresponding dehazing processing module, ghosting elimination processing module, or dual-light fusion processing module based on the identification results, realizing collaborative processing of different interference types under a unified framework; and the image quality evaluation and visualization module objectively quantifies and visualizes the processing results, solving the problem of separation between processing and evaluation in existing technologies, and providing a valid basis for algorithm parameter optimization, system performance evaluation, and consistency verification of batch processing effects.

[0016] In one embodiment, the image quality features include image contrast features, edge sharpness features, and frequency domain high-frequency energy features; The image contrast feature is ; in, Indicates the input image The average gray level, , Represents the pixel coordinates of the image, and the image size is... ; Image contrast reflects the degree of dispersion of image grayscale distribution. When an image is affected by haze, the overall grayscale distribution of the image tends to be concentrated due to atmospheric scattering, which leads to a decrease in image contrast.

[0017] The edge sharpness feature is ; in, and These represent the gradients of the image in the horizontal and vertical directions, respectively. Motion blur or degraded blur can cause the edge structure of an image to become smooth, thereby reducing the edge strength of the image. Therefore, calculating the edge sharpness of an image can reflect whether there is motion blur.

[0018] The frequency domain high-frequency energy characteristics are ; Among them, the two-dimensional Fourier transform of the input image is obtained , This represents the image in the frequency domain. The high-frequency region is defined as: ; in The set frequency threshold.

[0019] The detailed information in an image is mainly concentrated in the high-frequency part of the frequency domain. When the image is blurred or affected by haze, the high-frequency information will be significantly reduced. Therefore, by calculating the proportion of high-frequency energy in the image, the degree of image degradation can be further described.

[0020] Specifically, by extracting the aforementioned contrast features, edge sharpness features, and high-frequency energy features, the degree of image degradation can be comprehensively characterized from multiple dimensions, providing a reliable feature basis for the subsequent automatic determination of interference types.

[0021] In one embodiment, automatically determining the image interference mode based on the image quality characteristics includes: Based on the image quality features, an image degradation feature vector is constructed as follows: ; Construct a classification function based on image degradation feature vectors. The classification function outputs the classification result, and the type of interference is determined to be either cloud or ghosting based on the classification result. The dual-light mode is determined by analyzing the input image file path and folder structure. If there is another modal image corresponding to the current image, it is determined to be a dual-light mode; otherwise, it is determined to be a single-modal image and the determination of cloud mode or ghosting mode continues.

[0022] Specifically, image degradation feature vector The three features described in the data depict the degree of image degradation from different dimensions. Among them, the contrast index... This reflects the dispersion of the image's grayscale distribution: when an image is affected by haze, due to atmospheric scattering, the overall grayscale distribution tends to concentrate, leading to a decrease in the contrast index. A significant decrease; while motion blur also affects image quality, its impact on the centralization of grayscale distribution is relatively small. Average edge intensity This reflects the sharpness of edge structures in an image: motion blur causes edges to smooth along the direction of motion, reducing the edge gradient magnitude and average edge intensity. A significant decrease; while smog primarily reduces overall contrast, with a relatively weaker weakening effect on edge gradients. High-frequency energy ratio This reflects the richness of detail in the image: both haze and motion blur lead to the loss of high-frequency information, but the two types of interference have different attenuation patterns of high-frequency energy—the high-frequency attenuation caused by haze is usually more uniform, while the high-frequency attenuation caused by motion blur has directional characteristics.

[0023] Based on the differences in sensitivity to different types of interference based on the above three features, a classification function is constructed. The classification function can be implemented using various classifiers, such as support vector machines, decision trees, or neural networks. During the classifier training phase, a large number of sample images with known interference types are collected beforehand, and the degradation feature vector of each sample image is extracted. And label the corresponding interference type. The classifier is trained using machine learning algorithms, enabling it to learn the mapping relationship between feature vectors and interference types.

[0024] In the classification decision stage, the degraded feature vector extracted from the image to be processed is input into the trained classification function. In a classification function, the probability or distance of a sample belonging to each interference type is calculated based on the position of the feature vector in the high-dimensional feature space. For example, when using a support vector machine classifier, the classification function outputs the class label based on the position of the feature vector relative to the classification hyperplane; when using a decision tree classifier, the classification function makes decisions layer by layer based on the judgment results of the feature vector at each node, and finally outputs the class label corresponding to the leaf node.

[0025] Output of the classification function For discrete category identifiers, for example, setting Indicates fog mode, This indicates a motion blur mode. Based on the results of the classification function, the system determines whether the interference type in the current image to be processed is a cloud / fog mode or a motion blur mode.

[0026] Through the aforementioned classification and judgment mechanism based on image degradation feature vectors, this system can automatically identify the type of interference an image is subjected to, without the need for manual pre-judgment. Compared to existing technologies that require users to manually select processing modes, this system significantly improves the automation and efficiency of image processing. It is particularly suitable for batch image processing scenarios where different images may be subject to different types of interference, and can automatically match the optimal anti-interference processing flow for each image.

[0027] In addition, the system reads the image file names. In multimodal imaging datasets, different imaging modalities are typically identified by folders: "i" folders represent infrared images, and "v" folders represent visible light images. The system searches the input folders for an image of another modality corresponding to the current image. For example, when the current image is an infrared image, the system searches the same folder for an image file with the same ID but whose path contains a "v" folder; when the current image is a visible light image, it searches for a corresponding infrared image file. If the corresponding modal image exists in the folder, the system determines that the current input data is multimodal image data and automatically enters the infrared and visible light image fusion processing mode; if no corresponding modal image is detected, the system determines that the input data is a single-modal image, and performs corresponding single-modal anti-interference processing based on image degradation characteristics. Through the above methods, this system can automatically select the corresponding anti-interference processing flow for different types of interference. In the case of single-modal interference, it can determine the cloud or fog mode or the trailing shadow mode through the classification function and call the corresponding module. In the case of multi-modal interference, it can automatically identify and enter the dual-light fusion processing mode, thereby avoiding the problem that a single processing method is not applicable enough in complex scenes. It also solves the problem that existing image processing systems usually require manual specification of image modality and interference type.

[0028] In one embodiment, the dehazing module is used to calculate the structural intensity distribution of the input image; perform Gaussian smoothing on the structural intensity distribution to obtain a structural guidance map; perform weighted fusion of the structural guidance map and the original image and design an adaptive structural weight mechanism to adjust the structural enhancement intensity to construct a structural guidance feature map; input the structural guidance feature map into the Swin-Transformer-based dehazing model DehazeFormer to obtain a clear image after dehazing.

[0029] Because remote sensing or aerial images are captured at high altitudes, atmospheric scattering significantly affects the brightness distribution over a wide area. This often results in uneven spatial distribution, reduced contrast, and weakened edge details in haze degradation within remote sensing images. Furthermore, remote sensing images commonly contain numerous ground features with distinct structural characteristics, such as roads, building edges, river boundaries, and ridgelines. These structural features exhibit strong spatial stability and continuity. Existing deep learning-based dehazing models typically use the original image as input and automatically learn haze degradation features through the network. However, in remote sensing or aerial photography scenarios, the mixture of low-frequency haze distribution and high-frequency ground feature structures makes the network susceptible to interference from low-frequency haze information during feature extraction, thus affecting its ability to recover true structural information.

[0030] This embodiment enhances the network's ability to preserve the true structure of ground features by explicitly extracting structural information from the image and using this information to guide the input features of the dehazing network. By introducing ground feature structure guidance information during the network input stage, the network can focus more on areas with real structural significance during feature extraction, thereby reducing the interference of low-frequency components of haze on feature learning and improving the restoration of key details such as road, building boundaries, and terrain structures in remote sensing images. Compared with directly using the original image as input, this embodiment, by introducing structure-guided features, not only improves the clarity and contrast of the dehazed image but also effectively maintains the integrity of ground feature structure information, thus enhancing the application value of remote sensing images in subsequent tasks.

[0031] Specifically, the constructed structure-guided feature map The DehazeFormer model is used for image dehazing. The DehazeFormer network uses its multi-layer U-Net model to extract features at multiple scales and leverages the Transformer structure to model global dependencies in the image, thereby restoring haze degradation in complex scenes. The final restored clear image can be represented as: ; in This invention represents a clear image after dehazing. By introducing ground feature structure guidance information during the network input stage, the network can focus more on areas with real structural significance during feature extraction, thereby reducing the interference of low-frequency components of haze on feature learning and improving the restoration of key details such as road, building boundaries, and terrain structure in remote sensing images. Compared with directly using the original image as input, this application, by introducing structure guidance features, not only improves the clarity and contrast of the dehazed image but also effectively maintains the integrity of ground feature structure information, thus enhancing the application value of remote sensing images in subsequent tasks.

[0032] In one embodiment, the calculated structural intensity distribution of the input image is as follows: ; in, Indicates the input image. , Represents the pixel coordinates of the image.

[0033] In one embodiment, the structure guidance map is weighted and fused with the original image, and an adaptive structure weighting mechanism is designed to adjust the structure enhancement intensity to construct a structure guidance feature map, including: The structure guidance map is weighted and fused with the original image, and an adaptive structure weight mechanism is designed to adjust the structure enhancement intensity to construct the structure guidance feature map. ; ; in, This represents the input image, i.e., the original image. This represents a structural guidance diagram. This is the structural reinforcement factor.

[0034] Specifically, using fixed weights to enhance structural features across the entire image can easily lead to two types of problems: firstly, insufficient enhancement may occur in areas with strong structure, failing to fully utilize edge information to guide the dehazing process; secondly, over-enhancement may occur in areas with weak or smooth structure, introducing noise or false edges and affecting the stability of the dehazing results. Therefore, in the process of dehazing remote sensing images, it is necessary to adaptively adjust the structural guidance information according to the strength of structural features in different regions, so that areas with obvious structure receive stronger guidance, while areas with weak structure maintain a smoother feature representation.

[0035] This embodiment calculates and normalizes an image structure intensity map to obtain pixel-level structure weight coefficients, enabling the structure enhancement intensity to dynamically adjust according to changes in the structural information at the pixel location. This normalization process limits the value range of the structure weights to [0,1], thus preventing excessive structure enhancement from causing image distortion. By using the structure guidance map as auxiliary information to perform weighted fusion on the original image, regions with obvious structural features are enhanced, while low-contrast regions significantly affected by haze are more accurately identified under the guidance of structural information.

[0036] In one embodiment, the ghosting removal processing module is used to construct a local structure tensor matrix based on the horizontal and vertical gradients of the input image; perform eigenvalue decomposition on the local structure tensor matrix to estimate the ghosting direction angle; construct a direction-guided weight function based on the ghosting direction angle; construct a ghosting direction-guided feature map based on the direction-guided weight function; and input the ghosting direction-guided feature map into a multi-scale end-to-end recovery network MIMO-UNet to obtain a clear image after ghosting removal.

[0037] Specifically, gradient calculation is performed on the image to obtain the gradients in the horizontal and vertical directions. and Based on the horizontal and vertical gradients, construct the local structure tensor matrix: ; By performing eigenvalue decomposition on the local structure tensor matrix, the dominant local direction can be obtained, thereby estimating the trailing direction angle. Subsequently, a directional guiding weight function is constructed to reflect the structural response intensity along the trailing direction. When the trailing image extends along a certain direction, the structural response in that direction will be significantly enhanced.

[0038] After obtaining the motion blur direction guidance feature map, it is input into the motion blur removal network for restoration. This invention employs the MIMO-UNet multi-scale end-to-end restoration network based on an encoder-decoder structure as the core restoration model. This network achieves progressive restoration from coarse to fine through a multi-input multi-output structure, extracting image features at different scales during the encoding stage and gradually restoring image detail structures during the decoding stage. Simultaneously, an asymmetric feature fusion module (AFF) is introduced during the multi-scale feature fusion process. By weighted fusion of features at different scales, effective information interaction between the encoder and decoder is achieved, thereby enhancing the network's ability to restore structural information of the motion blur region. The final restored clear image is as follows: ; This embodiment enhances the network's ability to recover structural information along the trailing direction by explicitly extracting the dominant direction of the trailing shadow in the image and constructing a direction-guided feature map, thereby improving the effect of trailing shadow removal and edge sharpness in remote sensing images. By introducing a trailing shadow direction estimation and adaptive direction-guided feature construction mechanism, this embodiment enables the network to pay more attention to the structural information distributed along the trailing shadow direction during the trailing shadow recovery process, thus significantly improving the image edge sharpness and detail integrity. Compared with methods that rely solely on deep networks for direct recovery, this method can achieve more stable and clearer recovery results in remote sensing or aerial images, and is particularly suitable for trailing shadow removal processing in scenarios with high platform movement speed or multi-target movement.

[0039] In one embodiment, a direction-guiding weight function is constructed based on the motion blur direction angle, and a motion blur direction-guiding feature map is constructed based on the direction-guiding weight function, including: The direction-guided weight function is constructed based on the direction and angle of the trailing shadow. ; in, and These are the horizontal and vertical gradients of the input image, respectively. The direction and angle of the motion blur; Constructing a trail direction guidance feature map based on the direction guidance weight function ; ; ; in, For adaptive adjustment coefficient, The image size is the input image size. This represents the input image. In this method, in areas with significant motion blur, the directional response is stronger, resulting in a more pronounced enhancement effect, thus strengthening structural features along the motion blur direction. Conversely, in areas with weak motion blur, the directional response is smaller, leading to a weaker enhancement effect, thereby avoiding unnecessary noise amplification in flat areas.

[0040] Specifically, by constructing a direction-guided weighting function, the structural response in a certain direction will be significantly enhanced when the trail extends along that direction. To avoid over-enhancing or under-enhancing the directional features in different images or regions, this embodiment further introduces an adaptive adjustment coefficient. The directional guidance intensity is automatically determined by the overall image motion blur intensity. In areas with significant motion blur, the directional response is stronger, resulting in a more pronounced enhancement effect and thus strengthening structural features along the motion blur direction. Conversely, in areas with weak motion blur, the directional response is smaller, leading to a weaker enhancement effect and avoiding unnecessary noise amplification in flat areas.

[0041] In one embodiment, the dual-light fusion processing module is used to perform feature point matching between the infrared image and the visible light image using the SuperPoint and SuperGlue methods; to calculate the homography matrix based on the matching results and register the infrared image and the visible light image; and to perform multi-scale fusion of the registered infrared image and the visible light image using the stationary wavelet transform (SWT) algorithm to obtain the dual-light fusion result.

[0042] Specifically, the multi-scale fusion process of the stationary wavelet transform (SWT) algorithm includes: performing stationary wavelet transforms on the registered infrared and visible light images respectively to decompose them into low-frequency and high-frequency components; applying a weighted average fusion rule to the low-frequency components and a maximum absolute value fusion rule to the high-frequency components; and performing inverse stationary wavelet transforms on the fused low-frequency and high-frequency components to reconstruct the dual-light fusion result. Through multi-scale decomposition and reconstruction using stationary wavelet transform, the thermal target information of the infrared image and the texture details of the visible light image can be effectively preserved, achieving information complementarity and enhancing the observability of the target or scene.

[0043] In one embodiment, the image quality evaluation and visualization module includes a full-reference image quality evaluation unit and a no-reference image quality evaluation unit. The full-reference image quality evaluation unit performs full-reference image quality evaluation index calculation when a clear image corresponding to the image to be processed exists, and performs quality evaluation based on the full-reference image quality evaluation index. The full-reference image quality evaluation index includes peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), endpoint error index (EPE), edge preservation index (EPI), image registration rate, and target detail retention. The no-reference image quality evaluation unit performs no-reference image quality evaluation index calculation when a clear image does not exist, and performs quality evaluation based on the no-reference image quality evaluation index. The no-reference image quality evaluation index includes the mean and standard deviation of the grayscale histogram, edge sharpness, texture clarity, and high-frequency energy percentage.

[0044] Specifically, when a clear image corresponding to the image to be processed exists, this application has the function of calculating a full-reference image quality evaluation index between the result image and the clear image, and evaluating the recovery processing capability of each anti-interference algorithm through each full-reference image quality evaluation index. This function is applicable to the algorithm tuning process for different scenarios.

[0045] Full reference image quality evaluation metrics include: Peak Signal-to-Noise Ratio (PSNR): One of the most classic objective image quality metrics in the field of digital image processing. Its core idea is to calculate the mean square error between the original image and the distorted image. MSE The degree of distortion is measured by the ratio of the maximum signal power to the noise power, and then quantitatively evaluated by the ratio of the maximum signal power to the noise power. The formula is expressed as follows: ; in, MAX It is the maximum possible pixel value in the image. MSE It is the average difference in pixel values ​​between two images. A higher PSNR indicates that the two images are more similar and that there is less quality loss.

[0046] Structural Similarity Index Measure (SSIM): It can measure the degree of distortion in an image, as well as the similarity between two images. (And...) MSE Unlike PSNR, which measures absolute error, SSIM is a perceptual model, meaning it aligns more closely with human visual perception. It primarily considers three key image features: Luminance, Contrast, and Structure. Luminance is measured by average grayscale, obtained by averaging the values ​​of all pixels, as shown in the following formula: ; in Representing an image x The average value of all pixels; Representing an image y The average value of all pixels; This represents a small constant to prevent division by zero.

[0047] Contrast ratio is measured by the standard deviation of gray levels, and its calculation formula is shown below: ; in Representing an image x The variance; Representing an image y The variance; This represents a small constant to prevent division by zero.

[0048] The structure reflects the geometric structure and texture features of objects in an image, and its calculation formula is shown below: ; in Representing an image x and y covariance, .

[0049] The final SSIM is shown in the formula below: ; generally, ,therefore: ; Combining the above formulas, we can obtain the complete SSIM formula, as shown below: ; A higher SSIM value indicates that the two images are closer.

[0050] Endpoint error (EPE) measures the accuracy of optical flow estimation by calculating the average Euclidean distance between the predicted and actual optical flow fields. It can be expressed as the average error per pixel or the average error over the entire image. In the image ghosting removal task, this application uses EPE to evaluate the error of the average motion vector estimation, and its calculation is as follows:

[0051] in To predict the optical flow vector, For the actual optical flow vector, Indicates the number of pixels. When If the value is less than the threshold, the estimation is considered successful; otherwise, it fails. Finally, the motion vector estimation error rate is calculated.

[0052] Pixel error is used to evaluate the dynamic blur kernel estimation error. In this application, the pixel error is calculated by comparing the image obtained after binarizing the dynamic blur kernel with the binarized image of the real blur kernel. The final number of inconsistent pixels is divided by the pixel size of the real blur kernel to obtain the dynamic blur kernel estimation error.

[0053] The Edge Preservation Index (EPI) is used to quantitatively evaluate the ability of image deblurring to preserve target edges. This application uses EPI based on gradient magnitude to measure the edge preservation ability of an image. For the input image and the restored image, the color image is first converted to grayscale, and then the horizontal and vertical gradients are calculated using the Sobel operator.

[0054] The sum of the absolute values ​​of gradient magnitudes reflects the edge strength of the image. EPI is defined as the ratio of the sum of gradient magnitudes of the reconstructed image to that of the original image; the closer the EPI is to 1, the better the edges are preserved.

[0055] Image registration is a crucial step in dual-light fusion processing, ensuring spatial alignment of fusionable images. In the image registration stage, the SuperPoint+SuperGlue method is used for feature point matching, followed by pixel-level alignment of infrared and visible light images through stationary wavelet transform. This application obtains the number of matched feature point pairs using the SuperPoint+SuperGlue method, estimates the homography matrix H using RANSAC, and removes incorrectly matched points. The image registration rate is then obtained by dividing the number of correctly matched feature point pairs by the total number of feature point pairs.

[0056] To evaluate the performance of the two-light fused image in preserving details of the original image, this application employs a detail preservation score based on image gradient variance. For both the source and fused images, the color images are first converted to grayscale, and then the Laplacian operator is used to calculate the image gradient. The image gradient variance reflects the richness of texture and edge details in the image. The fused detail score is calculated by comparing the gradient variance of the fused image with the larger variance of the two source images, yielding the degree of detail preservation of the fused image relative to the source images: a score close to 100% indicates that the fused image preserves the details of the original image very well.

[0057] In practical engineering applications, obtaining ideally clear images is often difficult, posing a challenge to the objective evaluation of image restoration results. To address this issue, this application employs a reference-free image quality evaluation method, constructing an evaluation index system encompassing grayscale statistical characteristics, edge information, texture structure, and frequency domain features. This index system can quantitatively analyze dehazing, de-ghosting, and two-light fusion results without the need for a reference image, providing a valid basis for algorithm performance evaluation and engineering parameter optimization.

[0058] In dehazing and two-light fusion processing, the grayscale histogram directly affects the visual effect and target recognizability due to changes in image brightness distribution and contrast. To analyze the overall grayscale characteristics of an image, this application extracts the grayscale mean and grayscale standard deviation from the grayscale histogram as auxiliary evaluation indicators.

[0059] The grayscale mean describes the overall brightness level of an image and can reflect whether there are excessively dark or bright areas after dehazing or fusion. The grayscale standard deviation measures the dispersion of the grayscale distribution and is closely related to image contrast and depth. By analyzing the statistical characteristics of grayscale, a supplementary evaluation of image enhancement and fusion effects can be provided from a global perspective, improving the completeness and reliability of engineering evaluation.

[0060] In image dehazing and de-ghosting, the degree of edge information recovery directly reflects the improvement in image sharpness; in the fusion of infrared and visible light, a clear edge structure helps to accurately represent the target contour. Therefore, this application employs an edge sharpness evaluation index based on the Laplacian operator to quantitatively analyze high-frequency edge information in images. By performing Laplacian operations on the image and calculating its variance, the distribution characteristics of edge energy can be effectively reflected. A higher edge sharpness value indicates richer edge details and more effective suppression of blurring effects in the image. The calculation is as follows: ; in This indicates that the input is a grayscale image. This represents variance calculation. This index has good sensitivity to haze residue, motion blur, and edge blurring during fusion, and is suitable for non-reference sharpness evaluation in engineering scenarios.

[0061] Texture detail is a crucial factor in evaluating the effectiveness of dehazing and deghosting algorithms, and it is also a key indicator of the degree of information complementarity in infrared / visible light fused images. This application combines Local Binary Pattern (LBP) and Gray-Level Co-occurrence Matrix (GLCM) features to comprehensively evaluate image texture sharpness. LBP is used to characterize local micro-texture structures, and the discreteness of its pattern distribution reflects the richness of detail information. The contrast and homogeneity features extracted by GLCM statistically describe the magnitude of texture variation and structural consistency, and their calculation is shown below: ; in, , , The weights of each feature are used to balance their contribution to the evaluation results. A comprehensive texture sharpness index is constructed through weighted fusion to measure the enhancement and preservation of texture details during dehazing, deghosting, and two-light fusion processes.

[0062] One of the core objectives of dehazing, de-ghosting, and two-light fusion is to enhance the detail information in an image, which is mainly reflected in the high-frequency components in the frequency domain. To evaluate image quality changes from a frequency domain perspective, this application constructs a high-frequency energy ratio evaluation index based on two-dimensional Fourier transform. By treating the central region of the spectrum as a low-frequency component and the peripheral region as a high-frequency component, calculating their energy and determining their proportional relationship, the degree of enhancement of high-frequency information in the image can be quantitatively reflected. The higher the high-frequency energy ratio, the more significant the dehazing and de-ghosting effects, or the more fully the detail information is expressed after two-light fusion; this index can effectively reflect the image detail restoration and enhancement effects.

[0063] In summary, Peak Signal-to-Noise Ratio (PSNR) measures the degree of distortion by calculating the mean squared error between the original and distorted images. A higher PSNR indicates greater similarity between the two images and less quality loss. Structural Similarity (SSIM) considers three key features of an image: brightness, contrast, and structure. A higher SSIM value indicates greater similarity between the two images. Endpoint Error (EPE) is used to evaluate the error of the average motion vector estimation in image ghosting removal tasks. Edge Preservation Index (EPI) quantitatively evaluates the ability of image deblurring to preserve target edges; an EPI closer to 1 indicates good edge preservation. Image Registration Rate is obtained by dividing the number of correctly matched feature point pairs by the total number of feature point pairs. Target Detail Retention is calculated by comparing the gradient variance of the fused image with the larger variance of the two source images, thus determining the degree of detail retention of the fused image relative to the source images.

[0064] The mean gray level of the gray-level histogram describes the overall brightness level of the image, reflecting whether there are excessively dark or bright areas after dehazing or fusion. The standard deviation of gray levels measures the dispersion of the gray-level distribution and is closely related to image contrast and layering. Edge sharpness is calculated based on the Laplacian operator and its variance, effectively reflecting the distribution characteristics of edge energy. The larger the edge sharpness value, the richer the edge details in the image and the more effectively the blurring effect is suppressed. Texture sharpness combines the features of Local Binary Pattern (LBP) and Gray-Level Co-occurrence Matrix (GLCM) to comprehensively evaluate the texture sharpness of the image, constructing a comprehensive texture sharpness index through weighted fusion. High-frequency energy proportion is constructed based on two-dimensional Fourier transform. By treating the central region of the spectrum as a low-frequency component and the outer region as a high-frequency component, its energy is calculated separately and the proportional relationship is obtained, quantitatively reflecting the degree of enhancement of high-frequency information in the image.

[0065] By comprehensively applying the above-mentioned full-reference and no-reference image quality evaluation indicators, this system can provide a comprehensive and objective quantitative evaluation of the image anti-interference processing effect in different application scenarios, providing a valid basis for algorithm performance evaluation and engineering parameter optimization.

[0066] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0067] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An integrated system for image anti-interference processing and quality evaluation, characterized in that, The system includes an image interference mode determination module, an image anti-interference module, and an image quality evaluation and visualization module: the image anti-interference module includes a dehazing module, a ghosting elimination module, and a dual-light fusion module; The image interference mode determination module is used to acquire the image to be processed; extract image quality features from the image to be processed; and automatically determine the image interference mode based on the image quality features. The image interference mode includes cloud and fog mode, ghosting mode, or dual-light mode. The image anti-interference module is used to call the corresponding processing module from the dehazing processing module, the ghosting elimination processing module or the dual-light fusion processing module according to the determined image interference mode, to perform anti-interference enhancement processing on the image to be processed, and output the processed result image. The image quality evaluation and visualization module is used to extract and evaluate image quality features from the processed image, output image quality evaluation indicators, and visualize the image to be processed, the processed image, and the image quality evaluation indicators. The image quality features include image contrast features, edge sharpness features, and frequency domain high-frequency energy features; The image contrast feature is in, Indicates the input image The average gray level, , Represents the pixel coordinates of the image, and the image size is... ; The edge sharpness feature is in, and These represent the gradients of the image in the horizontal and vertical directions, respectively. The frequency domain high-frequency energy characteristics are in, This represents the image in the frequency domain. Indicates the high-frequency region. This indicates the set frequency threshold; Automatically determine image interference modes based on the image quality characteristics, including: Based on the image quality features, an image degradation feature vector is constructed as follows: ; Construct a classification function based on image degradation feature vectors. The classification function outputs the classification result, and the type of interference is determined to be either cloud or ghosting based on the classification result. The dual-light mode is determined by judging whether there is another modal image corresponding to the current image according to a preset folder naming rule. The imaging modality is distinguished by folder identifiers: "i" folder represents infrared images and "v" folder represents visible light images. If there is an image file with the same main file name as the current image in the folder path of the current image and located in another modal identifier folder, it is determined to be a dual-light mode; otherwise, it is determined to be a single-modal image, and it is further determined to be a cloud or fog mode or a motion blur mode according to the image quality characteristics.

2. The system according to claim 1, characterized in that, The dehazing module is used to calculate the structural intensity distribution of the input image; Gaussian smoothing is applied to the structural intensity distribution to obtain a structural guidance map; the structural guidance map is weighted and fused with the original image, and an adaptive structural weight mechanism is designed to adjust the structural enhancement intensity to construct a structural guidance feature map; The structure-guided feature map is input into the Swin-Transformer-based dehazing model DehazeFormer to obtain a clear image after dehazing.

3. The system according to claim 2, characterized in that, The structural strength distribution of the computational input image is as follows: in, Indicates the input image. , Represents the pixel coordinates of the image.

4. The system according to claim 2, characterized in that, The structure-guided map is weighted and fused with the original image, and an adaptive structure weighting mechanism is designed to adjust the structure enhancement intensity to construct a structure-guided feature map, including: The structure guidance map is weighted and fused with the original image, and an adaptive structure weight mechanism is designed to adjust the structure enhancement intensity to construct the structure guidance feature map. in, This represents the input image, i.e., the original image. This represents a structural guidance diagram. This is the structural reinforcement factor.

5. The system according to claim 1, characterized in that, The ghosting removal processing module is used to construct a local structure tensor matrix based on the horizontal and vertical gradients of the input image; perform eigenvalue decomposition on the local structure tensor matrix to estimate the ghosting direction angle; construct a direction-guided weight function based on the ghosting direction angle; construct a ghosting direction-guided feature map based on the direction-guided weight function; and input the ghosting direction-guided feature map into the multi-scale end-to-end recovery network MIMO-UNet to obtain a clear image after ghosting removal.

6. The system according to claim 5, characterized in that, A direction-guided weight function is constructed based on the motion blur direction angle, and a motion blur direction-guided feature map is constructed based on the direction-guided weight function, including: The direction-guided weight function is constructed based on the direction and angle of the trailing shadow. in, and These are the horizontal and vertical gradients of the input image, respectively. The direction and angle of the motion blur; Constructing a trail direction guidance feature map based on the direction guidance weight function in, For adaptive adjustment coefficient, The image size is the input image size. This represents the input image.

7. The system according to claim 1, characterized in that, The dual-light fusion processing module is used to perform feature point matching between infrared and visible light images using the SuperPoint and SuperGlue methods; to calculate the homography matrix based on the matching results and register the infrared and visible light images; and to perform multi-scale fusion of the registered infrared and visible light images using the stationary wavelet transform (SWT) algorithm to obtain the dual-light fusion result.

8. The system according to claim 1, characterized in that, The image quality evaluation and visualization module includes a full-reference image quality evaluation unit and a no-reference image quality evaluation unit; the full-reference image quality evaluation unit performs full-reference image quality evaluation index calculation when a clear image corresponding to the image to be processed exists, and performs quality evaluation based on the full-reference image quality evaluation index. The full reference image quality evaluation metrics include peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), endpoint error (EPE), edge preservation index (EPI), image registration rate, and target detail retention. The no-reference image quality evaluation unit performs no-reference image quality evaluation index calculation when no clear image exists, and performs quality evaluation based on the no-reference image quality evaluation index; the no-reference image quality evaluation index includes the mean and standard deviation of the grayscale histogram, edge sharpness, texture clarity, and high-frequency energy ratio.

Images