Medium-short wave high-resolution on-chip polarization imaging processing method suitable for low-altitude unmanned aerial vehicle detection

Abstract

The invention provides a medium-short wave high-resolution on-chip polarization imaging processing method suitable for low-altitude unmanned aerial vehicle detection, which solves the technical problems of non-uniform noise, blind pixel and the like of a medium-short wave detector, and comprises the following steps: S1, adopting a medium-short wave focal plane detector integrated by a micro polarizer array to realize simultaneous imaging in four polarization directions; s2, performing multi-segment and scene joint non-uniformity correction on the four-polarization direction image obtained in the S1; s3, blind pixel detection and replacement of weak and small target protection are carried out on the image corrected in the S2; s4, super-resolution reconstruction is carried out on the image processed in the step S3 by adopting an attention enhanced residual network; s5, obtaining a polarization degree image and a polarization angle image through Stokes vector calculation according to the super-resolution image in the S4, and constructing a multi-feature fusion matrix in combination with infrared radiation intensity information; s6, integrating the polarization characteristics and the super-resolution image by adopting a self-adaptive weighted fusion strategy; and S7, 8-bit dynamic mapping based on human visual characteristics is performed on the fused image in the S6.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and more specifically, relates to a medium- and short-wave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection. Background Technology

[0002] With the rapid development of drone technology, low-altitude, slow-speed, small drones are increasingly used in both civilian and military fields, but this has also brought security threats such as illegal intrusion, intelligence gathering, and terrorist attacks. According to industry reports, global drone security incidents increased by 37% year-on-year in 2024, with early warning failures due to low-altitude detection blind spots accounting for 42% of these incidents. Existing detection technologies mainly rely on radar, acoustic sensors, and visible light imaging, but these suffer from problems such as short detection range, high false alarm rates, and weak target recognition capabilities in complex electromagnetic environments and adverse weather conditions.

[0003] Mid- and short-wave infrared imaging technology has become a core means of low-altitude UAV detection due to its advantages such as all-weather operation and strong anti-interference capabilities. However, current mid- and short-wave imaging systems face three major technical bottlenecks: 1. Conflict between spatial resolution and polarization information acquisition: Traditional focal plane arrays require mechanical rotation of polarizers to achieve polarization imaging, which increases the system size and reduces the frame rate, making it impossible to meet the real-time detection requirements of high-speed maneuvering targets of UAVs.

[0004] 2. Image quality degradation problem: Medium and short wave detectors have hardware defects such as non-uniform noise and blind elements. The signal-to-noise ratio of weak targets in complex backgrounds is usually less than 3dB.

[0005] 3. Lagging data processing links: Existing systems mostly adopt a serial processing architecture of "correction-enhancement-recognition". Dynamic range compression during 8-bit image mapping results in the loss of more than 30% of details. The super-resolution reconstruction algorithm is highly complex and difficult to implement in real time on embedded platforms.

[0006] To address the aforementioned issues, this invention proposes a medium- and short-wave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection. Through multi-dimensional information fusion and intelligent processing algorithms, it achieves high-sensitivity, high-resolution detection and identification of low-altitude UAV targets. Summary of the Invention

[0007] The purpose of this application is to provide a mid-to-shortwave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection, breaking through the volume and weight limitations of traditional mechanical rotation polarization imaging, and realizing real-time acquisition of polarization information in the mid-to-shortwave band; solving the problems of non-uniform noise and blind elements in mid-to-shortwave detectors, and improving the signal-to-noise ratio of weak targets; constructing an integrated parallel processing architecture of "correction-enhancement-fusion" to achieve end-to-end real-time processing on an embedded platform; and improving the detection range and recognition accuracy of UAV targets in complex low-altitude backgrounds through multi-dimensional information fusion.

[0008] To achieve the above objectives, the technical solution adopted in this application is: a medium- and short-wave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection, comprising the following steps: S1. Employs a mid-to-short-wave focal plane detector integrated with a micro-polarizer array to achieve simultaneous imaging in four polarization directions; S2. Perform multi-segment and scene joint non-uniform correction on the four polarization direction image obtained in S1; S3. Perform blind pixel detection and replacement for weak target protection on the image corrected by S2; S4. Super-resolution reconstruction of the image processed by S3 is performed using an attention-enhanced residual network. S5. Based on the super-resolution image in S4, obtain the polarization degree (DoP) and polarization angle (AoP) images through Stokes vector calculation, and construct a multi-feature fusion matrix by combining infrared radiation intensity information; S6. An adaptive weighted fusion strategy is used to integrate polarization features and super-resolution images; S7. Perform an 8-bit dynamic mapping based on the characteristics of human visual perception on the fused image from S6.

[0009] Preferably, S2 includes the following steps: S2.1 Based on the blackbody calibration data, establish an initial correction coefficient table and calculate the pixel gain coefficient G(i,j) and offset coefficient O(i,j); S2.2 calculates the scene motion vectors of adjacent frames using the optical flow method to divide the static and dynamic regions; S2.3 In the static region, a time-domain high-pass filter is used to suppress 1 / f noise, while in the dynamic region, a guided filter is used for spatial correction.

[0010] Preferably, the multi-segment and scene joint non-uniform correction in S2 also includes temperature drift compensation.

[0011] Preferably, the attention-enhanced residual network in S4 includes a feature extraction layer, a cross-scale attention module, and an upsampling layer. The feature extraction layer contains three convolutional blocks, each consisting of a 3×3 convolutional layer, a ReLU activation function, and a batch normalization layer. The cross-scale attention module dynamically weights the feature maps at low, medium, and high scales.

[0012] Preferably, the cross-scale attention module of the attention-enhanced residual network in S4 includes the following steps: S4.1 Downsample the super-resolution feature map at 1 / 2, 1 / 4, and 1 / 8 resolution; S4.2 Perform 3×3 convolution on the feature maps at each scale to reduce the dimensionality to 64 channels; S4.3 generates attention weight maps at various scales using the Sigmoid activation function; S4.4 multiplies the attention weight map with the corresponding feature map, upsamples it to the original resolution, and then performs pixel-level additive fusion.

[0013] Preferably, the adaptive weighted fusion strategy in S6 is as follows: calculate the gradient, contrast and entropy of each feature map, construct a weight function W=0.3×gradient+0.5×contrast+0.2×entropy, and achieve pixel-level fusion through weight normalization.

[0014] Preferably, the 8-bit dynamic mapping in S7 adopts a non-linear mapping function f(V)=255 / (1+exp(-k(V-Vmean) / Vstd)), where k=0.8 is the contrast adjustment coefficient, Vmean is the scene gray mean, and Vstd is the scene gray standard deviation.

[0015] A mid-to-shortwave high-resolution on-chip polarization imaging processing system suitable for low-altitude UAV detection includes: (1) A 2×2 on-chip polarization imaging module, used to perform image acquisition and processing of S1; (2) FPGA preprocessing unit, used to perform S2-S3 correction and blind cell processing; (3) GPU acceleration unit, used to perform S4-S6 super-resolution reconstruction, polarization calculation and image fusion; (4) Image output unit, used to perform 8-bit dynamic mapping of S7 and output the result.

[0016] The beneficial effects provided by this application are as follows: (1) This invention provides a medium- and short-wave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection. It breaks through the volume and weight limitations of traditional mechanical rotation polarization imaging, and invents a 2×2 micro polarizer array integrated focal plane detector to achieve full-frame simultaneous polarization imaging, eliminate image registration errors caused by mechanical motion, utilize the penetration capability of medium- and short-wave infrared under complex weather conditions, and combine the polarization characteristics difference between the target and the background to construct an "intensity + polarization" dual-channel detection model to achieve real-time acquisition of polarization information in the medium- and short-wave bands, enhance all-weather detection capability, and achieve 24-hour uninterrupted detection.

[0017] (2) This invention provides a medium- and short-wave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection. It proposes a scene motion partitioning correction algorithm. The static region adopts two-point correction based on blackbody radiation source, and the dynamic region introduces time domain filtering. It constructs an integrated parallel processing architecture of "correction-enhancement-fusion" and realizes end-to-end real-time processing on an embedded platform. It solves the non-uniform noise and blind element problem of medium- and short-wave detectors and improves the signal-to-noise ratio of weak targets to more than 5dB.

[0018] (3) This invention provides a medium-shortwave high-resolution on-chip polarization imaging processing method suitable for low-altitude UAV detection. It proposes an attention-enhanced residual network super-resolution architecture (A-ResNet), introduces a cross-scale attention module (CSAM), dynamically weights feature maps at different levels, constructs an enhanced processing chain for multi-modal information fusion, and innovates a polarization-intensity-super-resolution multi-feature fusion decision model to improve the detection range and recognition accuracy of UAV targets in complex low-altitude backgrounds through multi-dimensional information fusion. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the mid-to-shortwave high-resolution on-chip polarization imaging processing method for low-altitude UAV detection provided in this embodiment of the invention. Figure 2 This is an on-chip polarization imaging diagram in an embodiment of the present invention; Figure 3 This is an intensity imaging diagram from an embodiment of the present invention; Figure 4 This is the fused image after undergoing a multi-feature fusion algorithm in this embodiment of the invention; Detailed Implementation To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0021] like Figure 1-4 As shown, a method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection is provided, including the following steps: S1. Employs a mid-to-short-wave focal plane detector integrated with a micro-polarizer array to achieve simultaneous imaging in four polarization directions; S2. Perform multi-segment and scene joint non-uniform correction on the four polarization direction image obtained in S1; S3. Perform blind pixel detection and replacement for weak target protection on the image corrected by S2; S4. Super-resolution reconstruction of the image processed by S3 is performed using an attention-enhanced residual network. S5. Based on the super-resolution image in S4, the polarization degree and polarization angle images are obtained through Stokes vector calculation, and a multi-feature fusion matrix is ​​constructed by combining the infrared radiation intensity information; S6. An adaptive weighted fusion strategy is used to integrate polarization features and super-resolution images; S7. Perform an 8-bit dynamic mapping based on the characteristics of human visual perception on the fused image from S6.

[0022] In some embodiments, S1 includes a mid-to-short-wave focal plane detector integrated with a 2×2 micro-polarizer array to collect scene radiation and simultaneously acquire images in four polarization directions: 0°, 45°, 90°, and 135°. The substrate material of the 2×2 micro-polarizer array is a germanium substrate, the polarization film layer adopts a metal grating structure with a period of 1.2 μm, a linewidth of 0.6 μm, a polarization extinction ratio >100:1, a pixel size of 15 μm × 15 μm, a fill factor ≥92%, a detector resolution of 1280 × 1024, a spectral response range of 3.7-4.8 μm, and a frame rate ≥60 fps.

[0023] On a 1280×1024 resolution focal plane, each 2×2 pixel unit integrates 0° / 45° / 90° / 135° micro-polarizing filters, achieving full-frame simultaneous polarization imaging and eliminating image registration errors caused by mechanical motion. Breaking through the size and weight limitations of traditional mechanically rotating polarization imaging, a 2×2 micro-polarizer array integrated focal plane detector was invented, enabling real-time acquisition of polarization information in the mid- and short-wave bands. The penetration capability of the mid- and short-wave bands (3.7-4.8μm) under complex weather conditions such as fog and smoke is 5-8 times higher than visible light and 30% higher than traditional long-wave infrared (8-14μm), enhancing all-weather detection capability and enabling 24-hour uninterrupted detection with a day-night detection distance ratio of 1:0.95.

[0024] In some embodiments, S2 specifically includes the following steps: S2.1 Based on the blackbody calibration data, establish an initial correction coefficient table and calculate the pixel gain coefficient G(i,j) and offset coefficient O(i,j); S2.2 calculates the scene motion vectors of adjacent frames using the optical flow method to divide the static and dynamic regions; S2.3 In the static region, a time-domain high-pass filter is used to suppress 1 / f noise, while in the dynamic region, a 5×5 guided filter is used for spatial correction. The guided image is the intensity image after static correction, with a filter radius of 5 pixels and a regularization parameter of 0.1.

[0025] Through calculations based on the blackbody radiation law and atmospheric transmittance simulation, the optimal detection window was determined to be 1.7-4.8 μm, resulting in a 30% increase in detection distance compared to traditional 8-14 μm long-wavelength systems. Furthermore, the multi-segment and scene-based joint non-uniform correction in S2 also includes temperature drift compensation. By dynamically adjusting the correction coefficient table based on real-time detector operating temperature, the temperature compensation accuracy is ≤ ±0.1℃.

[0026] In some embodiments, S3 specifically includes the following steps: S3.1 uses a three-threshold method to identify blind cells: |I(i,j)-neighborhood mean|>3×global standard deviation, relative deviation>25% or neighborhood correlation coefficient<0.6; S3.2 Background region blind pixels: 5×5 neighborhood median filtering is used for replacement, and multi-frame temporal mean value replacement is used for blind pixels in weak target regions, with a replacement error ≤1.2%.

[0027] This invention constructs an enhanced processing chain for multimodal information fusion: it proposes a scene motion partitioning correction algorithm, which uses two-point correction based on a blackbody radiation source for static regions and introduces temporal filtering for dynamic regions, improving the signal-to-noise ratio by 8dB compared to the traditional global correction algorithm; it designs a three-level processing flow of "detection-classification-replacement", which uses multi-frame temporal replacement for weak target regions and spatial interpolation for background regions, achieving a blind pixel repair accuracy of 99.8%, which is 12% higher than the industry standard (IRIG 106).

[0028] In some embodiments, S4 uses an attention-enhanced residual network to perform super-resolution reconstruction on the image processed by S3, increasing the resolution by 2 times, with a processing latency of ≤30ms and a peak signal-to-noise ratio of ≥38dB. The attention-enhanced residual network in S4 includes a feature extraction layer, a cross-scale attention module, and an upsampling layer. The feature extraction layer contains three convolutional blocks, each consisting of a 3×3 convolutional layer, a ReLU activation function, and a batch normalization layer. The cross-scale attention module dynamically weights the feature maps at low, medium, and high scales.

[0029] The cross-scale attention module of the attention-enhanced residual network in S4 includes the following steps: S4.1 Downsample the super-resolution feature map at 1 / 2, 1 / 4, and 1 / 8 resolution; S4.2 Perform 3×3 convolution on the feature maps at each scale to reduce the dimensionality to 64 channels; S4.3 generates attention weight maps at various scales using the Sigmoid activation function; S4.4 multiplies the attention weight map with the corresponding feature map, upsamples it to the original resolution, and then performs pixel-level additive fusion.

[0030] We propose an attention-enhanced residual network super-resolution architecture (A-ResNet) that introduces a cross-scale attention module (CSAM) to dynamically weight feature maps at different levels. The weights of low-level detail features are 0.7-0.9, and the weights of high-level semantic features are 0.3-0.5, resulting in a 2.3-fold improvement in the feature response of weak target regions.

[0031] In some embodiments, in S5, the Stokes vector (I, Q, U) is calculated based on the super-resolution image from S4, and the polarization degree DoP = √(Q² + U²) / I and the polarization angle AoP = 0.5 × arctan(U / Q) are further obtained. The Stokes vector calculation uses double-precision floating-point arithmetic (64-bit), with the polarization degree DoP retained to 3 decimal places and the polarization angle AoP retained to 1 decimal place, resulting in a calculation accuracy error ≤ 1e-6.

[0032] In some embodiments, the adaptive weighted fusion strategy in S6 is as follows: calculate the gradient, contrast and entropy of each feature map, construct a weight function W=0.3×gradient+0.5×contrast+0.2×entropy, and achieve pixel-level fusion through weight normalization.

[0033] In some embodiments, the 8-bit dynamic mapping in S7 uses a non-linear mapping function f(V)=255 / (1+exp(-k(V-Vmean) / Vstd)), where k=0.8 is the contrast adjustment coefficient, Vmean is the scene grayscale mean, and Vstd is the scene grayscale standard deviation.

[0034] Example 2 A mid-to-shortwave high-resolution on-chip polarization imaging processing system suitable for low-altitude UAV detection includes: (1) A 2×2 on-chip polarization imaging module, used to perform image acquisition and processing in S1; (2) FPGA preprocessing unit, used for S2-S3 correction and blind cell processing; (3) GPU acceleration unit, used to perform S4-S6 super-resolution reconstruction, polarization calculation and image fusion; (4) Image output unit, used to perform 8-bit dynamic mapping of S7 and output the result.

[0035] In some embodiments, the FPGA preprocessing unit uses a Xilinx Zynq UltraScale+ device, the GPU acceleration unit uses an NVIDIA Jetson AGX platform, and the data interaction between the FPGA preprocessing unit and the GPU acceleration unit uses a PCIe 4.0 x4 connection.

[0036] Example 3 A drone target (DJI Phantom 4) was tested against an urban background at a distance of 100m-500m and a flight speed of 0-30m / s. The system utilizing the method of Embodiment 1 of this invention was compared with a traditional long-wave infrared imaging system (8-14μm, 640×512 resolution) and a mechanically rotating polarization imaging system (3-5μm, 1024×768 resolution). The test results are as follows:

[0037] Example 4 Tests were conducted in a foggy, nighttime environment (visibility 1.5km). Compared to traditional long-wave systems, the system of Embodiment 2 of this invention clearly distinguished UAV targets at a distance of 500m, with a target-to-background contrast ratio of 3.2 in the DoP image. Regarding the detection of small targets (10×10 pixel targets): before blind pixel repair, the target area contained 3 blind pixels with a signal-to-noise ratio of 3.1dB; after blind pixel repair, the blind pixel recognition accuracy reached 100%, and the signal-to-noise ratio improved to 5.7dB.

[0038] Example 5 Under the conditions of a DJI Phantom 4 at a flight altitude of 100m and a speed of 15m / s, the system of Embodiment 2 of this invention was tested against a conventional system under complex weather conditions. The test results are as follows:

[0039] Example 6 Tests were conducted at a distance of 500m, in clear weather, with a background of urban buildings. The results for different target types are as follows:

[0040] Example 7 The system of Embodiment 2 of the present invention underwent interference testing under clear weather conditions with a target of DJI Phantom 4. The test results are as follows:

[0041] Example 8 A test was conducted against a very small target (micro-drone) at a distance of 800m in an urban setting. The results are as follows:

[0042] Example 9 The power consumption and environmental adaptability of the system in Embodiment 2 of this invention under full load operation were tested, and the test results are as follows:

[0043] In summary, this invention constructs an imaging system based on a 2×2 on-chip focal plane array (resolution 1280×1024). Through key technologies such as multi-segment and scene joint non-uniform correction, blind pixel detection and replacement for weak targets, image super-resolution reconstruction, polarization image processing, 8-bit image dynamic mapping, and multimodal image fusion, it achieves high-sensitivity and high-resolution detection and identification of low-altitude UAV targets.

[0044] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection, characterized in that, Includes the following steps: S1. Employs a mid-to-short-wave focal plane detector integrated with a micro-polarizer array to achieve simultaneous imaging in four polarization directions; S2. Perform multi-segment and scene joint non-uniform correction on the four polarization direction image obtained in S1; S3. Perform blind pixel detection and replacement for weak target protection on the image corrected by S2; S4. Super-resolution reconstruction of the image processed by S3 is performed using an attention-enhanced residual network. S5. Based on the super-resolution image in S4, the polarization degree and polarization angle images are obtained through Stokes vector calculation, and a multi-feature fusion matrix is ​​constructed by combining the infrared radiation intensity information; S6. An adaptive weighted fusion strategy is used to integrate polarization features with super-resolution images; S7. Perform an 8-bit dynamic mapping based on the characteristics of human visual perception on the fused image from S6.

2. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 1, characterized in that, S2 includes the following steps: S2.1 Based on the blackbody calibration data, establish an initial correction coefficient table and calculate the pixel gain coefficient G(i,j) and offset coefficient O(i,j); S2.2 calculates the scene motion vectors of adjacent frames using the optical flow method to divide the static and dynamic regions; S2.3 In the static region, a time-domain high-pass filter is used to suppress 1 / f noise, while in the dynamic region, a guided filter is used for spatial correction.

3. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 1, characterized in that, The multi-segment and scene joint non-uniform correction described in S2 also includes temperature drift compensation.

4. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 1, characterized in that, The attention-enhanced residual network described in S4 includes a feature extraction layer, a cross-scale attention module, and an upsampling layer. The feature extraction layer contains three convolutional blocks, each consisting of a 3×3 convolutional layer, a ReLU activation function, and a batch normalization layer. The cross-scale attention module dynamically weights the feature maps at low, medium, and high scales.

5. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 4, characterized in that, The cross-scale attention module of the attention-enhanced residual network described in S4 includes the following steps: S4.1 performs downsampling of the super-resolution feature map at 1 / 2, 1 / 4, and 1 / 8 resolution; S4.2 performs 3×3 convolutions on feature maps at each scale to reduce the dimensionality to 64 channels; S4.3 generates attention weight maps at various scales using the Sigmoid activation function; S4.4 multiplies the attention weight map with the corresponding feature map, upsamples it to the original resolution, and then performs pixel-level additive fusion.

6. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 1, characterized in that, The adaptive weighted fusion strategy in S6 is as follows: calculate the gradient, contrast and entropy of each feature map, construct a weight function W=0.3×gradient+0.5×contrast+0.2×entropy, and achieve pixel-level fusion through weight normalization.

7. The method for processing medium- and short-wave high-resolution on-chip polarization imaging suitable for low-altitude UAV detection according to claim 1, characterized in that, The 8-bit dynamic mapping in S7 uses a non-linear mapping function f(V)=255 / (1+exp(-k(V-Vmean) / Vstd)), where k=0.8 is the contrast adjustment coefficient, Vmean is the scene gray mean, and Vstd is the scene gray standard deviation.

8. A mid-to-shortwave high-resolution on-chip polarization imaging processing system suitable for low-altitude UAV detection, utilizing the method described in claims 1-7, characterized in that, include: (1) A 2×2 on-chip polarization imaging module, used to perform image acquisition and processing of S1; (2) FPGA preprocessing unit, used to perform S2-S3 correction and blind cell processing; (3) GPU acceleration unit, used to perform S4-S6 super-resolution reconstruction, polarization calculation and image fusion; (4) Image output unit, used to perform 8-bit dynamic mapping of S7 and output the result.

Images