Target Detection Method Based on Snapshot Spectro-Polarimetric Camera

Abstract

The present invention relates to a target detection method based on a snapshot spectral polarization camera, which is characterized by comprising the following steps: Step S1: Using a snapshot spectral camera and a snapshot polarization camera, obtain the spectral and polarization images of the detection target; Step S2: Screen the characteristic bands for the spectral characteristics of the detection target, perform PCA transformation on the spectral images of the characteristic bands to obtain the spectral preprocessing result SP 0 Figure, and at the same time calculate the total light intensity image of polarization S 0 and new polarization parameters I s Figure, and combine S 0 with I s Figure to perform PCA transformation to obtain the polarization preprocessing result Po 0 Figure; Step S3: Perform registration and fusion on SP 0 Figure and Po 0 Figure to obtain the fused image F ; Step S4: According to the target feature information, perform target detection on F Figure. Through the complementarity of spectral and polarization information, the present invention fuses spectral and polarization information, thereby improving the detection performance during multi-target detection.

Description

Technical Field

[0001] The present invention belongs to the fields of image fusion technology and computer vision, and particularly relates to a target detection method based on a snapshot spectral polarization camera. Background Art

[0002] The target camouflage and recognition technologies promote and develop each other in continuous confrontation. The use of advanced camouflage coatings and camouflage nets enables the target to better blend into the surrounding environment, resulting in errors in traditional optical target detection methods and making it difficult to accurately distinguish the target. Light intensity, spectral, and polarization information reveal different characteristics of objects and backgrounds, and it has been proven that good detection results can be obtained by using spatial and spectral information, or spatial and polarization information. However, spectral imaging is affected by the spectral range, spectral resolution, and band extraction algorithm, and there are certain errors in the detection of artificial targets with spectral curves similar to the background. Although polarization imaging has obvious advantages in detecting and recognizing artificial targets, its results are greatly affected by the light angle. Therefore, using only spectral or polarization information to detect multiple camouflaged targets made of different materials may result in missed detections. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a target detection method based on a snapshot spectral polarization camera, which fuses spectral and polarization information through the complementarity of spectral and polarization information, thereby improving the detection performance during multi-target detection.

[0004] To achieve the above purpose, the present invention adopts the following technical solutions:

[0005] A target detection method based on a snapshot spectral polarization camera, characterized by including the following steps:

[0006] Step S1: Use a snapshot spectral camera and a snapshot polarization camera to obtain the spectral and polarization images of the detection target;

[0007] Step S2: Screen the characteristic bands of the spectral characteristics of the detection target, perform PCA transformation on the spectral map of the characteristic bands to obtain the spectral preprocessing result SP0 map, and at the same time calculate the total light intensity image S0 of polarization and the new polarization parameter I s graph, and perform PCA transformation on the S0 and Is graphs to obtain the polarization preprocessing result Po0 graph;

[0008] Step S3: Register and fuse the SP0 graph and the Po0 graph to obtain the fused image F;

[0009] Step S4: Perform target detection on the F graph according to the target feature information.

[0010] Further, the specific content of step S2 is:

[0011] 1) Filter the spectral source image according to the spectral characteristics of the target to obtain a waveband subset;

[0012] 2) Process the waveband subset using PCA transformation to obtain the SP0 map;

[0013] 3) Calculate the polarized S0 and Is maps:

[0014] S0 = I0 + I 90

[0015]

[0016] where, I0, I 90 represent the light intensity values in two polarization directions of 0° and 90°, S1 is the linearly polarized light component between the horizontal and vertical directions; S2 is the linearly polarized light component in the direction of 45° to 135°.

[0017] 4) Process the polarized S0 and Is maps using PCA transformation to obtain the Po0 map.

[0018] Further, the specific steps of step S3 are as follows:

[0019] 1) Register the SP0 map and the Po0 map;

[0020] 2) Use weighted least squares filtering to separate the base layer from the source image:

[0021] B n = F λ (I n )

[0022] where, I n is the nth source image, F λ is the weighted least squares filter, and B n is the base layer image;

[0023] 3) Calculate the detail layer image D n :

[0024] D n = I n - B n

[0025] 4) Filter the source image I using the image filter L and n to obtain the saliency map S n , and compare the saliency maps to determine the initial weight map P n :

[0026]

[0027]

[0028] 5) Using the corresponding source image I n as the guidance image to perform guided filtering on each weight map P n to obtain the weight map:

[0029]

[0030]

[0031] where r1, ε1, r2, and ε2 are the parameters of the guided filter, and W B n and W D n are the weight maps of the base layer and the detail layer;

[0032] 6) Based on the weight image, perform weighted fusion on the base layer and the detail layer images of the decomposed spectral and polarization images, and superimpose the fused base layer and detail layer images to obtain the final fused image F:

[0033]

[0034] where α n is the weighting coefficient, B n , D n are the base layer and the detail layer images of the spectral and polarization images, and F is the fused image.

[0035] Furthermore, in step S4, according to the target features, the fused image F is subjected to target detection by using methods such as template matching, RCNN, YOLO, or SSD.

[0036] The present invention has the following beneficial effects compared with the prior art:

[0037] The present invention uses a snapshot spectral and polarization camera to acquire images, improving the real-time performance of image acquisition, achieving synchronous acquisition between different cameras, and at the same time, taking advantage of the portability of the camera to expand the application scenarios of the algorithm; using an optimal clustering framework to perform band selection on the multispectral image, reducing data redundancy and having high generality; introducing the I S polarization parameter to improve the target contrast of the polarization PCA transformed image, thereby improving the quality of subsequent image fusion; using weighted least squares filtering to optimize the fusion method based on guided filtering, so as to improve the detail expression ability of the fused image, preserve more target surface detail information, increase the similarity between the target and the template, and further improve the detection accuracy during target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is the flowchart of the method of the present invention;

[0039] Figure 2 is the spectral image registration result in an embodiment of the present invention;

[0040] Figure 3 is the polarization image registration result in an embodiment of the present invention;

[0041] Figure 4 is the base layer image of the spectral image in an embodiment of the present invention;

[0042] Figure 5 is the detail layer image of the spectral image in an embodiment of the present invention;

[0043] Figure 6 is the base layer image of the polarization image in an embodiment of the present invention;

[0044] Figure 7 is the detail layer image of the polarization image in an embodiment of the present invention;

[0045] Figure 8 is the fused image in an embodiment of the present invention;

[0046] Figure 9 is the target detection image in an embodiment of the present invention. Detailed implementation manners

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0048] Please refer to Figure 1 , the present invention provides a snapshot spectral polarization multimodal target detection method, including the following steps:

[0049] Step S1: Synchronously collect and obtain a multi-spectral and polarization image matrix by using a snapshot spectral camera and a snapshot polarization camera.

[0050] Step S2: Screen the characteristic bands according to the spectral characteristics of the detection target, perform PCA transformation on the spectral maps of the characteristic bands to obtain the spectral preprocessing result SP0 map, and at the same time calculate the total light intensity image S0 of polarization and the new polarization parameter I s Figure, perform PCA transformation on the S0 and I s Figure to obtain the polarization preprocessing result Po0 map;

[0051] Step S3: Register and fuse the SP0 map and the Po0 map to obtain the fused image F;

[0052] Step S4: Perform target detection on the F map according to the target characteristics.

[0053] The following are specific implementation examples of the present invention.

[0054] In this embodiment, the spectral camera selects a snapshot multispectral camera (MQ022HG-IM-SM5X5-NIR, XIMEA) that can obtain spectral images of 25 bands in real time within the range of 660 - 975 nm, and the polarization camera selects a polarization filter array camera (BLACKFLYS BFS-U3-51S5PC, FLIR, CANADA) that can obtain images at four angles of polarization 0°, 45°, 90°, and 135° in real time. The two are placed in parallel; both use lenses with a focal length of 35 mm (VIS-NIR, #67-716, EDMUND). The camouflage targets are a spectral reconnaissance camouflage net with a small mosaic pattern that is difficult to detect with a single polarization information and an aluminum alloy plate (30 cm × 30 cm) coated with a grassland camouflage pattern that is difficult to detect with a single spectral information.

[0055] In this embodiment, the pixel filter detector array type fast camera used in step S1 integrates a single-chip filter array onto a standard CMOS detector, greatly reducing system stray light, improving sensitivity and imaging speed. At the same time, the wafer-level design of each pixel filter enables compact snapshot spectral and polarization acquisition, greatly increasing the portability and flexibility of the camera, and is particularly suitable for application fields with limited size and quality, such as small unmanned aerial vehicle camouflage target reconnaissance and identification applications. At the same time, the combined use of a snapshot spectral camera and a snapshot polarization camera can make up for the defect of the pixel filter detector array type spectral fast camera in spatial resolution and reduce the influence of light on polarization characteristics.

[0056] In this embodiment, the optimal clustering framework used in step S2 can give play to the advantages of clustering and sorting methods, obtain a waveband subset with lower correlation and more distinguishable information, thereby reducing the waveband redundancy while maintaining a high amount of information.

[0057] Preferably, the polarization parameter I S is a new polarization parameter proposed based on the Stokes vector S = (S0, S1, S2, S3) T :

[0058]

[0059] where S1 is the linearly polarized light component between the horizontal and vertical directions; S2 is the linearly polarized light component in the direction of 45° - 135°.

[0060] It not only has better performance than the polarization degree and linear polarization angle parameter images in terms of target / background contrast, but also has much lower noise than the two. S0 is the total light intensity image with rich detail information. Extracting the main features of the two and participating in spectral polarization fusion can significantly improve the quality of the fused image and the target detection performance.

[0061] In this embodiment, in step S3, a spectral image and a polarization image are fused using an image fusion algorithm. As Figure 4 and Figure 5 shown, in the base layer and detail layer images of the spectrum extracted by this algorithm, the characteristics of the camouflage net are more obvious, but the features of the camouflage board are less; as Figure 6 and Figure 7 shown, in the base layer and detail layer images of the polarization extracted by this algorithm, the characteristics of the camouflage board are more obvious, but the features of the camouflage net are less.

[0062] To integrate the advantages of the spectral image and the polarization image and highlight the two types of camouflage targets, an initial weight mapping P n is calculated for all the base layer and detail layer images in step S3. Based on all the base layer and detail layer images, the initial weight mapping P n is calculated as follows:

[0063]

[0064]

[0065] However, the obtained weight mapping is usually noisy and not aligned with the object boundaries, which may cause artifacts in the fused image. Therefore, a guided filter is applied to each weight mapping P n , and the corresponding source image I n is used as the guidance image

[0066]

[0067]

[0068] where r1, ε1, r2, and ε2 are the parameters of the guided filter, and W B n and W D n are the weight maps of the base layer and the detail layer. Moreover, the regularization parameter ε and the window size r have a greater impact on the final filtering result. The larger the window, the more obvious the smoothing effect; the smaller the window, the more details are retained; the larger the regularization parameter, the stronger the regularization ability, but the impact on the filtering effect is limited.

[0069] Then, the base layer and detail layer images of the spectral image and the polarization image are weighted and fused using the weight maps, and superimposed to obtain the final fusion result:

[0070]

[0071] where α n is the weighting coefficient, B n , D n are the base layer and detail layer images of the spectral and polarization images, and F is the fused image. The result is as shown in Figure 8as shown

[0072] Finally, according to the target features, methods such as template matching, RCNN, YOLO, and SSD are used to perform target detection on the fused image F, and the results are as Figure 9 shown. The fused image successfully highlights the details and features of the camouflage net and the camouflage board, and both camouflage targets are detected, proving that the fusion algorithm of the present invention realizes the complementarity of spectral and polarization information and improves the camouflage target detection performance.

[0073] The above are only the preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope covered by the present invention.

Claims

1. A target detection method based on a snapshot spectral polarization camera, characterized in that It includes the following steps: Step S1: Use a snapshot spectral camera and a snapshot polarization camera to obtain the spectral and polarization images of the detection target; Step S2: Screen the characteristic bands of the spectral characteristics of the detection target, perform PCA transformation on the spectral maps of the characteristic bands to obtain the spectral preprocessing result SP0 map, and at the same time calculate the total light intensity image S0 of polarization and the new polarization parameter I s Figure, and perform PCA transformation on S0 and I s Figure to obtain the polarization preprocessing result Po0 map; Step S3: Register and fuse the SP0 image and the Po0 image to obtain the fused image F; Step S4: Perform target detection on the F image according to the target feature information; The specific content of step S2 is as follows: 1) Screen the spectral source image according to the spectral characteristics of the target to obtain a waveband subset; 2) Process the waveband subset using PCA transformation to obtain the SP0 image; 3) Calculate S0 and I of polarization s Figure: S0 = I0 + I 90 wherein, I0 and I 90 represent the light intensity values in two polarization directions of 0° and 90°, S1 is the linearly polarized light component between the horizontal direction and the vertical direction; S2 is the linearly polarized light component in the direction of 45° to 135°; 4) Use PCA transformation to process the polarization S0 and I s to obtain the Po0 diagram; The specific content of step S3 is as follows: 1) Register the SP0 image and the Po0 image; 2) Use weighted least squares filtering to separate the base layer from the source image: B n = F λ (I n ) Among them, I n is the nth source image, F λ is the weighted least squares filter, and B n is the base layer image; 3) Calculate the detail layer image D n : D n = I n - B n 4) Use the image filter L and filter the source image I n to obtain the saliency map S n , and compare the saliency maps to determine the initial weight map P n : 5) Using the corresponding source image I n as the guiding image to perform guided filtering on each weight map P n to obtain the weight map: where r1, ε1, r2, and ε2 are the parameters of the guiding filter, and W B n and W D n are the weight maps of the base layer and the detail layer; 6) Based on the weight image, perform weighted fusion on the base layer and detail layer images of the decomposed spectral and polarization images, and superimpose the fused base layer and detail layer images to obtain the final fused image F: where α n is a weighting coefficient, B n , D n are the base layer and detail layer images of the spectral and polarization images, and F is the fused image.

2. The object detection method based on a snapshot spectral polarization camera according to claim 1, wherein In step S4, according to the target features, use template matching, RCNN, YOLO or SSD methods to perform target detection on the fused image F.

Images