Polarization visual detection method based on feature point and signal-to-noise ratio dual enhancement

Abstract

The invention provides a polarization visual detection method based on feature point and signal-to-noise ratio dual enhancement, and belongs to the technical field of polarization visual detection, and the method comprises the steps: obtaining two types of image information: a polarization image and a visible light image through a polarization camera; constructing a polarization image signal-to-noise ratio enhancement network, and separating noise from an original scene by learning correlation among multi-angle polarization information to realize signal-to-noise ratio enhancement; a polarization image feature point enhancement network is constructed, the number and quality of detected feature points are improved by learning the feature fusion relation between the total light intensity image and the polarization degree image, and feature point enhancement is achieved; and finally, the outputs of the two networks intersect with each other and are used as inputs to form a double-loop closed-loop learning network. According to the method, a polarization image signal-to-noise ratio enhancement and feature point enhancement network is constructed, a signal-to-noise ratio and feature point loopback enabling integrated enhancement method is provided, and the precision and environmental adaptability of a polarization visual detection system in an interference environment can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of polarization vision detection, and in particular relates to a polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio. Background Art

[0002] Navigation and detection systems provide spatial motion information and target feature information, and are core components of unmanned systems. Research has found that organisms in nature possess remarkable navigation and detection capabilities, utilizing polarization vision to perceive environmental and target information. This system offers advantages such as strong autonomy, high target recognition, and rich image information, providing a technical means for autonomous navigation and detection in complex interference environments. A common technology in polarization vision and detection is the processing and information extraction of polarization images. Therefore, improving the accuracy of information extraction from polarization images is a key challenge for polarization vision navigation and detection.

[0003] In recent years, many research institutions have conducted extensive research on polarization image processing and information extraction methods. For example, existing technologies have proposed the use of a convolutional feature enhancement module, which enhances the image feature expression capability through a combination of central differential convolution and activation followed by pooling, effectively capturing local details and edge information. In the invention application "Method and System for Adaptive Dynamic Range Enhancement of Infrared Images Based on Signal-to-Noise Ratio Perception" (Application No.: CN202410359705.6), an encoder is used to extract features from the original image and calculate the signal-to-noise ratio distribution map of the original infrared image. The feature map is divided into high-signal-to-noise ratio regions and low-signal-to-noise ratio regions and then fused, which can enhance the image signal-to-noise ratio and restore more image detail information. Prior art also proposes designing texture enhancement preprocessing methods that analyze the statistical information of image texture features to improve the representation of texture information. The ORB algorithm is then applied to texture and color-enhanced images to extract feature points, resulting in more prominent and diverse texture feature points. In the invention application "A Visual SLAM Method and Apparatus Based on Feature Enhancement Network" (Application No. CN202311404737.5), the geometric information and descriptors of the extracted feature points are used as network inputs to generate enhanced feature descriptors through the feature enhancement network. These enhanced feature descriptors are then used for feature point matching, improving feature point detectability. In the invention application "A Method for Integrated Navigation of Unmanned Aerial Vehicles Using Compound Eye Polarization Vision in Low-Light Environments" (Application No. CN202310474059.3), addressing the issue of unclear feature points in low-light scene images, the method enhances image features by performing mean filtering and opening operations on polarization degree and polarization angle images, followed by weighted fusion. This method improves the number of detected feature points in low-light scenes.

[0004] The above existing technologies only focus on enhancing either feature points or signal-to-noise ratio, failing to fully combine the advantages of both. They also fail to effectively utilize the image's polarization information, limiting the accuracy of feature point and signal-to-noise ratio extraction in complex environments such as fog and uneven lighting. Summary of the Invention

[0005] In order to solve the above problems and overcome the shortcomings of the existing technology, the present invention proposes a polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio, using a polarization camera to obtain two types of image information: polarization image and visible light image; constructing a polarization image signal-to-noise ratio enhancement network, by learning the correlation between multi-angle polarization information, separating the noise from the original scene, and achieving signal-to-noise ratio enhancement; constructing a polarization image feature point enhancement network, by learning the feature fusion relationship between the total light intensity image and the polarization degree image, improving the number and quality of detected feature points, and achieving feature point enhancement; finally, the outputs of the two networks are crossed with each other as input to form a dual-loop closed-loop learning network, realizing integrated loop empowerment of signal-to-noise ratio and feature points.

[0006] The technical solution of the present invention is: a polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio, which is implemented in the following steps:

[0007] Step (1) Use polarization camera to obtain two types of image information: polarization image and visible light image; polarization image includes polarization degree images and polarized light intensity images; visible light images include total light intensity images;

[0008] Step (2), constructing a polarization image signal-to-noise ratio enhancement network; the signal-to-noise ratio enhancement network inputs a polarization light intensity image, uses a clear image as a true value constraint, uses an image sequence containing polarization information to learn the correlation between multi-angle polarization information, reconstructs the polarization light intensity image, and separates the background noise containing polarization characteristics of the scattered light from the original scene;

[0009] Step (3), construct a polarization image feature point enhancement network; the feature point enhancement network input is the total light intensity image and polarization degree Image, the network output is doubly constrained by the feature point loss function and the multi-scale image structure similarity loss function, where the feature point loss function is used to fuse the original scene information and polarization degree in the total light intensity image The edge feature information of the image and the multi-scale image structure similarity loss function are used to improve the detection quantity and quality of feature points, thereby learning the total light intensity image and polarization degree. Feature fusion relationship between images;

[0010] Step (4): The outputs of the signal-to-noise ratio enhancement network and the feature point enhancement network are cross-inputted into the two networks to form a signal-to-noise ratio and feature point dual-loop closed-loop learning network to generate a feature point and signal-to-noise ratio dual-enhanced image.

[0011] The advantages of the present invention compared with the prior art are:

[0012] This paper proposes, for the first time, a polarization-based visual detection method based on dual enhancement of feature points and signal-to-noise ratio. Based on polarization image information, a polarization image signal-to-noise ratio enhancement and feature point enhancement network are constructed. Furthermore, combining the input and output information of these two networks, an integrated enhancement method using polarization signal-to-noise ratio and feature point loopback is proposed. This method improves the detection accuracy of motion and target information in interfering environments such as fog and uneven lighting, providing technical support for high-precision polarization-based visual detection and navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of a polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to the present invention;

[0014] Figure 2 is the original image after SURF feature point detection;

[0015] Figure 3 It is a double enhanced image after SURF feature point detection. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The present invention proposes a polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio, such as Figure 1 As shown, the specific implementation steps of the present invention are as follows:

[0018] Step 1: Use a polarization camera and calculate the Stokes vector to obtain two types of image information: polarization image and visible light image; the polarization image includes polarization degree image and polarized light intensity images ; Visible light images include total light intensity images ;

[0019] The Stokes vector consists of four components and can be expressed as:

[0020] ,

[0021] The four components of the Stokes vector can be determined by the relationship between the polarized light intensity and the following formula, thereby describing the polarization state of light:

[0022] ,

[0023] in, 、 、 and Represents 4 polarized light intensity images along different directions respectively; represents the right-handed circularly polarized light intensity image, Represents the left-handed circularly polarized light intensity image; Indicates the total light intensity; Indicates the difference in light intensity between horizontal and vertical polarization directions; Indicates the difference in light intensity between 45° and 135° polarization directions; Represents the circular polarization component of light, which can be considered as 0 for linearly polarized light.

[0024] The polarization degree of the scene can be calculated based on the Stokes vector :

[0025] ,

[0026] Step 2: Construct a polarization image signal-to-noise ratio enhancement network. This network takes a polarization intensity image as input, uses a high-SNR, clear image as a ground truth constraint, and uses image sequences containing polarization information to learn the correlation between multi-angle polarization information. It then reconstructs the polarization intensity image and separates background noise containing polarization characteristics, such as scattered light, from the original scene.

[0027] The main body of the polarization image signal-to-noise ratio enhancement network is an encoder-decoder structure with an embedded signal-to-noise ratio enhancement module. The encoder is responsible for extracting the multi-level features of the input image, and the decoder restores the details by gradually reconstructing the image. The signal-to-noise ratio enhancement module first divides the input feature map into high-frequency and low-frequency parts through multi-scale convolution, and extracts the spatial polarization characteristics of the image at different scales; then, through the channel attention mechanism and the spatial attention mechanism, the importance of different channels and spatial distributions of the image is weighted, and then the multi-scale convolution layer is used to learn the correlation between multi-angle polarization information, separate the background noise containing polarization characteristics from the original scene, improve the feature expression of the signal part, and obtain an image with enhanced signal-to-noise ratio. The channel attention mechanism can be expressed as:

[0028] ,

[0029] in, and Represent the input and output of the channel attention mechanism respectively, and Represents the height and width of the image respectively, represents the Sigmoid activation function, represents the Hadamard product, represents a mapping operation consisting of two fully connected layers in the channel attention mechanism network, represent No. Rank Liedi The spatial attention mechanism can be expressed as:

[0030] ,

[0031] in, and Represent the input and output of the spatial attention mechanism, represents the Sigmoid activation function, represents the Hadamard product, represents the convolution operation, Represents the max pooling operation.

[0032] The polarization image SNR enhancement network uses supervised learning. The network compares the enhanced image with the labeled high SNR ground truth image to ensure the SNR enhancement effect. The loss function for constructing the polarization image SNR enhancement network is as follows:

[0033] ,

[0034] in, Represents the loss function of the polarization image SNR enhancement network, represents the input polarized light intensity image, represents a true image with high signal-to-noise ratio, Represents the signal-to-noise ratio enhanced image output by the polarization image signal-to-noise ratio enhancement network.

[0035] Step 3: Construct a polarization image feature point enhancement network. The network input is the total light intensity image and polarization degree. Image, the network output is doubly constrained by the feature point loss function and the multi-scale image structure similarity loss function, where the feature point loss function is used to fuse the original scene information and polarization degree in the total light intensity image The edge feature information of the image and the multi-scale structural similarity loss are used to further improve the detection quantity and quality of feature points, thereby learning the total light intensity image and polarization degree. The feature fusion relationship between images improves the number and quality of feature points of the network output image.

[0036] The polarization image feature point enhancement network is mainly composed of an encoder-decoder structure, with a feature point enhancement module embedded inside. The encoder is responsible for extracting the multi-level features of the input image, and the decoder restores the details by gradually reconstructing the image. The feature point enhancement module first extracts the spatial features of the image at multiple scales through a series of convolutional layers, capturing the feature points of the image at different scales to ensure that the extraction results are sensitive to both large-scale and local feature points; then, under the constraint of the feature point loss function, the original scene information in the total light intensity image and the key edge contour information in the polarization image are fully integrated through operations such as convolution and self-attention mechanism, thereby enhancing the detectability and robustness of the feature points without destroying the original scene information. At the same time, the multi-scale image structure similarity loss function constraint is added to improve the quality of the feature points detected in the image. The self-attention mechanism can be expressed as:

[0037] ,

[0038] in, and Represents input and output respectively, represents the dimension of the input, represents the normalized exponential function, which can be expressed as:

[0039] ,

[0040] The polarization image feature point enhancement network uses unsupervised learning. To ensure that the enhanced image output by the network can maintain structural consistency with the original image at different resolutions and levels of detail, to ensure that the network can effectively capture the significant features in the polarization image while retaining the original scene information, and to ensure that the enhanced image can detect more effective feature points, the loss function of the polarization image feature point enhancement network is constructed as follows:

[0041] ,

[0042] in, Represents the loss function of the polarization image feature point enhancement network, and is the weighting coefficient, represents the multi-scale weighted structural similarity loss function, which can be expressed as:

[0043] ,

[0044] in, is the weight coefficient, represents the window size, Represents the window Previous Image and The similarity between them can be expressed as:

[0045] ,

[0046] in, Indicates that the image is in the window The inner part, 、 represent the image mean and variance respectively, and is a constant.

[0047] represents the polarization intensity loss function, which can be expressed as:

[0048] ,

[0049] in, and Represents the height and width of the image respectively, Represents the matrix norm.

[0050] Represents the feature point loss function, which can be expressed as:

[0051] ,

[0052] in, Indicates the number of feature points detected for the image using the SURF feature point detection method.

[0053] Step 4: The outputs of the SNR enhancement network and the feature point enhancement network are cross-inputted into the two networks, forming a dual-loop closed-loop learning network for SNR and feature points. This achieves an integrated loop-back empowerment of polarization SNR and feature points. Ultimately, a dual-enhanced image of feature points and SNR is generated, providing high-precision target and motion perception information for polarization detection and navigation.

[0054] Example:

[0055] In this embodiment, a bionic polarization vision / inertial navigation detection system is used as an example. Considering that interference environments such as clouds, dust, and uneven lighting will affect the system's detection accuracy of motion information and target information, it is necessary to design a polarization vision detection method based on dual enhancement of feature points / signal-to-noise ratio to provide technical support for high-precision polarization vision navigation and detection.

[0056] The specific implementation process includes:

[0057] Step 1: Use a polarization camera to obtain two types of image information: polarization image and visible light image. The polarization image includes polarization degree image and polarization intensity image, and the visible light image includes total light intensity image.

[0058] Step 2: Construct a polarization image signal-to-noise ratio enhancement network. The SNR enhancement network takes the polarization intensity image as input, uses the clear image as the ground truth constraint, and uses the image sequence containing polarization information to learn the correlation between multi-angle polarization information. It then reconstructs the polarization intensity image and separates the background noise containing the polarization characteristics of the scattered light from the original scene.

[0059] Step 3: Construct a polarization image feature point enhancement network. The input of the feature point enhancement network is the total light intensity image and the polarization degree image. The network output is doubly constrained by the feature point loss function and the multi-scale image structure similarity loss function. The feature point loss function is used to fuse the original scene information in the total light intensity image and the edge feature information in the polarization degree image. The multi-scale image structure similarity loss function is used to improve the number and quality of feature point detection, thereby learning the feature fusion relationship between the total light intensity image and the polarization degree image.

[0060] Finally, the outputs of the signal-to-noise ratio enhancement network and the feature point enhancement network are cross-inputted into the two networks to form a signal-to-noise ratio and feature point dual-loop closed-loop learning network to generate a feature point and signal-to-noise ratio dual-enhanced image.

[0061] In order to prove the effect of this method on improving system performance under environmental interference, the Cityscapes dataset was selected for simulation verification. According to the designed method, the polarization image and visible light image were calculated, and then the polarization image signal-to-noise ratio enhancement and feature point enhancement network were constructed. The polarization image and visible light image were input into the network, and the output of the network was cross-inputted to the two networks to form a dual-loop closed-loop learning network, realizing the integrated loop empowerment of polarization signal-to-noise ratio and feature points. The original image after SURF feature point detection is shown in the figure below. Figure 2 As shown in the figure, the double enhanced image after SURF feature point detection is as follows Figure 3 shown.

[0062] Image analysis reveals that the original polarization image is subject to cloud and dust interference, significantly impacting the system's accuracy in detecting motion and target information. However, dual feature point / signal-to-noise ratio (SNR) enhancement significantly improves both the SNR and the number of detectable feature points. Image evaluation metrics are then used to calculate the peak signal-to-noise ratio (PNSR) between the original polarization image, the dual-enhanced image, and the ground-truth reference image. The PSNR for the original polarization image is 17.49, while the PSNR for the dual-enhanced image is 31.60, representing a 44.65% improvement. Using the SURF feature point detection method, the number of feature points in the original polarization image and the dual-enhanced image is 145, while the number in the dual-enhanced image is 334, a 56.69% improvement. Thus, the polarization vision detection method based on dual feature point / SNR enhancement achieves dual enhancement of feature points and SNR in polarization images.

[0063] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, and it should be clear that the present invention is not limited to the scope of the specific embodiments, it is obvious to those skilled in the art that as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

Claims

1. A polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio, characterized in that: The implementation steps are as follows: Step (1) Use polarization camera to obtain two types of image information: polarization image and visible light image; polarization image includes polarization degree Images and polarized light intensity images, visible light images include total light intensity images; Step (2), constructing a polarization image signal-to-noise ratio enhancement network; the signal-to-noise ratio enhancement network inputs a polarization light intensity image, uses a clear image as a true value constraint, uses an image sequence containing polarization information to learn the correlation between multi-angle polarization information, reconstructs the polarization light intensity image, and separates the background noise containing polarization characteristics of the scattered light from the original scene; Step (3), construct a polarization image feature point enhancement network; the feature point enhancement network input is the total light intensity image and polarization degree Image, the network output is doubly constrained by the feature point loss function and the multi-scale image structure similarity loss function, where the feature point loss function is used to fuse the original scene information and polarization degree in the total light intensity image The edge feature information of the image and the multi-scale image structure similarity loss function are used to improve the detection quantity and quality of feature points, thereby learning the total light intensity image and polarization degree. Feature fusion relationship between images; Step (4): The outputs of the signal-to-noise ratio enhancement network and the feature point enhancement network are cross-inputted into the two networks to form a signal-to-noise ratio and feature point dual-loop closed-loop learning network to generate a feature point and signal-to-noise ratio dual-enhanced image.

2. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 1, characterized in that: In the step (1), a polarization camera is used to obtain two types of image information based on Stokes vector calculation: a polarization image and a visible light image; the polarization image includes polarization degree image and polarized light intensity images ; Visible light images include total light intensity images ; Stokes vector Contains four components, expressed as: ， The four components of the Stokes vector are determined by the relationship between the polarized light intensity and the following formula, thereby describing the polarization state of light: ， in, 、 、 and Represents 4 polarized light intensity images along different directions; represents the right-handed circularly polarized light intensity image, Represents the left-handed circularly polarized light intensity image; Indicates the total light intensity; Indicates the difference in light intensity between horizontal and vertical polarization directions; Indicates the difference in light intensity between 45° and 135° polarization directions; Represents the circular polarization component of light, which is considered to be 0 for linearly polarized light; The polarization degree of the scene is obtained by solving the Stokes vector : 。 3. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 2, characterized in that: In step (2), the polarization image signal-to-noise ratio enhancement network is an encoder-decoder structure with an embedded signal-to-noise ratio enhancement module. The encoder is used to extract multi-level features of the input image, and the decoder restores details by gradually reconstructing the image. The signal-to-noise ratio enhancement module first divides the input feature map into high-frequency and low-frequency parts through multi-scale convolution, and extracts the spatial polarization features of the image at different scales. Subsequently, the importance of different channels and spatial distributions of the image are weighted through the channel attention mechanism and the spatial attention mechanism. Then, the correlation between multi-angle polarization information is learned through the multi-scale convolutional layer, and the background noise containing polarization characteristics is separated from the original scene to obtain an image with enhanced signal-to-noise ratio.

4. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 3, characterized in that: The channel attention mechanism is expressed as: ， in, and Represent the input image and output image of the channel attention mechanism, and Represents the height and width of the image respectively, represents the Sigmoid activation function, represents the Hadamard product, represents a mapping operation consisting of two fully connected layers in the channel attention mechanism, represent No. Rank Liedi The value of each channel.

5. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 3, characterized in that: The spatial attention mechanism is expressed as: ， in, and Represent the input image and output image of the spatial attention mechanism, represents the Sigmoid activation function, represents the Hadamard product, represents the convolution operation, Represents the max pooling operation.

6. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 3, characterized in that: The polarization image SNR enhancement network uses supervised learning. By comparing the enhanced image with the labeled high SNR ground truth image, the loss function of the polarization image SNR enhancement network is constructed as follows: ， in, Represents the loss function of the polarization image SNR enhancement network, represents the input polarized light intensity image, represents a true image with high signal-to-noise ratio, Represents the signal-to-noise ratio enhanced image output by the polarization image signal-to-noise ratio enhancement network.

7. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 2, characterized in that: In the step (3), the polarization image feature point enhancement network is an encoder-decoder structure with a feature point enhancement module embedded inside; the encoder is used to extract multi-level features of the input image, and the decoder restores details by gradually reconstructing the image. The feature point enhancement module first extracts the spatial features of the image at multiple scales through a series of convolutional layers, and captures the feature points of the image at different scales, so that the extraction result is sensitive to both large-scale and local feature points; then, under the constraint of the feature point loss function, the original scene information in the total light intensity image and the key edge contour information in the polarization image are fully integrated through convolution and self-attention mechanism operations, and the multi-scale image structure similarity loss function constraint is added.

8. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 7, characterized in that: The self-attention mechanism is expressed as: ， in, and Represent the input image and output image of the self-attention mechanism, represents the dimension of the input, represents the normalized exponential function, expressed as: 。 9. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 7, characterized in that: The polarization image feature point enhancement network adopts unsupervised learning. The loss function of the polarization image feature point enhancement network is constructed as follows: ， in, Represents the loss function of the polarization image feature point enhancement network, and is the weighting coefficient, represents the multi-scale weighted structural similarity loss function, represents the polarization intensity loss function, Represents the feature point loss function.

10. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 9, characterized in that: Expressed as: ， in, is the weight coefficient, represents the window size, Represents the window Previous Image and The similarity between them is expressed as: ， in, Indicates that the image is in the window The inner part, 、 represent the image mean and variance respectively, and is a constant.

11. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 9, characterized in that: Expressed as: ， in, and Represents the height and width of the image respectively, Represents the matrix norm.

12. The polarization vision detection method based on dual enhancement of feature points and signal-to-noise ratio according to claim 9, characterized in that: Expressed as: ， in, Indicates the number of feature points detected for the image using the SURF feature point detection method.

Images