Polarization intelligent sensing system and polarization intelligent sensing method
Patent Information
- Application Number
- MYPI2025000417
- Authority / Receiving Office
- MY · MY
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-07-14
- Filing Date
- 2023-06-09
- Publication Date
- 2024-01-14
AI Technical Summary
Existing optical equipment is difficult to achieve real-time information search and processing in harsh marine environments, and lacks environmental adaptability, which affects target recognition and image quality.
Combining polarization imaging technology and artificial intelligence, through the polarization optical imaging module, image processing module, image enhancement module and interpretation module, the neural network is used to process polarization images, obtain polarization degree and phase angle information, and achieve image enhancement and target recognition.
It realizes real-time information processing in harsh environments, obtains high-quality target scene images, and enhances target recognition capabilities. It is suitable for ocean, low illumination and water mist environments.
Smart Images

Figure PI2025000417_DRAWING_001 
Figure PI2025000417_DRAWING_002 
Figure PI2025000417_DRAWING_003
Abstract
Description
Polarization intelligent sensing system and sensing method
[0001] This application claims priority to the Chinese patent application filed with the Patent Office of China on July 14, 2022, with application number 202210825609.7 and invention name “Polarization Intelligent Sensing System and Perception Method”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present invention relates to the field of polarization imaging technology, and in particular to a polarization intelligent sensing system and sensing method. Background Art
[0003] As humanity's exploration of nature deepens, the demand for optical imaging in harsh environments is increasing, requiring optical equipment to possess increasingly strong environmental adaptability. For example, the ocean and its surroundings cover approximately 70% of the Earth's surface, yet humanity has relatively little understanding of them. Currently, countries around the world are conducting research on the development of marine resources. Effectively acquiring information in marine environments is fundamental to further scientific research and practical applications. Furthermore, obtaining clearer illuminated images of target scenes in dim lighting conditions facilitates accurate target recognition by machine vision.
[0004] The scattering characteristics of my country's coastal waters are highly complex, and the environment is complex and ever-changing. The air contains high levels of moisture, man-made targets are rare, target images are limited, and the impact of sea ice and waves is significant. Light intensity imaging technology is generally affected by environmental factors, and varying lighting conditions, such as strong light, weak light, and shadows, can interfere with imaging. The combination of oceanic and weak light conditions places increasingly stringent demands on the environmental adaptability of optical imaging equipment, necessitating greater environmental adaptability for optical observation equipment.
[0005] Polarization is one of the fundamental dimensions of light. Imaging technology that relies on the polarization information of target scenes is unique and relatively environmentally adaptable. Because light of different polarization directions has different penetration abilities through water vapor, polarization information can be used to remove fog. Because polarization interpretation enhances low-contrast data, it is suitable for acquiring long-range image information in harsh environments, offering advantages in suppressing background noise, improving detection range, enhancing detail, and recognizing camouflaged targets. Polarization imaging technology can also suppress solar glare and highlight man-made targets. These advantages lay the foundation for its widespread application in harsh environments, such as ocean scenes, and thus offer improved environmental adaptability.
[0006] At the same time, in order to obtain a better target scene restoration image, this paper proposes a polarization image enhancement model, which adopts the comprehensive calculation and processing method of intensity image, polarization degree image and polarization phase angle image to obtain a fused polarization image with better target highlighting effect, which can be used as one of the basic input data of the neural network.
[0007] Human information processing capabilities are limited, making them incompatible with the demands of processing massive amounts of information. The emergence of artificial intelligence (AI) can effectively address these inherent limitations and enable rapid image interpretation. Currently, optical devices with robust adaptability to harsh environments are not widely available on the market. Ocean remote sensing primarily relies on the high water vapor penetration of the infrared optical spectrum. Unlike these approaches, the present invention combines polarization imaging with AI to enhance the system's environmental adaptability.
[0008] Summary of the Invention
[0009] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a polarization intelligent perception system and perception method. The system combines polarization imaging technology with artificial intelligence technology to carry out information search and processing at a computing speed close to "real time". It is suitable for various application scenarios such as sea situation perception, target search, sea ice detection, etc., and is especially suitable for harsh conditions such as marine environment, low illumination environment, and water fog environment.
[0010] To achieve the above objectives, the present invention adopts the following specific technical solutions:
[0011] The polarization intelligent sensing system provided by the present invention includes:
[0012] A polarization optical imaging module is used to perform polarization imaging on a target scene to obtain a polarization image;
[0013] Polarization image processing module, used to solve the polarization image and obtain the polarization information of the target scene;
[0014] A target scene image enhancement module is used to generate image information of the target scene to be restored based on polarization information;
[0015] The target scene interpretation module is used to apply a neural network to obtain interpretation information of the target scene based on image information, spectral information or intensity information.
[0016] Preferably, the polarization image processing module includes:
[0017] A polarization image preprocessing unit, used for preprocessing the polarization image;
[0018] The polarization information calculation unit is used to calculate the pre-processed polarization image to obtain the polarization degree information and polarization phase angle information of the pixel.
[0019] Preferably, the target scene image enhancement module adopts a multi-dimensional target detection neural network based on the DETR network, and the multi-dimensional target detection neural network includes:
[0020] The preprocessing submodule takes the intensity image, polarization degree, polarization phase angle, restored target scene image, and spectral information as input or a partial combination thereof, and combines the denoising module and the CNN neural network to perform denoising and feature fusion on the polarization image.
[0021] The transformer submodule uses the Transformer model to extract relevant data of the target object;
[0022] The prediction output submodule uses the FNN neural network to process the data output by the converter submodule to obtain the multi-target detection results.
[0023] Preferably, the target scene image enhancement module adopts a polarization image enhancement model: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135}+k2*C2{IMG(D)}+k3*C3{IMG(A)}
[0024] Wherein, IMG(O) represents the image information of the target scene to be restored, C1{I0,I 45 , I 90 , I 135} represents the calculated preferred image of the intensity image 0°, 45°, 90° and 135° polarization images, which is a single polarization angle image or a multi-angle calculated image; IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image containing the edge highlighting information of the target space obtained by using the polarization degree image; IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image containing the surface information of the target object obtained by using the polarization phase angle image; k1, k2 and k3 respectively represent the intensity coefficients of the calculated preferred image of the polarization image, the polarization degree calculated image and the polarization phase angle calculated image.
[0025] Preferably, the polarization optical imaging module includes a matching polarization optical lens and a polarization detector.
[0026] The polarization intelligent sensing method provided by the present invention is implemented using a polarization intelligent sensing system and includes the following steps:
[0027] S1. Perform polarization imaging on the target scene to obtain a polarization image;
[0028] S2. Solve the polarization image to obtain polarization information of the target scene;
[0029] S3, generating image information of the target scene to be restored according to the polarization information;
[0030] S4. Apply a neural network to obtain interpretation information of the target scene based on the image information, spectral information or intensity information.
[0031] Preferably, step S2 specifically includes the following sub-steps:
[0032] S21, preprocessing the polarization image;
[0033] S22, obtaining the Stokes vector of the target scene from the preprocessed polarization image;
[0034] S23, calculating the polarization information of the target scene based on the Stokes vector;
[0035] Polarization information includes polarization degree information and polarization phase angle information;
[0036] The calculation formula of the degree of polarization information DoP is:
[0037] Or the calculation formula of the linear polarization degree information DoLP is:
[0038] Among them, I, Q, U, and V are the Stokes vectors of the target scene respectively;
[0039] The calculation formula of polarization angle information AoLP is:
[0040] Preferably, in step S21, the four adjacent pixels are defined as a super pixel I = [I0, I 45 , I 90 , I 135 ], the detection intensity is defined as I d , the true target strength is I t , the detection intensity is calibrated by polynomial fitting to achieve denoising of the polarization image. The polynomial fitting method is: I t =a0+a1I d +a2I d 2 …a n I d n ;
[0041] Among them, a0…a n are the coefficients of the polynomial.
[0042] Preferably, in step S3, a multi-dimensional target detection neural network based on the DETR network is constructed, and the multi-dimensional target detection neural network includes:
[0043] The preprocessing submodule takes the intensity image, polarization degree, polarization phase angle, restored target scene image, and spectral information as input or a partial combination as input, and combines the denoising module and CNN neural network to perform denoising and feature fusion on the polarization image.
[0044] The transformer submodule uses the Transformer model to extract relevant data of the target object;
[0045] The prediction output submodule uses the FNN neural network to process the data output by the converter submodule to obtain the multi-target detection results.
[0046] Preferably, in step S3, a polarization image enhancement model is constructed: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135}+k2*C2{IMG(D)}+k3*C3{IMG(A)}
[0047] Wherein, IMG(O) represents the image information of the target scene to be restored, C1{I0,I 45 , I 90 , I 135} represents the calculated preferred image of the intensity image 0°, 45°, 90° and 135° polarization images, which is a single polarization angle image or a multi-angle calculated image; IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image containing the edge highlighting information of the target space obtained by using the polarization degree image; IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image containing the surface information of the target object obtained by using the polarization phase angle image; k1, k2 and k3 respectively represent the intensity coefficients of the calculated preferred image of the polarization image, the polarization degree calculated image and the polarization phase angle calculated image.
[0048] The present invention can achieve the following technical effects:
[0049] 1. This invention can be widely applied to various carrier platform environments, has strong environmental adaptability, and can acquire target scene information that conventional optical sensors cannot perceive, such as polarization degree information, polarization phase angle information, image enhancement information, etc. The acquisition of new information dimensions lays a technical foundation for the widespread application of this system, and is suitable for a variety of application scenarios such as marine situation perception, target search, and sea ice detection, especially in harsh environments such as marine environments, low-light environments, and water fog environments.
[0050] 2. This invention combines polarization imaging technology with artificial intelligence technology to conduct information search and processing at a computing speed close to "real time";
[0051] 3. Obtain high-quality target scene image information;
[0052] 4. More accurately interpret the target scene image, classify typical objects in the image, and obtain the location information of the target object. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] FIG1 is a schematic diagram of the logical structure of a polarization intelligent sensing system provided according to an embodiment of the present invention;
[0054] FIG2 is a schematic structural diagram of a Cassegrain catadioptric polarization optical system according to an embodiment of the present invention;
[0055] FIG3 is a schematic diagram showing a comparison of polarization aberrations before and after optimization according to an embodiment of the present invention;
[0056] FIG4 is a schematic flow chart of a polarization intelligent sensing method according to an embodiment of the present invention;
[0057] FIG5 is a schematic diagram of the logical structure of a multi-dimensional target detection neural network based on a DETR network according to an embodiment of the present invention.
[0058] The reference numerals include: a polarization intelligent sensing system 100 , a polarization optical imaging module 101 , a polarization image processing module 102 , a target scene image enhancement module 103 , and a target scene interpretation module 104 . DETAILED DESCRIPTION
[0059] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.
[0060] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.
[0061] FIG1 shows the logical structure of a polarization intelligent sensing system according to an embodiment of the present invention.
[0062] As shown in Figure 1, the polarization intelligent perception system 100 provided by an embodiment of the present invention includes a polarization optical imaging module 101, a polarization image processing module 102, a target scene image enhancement module 103 and a target scene interpretation module 104. The polarization optical imaging module 101 is used to perform polarization imaging of the target scene to obtain a polarization image; the polarization image processing module 102 is used to interpret the polarization image to obtain polarization information of the target scene; the target scene image enhancement module 103 is used to generate image information of the target scene to be restored based on the polarization information; the target scene interpretation module 104 is used to apply a neural network to obtain interpretation information of the target scene based on image information, spectral information or intensity information.
[0063] The polarization optical imaging module 101 is composed of a polarization optical lens and a polarization detector. Polarization optical lenses generally refer to any optical system capable of imaging. Preferably, they must meet the requirements of the polarization detector. For example, the lens's spatial resolution must have an MTF value better than 0.3 at the polarization detector's spatial frequency. They must also be able to suppress polarization aberrations, effectively preserving the polarization information of the incident light without introducing additional aberrations. Polarization optical lenses include, but are not limited to, fixed-focus lenses, zoom lenses, refractive lenses, reflective lenses, and catadioptric lenses. A polarization detector is an image sensor capable of acquiring the polarization characteristics of a target scene. These can be focal plane cameras, amplitude cameras, aperture cameras, time-sharing imaging cameras, and other image sensors. The specific selection must be compatible with the structure of the optical lens.
[0064] Different polarization realization mechanisms can have different polarization optical lens structures. Here, the catadioptric polarization lens is mainly used for explanation. For example, in the Cassegrain catadioptric polarization optical system shown in Figure 2, the transmissive polarization compensation lens group 3 is located between the primary mirror 3 and the secondary mirror 1. The transmissive polarization compensation lens group 3 includes first to fourth lenses 21-24 arranged in sequence from the primary mirror 3 to the secondary mirror 1. The front surface of the first lens 21 (the side facing the primary mirror 3) is convex, the rear surface of the first lens 21 is concave, the front and rear surfaces of the second lens 22 are both concave, the front surface of the third lens 23 is concave, the rear surface of the third lens 23 is convex, and the front and rear surfaces of the second lens 22 are both convex.
[0065] Cassegrain catadioptric polarization optical systems use a combination of reflection and refraction to correct polarization aberration while meeting the system's spatial resolution requirements and adapting to the image plane polarization detector. During the design of such polarization lenses, the polarization aberration distribution at the detector surface should be calculated in detail to derive the initial polarization aberration distribution characteristics of the image.
[0066] Conventional lenses produce strong polarization aberrations, which significantly affect the polarization properties of incident light. Therefore, polarization aberration optimization is necessary for polarization remote sensing lenses. Conventional lenses that have not undergone polarization aberration optimization often have large polarization aberrations, as shown in the left image below in Figure 3. Polarization aberration optimization is achieved by combining transmissive and reflective elements, as shown in the right image in Figure 3.
[0067] The polarization image processing module 102 includes: a polarization image preprocessing unit and a polarization information calculation unit. The polarization image preprocessing unit is used to preprocess the polarization image, such as grayscale value stretching, denoising, sharpening, etc., to better carry out subsequent work; the polarization information calculation unit is used to calculate the preprocessed polarization image to obtain the polarization degree information and polarization phase angle information of the pixel.
[0068] After obtaining the original polarization image, the polarization information needs to be solved according to the requirements, including but not limited to polarization information, polarization phase angle information and intensity information. Taking the focal plane detector as an example, the original polarization image includes I0, I 45 , I 90 , I 135 The intensity information at different angles can be used to calculate the Stokes vector:
[0069] According to the calculated Stokes vector, the polarization degree information and polarization phase angle information of each pixel are solved.
[0070] The image composed of polarization information is called polarization image IMG(D);
[0071] The image formed by the polarization phase angle information is called the polarization phase angle image IMG(A).
[0072] The target scene image enhancement module 103 performs image enhancement and fusion based on user needs. It also builds a neural network model based on the characteristics of the input data, enabling near-real-time processing of the input image, generating information of interest to the next node. For example, after sample training, it can intelligently identify and perceive targets such as marine vessels, sea ice, shallow reefs, islands, and people.
[0073] The target scene image enhancement module 103 adopts the polarization image enhancement model: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135}+k2*C2{IMG(D)}+k3*C3{IMG(A)}
[0074] Wherein, IMG(O) represents the image information of the target scene to be restored, C{I0,I 45 , I 90 , I 135} represents the preferred image after calculation of the intensity image 0°, 45°, 90° and 135° polarization images, and the preferred image is a single polarization angle image or a calculated image of multiple angles, and the calculation process includes but is not limited to the weighted average, filtering, noise reduction and other processes in the conventional algorithm; IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image obtained by using the polarization degree image (containing the edge highlighting information of the target space); IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image obtained by using the polarization phase angle image (containing the surface information of the target object); k1, k2 and k3 respectively represent the intensity coefficients of the three images.
[0075] Alternatively, the target scene image enhancement module 103 adopts a multi-dimensional target detection neural network based on the DETR network, and the multi-dimensional target detection neural network includes a preprocessing submodule, a converter submodule and a prediction output submodule; the preprocessing submodule uses the intensity image, polarization degree, polarization phase angle, restored target scene image, and spectral information as input or a partial combination as input, and combines the noise reduction module and the CNN neural network to perform noise reduction and feature fusion on the polarization image; the converter submodule uses the Transformer model to extract relevant data of the target object; the prediction output submodule uses the FNN neural network to process the data output by the converter submodule to obtain multi-target detection results.
[0076] The above content describes in detail the logical structure of the polarization intelligent sensing system provided by the embodiment of the present invention. Corresponding to the polarization intelligent sensing system, the embodiment of the present invention also provides a polarization intelligent sensing method implemented by using the polarization intelligent sensing system.
[0077] FIG4 shows a process of a polarization intelligent sensing method provided according to an embodiment of the present invention.
[0078] As shown in FIG4 , the polarization intelligent sensing method provided by an embodiment of the present invention includes the following steps:
[0079] S1. Perform polarization imaging on the target scene to obtain a polarization image.
[0080] S2. Solve the polarization image to obtain polarization information of the target scene.
[0081] Step S2 specifically includes the following sub-steps:
[0082] S21. Preprocess the polarization image.
[0083] This process mainly realizes the calibration of the sensor detection intensity. Taking the conventional four-linear polarization direction detector as an example (it can be expanded to a full polarization detector in the future), the four adjacent pixels are defined as a super pixel I = [I0, I 45 , I 90 , I 135 ], the detection intensity is defined as I d , the true target strength is I t The core purpose of the denoising process is to detect the intensity I d Get the target's true strength I t In the present invention, a polynomial fitting method is used to achieve the calibration of the detection intensity. The components of noise include but are not limited to crosstalk in the polarization extinction direction, detector noise such as dark current noise, shot noise, 1 / f noise, thermal noise, etc. The calibration process can be expressed as the following formula: t =a0+a1I d +a2I d 2 …a n I d n ;
[0084] Among them, a0…a n are the coefficients of the polynomial, which can be calibrated through laboratory and field tests. The mathematical methods can be the least squares method, multiple linear regression method, etc.
[0085] S22. Obtain the Stokes vector (I, Q, U, V) of the target scene from the preprocessed polarization image.
[0086] S23. Calculate the polarization information of the target scene based on the Stokes vector.
[0087] Polarization information includes polarization degree information and polarization phase angle information;
[0088] The calculation formula of the degree of polarization information DoP is:
[0089] Or the calculation formula of the linear polarization degree information DoLP is:
[0090] Among them, I, Q, U, and V are the Stokes vectors of the target scene respectively;
[0091] The calculation formula of polarization angle information AoLP is:
[0092] For the split-focal-plane polarization detection system, since it senses linear polarization information (I, Q, U) but cannot sense circular polarization information, the calculation formula for its linear polarization degree is: The formula for calculating the polarization phase angle is still:
[0093] Different polarization information is adaptively calculated based on the target scenario. For example, for targets with distinct edge features, the degree of polarization information is primarily extracted, while for targets with distinct face features, the polarization phase angle information is primarily extracted. The weighting of the two in the intelligent recognition algorithm is intelligently selected here. After intelligent-driven calculation, the weight ratios assigned to the degree of polarization image and the polarization phase angle image are K1 and K2, respectively.
[0094] S3. Generate image information of the target scene to be restored according to the polarization information.
[0095] Combined with the polarization characteristics of the target, neural networks can be used to carry out artificial intelligence recognition, which has significant advantages over traditional intensity images. Conventional artificial intelligence processing technology is mainly based on intensity images and has relatively mature artificial intelligence frameworks, such as the Yolo series, but this system mainly uses intensity images. Polarization information is an independent information dimension of the target scene. There is no artificial intelligence processing framework that is compatible with the target polarization image on the market. Therefore, the "polarization camera" + "artificial intelligence" model proposed in the present invention is innovative. In terms of practicality, the present invention has a very broad application prospect, such as: search for artificial targets on the sea surface, detection and identification of sea surface and underwater targets, navigation under smog climate conditions, satellite or airborne remote sensing, etc.; at the same time, it can be combined with other technologies, such as conventional intensity image information acquisition technology, spectrum acquisition technology, radar technology, etc.
[0096] In terms of AI recognition, we conduct image enhancement and fusion based on user needs. We also build neural network models based on the characteristics of input data, enabling near-real-time processing of input images, generating information of interest to the next node. For example, after sample training, we can intelligently identify and perceive targets such as marine vessels, sea ice, shallow reefs, islands, and people.
[0097] Neural network models include but are not limited to the following two, both of which can be used to restore images that are closer to the actual information of the target scene:
[0098] 1. Construct a polarization image enhancement model. For example, a model can be expressed as: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135}+k2*C2{IMG(D)}+k3*C3{IMG(A)}
[0099] Wherein, IMG(O) represents the image information of the target scene to be restored, C1{I0,I 45 , I 90 , I 135} represents the calculated preferred image of the intensity image 0°, 45°, 90° and 135° polarization images, which is a single polarization angle image or a multi-angle calculated image; IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image containing the edge highlighting information of the target space obtained by using the polarization degree image; IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image containing the surface information of the target object obtained by using the polarization phase angle image; k1, k2 and k3 respectively represent the intensity coefficients of the calculated preferred image of the polarization image, the polarization degree calculated image and the polarization phase angle calculated image.
[0100] 2. Construct a CNN neural network, take polarization images (including polarization degree image A, polarization phase angle image B, and images of various polarization angles) as input, use the corresponding artificially optimized IMG(O) as labels, and train it around typical target scenes to ultimately obtain a network structure that can intelligently output the ideal IMG(O).
[0101] To optimize target detection tasks by leveraging the ability of polarization cameras to obtain target-related information that differs from that of ordinary visible light cameras, a multi-dimensional target detection neural network based on the DETR network is proposed, as shown in Figure 5. Specifically, it includes three sub-modules: preprocessing, transformer, and prediction output. In the preprocessing sub-module, the intensity image, degree of polarization, polarization phase angle, restored target scene image, and spectral information are used as inputs, or in combination. A denoising module and a CNN neural network are used to perform denoising and feature fusion on the multi-channel image. In the transformer sub-module, a Transformer model is used to extract relevant data about the target object. In the prediction output sub-module, an FNN neural network is used to process the transformer sub-module output to obtain multi-target detection results.
[0102] As shown in Figure 5, images of four different dimensions, namely visible light intensity, polarization degree, polarization phase angle, and infrared, from the same viewing angle are combined together as a 4-channel image and input into a specially trained DETR network. In this network, it is first pre-processed for denoising, and then Down-CNN (composed of convolutional layers and pooling layers) is used to fuse the image features of each channel and reduce the resolution to reduce subsequent memory requirements; then, four transform blocks are used to extract relevant features of each object, and finally, two FNNs are used to obtain the object type contained in the image and the location of the corresponding type. Its training label is generated by labelImg, and the training task is the bipartite graph matching problem between the prediction results and the training labels. The Hungarian algorithm is used to optimize and solve the minimum loss, and thereby obtain the trained network. Among them, the loss function is defined as follows: the prediction result is regarded as a sequence of fixed length N Labels are represented as O:O i=(c i , b i ), where i∈(1, N), c i Indicates the object type, b i If is a four-tuple, the minimum matching strategy defined by the Hungarian algorithm is as follows:
[0103] in, Considering the similarity prediction between the category and the ground-truth box, c i The confidence level of the category prediction is The bounding box prediction is So It can be expressed as:
[0104] in, To linearly combine L1 loss and iou loss, we get:
[0105] Then the overall loss is expressed as:
[0106] The training goal is to minimize the overall loss.
[0107] Due to the significant differences between polarization images and conventional images, polarization images are highly sensitive to incident light with strong polarization information, achieving the effect of "brightening weak light and weakening strong light." The aforementioned scenes, such as ships, sea ice, shallow reefs, islands, and people, all have distinct polarization characteristics compared to the background, and thus appear distinctly different in polarization images. This difference, unlike traditional intensity images, is the starting point of this invention.
[0108] At the specific implementation level, there are a variety of flexible technical solutions, for example: using polarization images alone to identify the outlines of man-made structures in the target scene (such as buildings), smooth surfaces (such as glass), and moving man-made objects (such as cars); for example: using the fusion of polarization images and polarization phase angle images to achieve dark scene enhancement (such as shadow removal); for example: using polarization images to estimate the tilt angle of the target object, etc.
[0109] Using polarization images as a sample set for artificial intelligence recognition will better identify targets with polarization characteristics than conventional intensity images.
[0110] S4. Apply a neural network to obtain interpretation information of the target scene based on the image information, spectral information or intensity information.
[0111] The present invention can not only generate image information of the target scene to be restored through polarization information as input of the neural network to obtain the interpretation of the target scene, but also obtain the interpretation information of the target scene through spectral information or intensity information as input of the neural network, and store the comprehensive calculation data.
[0112] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.
[0113] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
[0114] The above specific embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.
Claims
1. A polarization intelligent sensing system, characterized in that: include: A polarization optical imaging module is used to perform polarization imaging on a target scene to obtain a polarization image; A polarization image processing module, configured to process the polarization image to obtain polarization information of the target scene; a target scene image enhancement module, configured to generate image information of the target scene to be restored according to the polarization information; The target scene image enhancement module adopts a multi-dimensional target detection neural network based on the DETR network, and the multi-dimensional target detection neural network includes: A preprocessing submodule, which takes the intensity image, polarization degree, polarization phase angle, restored target scene image, and spectral information as input or a partial combination thereof, and performs noise reduction and feature fusion on the polarization image in combination with a noise reduction module and a CNN neural network; The transformer submodule uses the Transformer model to extract relevant data of the target object; The prediction output submodule uses an FNN neural network to process the data output by the converter submodule to obtain a multi-target detection result; A target scene interpretation module is used to apply a neural network to obtain interpretation information of the target scene based on the image information, spectral information or intensity information.
2. The polarization intelligent sensing system according to claim 1, wherein: The polarization image processing module includes: a polarization image preprocessing unit, configured to preprocess the polarization image; The polarization information calculation unit is used to calculate the pre-processed polarization image to obtain the polarization degree information and polarization phase angle information of the pixel.
3. The polarization intelligent sensing system according to claim 1, wherein: The target scene image enhancement module adopts the polarization image enhancement model: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135 }+k2*C2{IMG(D)}+k3*C3{IMG(A)}; Wherein, IMG(O) represents the image information of the target scene to be restored, C1{I0,I 45 , I 90 , I 135 } represents the calculated preferred image of the intensity image 0°, 45°, 90° and 135° polarization images, and the preferred image is a single polarization angle image or a calculated image of multiple angles; IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image containing edge highlighting information of the target space obtained using the polarization degree image; IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image containing target surface information obtained using the polarization phase angle image; k1, k2 and k3 represent the intensity coefficients of the calculated preferred image of the polarization image, the polarization degree calculated image and the polarization phase angle calculated image, respectively.
4. The polarization intelligent sensing system according to claim 1, wherein: The polarization optical imaging module includes a matching polarization optical lens and a polarization detector, and the polarization optical lens has a weak polarization modulation characteristic.
5. A polarization intelligent sensing method implemented by using the polarization intelligent sensing system according to any one of claims 1 to 4, characterized in that: The steps include: S1. Perform polarization imaging on the target scene to obtain a polarization image; S2. Solving the polarization image to obtain polarization information of the target scene; S3, generating image information of the target scene to be restored according to the polarization information; Construct a multi-dimensional target detection neural network based on the DETR network, the multi-dimensional target detection neural network includes: A preprocessing submodule, which takes the intensity image, polarization degree, polarization phase angle, restored target scene image, and spectral information as input or a partial combination thereof, and performs noise reduction and feature fusion on the polarization image in combination with a noise reduction module and a CNN neural network; The transformer submodule uses the Transformer model to extract relevant data of the target object; The prediction output submodule uses an FNN neural network to process the data output by the converter submodule to obtain a multi-target detection result; S4. Apply a neural network to obtain interpretation information of the target scene based on the image information, spectral information, or intensity information.
6. The polarization intelligent sensing method according to claim 5, wherein: Step S2 specifically includes the following sub-steps: S21, preprocessing the polarization image; S22, acquiring the Stokes vector of the target scene from the preprocessed polarization image; S23, calculating polarization information of the target scene based on the Stokes vector; The polarization information includes polarization degree information and polarization phase angle information; The calculation formula of the polarization degree information DoP is: Or the calculation formula of the linear polarization degree information DoLP is: Wherein, I, Q, U, and V are the Stokes vectors of the target scene respectively; The calculation formula of the polarization angle information AoLP is:
7. The polarization intelligent sensing method according to claim 6, wherein: In step S21, the four adjacent pixels are defined as a super pixel I = [I0, I 45 , I 90 , I 135 ], the detection intensity is defined as I d , the true target strength is I t , the detection intensity is calibrated by a polynomial fitting method to achieve denoising of the polarization image, and the polynomial fitting method is: I t =a0+a1I d +a2I d 2 …a n I d n ; Among them, a0…a n are the coefficients of the polynomial.
8. The polarization intelligent sensing method according to claim 5, wherein: In step S3, a polarization image enhancement model is constructed: IMG(O)=k1*C1{I0,I 45 , I 90 , I 135 }+k2*C2{IMG(D)}+k3*C3{IMG(A)} Wherein, IMG(O) represents the image information of the target scene to be restored, C1{I0,I 45 , I 90 , I 135 } represents the calculated preferred image of the intensity image 0°, 45°, 90° and 135° polarization images, which is a single polarization angle image or a multi-angle calculated image. IMG(D) represents the polarization degree image, C2{IMG(D)} represents the calculated image containing edge highlighting information of the target space obtained using the polarization degree image; IMG(A) represents the polarization phase angle image, C3{IMG(A)} represents the calculated image containing target surface information obtained using the polarization phase angle image; k1, k2 and k3 represent the intensity coefficients of the calculated preferred image of the polarization image, the polarization degree calculated image and the polarization phase angle calculated image, respectively.
