Image fusion method and device based on infrared polarization imaging

Abstract

The invention discloses an image fusion method and device based on infrared polarization imaging, and relates to the technical field of image processing. The method comprises the following steps: acquiring infrared polarization image data of a target scene; based on a polarization bidirectional reflection distribution function, decomposing the image into three kinds of feature original element images including total radiation intensity, linear polarization degree and polarization angle; processing according to a preset task mode to obtain a feature primitive image; calculating the regional information abundance measurement of each feature primitive image and the regional complementarity measurement between every two feature primitive images; and generating a fusion weight map through a normalized exponential function, and outputting a fusion image after carrying out weighted fusion on the feature primitive image. Through physical feature decomposition, task targeted processing and quantitative measurement fusion, the problems of low richness and low accuracy of fused image information in a traditional method are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image fusion method and device based on infrared polarization imaging. BACKGROUND

[0002] In the field of image fusion technology, with the continuous development of science and technology, its application range is increasingly wide. Image fusion technology can integrate the information in multi-source images to provide more abundant and accurate data support for subsequent analysis and decision-making. In the aspect of image fusion based on infrared polarization imaging, traditional image fusion is usually based on feature extraction, that is, the feature information of images such as edges and textures is extracted, and then image fusion is performed according to these features.

[0003] However, the image information fused by the above image fusion method has relatively low richness and accuracy, and is difficult to adapt to the needs of various application scenarios. SUMMARY

[0004] In order to solve the above technical problems, the present application provides an image fusion method and device based on infrared polarization imaging.

[0005] In a first aspect of the present application, an image fusion method based on infrared polarization imaging is provided: obtaining infrared polarization image data of a target scene; based on the polarization bidirectional reflectance distribution function, the infrared polarization image data is decomposed into a first characteristic primitive image, a second characteristic primitive image and a third characteristic primitive image; According to a preset task mode, the first characteristic primitive image, the second characteristic primitive image and the third characteristic primitive image are respectively subjected to feature optimization processing to obtain a first characteristic base image, a second characteristic base image and a third characteristic base image; respectively calculating a first regional information richness metric of the first characteristic base image, a second regional information richness metric of the second characteristic base image and a third regional information richness metric of the third characteristic base image; calculating a first regional complementarity metric between the first characteristic base image and the second characteristic base image, a second regional complementarity metric between the first characteristic base image and the third characteristic base image, and a third regional complementarity metric between the second characteristic base image and the third characteristic base image; processing the first regional information richness metric, the second regional information richness metric, the third regional information richness metric, the first regional complementarity metric, the second regional complementarity metric and the third regional complementarity metric by a normalization exponential function to generate a fusion weight map; The first feature primitive image, the second feature primitive image and the third feature primitive image are fused by using the fusion weight map, and a fused image is output.

[0006] By adopting the technical solution, the infrared polarization data is decomposed based on the physical model of the polarization bidirectional reflectance distribution function, three original primitive images of three physical properties of complementarity, i.e., total radiation intensity (first feature), linear polarization degree (second feature) and polarization angle (third feature), are separated from the source, and the physical authenticity and completeness of information representation are ensured; the primitive images are processed for targeted enhancement through a preset task mode, and the saliency of key task features is significantly improved; the contribution value of each primitive image in different spatial regions is accurately described through quantization of regional information abundance and cross-feature complementarity; the fusion weight map is dynamically generated by using a normalization exponential function, and optimal weights are adaptively allocated in a data-driven manner, so that the high information abundance regions of the primitive images are retained while the complementary information between features is maximized. The technical chain cooperates to make the fused image have high information entropy (richness), strong edge fidelity (accuracy) and task-related feature enhancement characteristics, thereby meeting the stringent requirements of image quality in scenes such as high-precision target recognition and complex environment detection.

[0007] Optionally, the decomposing, by the polarization bidirectional reflectance distribution function, of the infrared polarization image data into the first feature original primitive image, the second feature original primitive image and the third feature original primitive image comprises: establishing a physical decomposition model through the polarization bidirectional reflectance distribution function; extracting total radiation intensity information from the infrared polarization image data according to the physical decomposition model, and constructing the first feature original primitive image according to the total radiation intensity information; extracting a linear polarization degree parameter from linear polarization state component information of the infrared polarization image data according to the physical decomposition model, and constructing the second feature original primitive image according to the linear polarization degree parameter; extracting a polarization angle parameter from the linear polarization state component information according to the physical decomposition model, and constructing the third feature original primitive image according to the polarization angle parameter.

[0008] By adopting the technical solution, the physical decomposition model established based on the polarization bidirectional reflectance distribution function can accurately strip the three core polarization features of total radiation intensity, linear polarization degree and polarization angle from the infrared polarization image data: the total radiation intensity reflects the thermal radiation characteristics of a target, and the linear polarization degree and the polarization angle reflect physical properties such as the surface material and roughness of the target. This decomposition method ensures that the three original primitive images completely retain the key information of infrared polarization imaging and avoid feature confusion or loss.

[0009] Optionally, the first feature primitive image, the second feature primitive image and the third feature primitive image are respectively subjected to feature optimization processing according to the preset task mode, to obtain a first feature primitive image, a second feature primitive image and a third feature primitive image. According to the preset task mode, a mapping relationship between a pixel original gray value and a target gray value in the first feature primitive image is established, and the gray value of the target region is adjusted according to the mapping relationship, to obtain the first feature primitive image. The second feature primitive image is decomposed into a plurality of feature layers with different spatial resolutions according to a preset scale parameter, edge features and texture features are extracted from each feature layer, and the edge features and the texture features are superimposed and combined according to a preset weight ratio, to obtain the second feature primitive image. Noise pixels in the third feature primitive image are identified through a preset noise judgment condition, all non-noise pixels in a preset range around the noise pixels are selected, a statistical value of all the non-noise pixels is calculated, and the statistical value is used to replace the pixel value of the noise pixels, to obtain the third feature primitive image.

[0010] By using the above technical solution, the target region gray value of the first feature primitive image can be adjusted in a targeted manner, and the distinction between the target region and the background region is enhanced. Through multi-scale feature layer decomposition and weighted combination, the edge and texture features in the second feature primitive image under different scales can be fully retained and strengthened, and the clarity of the target contour is improved. Based on the replacement of the noise pixels by the statistical value of the non-noise pixels in the preset range, the noise in the third feature primitive image can be effectively removed while the effective feature information is maximally retained.

[0011] Optionally, the first region information abundance metric of the first feature primitive image, the second region information abundance metric of the second feature primitive image and the third region information abundance metric of the third feature primitive image are respectively calculated. The first feature primitive image, the second feature primitive image and the third feature primitive image are divided into a plurality of image blocks according to a preset size, to obtain a first number of first image blocks of the first feature primitive image, a second number of second image blocks of the second feature primitive image and a third number of third image blocks of the third feature primitive image. weighting and summing the gray level entropy value and the gradient amplitude mean value of the first image block to obtain a first weighted sum, and arranging the first quantity of the first weighted sums according to the spatial coordinate positions of the first image blocks in the first feature primitive image to obtain the first regional information abundance measure, the gray level entropy value representing the complexity of the gray level distribution in the image block, and the gradient amplitude mean value representing the intensity of the edge feature in the image block; weighting and summing the gray level entropy value and the gradient amplitude mean value of the second image block to obtain a second weighted sum, and arranging the second quantity of the second weighted sums according to the spatial coordinate positions of the second image blocks in the second feature primitive image to obtain the second regional information abundance measure; weighting and summing the gray level entropy value and the gradient amplitude mean value of the third target image block to obtain a third weighted sum, and arranging the third quantity of the third weighted sums according to the spatial coordinate positions of the third image blocks in the third feature primitive image to obtain the third regional information abundance measure.

[0012] By adopting the technical solution, the feature primitive image is divided into image blocks for regional analysis, and the gray level entropy value and the gradient amplitude mean value are combined for weighted calculation, so that the local regional information value of each feature primitive image is quantified. This regional measurement method not only retains the spatial position correlation, but also comprehensively describes the information abundance of different regions through the fusion of double indicators. The gray level entropy value highlights the value of information-intensive regions, and the gradient amplitude mean value emphasizes the importance of edge feature significant regions. The finally formed regional information abundance measure can accurately reflect the information quality of each local region.

[0013] Optionally, the calculation of the first regional complementarity measure between the first feature primitive image and the second feature primitive image, the second regional complementarity measure between the first feature primitive image and the third feature primitive image, and the third regional complementarity measure between the second feature primitive image and the third feature primitive image comprises: calculating a first gray level difference entropy of a first target image block of the first feature primitive image and a second target image block of the second feature primitive image, taking the first gray level difference entropy as the first regional complementarity measure, the first target image block being any one of the first quantity of the first image blocks, and the second target image block being an image block of the second quantity of the second image blocks at the same spatial coordinate as the first target image block; calculating a second gray level difference entropy of the first target image block and a third target image block of the third feature primitive image, taking the second gray level difference entropy as the second regional complementarity measure, the third target image block being an image block of the third quantity of the third image blocks in the same spatial coordinate range as the first target image block. calculate a third gray difference entropy of the second target image block and the third target image block, and take the third gray difference entropy as the third regional complementarity measure.

[0014] By using the above technical solution, the gray difference entropy is calculated in units of image blocks with the same spatial coordinates, and the information difference and complementarity of different feature primitive images in the same region are accurately quantified. The gray difference entropy captures the numerical difference between features and reflects the complexity of the difference distribution by statistical gray difference value distribution probability, can effectively represent the information complementarity degree of different feature primitives in the same region, and the higher the difference entropy, the stronger the feature complementarity of the region. This regional complementarity measurement method ensures that the fusion process can integrate the complementary advantages of different features in each region, and avoids information redundancy or conflict.

[0015] Optionally, the calculating the first gray difference entropy of the first target image block of the first feature primitive image and the second target image block of the second feature primitive image, and taking the first gray difference entropy as the first regional complementarity measure comprises: calculating the difference value of the gray values of each pair of pixels in the first target image block and the second target image block and taking the absolute value to obtain a gray difference data set, each pair of pixels comprising a first pixel in the first target image block and a second pixel in the second target image block, the position of the first pixel in the first target image block being the same as the position of the second pixel in the second target image block; statistically counting the frequency of each difference value in the gray difference data set, dividing the frequency of each difference value by the total number of pixels of the first image block to obtain the probability of each difference value; multiplying the probability of each difference value by the logarithm of the probability to obtain an intermediate value, and then taking the negative value of the sum of all intermediate values to obtain the first gray difference entropy; determining the first gray difference entropy as the first regional complementarity measure between the first target image block and the second target image block.

[0016] By using the above technical solution, the gray difference of two target image blocks is calculated from the pixel level, the difference amplitude information is retained by absolute value processing, and then the discrete gray difference is converted into quantifiable distribution characteristics through frequency statistics and probability conversion. By using entropy value calculation, not only the size of the difference can be reflected, but also the complexity of the difference distribution can be accurately described. The more diverse the difference is and the more uniform the distribution is, the higher the entropy value is, which directly reflects the information complementarity degree of two feature primitives in the region.

[0017] Optionally, the processing the first regional information abundance metric, the second regional information abundance metric, the third regional information abundance metric, the first regional complementarity metric, the second regional complementarity metric and the third regional complementarity metric by a normalized exponential function to generate a fusion weight map further comprises: weighting and fusing the first regional information abundance metric, the first regional complementarity metric and the second regional complementarity metric to obtain a first comprehensive feature weight; weighting and fusing the second regional information abundance metric, the first regional complementarity metric and the third regional complementarity metric to obtain a second comprehensive feature weight; weighting and fusing the third regional information abundance metric, the second regional complementarity metric and the third regional complementarity metric to obtain a third comprehensive feature weight; inputting the first comprehensive feature weight, the second comprehensive feature weight and the third comprehensive feature weight into a normalized exponential function to calculate a first weight value matrix corresponding to the first comprehensive feature weight, a second weight value matrix corresponding to the second comprehensive feature weight and a third weight value matrix corresponding to the third comprehensive feature weight; generating a fusion weight map according to the first weight value matrix, the second weight value matrix and the third weight value matrix, the fusion weight map having the same spatial size as the first feature primitive image, the second feature primitive image and the third feature primitive image.

[0018] By adopting the technical solution, the regional information abundance and complementarity metrics are first weighted and fused to form a comprehensive weight that can comprehensively reflect the information value of each feature primitive and the complementary relationship between features, and then the normalized exponential function is used to convert the comprehensive weight into a weight matrix for spatial matching. This processing integrates the information quality of single features and the complementary advantages of multiple features, and retains the spatial position correlation in the form of a matrix, so that the finally generated fusion weight map can be accurately aligned with the feature primitive image.

[0019] In a second aspect of the present application, an image fusion system based on infrared polarization imaging is provided, which specifically comprises: an image data acquisition module configured to acquire infrared polarization image data of a target scene; an image decomposition module configured to decompose the infrared polarization image data into a first feature primitive image, a second feature primitive image and a third feature primitive image based on a polarization bidirectional reflectance distribution function; a task processing module, configured to perform feature optimization processing on the first feature raw primitive image, the second feature raw primitive image and the third feature raw primitive image respectively according to a preset task mode, to obtain a first feature primitive image, a second feature primitive image and a third feature primitive image; an information abundance metric calculation module, configured to calculate a first region information abundance metric of the first feature primitive image, a second region information abundance metric of the second feature primitive image and a third region information abundance metric of the third feature primitive image respectively; a complementarity metric calculation module, configured to calculate a first region complementarity metric between the first feature primitive image and the second feature primitive image, a second region complementarity metric between the first feature primitive image and the third feature primitive image, and a third region complementarity metric between the second feature primitive image and the third feature primitive image; a fusion weight map generation module, configured to generate a fusion weight map by processing the first region information abundance metric, the second region information abundance metric, the third region information abundance metric, the first region complementarity metric, the second region complementarity metric and the third region complementarity metric through a normalized exponential function; a fused image output module, configured to perform weighted fusion on the first feature primitive image, the second feature primitive image and the third feature primitive image by using the fusion weight map, and output a fused image.

[0020] In a third aspect of the present application, an electronic device is provided, which includes a processor, a memory, a user interface and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method according to any one of the preceding aspects.

[0021] In a fourth aspect of the present application, a computer readable storage medium is provided, which stores instructions, when the instructions are executed, the method according to any one of the preceding aspects is performed.

[0022] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: From the physical essence, the infrared polarization data is decomposed into three core feature primitives of total radiation intensity, linear polarization degree and polarization angle based on the polarization bidirectional reflectance distribution function, which completely retains the multi-dimensional information of target thermal radiation characteristics and surface physical properties, and lays a high-quality foundation for subsequent processing. At the same time, the feature primitives are differentiated and enhanced according to the preset task mode, through operations such as contrast enhancement, edge strengthening and noise suppression. Through the quantitative regional information abundance and regional complementarity, combined with the gray difference entropy, the fineness measurement from pixels to regions is realized, which provides an objective basis for weight allocation and avoids subjective bias. Finally, based on these quantitative results, a fusion weight map is dynamically generated, which takes into account the information quality of single features and the complementary advantages of multiple features, so that the fused image has high information richness and ensures the accuracy of information integration, thus effectively meeting the scene requirements of high-precision target recognition, complex environment detection and other high-quality image requirements. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a system architecture schematic diagram of an embodiment of an image fusion method or an image fusion system based on infrared polarization imaging applied in the present application; Figure 2 is a flowchart of an image fusion method based on infrared polarization imaging disclosed in an embodiment of the present application; Figure 3 is a module schematic diagram of an image fusion system based on infrared polarization imaging disclosed in an embodiment of the present application; Figure 4 is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0024] Mark explanation: 301, image data acquisition module; 302, image decomposition module; 303, task processing module; 304, information abundance measurement calculation module; 305, complementarity measurement calculation module; 306, fusion weight map generation module; 307, fusion image output module; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION

[0025] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in the specification, with reference to the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments.

[0026] As Figure 1As shown, the system architecture 100 can include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0027] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as model training applications, video recognition applications, web browser applications, social platform software, etc.

[0028] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. They can be implemented as multiple software or software modules (such as multiple software or software modules for providing distributed services), or as a single software or software module. No specific limitation is made herein.

[0029] The embodiment discloses an image fusion method based on infrared polarization imaging, Figure 2 is a flowchart of an image fusion method based on infrared polarization imaging according to an embodiment of the present application, as Figure 2 shown, the method includes the following steps: S201, acquiring infrared polarization image data of a target scene; Specifically, an infrared polarization imaging camera with a 12 μm infrared detector is used. The camera is fixed on a tripod and calibrated to be horizontal, ensuring that the lens optical axis is aligned with the target scene (specifically including artificial buildings, natural vegetation, and moving or stationary vehicles with a height of 1-5 m, and the scene environment is a clear night, with moonlight and environmental scattered light as the main light source). The spectral response range is set to 8-14 μm, the frame rate is 15 fps, and the exposure time is 50 ms. The camera acquisition program is started, and the polarizer is switched to 0°, 45°, 90°, and 135° directions in turn (0° direction is parallel to the ground and along the horizontal center axis of the camera lens (i.e. parallel to the horizontal direction of the camera body), and 90° direction is perpendicular to 0° direction (i.e. perpendicular to the ground, along the vertical center axis of the camera lens)). Five frames of images are continuously collected in each direction to reduce random noise. During the collection process, the camera and the target scene remain relatively stationary. The collected image data is stored in the camera's built-in storage card in 16-bit.raw format, and the metadata such as collection time, environmental temperature, and target distance are recorded. After the collection is completed, the raw data is transmitted to the server through the USB (Universal Serial Bus, Universal Serial Bus) interface, and the infrared polarization image data set containing four direction polarizations is obtained.

[0030] S202, based on the polarization bidirectional reflectance distribution function, the infrared polarization image data is decomposed into a first characteristic raw primitive image, a second characteristic raw primitive image, and a third characteristic raw primitive image; Specifically, four-direction (0°, 45°, 90°, 135°) infrared polarization image data (640×512 pixels, 16-bit.raw format) is loaded, and a physical decomposition model is established based on the polarization bidirectional reflectance distribution function (combining the target surface diffuse reflection and specular reflection characteristics). Three characteristic raw primitive images are obtained by decomposition: the first characteristic raw primitive image is formed by calculating the average value of the gray scale of the corresponding pixels in the four directions; the second characteristic raw primitive image is obtained by calculating the square sum of the two linear polarization components from the gray scale difference between 0° and 90°, 45° and 135° direction images, and then dividing the square sum by the total radiation intensity (total radiation is 0); the third characteristic raw primitive image is formed by arranging the polarization angle calculated by arctangent based on the two linear polarization components (the angle is corrected to 0°-180° according to the sign of the component).

[0031] Optionally, the infrared polarization image data is decomposed into a first characteristic raw primitive image, a second characteristic raw primitive image, and a third characteristic raw primitive image based on the polarization bidirectional reflectance distribution function, which includes: A physical decomposition model is established by the polarized bidirectional reflectance distribution function; total radiation intensity information is extracted from the infrared polarized image data according to the physical decomposition model, and a first characteristic original primitive image is constructed according to the total radiation intensity information; a linear polarization degree parameter is extracted from linear polarization state component information of the infrared polarized image data according to the physical decomposition model, and a second characteristic original primitive image is constructed according to the linear polarization degree parameter; a polarization angle parameter is extracted from the linear polarization state component information according to the physical decomposition model, and a third characteristic original primitive image is constructed according to the polarization angle parameter.

[0032] Specifically, based on the characteristics of diffuse reflection (weak polarization) and specular reflection (strong polarization) in the polarized bidirectional reflectance distribution function and their association with target material, roughness, and illumination angle, a model containing four layers is constructed using Python programming language combined with Open Computer Vision Library (OpenCV) and Numerical Computing Extension Library (NumPy). First, the input preprocessing layer is built. The code reads the raw format data of the four-direction infrared polarization image, converts it into a 640x512 two-dimensional array through the NumPy library, and calls the medianBlur function in the OpenCV library to perform median filtering on the image to remove the salt and pepper noise introduced by the sensor. The processed data is used as the input of the model. Then, the reflection component separation layer is built. Based on the polarization difference characteristics of diffuse reflection and specular reflection in the polarized bidirectional reflectance distribution function, a separation algorithm is constructed: first, calculate the preliminary degree of polarization of each pixel (estimated by the gray value of the four-direction image), set the polarization threshold to 0.3, and determine the area with a polarization degree lower than the threshold as the diffuse reflection dominant area, and extract its gray value to form the diffuse reflection component matrix; the area with a polarization degree higher than or equal to the threshold is determined as the specular reflection dominant area, and its gray value is extracted to form the specular reflection component matrix. Then, the feature parameter mapping layer is designed, and the conversion formula is written to map the two components into intermediate parameters: total radiation intensity base value = diffuse reflection component x 0.6 + specular reflection component x 0.4 (the weight is determined by 10 standard sample images); horizontal and vertical components of linear polarization state (Q component) = 0° direction image gray value - 90° direction image gray value; oblique component of linear polarization state (U component) = 45° direction image gray value - 135° direction image gray value, and these parameters are stored as matrices with the same size as the input image. Finally, the output rule definition layer is built, and the calculation logic code of three types of feature parameters is written: the total radiation intensity is calculated by the mean value of the gray values of the corresponding pixels of the four-direction image; the linear polarization degree is calculated by the square root of the sum of the squares of the Q component and the U component divided by the total radiation intensity (the linear polarization degree is set to 0 when the total radiation intensity is 0); the polarization angle is calculated by the inverse tangent value of the ratio of the U component to the Q component multiplied by a fixed coefficient (for example, 0.5), and the angle is modified to the range of 0°-180° according to the signs of Q and U components. These calculation rules are packaged into callable functions to complete the model construction.

[0033] Further, call the input preprocessing layer and the feature parameter mapping layer in the physical decomposition model, load the preprocessed four-direction (0°, 45°, 90°, 135°) infrared polarization image data (640x512 pixel two-dimensional array, pixel value range 0-1); for each spatial coordinate (x, y) in the image, x represents the horizontal position index of the pixel in the image (horizontal direction), and y represents the vertical position index of the pixel in the image (vertical direction). For example, in a 640x512 pixel image, the value range of x is 0 to 639 (corresponding to each column of pixels from left to right), and the value range of y is 0 to 511 (corresponding to each row of pixels from top to bottom). Then, according to the total radiation intensity calculation rule defined by the model feature parameter mapping layer, the pixel values of the coordinate in the 0°, 45°, 90°, and 135° direction images are extracted. First, remove the outliers whose deviation from the pixel values of the remaining three directions is more than 30% (if there are outliers, replace them with the mean of the remaining three values), and then calculate the arithmetic mean of the pixel values of the four directions. The arithmetic mean is taken as the total radiation intensity value at (x, y). After traversing all coordinates to complete the calculation, arrange all total radiation intensity values in the original image spatial coordinate order to form a 640x512 pixel two-dimensional matrix. Through linear mapping, the total radiation intensity values (range 0-1) in the matrix are converted to 16-bit integers (range 0-65535), and the imwrite function of OpenCV is used to save the matrix as a label image file format, i.e. to obtain the first feature primitive image.

[0034] Further, call the feature parameter mapping layer and output rule definition layer of the physical decomposition model, load the preprocessed four-direction infrared polarization image data through the Python script, and for each pixel coordinate (x, y) (x is the horizontal index of 0-639, and y is the vertical index of 0-511), first extract the linear polarization state component information, and the model defines the calculation of Q component (0° direction pixel value minus 90° direction pixel value) and U component (45° direction pixel value minus 135° direction pixel value); then calculate the linear polarization degree parameter according to the output rule: use the sqrt function of the numerical calculation library (NumPy) to calculate the square root of the sum of the squares of the Q component and the U component (i.e. linear polarization intensity), and then divide by the total radiation intensity value of the coordinate (if the total radiation intensity is 0, the linear polarization degree is directly set to 0), to obtain a linear polarization degree value in the range of 0-1; after traversing all pixel coordinates to complete the calculation, arrange all linear polarization degree values in the original image spatial coordinate order to form a 640x512 pixel two-dimensional matrix; through linear mapping, the matrix value (0-1) is converted to a 16-bit integer (0-65535), and the imwrite function is used to save it as a label image file format, i.e. to obtain the second feature primitive image.

[0035] Further, the characteristic parameter mapping layer and the output rule definition layer of the physical decomposition model are called to load the extracted linear polarization state components (Q component and U component) and total radiation intensity map data through a Python script. For each pixel coordinate (x, y) (x is a horizontal index of 0-639, and y is a vertical index of 0-511), it is first judged whether the total radiation intensity of the coordinate is 0 (if 0, the polarization angle is directly set to 0°); if the total radiation intensity is not 0, the arctan2 function of NumPy is used to calculate the arctangent value of the ratio of the U component to the Q component based on the Q component and the U component (the return value range is -pi to pi radian), and then the result is multiplied by 0.5 to convert it into the radian value of the polarization angle, and then the angle conversion formula (radian x 180 / pi) is used to convert it into the angle value; the quadrant correction is performed according to the positive and negative of the Q component and the U component: when Q>0 and U>0, the angle remains unchanged; when Q<0, the angle is added by 90°; when U<0, the angle is added by 180°, to ensure that the final angle range is limited to 0°-180°; after the calculation of all pixel coordinates is completed, all polarization angle values are arranged in the original image space coordinate order to form a 640x512 pixel two-dimensional matrix; the angle value of 0°-180° is converted into a 16-bit integer (0-65535) through linear mapping, and the imwrite function of OpenCV is used to save it in the tag image file format (TIFF), that is, the third characteristic original primitive image (polarization angle map) is obtained, and a metadata file recording the angle mapping relationship is generated.

[0036] S203, according to the preset task mode, the first characteristic original primitive image, the second characteristic original primitive image and the third characteristic original primitive image are respectively processed, and the first characteristic primitive image, the second characteristic primitive image and the third characteristic primitive image are obtained. Specifically, when the preset task mode is the target detection mode, for the first feature original primitive image, the gray value range and the frequency of each gray value are counted, the gray value interval of the target region and the background region is set, the linear mapping relationship from the original gray value to the corresponding interval is established, the gray value of each pixel is adjusted according to the relationship to enhance the distinction between the target and the background, and the first feature primitive image is obtained; for the second feature original primitive image, the second feature original primitive image is decomposed into a plurality of feature layers with different spatial resolutions according to a preset scale parameter, for each layer, the edge feature is extracted by whether the gray difference between the pixel and the 8-neighborhood pixel exceeds a preset threshold, the texture feature is extracted by calculating the 3*3 neighborhood gray mean and variance, and then the edge feature and the texture feature are superimposed and combined at the corresponding pixel position according to a preset weight ratio (for example, the first scale edge and texture each account for 40%, and the second and third scales each account for 30%), and the second feature primitive image is obtained; for the third feature original primitive image, the noise pixels are identified by a preset noise judgment condition (for example, the absolute value of the difference between the pixel gray value and the 3*3 neighborhood mean value exceeds a preset threshold of 50), all non-noise pixels within a range of 5*5 around the noise pixels are selected, the gray mean value of the non-noise pixels is calculated as a statistical value, and the pixel value of the noise pixel is replaced by the statistical value, and the third feature primitive image is obtained.

[0037] Optionally, according to the preset task mode, the first feature original primitive image, the second feature original primitive image and the third feature original primitive image are respectively subjected to feature optimization processing, and the first feature primitive image, the second feature primitive image and the third feature primitive image are obtained. When the preset task mode is the target detection mode, the mapping relationship between the original gray value of the pixel in the first feature original primitive image and the target gray value is established, the gray value of the target region is adjusted according to the mapping relationship, and the first feature primitive image is obtained; the second feature original primitive image is decomposed into a plurality of feature layers with different spatial resolutions according to a preset scale parameter, the edge feature and the texture feature are extracted from each feature layer, the edge feature and the texture feature are superimposed and combined according to a preset weight ratio, and the second feature primitive image is obtained; the noise pixels in the third feature original primitive image are identified by a preset noise judgment condition, all non-noise pixels within a preset range around the noise pixels are selected, the statistical value of all the non-noise pixels is calculated, and the pixel value of the noise pixel is replaced by the statistical value, and the third feature primitive image is obtained.

[0038] Specifically, when the preset task mode is a target detection mode, first, the gray value range of the image and the frequency of each gray value are obtained by traversing all the pixel points of the first feature original primitive image, and a target gray value interval is set based on the statistical result, wherein the target region expected gray value interval and the background region expected gray value interval are non-overlapping and have a preset difference; then, a mapping relationship between the original gray value and the target gray value is established, specifically, the original gray value is linearly mapped to the corresponding target gray value interval in proportion, and for the pixels preliminarily determined as the target region through the previous image analysis, the original gray value is mapped to the target region expected gray value interval, and the remaining pixels are mapped to the background region expected gray value interval; finally, the gray value of each pixel in the image is converted according to the mapping relationship, the adjustment of the gray value of the target region is completed, and the first feature primitive image is obtained.

[0039] Further, the second feature original primitive image is decomposed into multiple feature image layers with different spatial resolutions according to preset scale parameters (for example, three scale levels are set, which are 1 times, 1 / 2 and 1 / 4 of the resolution of the original image), for each feature image layer, edge features are extracted by calculating the gray difference between a pixel and its neighborhood pixels (specifically, when the gray difference between a pixel and its 8-neighborhood pixels exceeds a preset threshold, the pixel is determined as an edge point and marked), texture features are extracted by counting the distribution of gray values in the neighborhood of the pixel (specifically, the mean and variance of the gray values in the 3x3 neighborhood of each pixel are calculated as texture feature values), and the edge features and texture features extracted from each scale image layer are numerically superimposed at the corresponding pixel positions according to a preset weight ratio (for example, the edge features and texture features of the first scale image layer each account for 40% of the weight, the edge features and texture features of the second scale image layer each account for 30% of the weight, and the edge features and texture features of the third scale image layer each account for 30% of the weight), to obtain the second feature primitive image.

[0040] Further, noise pixels in the third feature original primitive image are identified through a preset noise judgment condition (the preset noise judgment condition is that for each pixel in the image, the mean of the gray values of all pixels in its 3x3 neighborhood is calculated, and if the absolute value of the difference between the gray value of the pixel and the mean value exceeds a preset threshold of 50, the pixel is a noise pixel); all non-noise pixels within a 5x5 range around the noise pixel are selected, and the mean of the gray values of these non-noise pixels is calculated as a statistical value; the statistical value is used to replace the original pixel value of the noise pixel, and all identified noise pixels in the image are processed in turn to obtain the third feature primitive image.

[0041] S204, respectively calculating a first region information abundance measure of the first feature primitive image, a second region information abundance measure of the second feature primitive image, and a third region information abundance measure of the third feature primitive image; Specifically, the first feature primitive image, the second feature primitive image and the third feature primitive image are called, and each image is first divided into non-overlapping image blocks according to a preset size of 16*16 pixels; for each first image block, the calcHist function is called to calculate the gray level histogram, the probability of different gray levels is counted, and the gray entropy value reflecting the complexity of the gray level distribution is calculated through the probability; meanwhile, the Sobel operator is used to calculate the gradient of the image block in the horizontal and vertical directions respectively, the gradient values in the two directions are combined to calculate the gradient intensity of each pixel, and then the average value of the gradient intensity of all pixels is calculated to obtain the gradient amplitude average value reflecting the edge feature intensity; then the weighted sum is calculated according to the proportion of 0.6 (gray entropy value): 0.4 (gradient amplitude average value), and the first weighted sum is obtained; all the first weighted sums are arranged according to the spatial coordinate positions of the original image blocks to form a first regional information abundance measurement matrix consistent with the size of the original image; the same process is used to calculate the weighted sum of the gray entropy value and the gradient amplitude average value according to the weight of 0.5:0.5 for the second image block, to obtain the second weighted sum and arrange it as the second regional information abundance measurement matrix; the weighted sum of the two is calculated according to the weight of 0.4:0.6 for the third image block, to obtain the third weighted sum and arrange it as the third regional information abundance measurement matrix.

[0042] Optionally, the respective calculation of the first regional information abundance measurement of the first feature primitive image, the second regional information abundance measurement of the second feature primitive image and the third regional information abundance measurement of the third feature primitive image comprises: The first feature primitive image, the second feature primitive image and the third feature primitive image are divided into a plurality of image blocks according to a preset size, to obtain a first number of first image blocks of the first feature primitive image, a second number of second image blocks of the second feature primitive image and a third number of third image blocks of the third feature primitive image; the gray entropy value and the gradient amplitude average value of the first image block are weighted and summed to obtain a first weighted sum, and the first number of first weighted sums are arranged according to the spatial coordinate positions of the first image blocks in the first feature primitive image to obtain the first regional information abundance measurement, wherein the gray entropy value represents the complexity of the gray level distribution in the image block, and the gradient amplitude average value represents the strength of the edge feature in the image block; the gray entropy value and the gradient amplitude average value of the second image block are weighted and summed to obtain a second weighted sum, and the second number of second weighted sums are arranged according to the spatial coordinate positions of the second image blocks in the second feature primitive image to obtain the second regional information abundance measurement; the gray entropy value and the gradient amplitude average value of the third target image block are weighted and summed to obtain a third weighted sum, and the third number of third weighted sums are arranged according to the spatial coordinate positions of the third image blocks in the third feature primitive image to obtain the third regional information abundance measurement.

[0043] Specifically, the first, second and third feature primitive images are called, and a preset image block size of 16x16 pixels is set. For the first feature primitive image, starting from the top-left pixel (0, 0), a 16-pixel-wide column segment is taken every 16 pixels in the horizontal direction, and a 16-pixel-high row segment is taken every 16 pixels in the vertical direction, and the two are crossed to form a first image block of 16x16 pixels. The first image block is sequentially intercepted by sliding to the right and down. In the horizontal direction, 40 (640 ÷ 16) first image blocks can be intercepted, and in the vertical direction, 32 (512 ÷ 16) first image blocks can be intercepted, a total of 1280 first image blocks. For the image edge, if there is an area less than 16x16 pixels (such as non-integer division after resizing), 0 is filled to 16x16 pixels to ensure that all first image blocks have the same size. The second feature primitive image is divided in the same way to obtain 1280 second image blocks, and the third feature primitive image is divided to obtain 1280 third image blocks.

[0044] Further, for each first image block, the calcHist function of OpenCV is called to calculate the gray histogram, the number of occurrences of each gray level is counted and divided by 256 (total number of pixels) to obtain the probability, and the negative value of the product of each probability and the logarithm of the probability is obtained by accumulation to obtain the gray entropy value (representing the complexity of gray distribution). At the same time, the Sobel operator is used to calculate the horizontal and vertical gradients, and the average value of the gradient strength of each pixel in two directions is obtained by converting the two direction gradient values through squaring and square root. The average gradient amplitude value (representing the edge intensity) is obtained, and the first weighted sum is obtained by weighting and summing the gray entropy value by 60% and the gradient amplitude value by 40%. The 1280 first weighted sums are arranged according to the spatial coordinate positions of the first image blocks in the first feature primitive image (such as the image block corresponding to the (x x 16, y x 16) to ((x + 1) x 16 - 1, (y + 1) x 16 - 1) region of the original image), and a two-dimensional matrix of 640x512 pixels is reconstructed to obtain the first regional information abundance measure. The same method is used to calculate the gray entropy value and the gradient amplitude value for each second image block, and the second weighted sum is obtained by weighting and summing the gray entropy value by 50% and the gradient amplitude value by 50%. The 1280 second weighted sums are arranged according to the corresponding spatial coordinate positions, and a matrix of the same size is reconstructed to obtain the second regional information abundance measure. The same method is used to calculate the gray entropy value and the gradient amplitude value for each third image block, and the third weighted sum is obtained by weighting and summing the gray entropy value by 40% and the gradient amplitude value by 60%. The 1280 third weighted sums are arranged according to the corresponding spatial coordinate positions, and a matrix of the same size is reconstructed to obtain the third regional information abundance measure.

[0045] S205, calculate a first region complementarity measure between the first feature primitive image and the second feature primitive image, a second region complementarity measure between the first feature primitive image and the third feature primitive image, and a third region complementarity measure between the second feature primitive image and the third feature primitive image. Specifically, the first, second, and third region information abundance measure matrices are loaded, and for each pixel coordinate (x, y) (x is a horizontal index of 0-639, and y is a vertical index of 0-511), a first region complementarity measure is calculated. First, the absolute difference between the value of the first region information abundance measure at (x, y) and the value of the second region information abundance measure at (x, y) is calculated, and then divided by the maximum value of the two (if the maximum value is 0, the difference is directly set to 0), to obtain the first region complementarity value at this position. Using the same method, the absolute difference between the first region information abundance measure and the third region information abundance measure at (x, y) is divided by the maximum value of the two (set to 0 if the maximum value is 0), to obtain the second region complementarity value. The absolute difference between the second region information abundance measure and the third region information abundance measure at (x, y) is divided by the maximum value of the two (set to 0 if the maximum value is 0), to obtain the third region complementarity value. After traversing all pixel coordinates, the first region complementarity value is arranged according to the original coordinates to form a first region complementarity measure matrix consistent with the size of the original image. Similarly, the second and third region complementarity measure matrices are formed.

[0046] Optionally, the calculation of the first region complementarity measure between the first feature primitive image and the second feature primitive image, the second region complementarity measure between the first feature primitive image and the third feature primitive image, and the third region complementarity measure between the second feature primitive image and the third feature primitive image includes: A first gray difference entropy of a first target image block of the first feature primitive image and a second target image block of the second feature primitive image is calculated, and the first gray difference entropy is taken as the first region complementarity measure. The first target image block is any one of the first number of first image blocks, and the second target image block is an image block in the second number of second image blocks that is in the same spatial coordinate as the first target image block. A second gray difference entropy of the first target image block and a third target image block of the third feature primitive image is calculated, and the second gray difference entropy is taken as the second region complementarity measure. The third target image block is an image block in the third number of third image blocks that is in the same spatial coordinate range as the first target image block. A third gray difference entropy of the second target image block and the third target image block is calculated, and the third gray difference entropy is taken as the third region complementarity measure.

[0047] Specifically, when calculating the first region complementarity measure, first, select an arbitrary first target image block from the first feature primitive image, locate its spatial coordinate range in the image (such as the upper left corner coordinate (x1, y1) to the lower right corner coordinate (x2, y2)), and then select a second target image block from the second feature primitive image that completely coincides with the range; traverse all positionally identical pixel pairs in the two image blocks (i.e., the pixel at coordinate (i, j) in the first target image block and the pixel at coordinate (i, j) in the second target image block), calculate the difference in gray scale value of each pixel pair and take the absolute value to form a gray scale difference data set; count the frequency of occurrence of each difference value in the data set, divide the frequency by the total number of pixels in the image block (256) to obtain the occurrence probability of each difference value, and then pass through the formula: , H diff represents the gray scale difference entropy, k represents the index of the gray scale difference, and the value range is any integer in 1 to n (n is the total number of all possible gray scale differences, determined by the gray scale level range of the image, such as the gray scale difference range of a 16-bit image is 0-65535), p k represents the occurrence probability of the kth gray scale difference in the image block, and the first gray scale difference entropy is calculated, which is taken as the first region complementarity measure value of the corresponding spatial region of the first target image block and the second target image block.

[0048] Further, when calculating the second region complementarity measure and the third region complementarity measure, the same method as described above is used: for the second region complementarity measure, select a third target image block with the same spatial coordinate range as the first target image block, calculate the absolute value of the gray scale difference of the corresponding pixels and form a data set, and after frequency statistics, probability calculation, and entropy formula, the second gray scale difference entropy is obtained as the second region complementarity measure value; for the third region complementarity measure, take the second target image block and the third target image block as the object, calculate the absolute value of the gray scale difference of the corresponding pixels, and through the same process of frequency statistics, probability calculation, and entropy value, the third gray scale difference entropy is obtained as the third region complementarity measure value. After traversing all image blocks, arrange the measure values according to the original spatial coordinate position to form a first region complementarity measure, a second region complementarity measure, and a third region complementarity measure matrix with the same size as the feature primitive image.

[0049] Optionally, the calculation of the first gray scale difference entropy of the first target image block of the first feature primitive image and the second target image block of the second feature primitive image, and the taking of the first gray scale difference entropy as the first region complementarity measure comprise: calculating a difference value of a gray scale value of each pair of pixels in the first target image block and the second target image block and taking an absolute value to obtain a gray scale difference data set, the each pair of pixels including a first pixel in the first target image block and a second pixel in the second target image block, the first pixel being at the same position in the first target image block as the second pixel in the second target image block; counting a frequency of occurrence of each difference value in the gray scale difference data set, dividing the frequency of occurrence of each difference value by a total number of pixels of the first image block to obtain a probability of occurrence of each difference value; multiplying the probability of each difference value by a logarithm of the probability to obtain an intermediate value, and then adding all intermediate values to obtain a negative value, and the first gray scale difference entropy is obtained; determining the first gray scale difference entropy as the first regional complementarity metric between the first target image block and the second target image block.

[0050] Specifically, the imread function of OpenCV is called to load the image files of the first target image block and the second target image block respectively, and the image files are converted into two-dimensional arrays (the data type is uint16, and the pixel value range is 0-65535) through the NumPy library, wherein the array of the first target image block is denoted as arr1, and the array of the second target image block is denoted as arr2, and the dimensions of the two are both (16, 16), and the index (i, j) in the array corresponds to the pixel in the i-th row and the j-th column of the image block (i∈[0, 15], j∈[0, 15]); then, an empty list diff_list is defined to store the gray scale difference data, and a double-layer for loop is used to traverse all pixel coordinates: the outer loop variable i is from 0 to 15 (traverse the row), and the inner loop variable j is from 0 to 15 (traverse the column), in each loop, arr1[i, j] (the gray scale value of the pixel in the i-th row and the j-th column of the first target image block) and arr2[i, j] (the gray scale value of the pixel at the same position in the second target image block) are obtained through the index, the difference value of the two is calculated and the absolute value is taken, that is, diff=abs(arr1[i, j]-arr2[i, j]); the diff value obtained each time is added to diff_list in turn, and after the loop ends, diff_list will contain 256 elements (16×16=256), each element is an integer in the range of 0-65535; finally, the diff_list is converted into a one-dimensional array through the array function of the NumPy library, and the array is the gray scale difference data set.

[0051] Further, the generated gray-scale difference data set is obtained, the bincount function of the NumPy library is called to count the frequency of each possible difference value in the array (the minlength parameter is set to 65536), and a frequency array freq with a length of 65536 is obtained, where freq[d] represents the number of times the difference value d appears in the data set; the total number of pixels of the first image block is determined to be 256 (16x16), and each element in the frequency array freq is divided by 256 using the division operation of NumPy to obtain a probability array prob (data type is converted to float32), where prob[d]=freq[d] / 256, representing the probability of the difference value d appearing; all elements in prob greater than 0 and their corresponding difference values are filtered out through conditional judgment, and the results are stored as a tuple containing two lists (differences, probabilities), where differences records all the difference values that appear, and probabilities records the probabilities of the corresponding difference values; finally, it is verified whether the sum of all elements in probabilities is close to 1.0 to ensure the correctness of the probability statistics.

[0052] Further, the difference values and their corresponding probabilities that have been counted (stored as a data structure containing two arrays unique_diffs and probs, unique_diffs being the difference values that appear, and probs being the corresponding probabilities, i.e. the probability p of each difference value appearing) are obtained; the calculation method is as follows: for each non-zero probability p, first calculate the intermediate value mid=p x log2(p) (where p is the probability of a certain difference value appearing); initialize the cumulative variable sum_mid to 0.0, and use a for loop to iterate through all non-zero probabilities p: if p>0 (to avoid calculating the logarithm of 0 probability value, which would cause an error), then calculate mid and add it to sum_mid; after the loop ends, the first gray-scale difference entropy H=-sum_mid, which has a value range of 0 to 8; the H value is used as the first regional complementarity measure between the current first target image block and the second target image block, and is stored in the matrix position corresponding to the original image block spatial coordinates; it is verified whether H is within the range of 0≤H≤8; after traversing all image block pairs, the first regional complementarity measure matrix is obtained.

[0053] S206, processing the first regional information abundance measure, the second regional information abundance measure, the third regional information abundance measure, the first regional complementarity measure, the second regional complementarity measure and the third regional complementarity measure by a normalized exponential function to generate a fusion weight map; Specifically, the first, second, and third region information abundance metric matrices and the first, second, and third region complementarity metric matrices are loaded, and first, the comprehensive scores of the three feature primitive images are calculated respectively, and the comprehensive score of each feature is the product of the information abundance value of the feature and the sum of the complementarity values of the feature with the other two features; then, the three comprehensive scores are adjusted, and each score is subtracted by the maximum value in the three scores to obtain more stable adjusted scores; then, the weight of each feature is calculated by a normalization exponential function, and each weight represents the fusion proportion of the corresponding feature at the pixel position; finally, the three calculated weights are stored in a three-channel weight map according to the pixel position, and each channel corresponds to the weight of a feature, and is saved in a 32-bit floating-point multi-channel TIFF format, that is, a fusion weight map is obtained.

[0054] Optionally, the processing of the first region information abundance metric, the second region information abundance metric, the third region information abundance metric, the first region complementarity metric, the second region complementarity metric, and the third region complementarity metric by the normalization exponential function to generate the fusion weight map further includes: The first region information abundance metric, the first region complementarity metric, and the second region complementarity metric are weighted and fused to obtain a first comprehensive feature weight; the second region information abundance metric, the first region complementarity metric, and the third region complementarity metric are weighted and fused to obtain a second comprehensive feature weight; the third region information abundance metric, the second region complementarity metric, and the third region complementarity metric are weighted and fused to obtain a third comprehensive feature weight; the first comprehensive feature weight, the second comprehensive feature weight, and the third comprehensive feature weight are input into a normalization exponential function to calculate a first weight value matrix corresponding to the first comprehensive feature weight, a second weight value matrix corresponding to the second comprehensive feature weight, and a third weight value matrix corresponding to the third comprehensive feature weight; and a fusion weight map is generated according to the first weight value matrix, the second weight value matrix, and the third weight value matrix, and the fusion weight map has the same spatial size as the first feature primitive image, the second feature primitive image, and the third feature primitive image.

[0055] Specifically, load the first region information abundance metric matrix (A1), the second region information abundance metric matrix (A2), the third region information abundance metric matrix (A3), and the first region complementarity metric matrix (C12), the second region complementarity metric matrix (C13), and the third region complementarity metric matrix (C23), set three groups of fusion weights (such as 1 / 3), for each pixel coordinate (x, y) (x∈[0, 639], y∈[0, 511]), calculate the first comprehensive feature weight W1= (A1[x, y]×w_a+C12[x, y]×w_c1+C13[x, y]×w_c2), wherein w_a, w_c1, w_c2 are preset weighting coefficients; calculate the second comprehensive feature weight W2= (A2[x, y]×w_a+C12[x, y]×w_c1+C23[x, y]×w_c3), wherein w_a, w_c1, w_c3 are preset weighting coefficients; calculate the third comprehensive feature weight W3= (A3[x, y]×w_a+C13[x, y]×w_c2+C23[x, y]×w_c3), wherein w_a, w_c2, w_c3 are preset weighting coefficients; limit W1, W2, W3 in the range of 0-1 through the clip function of NumPy; after traversing all the pixels, store the three weights as matrices with the same size as the original image, that is, obtain the first, second, and third comprehensive feature weights.

[0056] Further, load the first comprehensive feature weight matrix (W1), the second comprehensive feature weight matrix (W2) and the third comprehensive feature weight matrix (W3); perform the following operations on each pixel coordinate (x, y) in the image (x∈[0, 639], y∈[0, 511]): extract the three comprehensive feature weight values, respectively denoted as w1=W1[x, y], w2=W2[x, y] and w3=W3[x, y] (where w1 represents the value of the first comprehensive feature weight at the pixel); first calculate the maximum value max_w among the three, and then adjust each weight value: w1'=w1-max_w, w2'=w2-max_w and w3'=w3-max_w; call the exp function to calculate the exponential value: exp1=exp(w1'), exp2=exp(w2') and exp3=exp(w3') (where exp represents the natural exponential function, which is used to amplify the difference between the adjusted weights, and exp1 represents the result of the natural exponential operation on the adjusted first comprehensive feature weight w1'); calculate the sum of the exponentials sum_exp=exp1+exp2+exp3; finally calculate the normalized weight value: the value of the first weight value matrix at (x, y) is w1_norm=exp1 / sum_exp, the value of the second weight value matrix at (x, y) is w2_norm=exp2 / sum_exp, and the value of the third weight value matrix at (x, y) is w3_norm=exp3 / sum_exp (where w1_norm, w2_norm and w3_norm are the first, second and third normalized weight values, respectively; after traversing all the pixels, store the three normalized weight values as 32-bit floating-point matrices with the same size as the original image, i.e., obtain the first weight value matrix, the second weight value matrix and the third weight value matrix.

[0057] Further, load the first, second and third weight value matrices (W1 norm, W2 norm, W3 norm); create an empty three-channel matrix fusion_weights with the size of 640x512x3 (heightxwidthxchannel number); for each pixel coordinate (x, y) (x∈[0, 639], y∈[0, 511]), assign the value W1 norm[x, y] of the first weight value matrix at this position to the corresponding position of the 0th channel of fusion_weights, that is, fusion_weights[x, y, 0]=W1 norm[x, y], assign the value W2 norm[x, y] of the second weight value matrix to the 1st channel, that is, fusion_weights[x, y, 1]=W2 norm[x, y], and assign the value W3 norm[x, y] of the third weight value matrix to the 2nd channel, that is, fusion_weights[x, y, 2]=W3 norm[x, y]; sum the three-channel values of each pixel position by using the sum function of NumPy; after traversing all pixels to complete the assignment, use the imwrite function of OpenCV to save the three-channel matrix fusion_weights in the 32-bit floating-point multi-channel TIFF format, that is, the fusion weight map.

[0058] S207, using the fusion weight map to perform weighted fusion on the first feature element image, the second feature element image and the third feature element image, and output a fusion image.

[0059] Specifically, load the fusion weight map and the first, second and third feature primitive images to ensure that all image spaces are completely matched in size; create an empty matrix fused_image with the same size as the feature primitive images to store the fusion result; for each pixel coordinate (x, y) (x∈[0, 639], y∈[0, 511]), extract the three-channel weight values at the corresponding position in the fusion weight map: w1=fusion_weights[x, y, 0] (weight of the first feature primitive image), w2=fusion_weights[x, y, 1] (weight of the second feature primitive image), w3=fusion_weights[x, y, 2] (weight of the third feature primitive image), and simultaneously extract the pixel values at the position in the three feature primitive images: p1=first feature primitive image[x, y], p2=second feature primitive image[x, y], p3=third feature primitive image[x, y]; calculate the fused pixel value by the weighted sum formula: fused_pixel=round(w1×p1+w2×p2+w3×p3), where the round function is used to convert the floating-point calculation result to an integer; use the clip function of NumPy to limit fused_pixel to the range of 0-65535 (consistent with the value range of a 16-bit image) to avoid pixel value overflow; assign the calculated fused_pixel to fused_image[x, y], and after completing the assignment of all pixels, save fused_image as a 16-bit single-channel TIFF format by using the imwrite function, which is the final output fusion image.

[0060] The embodiment also discloses an image fusion system based on infrared polarization imaging, Figure 3 is a module schematic diagram of an image fusion system based on infrared polarization imaging according to the embodiment of the application, as Figure 3 shown, the system comprises: an image data acquisition module 301 configured to acquire infrared polarization image data of a target scene; an image decomposition module 302 configured to decompose the infrared polarization image data into a first feature original primitive image, a second feature original primitive image and a third feature original primitive image based on a polarization bidirectional reflectance distribution function; a task processing module 303 configured to perform feature optimization processing on the first feature original primitive image, the second feature original primitive image and the third feature original primitive image respectively according to a preset task mode, to obtain a first feature primitive image, a second feature primitive image and a third feature primitive image; The information abundance metric calculation module 304 is configured to calculate a first regional information abundance metric of the first feature primitive image, a second regional information abundance metric of the second feature primitive image, and a third regional information abundance metric of the third feature primitive image, respectively. The complementarity metric calculation module 305 is configured to calculate a first regional complementarity metric between the first feature primitive image and the second feature primitive image, a second regional complementarity metric between the first feature primitive image and the third feature primitive image, and a third regional complementarity metric between the second feature primitive image and the third feature primitive image. The fusion weight map generation module 306 is configured to process the first regional information abundance metric, the second regional information abundance metric, the third regional information abundance metric, the first regional complementarity metric, the second regional complementarity metric, and the third regional complementarity metric by a normalized exponential function to generate a fusion weight map. The fused image output module 307 is configured to perform weighted fusion on the first feature primitive image, the second feature primitive image, and the third feature primitive image by using the fusion weight map, and output a fused image.

[0061] Optionally, the image decomposition module 302 is specifically configured to: establish a physical decomposition model based on the polarized bidirectional reflectance distribution function, extract total radiation intensity information from the infrared polarized image data according to the physical decomposition model, construct the first feature original primitive image according to the total radiation intensity information, extract a linear polarization degree parameter from linear polarization state component information of the infrared polarized image data according to the physical decomposition model, and construct the second feature original primitive image according to the linear polarization degree parameter, and extract a polarization angle parameter from the linear polarization state component information according to the physical decomposition model, and construct the third feature original primitive image according to the polarization angle parameter.

[0062] Optionally, the task processing module 303 is specifically configured to: When the preset task mode is a target detection mode, a mapping relationship between pixel original gray scale values and target gray scale values in the first feature original base element image is established, a gray scale value of a target region is adjusted according to the mapping relationship, and the first feature base element image is obtained; the second feature original base element image is decomposed into a plurality of feature layers with different spatial resolutions according to a preset scale parameter, edge features and texture features are extracted from each of the feature layers, the edge features and the texture features are superimposed and combined according to a preset weight ratio, and the second feature base element image is obtained; noise pixels in the third feature original base element image are identified through a preset noise judgment condition, all non-noise pixels in a preset range around the noise pixels are selected, statistical values of all the non-noise pixels are calculated, and pixel values of the noise pixels are replaced by the statistical values, and the third feature base element image is obtained.

[0063] Optionally, the information abundance metric calculation module 304 is specifically configured to: The first feature base element image, the second feature base element image and the third feature base element image are divided into a plurality of image blocks according to a preset size, a first number of first image blocks of the first feature base element image, a second number of second image blocks of the second feature base element image and a third number of third image blocks of the third feature base element image are obtained; a first weighted sum is obtained by weighted summing a gray scale entropy value and a gradient amplitude mean value of the first image block, the first number of the first weighted sums are arranged according to spatial coordinate positions of the first image blocks in the first feature base element image to obtain the first region information abundance metric, the gray scale entropy value represents a complexity of gray scale distribution in an image block, and the gradient amplitude mean value represents a strength of edge features in the image block; a second weighted sum is obtained by weighted summing a gray scale entropy value and a gradient amplitude mean value of the second image block, the second number of the second weighted sums are arranged according to spatial coordinate positions of the second image blocks in the second feature base element image to obtain the second region information abundance metric; a third weighted sum is obtained by weighted summing a gray scale entropy value and a gradient amplitude mean value of the third target image block, and the third number of the third weighted sums are arranged according to spatial coordinate positions of the third image blocks in the third feature base element image to obtain the third region information abundance metric.

[0064] Optionally, the complementarity metric calculation module 305 is specifically configured to: calculating a first gray difference entropy of the first target image block and a second target image block of the second feature base image, the first gray difference entropy being taken as the first regional complementarity measure, the first target image block being any one of the first image blocks, and the second target image block being an image block of the second image blocks which is in the same spatial coordinate as the first target image block; calculating a second gray difference entropy of the first target image block and a third target image block of the third feature base image, the second gray difference entropy being taken as the second regional complementarity measure, the third target image block being an image block of the third image blocks which is in the same spatial coordinate range as the first target image block; and calculating a third gray difference entropy of the second target image block and the third target image block, the third gray difference entropy being taken as the third regional complementarity measure.

[0065] Optionally, the complementarity measure calculation module 305 is specifically configured to: calculating a difference value of the gray values of each pair of pixels in the first target image block and the second target image block and taking an absolute value to obtain a gray difference data set, the each pair of pixels including a first pixel in the first target image block and a second pixel in the second target image block, the first pixel being in the same position in the first target image block as the second pixel in the second target image block; counting a frequency of occurrence of each difference value in the gray difference data set, dividing the frequency of occurrence of each difference value by a total number of pixels of the first image block to obtain a probability of occurrence of each difference value; multiplying the probability of each difference value by a logarithm of the probability to obtain an intermediate value, and then adding all the intermediate values to obtain the first gray difference entropy; and determining the first gray difference entropy as the first regional complementarity measure between the first target image block and the second target image block.

[0066] Optionally, the fusion weight map generation module 306 is specifically configured to: The first region information abundance metric, the first region complementarity metric and the second region complementarity metric are weighted and fused to obtain a first comprehensive feature weight; the second region information abundance metric, the first region complementarity metric and the third region complementarity metric are weighted and fused to obtain a second comprehensive feature weight; the third region information abundance metric, the second region complementarity metric and the third region complementarity metric are weighted and fused to obtain a third comprehensive feature weight; the first comprehensive feature weight, the second comprehensive feature weight and the third comprehensive feature weight are input into a normalization exponential function to obtain a first weight value matrix corresponding to the first comprehensive feature weight, a second weight value matrix corresponding to the second comprehensive feature weight and a third weight value matrix corresponding to the third comprehensive feature weight; and a fusion weight map is generated according to the first weight value matrix, the second weight value matrix and the third weight value matrix, wherein the fusion weight map has the same spatial size as the first feature primitive image, the second feature primitive image and the third feature primitive image.

[0067] The embodiment also discloses an electronic device, referring to Figure 4 The electronic device can include at least one processor 401, at least one communication bus 402, a user interface 403, a network interface 404, and at least one memory 405. The communication bus 402 is used to realize the connection and communication between the components. The user interface 403 can include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 can also include a standard wired interface and a wireless interface. The network interface 404 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The processor 401 can include one or more processing cores. The processor 401 connects various parts of the server through various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the processor 401 can be realized in at least one of the hardware forms of a digital signal processing (Digital Signal Processing, DSP), a field programmable gate array (Field-Programmable Gate Array, FPGA), and a programmable logic array (Programmable Logic Array, PLA). The processor 401 can be integrated with a combination of one or more of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem.

[0068] In Figure 4 In the electronic device shown, the user interface 403 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 401 can be used to call an application program stored in the memory 405, which is an image fusion method based on infrared polarization imaging, and when executed by one or more processors 401, causes the electronic device to perform the method of one or more of the above embodiments.

Claims

1. An image fusion method based on infrared polarization imaging, characterized in that, Applied to a server, the method comprises: acquiring infrared polarization image data of a target scene; based on a polarization bidirectional reflectance distribution function, decomposing the infrared polarization image data into a first characteristic raw primitive image, a second characteristic raw primitive image and a third characteristic raw primitive image; according to a preset task mode, performing feature optimization processing on the first characteristic raw primitive image, the second characteristic raw primitive image and the third characteristic raw primitive image respectively to obtain a first characteristic primitive image, a second characteristic primitive image and a third characteristic primitive image; calculating a first regional information abundance metric of the first characteristic primitive image, a second regional information abundance metric of the second characteristic primitive image and a third regional information abundance metric of the third characteristic primitive image respectively; calculating a first regional complementarity metric between the first characteristic primitive image and the second characteristic primitive image, a second regional complementarity metric between the first characteristic primitive image and the third characteristic primitive image, and a third regional complementarity metric between the second characteristic primitive image and the third characteristic primitive image; generating a fusion weight map based on the first regional information abundance metric, the second regional information abundance metric and the third regional information abundance metric and the first regional complementarity metric, the second regional complementarity metric and the third regional complementarity metric through a normalization exponential function; performing weighted fusion on the first characteristic primitive image, the second characteristic primitive image and the third characteristic primitive image using the fusion weight map to output a fused image.

2. The method of claim 1, wherein, The method based on the polarization bidirectional reflectance distribution function to decompose the infrared polarization image data into a first characteristic raw primitive image, a second characteristic raw primitive image and a third characteristic raw primitive image comprises: establishing a physical decomposition model through the polarization bidirectional reflectance distribution function; extracting total radiation intensity information from the infrared polarization image data according to the physical decomposition model, and constructing the first characteristic raw primitive image according to the total radiation intensity information; extracting a linear polarization degree parameter from linear polarization state component information of the infrared polarization image data according to the physical decomposition model, and constructing the second characteristic raw primitive image according to the linear polarization degree parameter; extracting a polarization angle parameter from the linear polarization state component information according to the physical decomposition model, and constructing the third characteristic raw primitive image according to the polarization angle parameter.

3. The method of claim 1, wherein, The method according to the preset task mode, performing feature optimization processing on the first characteristic raw primitive image, the second characteristic raw primitive image and the third characteristic raw primitive image respectively to obtain a first characteristic primitive image, a second characteristic primitive image and a third characteristic primitive image comprises: according to the preset task mode, establishing a mapping relationship between a pixel raw gray value in the first characteristic raw primitive image and a target gray value, adjusting the gray value of a target region according to the mapping relationship to obtain the first characteristic primitive image; the second feature primitive image is decomposed into a plurality of feature map layers with different spatial resolutions according to preset scale parameters, edge features and texture features are extracted from each of the feature map layers, the edge features and the texture features are superimposed and combined according to a preset weight ratio, and the second feature primitive image is obtained; noise pixels in the third feature primitive image are identified through a preset noise judgment condition, all non-noise pixels in a preset range around the noise pixels are selected, statistical values of all the non-noise pixels are calculated, and pixel values of the noise pixels are replaced by the statistical values, and the third feature primitive image is obtained.

4. The method of claim 1, wherein, the first region information abundance metric of the first feature primitive image, the second region information abundance metric of the second feature primitive image, and the third region information abundance metric of the third feature primitive image are calculated respectively, including: the first feature primitive image, the second feature primitive image, and the third feature primitive image are divided into a plurality of image blocks according to a preset size, and a first number of first image blocks of the first feature primitive image, a second number of second image blocks of the second feature primitive image, and a third number of third image blocks of the third feature primitive image are obtained; a first weighted sum is obtained by weighted summing a gray entropy value and a gradient amplitude mean value of the first image block, the first region information abundance metric is obtained by arranging the first number of the first weighted sums according to spatial coordinate positions of the first image blocks in the first feature primitive image, the gray entropy value represents a complexity of a gray distribution in an image block, and the gradient amplitude mean value represents a strength of an edge feature in the image block; a second weighted sum is obtained by weighted summing a gray entropy value and a gradient amplitude mean value of the second image block, and the second region information abundance metric is obtained by arranging the second number of the second weighted sums according to spatial coordinate positions of the second image blocks in the second feature primitive image; a third weighted sum is obtained by weighted summing a gray entropy value and a gradient amplitude mean value of the third target image block, and the third region information abundance metric is obtained by arranging the third number of the third weighted sums according to spatial coordinate positions of the third image blocks in the third feature primitive image.

5. The method of claim 4, wherein, the first region complementarity metric between the first feature primitive image and the second feature primitive image, the second region complementarity metric between the first feature primitive image and the third feature primitive image, and the third region complementarity metric between the second feature primitive image and the third feature primitive image are calculated, including: a first gray difference entropy of a first target image block of the first feature primitive image and a second target image block of the second feature primitive image is calculated, the first gray difference entropy is taken as the first region complementarity metric, the first target image block is any one of the first number of the first image blocks, and the second target image block is an image block of the second number of the second image blocks which is at the same spatial coordinate as the first target image block; calculate a second gray difference entropy of the first target image block and a third target image block of the third feature primitive image, the third target image block being an image block of the third number of third image blocks in the same spatial coordinate range as the first target image block, and take the second gray difference entropy as the second regional complementarity measure; calculate a third gray difference entropy of the second target image block and the third target image block, and take the third gray difference entropy as the third regional complementarity measure.

6. The method of claim 5, wherein, The calculating a first gray difference entropy of a first target image block of the first feature primitive image and a second target image block of the second feature primitive image, and taking the first gray difference entropy as the first regional complementarity measure comprises: calculating a difference value between gray values of each pair of pixels in the first target image block and the second target image block and taking an absolute value to obtain a gray difference data set, the each pair of pixels comprising a first pixel in the first target image block and a second pixel in the second target image block, the first pixel being in the same position in the first target image block as the second pixel in the second target image block; counting a frequency of occurrence of each difference value in the gray difference data set, and dividing the frequency of each difference value by a total number of pixels of the first image block to obtain a probability of occurrence of each difference value; multiplying the probability of each difference value by a logarithm of the probability to obtain an intermediate value, and then adding all intermediate values and taking a negative value to obtain the first gray difference entropy; determining the first gray difference entropy as the first regional complementarity measure between the first target image block and the second target image block.

7. The method of claim 1, wherein, The processing the first regional information abundance measure, the second regional information abundance measure, the third regional information abundance measure, the first regional complementarity measure, the second regional complementarity measure, and the third regional complementarity measure by a normalized exponential function to generate a fusion weight map further comprises: weighting fusion of the first regional information abundance measure, the first regional complementarity measure, and the second regional complementarity measure to obtain a first comprehensive feature weight; weighting fusion of the second regional information abundance measure, the first regional complementarity measure, and the third regional complementarity measure to obtain a second comprehensive feature weight; weighting fusion of the third regional information abundance measure, the second regional complementarity measure, and the third regional complementarity measure to obtain a third comprehensive feature weight; inputting the first comprehensive feature weight, the second comprehensive feature weight, and the third comprehensive feature weight into a normalized exponential function to calculate a first weight value matrix corresponding to the first comprehensive feature weight, a second weight value matrix corresponding to the second comprehensive feature weight, and a third weight value matrix corresponding to the third comprehensive feature weight; A fusion weight map is generated according to the first weight value matrix, the second weight value matrix and the third weight value matrix, and the fusion weight map has the same spatial size as the first feature primitive image, the second feature primitive image and the third feature primitive image.

8. An image fusion system based on infrared polarimetric imaging, characterized in that, Specifically comprising: An image data acquisition module is configured to acquire infrared polarization image data of a target scene; An image decomposition module is configured to decompose the infrared polarization image data into a first feature raw primitive image, a second feature raw primitive image and a third feature raw primitive image based on a polarization bidirectional reflectance distribution function; A task processing module is configured to perform feature optimization processing on the first feature raw primitive image, the second feature raw primitive image and the third feature raw primitive image respectively according to a preset task mode, to obtain a first feature primitive image, a second feature primitive image and a third feature primitive image; An information abundance metric calculation module is configured to calculate a first regional information abundance metric of the first feature primitive image, a second regional information abundance metric of the second feature primitive image and a third regional information abundance metric of the third feature primitive image respectively; A complementarity metric calculation module is configured to calculate a first regional complementarity metric between the first feature primitive image and the second feature primitive image, a second regional complementarity metric between the first feature primitive image and the third feature primitive image, and a third regional complementarity metric between the second feature primitive image and the third feature primitive image; A fusion weight map generation module is configured to process the first regional information abundance metric, the second regional information abundance metric, the third regional information abundance metric, the first regional complementarity metric, the second regional complementarity metric and the third regional complementarity metric by a normalized exponential function to generate a fusion weight map; A fused image output module is configured to perform weighted fusion on the first feature primitive image, the second feature primitive image and the third feature primitive image by using the fusion weight map, and output a fused image.

9. An electronic device, comprising: The electronic device includes a processor, a memory, a user interface and a network interface, the memory is configured to store instructions, the user interface and the network interface are configured to communicate with other devices, and the processor is configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed, perform the method of any one of claims 1-7.

Images