A method and system for day and night radiometric imaging fusion and switching based on dual spectra

Abstract

This invention provides a method and system for day-night radiation imaging fusion and switching based on dual-spectrum imaging, belonging to the field of image processing technology. The method includes extracting visible light brightness features and thermal infrared temperature gradient features, analyzing the spatial consistency of radiation intensity and gradient direction, determining the day / night time period type and the degree of scene dynamic change, performing feature domain transformation, constructing a cross-spectral feature alignment mapping relationship, performing spatial registration and semantic correspondence in the fused feature space, and determining mode switching and output based on a synergy index. This invention achieves efficient fusion and intelligent switching of dual-spectral images, improving imaging quality and stability in complex environments.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for day and night radiometric imaging fusion and switching based on dual spectra. Background Technology

[0002] Dual-spectral imaging is an advanced detection method that combines visible light imaging with thermal infrared imaging. By simultaneously acquiring visible light and thermal infrared radiation information of a scene, it can provide more comprehensive target information under different environmental conditions. Visible light imaging can provide rich detail and texture information in well-lit daytime environments, while thermal infrared imaging can capture the thermal radiation characteristics of targets at night or in low-light conditions. This complementarity makes dual-spectral imaging technology a promising field for applications in military reconnaissance, security monitoring, night vision driving, and infrared remote sensing.

[0003] Traditional dual-spectral image fusion methods typically employ fixed-weight fusion algorithms, which cannot adaptively adjust the fusion strategy according to changes in the day-night environment. This results in poor fusion performance in scenes with significant lighting variations, especially during day-night transitions, where a smooth transition cannot be achieved, impacting system stability and reliability. Existing feature extraction and matching methods perform poorly when processing cross-spectral images, struggling to establish accurate feature correspondences between visible and infrared images. This is because images from different spectra exhibit significant differences in physical properties, making it difficult for traditional pixel-level or feature-level fusion methods to effectively utilize the complementary information of the two modalities. This is particularly problematic in complex environments, where artifacts or detail loss are easily generated. Existing dual-spectral imaging systems lack intelligent mode-switching mechanisms, failing to automatically select the optimal operating mode based on dynamic scene changes. In practical applications, when environmental conditions or target characteristics change, the system cannot adjust its output strategy in a timely manner, often requiring manual intervention. This significantly reduces the system's automation and adaptability, limiting its effectiveness in complex and changing environments. Summary of the Invention

[0004] The embodiments of the present invention provide a method and system for day and night radiation imaging fusion and switching based on dual spectrum, which can solve the problems in the prior art.

[0005] A first aspect of this invention provides a method for day-night radiometric imaging fusion and switching based on dual spectra, comprising:

[0006] Extract the brightness distribution features of visible light radiation images and the temperature gradient distribution features of thermal infrared radiation images;

[0007] Based on the brightness distribution characteristics and the temperature gradient distribution characteristics, the spatial consistency analysis of radiation intensity and gradient direction is used to determine the day and night time period type and the degree of scene dynamic change of the current imaging environment, and the day and night environment discrimination result is obtained.

[0008] Based on the day and night environment discrimination results, feature domain transformations are performed on the visible light radiation image and the thermal infrared radiation image respectively to obtain visible light feature domain characterization and thermal infrared feature domain characterization.

[0009] Based on the day and night time period type, by constructing a cross-spectral feature alignment mapping relationship, the visible light feature domain representation and the thermal infrared feature domain representation are projected onto a unified fusion feature space, and spatial registration and semantic correspondence are performed in the fusion feature space. Then, the fusion radiation image is reconstructed through inverse transformation.

[0010] Based on the day / night time period type and the degree of dynamic change of the scene, the overlap and mutual information of the feature distribution of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space are calculated to obtain the coordination index. The coordination index is used as the criterion to determine the mode switching and image output.

[0011] Extracting the brightness distribution features of visible light radiation images and the temperature gradient distribution features of thermal infrared radiation images includes:

[0012] The visible light radiation image is transformed into a color space, converting it from the original color space to a target color space that includes a luminance component. The luminance component is separated from the target color space to obtain a luminance channel image. The luminance channel image is then decomposed into multiple scale-level luminance sub-images. The statistical distribution parameters of each luminance sub-image are calculated to obtain the luminance distribution characteristics.

[0013] The thermal infrared radiation image is subjected to a nonlinear mapping transformation between radiation intensity and temperature. Based on the nonlinear mapping transformation, the pixel grayscale values ​​of the thermal infrared radiation image are converted into temperature physical quantities to obtain the temperature field distribution. The gradient of the temperature field distribution is calculated in the horizontal and vertical directions to obtain the horizontal gradient component and the vertical gradient component. Based on the horizontal gradient component and the vertical gradient component, a gradient amplitude field and a gradient direction field are constructed to obtain the temperature gradient distribution characteristics.

[0014] By analyzing the spatial consistency between radiation intensity and gradient direction, the day / night time type and the degree of scene dynamic change in the current imaging environment are determined, and the day / night environment discrimination results are obtained, including:

[0015] The global brightness mean and local brightness variance are extracted from the brightness distribution characteristics. The light intensity level is determined based on the comparison between the global brightness mean and the preset brightness level interval. The light distribution uniformity index is determined based on the spatial distribution uniformity of the local brightness variance.

[0016] The gradient direction field is extracted from the temperature gradient distribution features. The gradient direction angle of each pixel in the gradient direction field is divided into intervals. The number of pixels in each interval is counted to obtain a direction distribution histogram. The direction distribution histogram is subjected to main direction clustering analysis to obtain the set of main directions of the temperature gradient.

[0017] The spatial location of the local brightness variance is matched with the spatial location of the gradient amplitude field at the pixel level. The angle deviation between the brightness change direction and the gradient direction at the matching location is calculated. The percentage of pixels whose angle deviation is less than a preset consistency angle threshold is counted to obtain the spatial consistency index.

[0018] Based on the light intensity level and the light distribution uniformity index, the day and night time period type is determined. Based on the spatial consistency index and the number of principal directions in the set of principal directions of the temperature gradient, the degree of dynamic change of the scene is determined, and the day and night environment discrimination result is obtained.

[0019] Based on the day / night environment discrimination result, feature domain transformations are performed on the visible light radiation image and the thermal infrared radiation image respectively to obtain visible light feature domain representation and thermal infrared feature domain representation, including:

[0020] Spectral analysis is performed on the visible light radiation image to obtain the energy distribution concentration of each frequency component and to fuse it with the light intensity level to obtain the frequency domain decomposition level parameters; the global gradient amplitude distribution of the thermal infrared radiation image is calculated, the pixel density with gradient amplitude exceeding a preset response threshold is counted to obtain the spatial gradient response activity index and to perform nonlinear mapping operation with the scene dynamic change degree to obtain the spatial filtering scale parameters.

[0021] According to the frequency domain decomposition level parameters, the visible light radiation image is subjected to multi-level spectral transformation. At each decomposition level, the image signal is decomposed into multiple coefficient components to obtain visible light frequency domain coefficients of multiple frequency sub-bands. Texture descriptors are extracted from each of the visible light frequency domain coefficients to obtain the visible light feature domain characterization.

[0022] Based on the spatial filtering scale parameters, multiple spatial filtering kernels of different scales are constructed. Each spatial filtering kernel is convolved with the thermal infrared radiation image to obtain thermal infrared filtering response images of multiple scale levels. Edge responders are extracted from each thermal infrared filtering response image to obtain the thermal infrared feature domain characterization.

[0023] Based on the day / night time period type, by constructing a cross-spectral feature alignment mapping relationship, the visible light feature domain representation and the thermal infrared feature domain representation are projected onto a unified fused feature space, including:

[0024] The dominant spectral type is determined from the day and night time period types. Based on the dominant spectral type, a corresponding feature space reference coordinate system is selected. The visible light feature domain representation is projected onto the feature space reference coordinate system to obtain the dimension-aligned visible light feature domain representation.

[0025] Calculate the Euclidean distance between each edge responder and each texture descriptor, and select edge responders and texture descriptors with an Euclidean distance less than a preset alignment distance threshold as spatially corresponding feature pairs. Calculate the intensity difference between the response intensity value of the edge responder and the texture intensity value of the texture descriptor in each spatially corresponding feature pair, construct an intensity mapping coefficient based on the intensity difference, and use the intensity mapping coefficient to perform amplitude scaling transformation on the response intensity value of each edge responder in the thermal infrared feature domain representation to obtain the amplitude-aligned thermal infrared feature domain representation.

[0026] The visible light feature domain characterization after dimensional alignment and the thermal infrared feature domain characterization after amplitude alignment are mapped in the feature space reference coordinate system.

[0027] The process involves spatial registration and semantic correspondence within the fused feature space, followed by inverse transformation to reconstruct a fused radiometric image, including:

[0028] Extract the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation and the spatial coordinates of each feature point in the amplitude-aligned thermal infrared feature domain representation, calculate the positional deviation, construct a spatial transformation matrix based on the positional deviation, and use the spatial transformation matrix to perform coordinate correction transformation on the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation to obtain a spatially registered fused feature set.

[0029] Semantic category labeling is performed on each feature point in the spatially registered fusion feature set. Feature points with the same semantic category are divided into the same semantic region. The number of feature points represented by the visible light feature domain after dimension alignment and the number of feature points represented by the thermal infrared feature domain after amplitude alignment are counted in each semantic region. The ratio of the number of feature points of the two types is calculated. The spectral contribution weight of each semantic region is determined according to the ratio. The feature points in each semantic region are fused according to the spectral contribution weight to obtain the semantically corresponding fusion feature representation.

[0030] The fused feature representation after semantic correspondence is subjected to inverse feature domain transformation and mapped back to the image pixel space to obtain the fused radiative image.

[0031] Based on the day / night time period type and the degree of dynamic change of the scene, the calculation of the feature distribution overlap and mutual information of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space includes:

[0032] The illumination stability coefficient is determined based on the day / night time period type, the motion disturbance coefficient is determined based on the dynamic change degree of the scene, and the illumination stability coefficient and the motion disturbance coefficient are combined to obtain the environmental correction factor.

[0033] The frequency of occurrence of each feature value in the fused feature space is statistically analyzed for the visible light feature domain characterization and the thermal infrared feature domain characterization, respectively, to construct the visible light feature distribution and the thermal infrared feature distribution. The area of ​​the intersection region and the area of ​​the union region of the visible light feature distribution and the thermal infrared feature distribution are calculated and the ratio is obtained to obtain the initial overlap. The initial overlap is corrected using the environmental correction factor to obtain the feature distribution overlap.

[0034] The visible light feature distribution and the thermal infrared feature distribution are subjected to joint probability distribution calculation to obtain a dual-spectral joint distribution. The joint entropy of the dual-spectral joint distribution is calculated. The edge entropy of the visible light feature distribution and the thermal infrared feature distribution are calculated respectively and summed to obtain the entropy sum value. The difference between the entropy sum value and the joint entropy is calculated to obtain the initial mutual information. The initial mutual information is corrected using the environmental correction factor to obtain the mutual information.

[0035] A second aspect of the present invention provides a dual-spectrum day-night radiometric imaging fusion and switching system, comprising:

[0036] The first unit is used to extract the brightness distribution features of visible light radiation images and the temperature gradient distribution features of thermal infrared radiation images.

[0037] The second unit is used to determine the day / night time type and the degree of scene dynamic change of the current imaging environment based on the brightness distribution characteristics and the temperature gradient distribution characteristics, through spatial consistency analysis of radiation intensity and gradient direction, and to obtain the day / night environment discrimination result.

[0038] The third unit is used to perform feature domain transformation on the visible light radiation image and the thermal infrared radiation image respectively based on the day and night environment discrimination result, so as to obtain the visible light feature domain characterization and the thermal infrared feature domain characterization.

[0039] The fourth unit is used to project the visible light feature domain representation and the thermal infrared feature domain representation onto a unified fusion feature space by constructing a cross-spectral feature alignment mapping relationship based on the day and night time period type, and to perform spatial registration and semantic correspondence in the fusion feature space, and then reconstruct a fusion radiometric image through inverse transformation;

[0040] The fifth unit is used to calculate the overlap and mutual information of the feature distributions of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space based on the day / night time period type and the degree of scene dynamic change, to obtain a coordination index, and to determine the mode switching and image output based on the coordination index.

[0041] A third aspect of the embodiments of the present invention,

[0042] An electronic device is provided, comprising:

[0043] processor;

[0044] Memory used to store processor-executable instructions;

[0045] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0046] Fourth aspect of the present invention,

[0047] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0048] The beneficial effects of this application are as follows:

[0049] The dual-spectrum day-night radiation imaging fusion and switching method provided by this invention can achieve intelligent fusion and switching of visible light and thermal infrared images under different day and night conditions through feature extraction and environmental discrimination, effectively solving the problem of unstable imaging quality of traditional imaging methods in complex environments.

[0050] This method utilizes cross-spectral feature alignment mapping technology to project visible light and thermal infrared images onto a unified feature space, achieving spatial registration and semantic correspondence. This ensures that the fused image enhances detail while preserving the original image information, thereby improving the adaptability and robustness of the imaging system under different lighting conditions.

[0051] The mode switching mechanism based on the coordination index enables the system to automatically select the optimal imaging mode according to scene changes, reducing the need for manual intervention, improving system operating efficiency and image quality, and providing more reliable visual information support for monitoring, recognition and other applications in various complex environments. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the day-night radiometric imaging fusion and switching method based on dual spectra according to an embodiment of the present invention.

[0053] Figure 2 This is a schematic diagram of the process for calculating the synergy of dual-spectral features. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0056] Figure 1 This is a flowchart illustrating the day-night radiometric imaging fusion and switching method based on dual spectra according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0057] Extract the brightness distribution features of visible light radiation images and the temperature gradient distribution features of thermal infrared radiation images;

[0058] Based on the brightness distribution characteristics and the temperature gradient distribution characteristics, the spatial consistency analysis of radiation intensity and gradient direction is used to determine the day and night time period type and the degree of scene dynamic change of the current imaging environment, and the day and night environment discrimination result is obtained.

[0059] Based on the day and night environment discrimination results, feature domain transformations are performed on the visible light radiation image and the thermal infrared radiation image respectively to obtain visible light feature domain characterization and thermal infrared feature domain characterization.

[0060] Based on the day and night time period type, by constructing a cross-spectral feature alignment mapping relationship, the visible light feature domain representation and the thermal infrared feature domain representation are projected onto a unified fusion feature space, and spatial registration and semantic correspondence are performed in the fusion feature space. Then, the fusion radiation image is reconstructed through inverse transformation.

[0061] Based on the day / night time period type and the degree of dynamic change of the scene, the overlap and mutual information of the feature distribution of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space are calculated to obtain the coordination index. The coordination index is used as the criterion to determine the mode switching and image output.

[0062] In one optional implementation, extracting the brightness distribution features of the visible light radiation image and the temperature gradient distribution features of the thermal infrared radiation image includes:

[0063] The visible light radiation image is transformed into a color space, converting it from the original color space to a target color space that includes a luminance component. The luminance component is separated from the target color space to obtain a luminance channel image. The luminance channel image is then decomposed into multiple scale-level luminance sub-images. The statistical distribution parameters of each luminance sub-image are calculated to obtain the luminance distribution characteristics.

[0064] The thermal infrared radiation image is subjected to a nonlinear mapping transformation between radiation intensity and temperature. Based on the nonlinear mapping transformation, the pixel grayscale values ​​of the thermal infrared radiation image are converted into temperature physical quantities to obtain the temperature field distribution. The gradient of the temperature field distribution is calculated in the horizontal and vertical directions to obtain the horizontal gradient component and the vertical gradient component. Based on the horizontal gradient component and the vertical gradient component, a gradient amplitude field and a gradient direction field are constructed to obtain the temperature gradient distribution characteristics.

[0065] The system receives a visible light radiation image and a corresponding thermal infrared radiation image of the region as input. For the visible light radiation image, a color space transformation operation is performed. Assuming the input visible light image is in RGB format, it is converted to the Lab color space, which includes a luminance component (L) and chromaticity components (a and b). A color space conversion algorithm is used to non-linearly map the RGB values ​​of each pixel to obtain the corresponding Lab value. For example, a pixel with RGB values ​​of (210, 150, 80) will have a Lab value of (65, 12, 45) after color space conversion.

[0066] From the converted Lab image, the luminance component L is separated to form a single-channel luminance image. The pixel values ​​of this luminance image typically range from 0 to 100, where 0 represents black and 100 represents white. To extract multi-scale features, a Gaussian pyramid or Laplacian pyramid method is used to perform multi-scale spatial decomposition on the luminance channel image. Taking the Gaussian pyramid as an example, if the original luminance image is 512×512 pixels, it is first Gaussian filtered and downsampled by a factor of 2 to obtain a first-level sub-image of 256×256 pixels. Then, the same operation is performed on the first-level sub-image to obtain a second-level sub-image of 128×128 pixels. This process continues until a sub-image with a minimum size of 32×32 pixels is obtained, resulting in a total of 5 luminance sub-images at different scales.

[0067] For each scale's brightness sub-image, statistical distribution parameters are calculated to characterize the brightness distribution features. These parameters include mean, standard deviation, histogram distribution, entropy, and energy. For example, for the first-level 256×256 pixel sub-image, the calculated brightness mean is 68.5, standard deviation is 15.3, and entropy is 6.8. For the second-level 128×128 pixel sub-image, the calculated brightness mean is 65.2, standard deviation is 12.7, and entropy is 6.2. These statistical parameters from each scale's sub-image are combined to form a feature vector, representing the brightness distribution features. For example, a complete feature vector contains 25 parameters, comprehensively characterizing the distribution of image brightness at different scales.

[0068] For thermal infrared radiation images, a nonlinear mapping transformation between radiation intensity and temperature is performed. The original pixel values ​​of the thermal infrared image represent the radiation intensity received by the detector and need to be converted into actual temperature values ​​for effective analysis. A mapping relationship is established based on the calibration parameters of the thermal infrared camera. For example, for a pixel with a grayscale value of 125, the corresponding temperature value after nonlinear mapping transformation is 35.6℃; for a pixel with a grayscale value of 180, the corresponding temperature value is 42.8℃. This mapping typically follows a modified Planck's law of radiation and considers atmospheric attenuation and equipment parameters.

[0069] Through the above transformation, a complete temperature field distribution map is obtained. Each pixel value in the map represents the actual temperature. On the temperature field distribution map, the temperature gradients in the horizontal and vertical directions are calculated respectively. The horizontal gradient is calculated using the horizontal difference operator. For example, for the pixel at coordinates (100, 150) in the temperature field, its value is 38.2℃, while the value of its right-hand neighbor (101, 150) is 38.5℃. Then the horizontal gradient component of this point is 0.3℃ / pixel. The vertical gradient is calculated using the vertical difference operator. For example, the value of the neighboring point below this point (100, 151) is 37.9℃. Then the vertical gradient component is -0.3℃ / pixel.

[0070] Based on the calculated horizontal and vertical gradient components, a gradient magnitude field and a gradient direction field are constructed. The gradient magnitude is calculated by taking the square root of the sum of the squares of the horizontal and vertical components. For the point in the example above, the gradient magnitude is 0.42℃ / pixel. The gradient direction is calculated by taking the arctangent of the horizontal and vertical components, representing the direction angle of temperature change; the gradient direction angle is 315 degrees. Statistical analysis is performed on the gradient magnitude and gradient direction fields to extract features such as gradient magnitude histograms, direction histograms, and principal gradient directions. For example, the average temperature gradient magnitude is 0.35℃ / pixel, the standard deviation is 0.12℃ / pixel, and the principal gradient direction is 225 degrees. These features collectively constitute the temperature gradient distribution characteristics, used to characterize the temperature change patterns in thermal infrared images.

[0071] Through the above steps, the brightness distribution characteristics of the visible light radiation image and the temperature gradient distribution characteristics of the thermal infrared radiation image were extracted, providing a foundation for subsequent image analysis and understanding.

[0072] In one optional implementation, the day / night time period type and the degree of scene dynamic change in the current imaging environment are determined by analyzing the spatial consistency between radiation intensity and gradient direction, resulting in day / night environment discrimination results including:

[0073] The global brightness mean and local brightness variance are extracted from the brightness distribution characteristics. The light intensity level is determined based on the comparison between the global brightness mean and the preset brightness level interval. The light distribution uniformity index is determined based on the spatial distribution uniformity of the local brightness variance.

[0074] The gradient direction field is extracted from the temperature gradient distribution features. The gradient direction angle of each pixel in the gradient direction field is divided into intervals. The number of pixels in each interval is counted to obtain a direction distribution histogram. The direction distribution histogram is subjected to main direction clustering analysis to obtain the set of main directions of the temperature gradient.

[0075] The spatial location of the local brightness variance is matched with the spatial location of the gradient amplitude field at the pixel level. The angle deviation between the brightness change direction and the gradient direction at the matching location is calculated. The percentage of pixels whose angle deviation is less than a preset consistency angle threshold is counted to obtain the spatial consistency index.

[0076] Based on the light intensity level and the light distribution uniformity index, the day and night time period type is determined. Based on the spatial consistency index and the number of principal directions in the set of principal directions of the temperature gradient, the degree of dynamic change of the scene is determined, and the day and night environment discrimination result is obtained.

[0077] A global brightness statistical analysis was performed on the visible light image. For a visible light image with a resolution of 1920×1080, its global average brightness was calculated, assuming the average brightness value to be 158. Simultaneously, the image was divided into a 12×12 grid, and the local brightness variance was calculated within each grid. For example, the local brightness variance calculated within the grid located at coordinates (5, 7) was 437.6. A comparison was made using preset brightness grading intervals, which were set as follows: 0-50 for low illumination, 51-150 for medium illumination, and 151-255 for high illumination. Since the global average brightness value of 158 falls within the 151-255 interval, the current illumination level was determined to be "high illumination".

[0078] The spatial distribution uniformity index of local brightness variance is calculated. The standard deviation analysis of the local brightness variance of all grids is performed, and the standard deviation value is 98.2. This standard deviation value is compared with the preset uniformity threshold of 120. Since 98.2 is less than 120, the current illumination distribution uniformity index is determined to be "uniform".

[0079] The gradient direction angle from 0 to 360 degrees is divided into 8 intervals: 0-45 degrees, 45-90 degrees, 90-135 degrees, 135-180 degrees, 180-225 degrees, 225-270 degrees, 270-315 degrees, and 315-360 degrees. The number of pixels in each interval is counted to form a histogram of direction distribution.

[0080] Assuming the statistical results are as follows: 75,846 pixels in the 0-45 degree range, 42,318 pixels in the 45-90 degree range, 98,754 pixels in the 90-135 degree range, 38,429 pixels in the 135-180 degree range, 71,248 pixels in the 180-225 degree range, 41,527 pixels in the 225-270 degree range, 96,241 pixels in the 270-315 degree range, and 34,037 pixels in the 315-360 degree range. Using a clustering threshold of 15% of the total pixels (72,000 pixels), a principal direction clustering analysis was performed, determining that the principal direction set includes three directions: 0-45 degrees, 90-135 degrees, and 270-315 degrees.

[0081] The spatial location of the local brightness variance is matched with the spatial location of the gradient magnitude field. Since thermal imaging data and visible light images have different resolutions, the thermal imaging data is first interpolated to a 1920×1080 resolution. For each pixel, the angular deviation between the brightness change direction and the temperature gradient direction is calculated. A consistency angle threshold of 30 degrees is set, and the number of pixels with an angle deviation less than 30 degrees is counted. In this example, a total of 1,562,743 pixels have an angle deviation less than 30 degrees, accounting for 75.4% of the total number of pixels (2,073,600). Therefore, the spatial consistency index is 0.754.

[0082] Based on the above analysis results, environmental classification is performed. Based on the light intensity level of "high illumination" and the uniformity index of light distribution of "uniformity," the current day / night period type is determined to be "daytime." Simultaneously, based on a spatial consistency index of 0.754, which is greater than the preset threshold of 0.7, and a set of main temperature gradient directions containing three main directions, which is greater than the preset threshold of 2, the scene's dynamic change level is determined to be "high dynamic scene." Finally, the day / night environment classification result is output as "daytime high dynamic scene."

[0083] This method is applicable to scenarios such as intelligent monitoring, nighttime navigation of drones, and night vision systems for autonomous driving. By fusing complementary information from visible light and thermal imaging data, it enables accurate judgment of environmental conditions, providing key reference for subsequent tasks such as image enhancement, target detection, and scene understanding.

[0084] In one optional implementation, based on the day / night environment discrimination result, feature domain transformation is performed on the visible light radiation image and the thermal infrared radiation image respectively to obtain visible light feature domain characterization and thermal infrared feature domain characterization, including:

[0085] Spectral analysis is performed on the visible light radiation image to obtain the energy distribution concentration of each frequency component and to fuse it with the light intensity level to obtain the frequency domain decomposition level parameters; the global gradient amplitude distribution of the thermal infrared radiation image is calculated, the pixel density with gradient amplitude exceeding a preset response threshold is counted to obtain the spatial gradient response activity index and to perform nonlinear mapping operation with the scene dynamic change degree to obtain the spatial filtering scale parameters.

[0086] According to the frequency domain decomposition level parameters, the visible light radiation image is subjected to multi-level spectral transformation. At each decomposition level, the image signal is decomposed into multiple coefficient components to obtain visible light frequency domain coefficients of multiple frequency sub-bands. Texture descriptors are extracted from each of the visible light frequency domain coefficients to obtain the visible light feature domain characterization.

[0087] Based on the spatial filtering scale parameters, multiple spatial filtering kernels of different scales are constructed. Each spatial filtering kernel is convolved with the thermal infrared radiation image to obtain thermal infrared filtering response images of multiple scale levels. Edge responders are extracted from each thermal infrared filtering response image to obtain the thermal infrared feature domain characterization.

[0088] The two-dimensional Fourier transform of the visible light radiation image is calculated to obtain the energy distribution of the image in the frequency domain. Specifically, after converting the image to the frequency domain, the percentages of energy in the low-frequency band (0-50Hz), mid-frequency band (51-150Hz), and high-frequency band (above 151Hz) are statistically analyzed. For example, in a certain test, the low-frequency band accounted for 65% of the energy, the mid-frequency band for 28%, and the high-frequency band for 7%. This concentration of frequency domain energy distribution is then fused with the previously obtained illumination intensity levels (e.g., levels 1-5, where level 1 represents extremely dark and level 5 represents extremely bright) using a weighted average. If the illumination intensity is level 4 (relatively bright), the frequency domain energy distribution concentration is assigned a weight of 0.7, and the illumination intensity is assigned a weight of 0.3. The calculated frequency domain decomposition level parameter is 3.5, indicating that the image needs to be decomposed into 4 frequency levels.

[0089] For thermal infrared radiation images, the global gradient magnitude distribution is calculated. Specifically, the Sobel operator is used to calculate the gradients in the x and y directions of the image, and then the total gradient magnitude is synthesized. A preset response threshold is set to 15% of the image's grayscale value range (e.g., 38 for an 8-bit image). The number of pixels with gradient magnitudes exceeding this threshold is counted and divided by the total number of pixels in the image to obtain the spatial gradient response activity index. In a test example, the spatial gradient response activity of a thermal infrared image is 0.23, indicating that 23% of the pixels have significant edge features. This index is non-linearly mapped to the scene's dynamic change level (expressed as the number of moving objects detected per second). The mapping uses a logarithmic function: if the scene's dynamic change level is 5, i.e., 5 moving objects are detected per second, the spatial filtering scale parameter is calculated to be 2.3, indicating that a spatial filtering kernel of 3 scales needs to be constructed.

[0090] Based on the frequency domain decomposition level parameters, multi-level spectral transformation is performed on the visible light radiation image. Taking wavelet transform as an example, when the frequency domain decomposition level parameter is 3.5, a 4-level wavelet decomposition is performed, resulting in 4 different frequency sub-bands. At each decomposition level, the image is decomposed into approximation coefficients (LL sub-band) and detail coefficients (LH, HL, HH sub-bands). Texture descriptors, including energy, entropy, contrast, and homogeneity, are extracted from each sub-band. For example, for the LL sub-band of the first-level decomposition, the extracted energy value is 0.85, entropy is 4.2, contrast is 0.32, and homogeneity is 0.78. These texture descriptors are combined to form a visible light feature domain representation vector with a length of 4×4×4=64 (4 features × 4 sub-bands × 4 decomposition levels).

[0091] Based on the spatial filtering scale parameter, multiple spatial filtering kernels of different scales are constructed. When the spatial filtering scale parameter is 2.3, three Gaussian filtering kernels are constructed with sizes of 3×3, 5×5, and 7×7, and standard deviations of 0.8, 1.2, and 1.8, respectively. These filtering kernels are convolved with the thermal infrared radiation image to obtain three thermal infrared filtered response images at different scale levels. Edge responders are extracted from each response image, including horizontal edge intensity, vertical edge intensity, edge direction consistency, and edge continuity. Taking the response image generated by the 7×7 filtering kernel as an example, the mean horizontal edge intensity is 0.43, the mean vertical edge intensity is 0.38, the edge direction consistency is 0.65, and the edge continuity is 0.72. These edge responders are combined to form a thermal infrared feature domain representation vector with a length of 4×3=12 (4 features × 3 scales).

[0092] Through the above processing, based on the day / night environment discrimination results, feature domain representations are adaptively extracted from visible light and thermal infrared images. These representations can effectively capture the significant characteristics of different spectral images under different environmental conditions. Experiments show that under low light conditions at night, the weight of the thermal infrared feature domain representation automatically increases; while under sufficient light conditions during the day, the weight of the visible light feature domain representation is correspondingly enhanced, thus ensuring the system's stable performance under all-weather conditions.

[0093] In one optional implementation, based on the day / night time period type, projecting the visible light feature domain representation and the thermal infrared feature domain representation onto a unified fused feature space by constructing a cross-spectral feature alignment mapping relationship includes:

[0094] The dominant spectral type is determined from the day and night time period types. Based on the dominant spectral type, a corresponding feature space reference coordinate system is selected. The visible light feature domain representation is projected onto the feature space reference coordinate system to obtain the dimension-aligned visible light feature domain representation.

[0095] Calculate the Euclidean distance between each edge responder and each texture descriptor, and select edge responders and texture descriptors with an Euclidean distance less than a preset alignment distance threshold as spatially corresponding feature pairs. Calculate the intensity difference between the response intensity value of the edge responder and the texture intensity value of the texture descriptor in each spatially corresponding feature pair, construct an intensity mapping coefficient based on the intensity difference, and use the intensity mapping coefficient to perform amplitude scaling transformation on the response intensity value of each edge responder in the thermal infrared feature domain representation to obtain the amplitude-aligned thermal infrared feature domain representation.

[0096] The visible light feature domain characterization after dimensional alignment and the thermal infrared feature domain characterization after amplitude alignment are mapped in the feature space reference coordinate system.

[0097] The dominant spectral type is determined from the defined day and night time periods. During the daytime, visible light is determined as the dominant spectral type due to its high imaging quality. During the nighttime, thermal infrared imaging is determined as the dominant spectral type due to its significant advantages in low-light environments.

[0098] Based on the determined dominant spectral type, a corresponding feature space reference coordinate system is selected. During the daytime, when visible light is determined to be the dominant spectral type, the coordinate system of the visible light feature domain is selected as the feature space reference coordinate system; during the nighttime, the coordinate system of the thermal infrared feature domain is selected as the feature space reference coordinate system. The eigenvectors in the visible light feature domain representation are directly mapped to this reference coordinate system, maintaining their original dimensions and structure. For the thermal infrared feature domain representation, a dimension transformation operation is required to make it consistent with the dimension of the visible light feature domain representation. Assuming the visible light feature domain representation has 128 dimensions and the thermal infrared feature domain representation has 96 dimensions, the thermal infrared feature domain representation is converted to 128 dimensions through interpolation or dimension upscaling techniques, thus obtaining the dimension-aligned visible light feature domain representation.

[0099] For each edge responder in the visible light feature domain, calculate its Euclidean distance with each texture descriptor in the thermal infrared feature domain. Assuming the edge responders in the visible light feature domain are [0.35, 0.42, 0.56, 0.21] and the texture descriptors in the thermal infrared feature domain are [0.32, 0.45, 0.58, 0.25], the Euclidean distance between them is 0.0583.

[0100] Edge responders and texture descriptors with an Euclidean distance less than a preset alignment distance threshold are selected as spatially corresponding feature pairs. For example, when the preset alignment distance threshold is set to 0.1, if the calculated Euclidean distance is 0.0583, which is less than the threshold of 0.1, then the edge responder and texture descriptor are selected as a pair of spatially corresponding feature pairs.

[0101] For each selected spatial corresponding feature pair, the intensity difference between the response intensity value of the edge responder and the texture intensity value of the texture descriptor is calculated. Taking the spatial corresponding feature pair in the example above as an example, the response intensity value of the edge responder is 0.56, the texture intensity value of the texture descriptor is 0.58, and the intensity difference is 0.02.

[0102] Based on the calculated intensity difference, an intensity mapping coefficient is constructed. The calculation of the intensity mapping coefficient takes into account the magnitude and direction of the intensity difference, as well as the current day / night time period. For daytime periods, the intensity mapping coefficient can be set to 1.0 minus the normalized intensity difference, for example, 1.0 - 0.02 / 0.5 = 0.96; for nighttime periods, the intensity mapping coefficient can be set to 1.0 plus the normalized intensity difference, for example, 1.0 + 0.02 / 0.5 = 1.04.

[0103] Using the constructed intensity mapping coefficients, the response intensity values ​​of each edge responder in the thermal infrared feature domain representation are subjected to amplitude scaling transformation. Taking a response intensity value of 0.72 for an edge responder in the thermal infrared feature domain as an example, after applying the intensity mapping coefficient 0.96 for amplitude scaling transformation, the resulting response intensity value is 0.72 × 0.96 = 0.6912. In this way, all edge responders in the thermal infrared feature domain representation are processed to obtain an amplitude-aligned thermal infrared feature domain representation.

[0104] The visible light feature domain representation, aligned by dimensions, and the thermal infrared feature domain representation, aligned by amplitude, are mapped onto a reference coordinate system in the feature space. During this mapping process, a weighted feature fusion strategy is employed, assigning different weights to the visible light and thermal infrared features based on the current day / night time and dominant spectral type. For example, during the daytime, the weight of the visible light feature is 0.7, and the weight of the thermal infrared feature is 0.3; during the nighttime, the weight of the visible light feature is 0.3, and the weight of the thermal infrared feature is 0.7. Through this weighted fusion method, the two feature representations are projected onto a unified fused feature space, achieving effective alignment and fusion of cross-spectral features.

[0105] In one optional implementation, spatial registration and semantic correspondence are performed in the fused feature space, and then the fused radiometric image is reconstructed through inverse transformation, including:

[0106] Extract the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation and the spatial coordinates of each feature point in the amplitude-aligned thermal infrared feature domain representation, calculate the positional deviation, construct a spatial transformation matrix based on the positional deviation, and use the spatial transformation matrix to perform coordinate correction transformation on the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation to obtain a spatially registered fused feature set.

[0107] Semantic category labeling is performed on each feature point in the spatially registered fusion feature set. Feature points with the same semantic category are divided into the same semantic region. The number of feature points represented by the visible light feature domain after dimension alignment and the number of feature points represented by the thermal infrared feature domain after amplitude alignment are counted in each semantic region. The ratio of the number of feature points of the two types is calculated. The spectral contribution weight of each semantic region is determined according to the ratio. The feature points in each semantic region are fused according to the spectral contribution weight to obtain the semantically corresponding fusion feature representation.

[0108] The fused feature representation after semantic correspondence is subjected to inverse feature domain transformation and mapped back to the image pixel space to obtain the fused radiative image.

[0109] For each feature point (x_v, y_v) in the dimension-aligned visible light feature domain representation and the corresponding feature point (x_t, y_t) in the amplitude-aligned thermal infrared feature domain representation, calculate their positional deviations Δx = x_v - x_t and Δy = y_v - y_t. Based on the positional deviations of all feature point pairs, construct the spatial transformation matrix T using the least squares method. For example, in practical applications, taking 100 SIFT feature points from both the visible light and thermal infrared images, the calculated average positional deviations are Δx = 5.3 pixels and Δy = 3.7 pixels. The spatial transformation matrix T constructed based on this can be represented as a combination of translation and rotation.

[0110] For each feature point (x_v, y_v) in the visible light feature domain representation, a coordinate transformation is performed using the spatial transformation matrix T to obtain the corrected coordinates (x_v', y_v'). For example, for the visible light feature point with coordinates (120, 85), after transformation by the transformation matrix T, its coordinates become (114.7, 81.3), which is very close to the corresponding thermal infrared feature point coordinates (115, 81), achieving the purpose of spatial registration. The corrected visible light feature domain representation and the thermal infrared feature domain representation are merged to form the spatially registered fused feature set F.

[0111] Semantic category labeling is performed on each feature point in the spatially registered fusion feature set. Feature points with the same semantic category are grouped into the same semantic region. A pre-trained semantic segmentation model is used to classify each feature point in the fusion feature set F, assigning it to a specific semantic category, such as "buildings," "vegetation," "roads," and "sky." In practical applications, for a 640×480 resolution image, five main semantic regions can be identified: "buildings" (35% of the image area), "vegetation" (25%), "roads" (20%), "sky" (15%), and "water" (5%).

[0112] For each semantic region S, calculate the number of visible light feature points N_v and the number of thermal infrared feature points N_t, then calculate the ratio r = N_v / N_t. Based on this ratio, determine the spectral contribution weights w_v and w_t for that semantic region, satisfying w_v + w_t = 1. For example, in the "building" region, the number of visible light feature points is 5600, the number of thermal infrared feature points is 4200, and the ratio r ≈ 1.33. Therefore, the visible light contribution weight w_v = 0.57 and the thermal infrared contribution weight w_t = 0.43 can be set for this region. In the "water" region, the number of visible light feature points is 800, the number of thermal infrared feature points is 1200, and the ratio r ≈ 0.67. Therefore, the visible light contribution weight w_v = 0.4 and the thermal infrared contribution weight w_t = 0.6 can be set for this region.

[0113] For each feature point p in the semantic region S, if it appears in both the visible light feature domain and the thermal infrared feature domain, its feature values ​​are weighted and averaged: f_p = w_v × f_v + w_t × f_t, where f_v and f_t are the feature values ​​of the point in the visible light and thermal infrared feature domains, respectively. If feature point p appears only in either the visible light or thermal infrared feature domain, the feature value is directly used. For example, for a feature point in the "building" region that appears in both feature domains, its fused feature value is calculated as: f_p = 0.57 × f_v + 0.43 × f_t.

[0114] The semantically mapped fused feature representation undergoes inverse feature domain transformation. A pre-trained feature decoder network transforms the fused feature representation back from the feature domain to the pixel domain. Specifically, the fused feature representation is input into the decoder network, which performs upsampling and convolution operations layer by layer, ultimately generating a fused radiometric image with the same resolution as the original image. For example, for a 640×480 resolution input image, the size of the fused feature representation is 80×60×256 (width×height×number of channels). After four upsampling operations by the decoder network, doubling the size each time, a 640×480×3 fused radiometric image is finally generated. In practical applications, the fused radiometric image retains the clear details of the visible light image and the temperature information of the thermal infrared image. For example, the "building" region displays both the exterior texture of the building and the temperature distribution, enhancing the information content and interpretability of the image.

[0115] In one optional implementation, calculating the feature distribution overlap and mutual information of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space, based on the day / night time period type and the degree of scene dynamic change, includes:

[0116] The illumination stability coefficient is determined based on the day / night time period type, the motion disturbance coefficient is determined based on the dynamic change degree of the scene, and the illumination stability coefficient and the motion disturbance coefficient are combined to obtain the environmental correction factor.

[0117] The frequency of occurrence of each feature value in the fused feature space is statistically analyzed for the visible light feature domain characterization and the thermal infrared feature domain characterization, respectively, to construct the visible light feature distribution and the thermal infrared feature distribution. The area of ​​the intersection region and the area of ​​the union region of the visible light feature distribution and the thermal infrared feature distribution are calculated and the ratio is obtained to obtain the initial overlap. The initial overlap is corrected using the environmental correction factor to obtain the feature distribution overlap.

[0118] The visible light feature distribution and the thermal infrared feature distribution are subjected to joint probability distribution calculation to obtain a dual-spectral joint distribution. The joint entropy of the dual-spectral joint distribution is calculated. The edge entropy of the visible light feature distribution and the thermal infrared feature distribution are calculated respectively and summed to obtain the entropy sum value. The difference between the entropy sum value and the joint entropy is calculated to obtain the initial mutual information. The initial mutual information is corrected using the environmental correction factor to obtain the mutual information.

[0119] like Figure 2 As shown, the method includes:

[0120] The illumination stability coefficient is determined based on the day / night cycle: 0.9 for daytime, 0.7 for dusk, 0.95 for night, and 0.6 for dawn. The motion disturbance coefficient is determined based on the degree of scene dynamics: 0.9 for low-dynamic scenes, 0.7 for medium-dynamic scenes, and 0.5 for high-dynamic scenes. The illumination stability coefficient is multiplied by the motion disturbance coefficient to obtain the environment correction factor; for example, for a low-dynamic scene during the day, the environment correction factor is 0.9 × 0.9 = 0.81.

[0121] Statistical analysis was performed on the extracted visible light and thermal infrared feature domain representations. Assuming the fused feature space has a dimension of 128, the visible light feature domain representation is a set of 128-dimensional vectors, and the thermal infrared feature domain representation is also a set of 128-dimensional vectors. The numerical distribution of these two sets of features in each dimension was statistically analyzed, and histograms of visible light and thermal infrared feature distributions were constructed. For example, the feature value interval in each dimension was divided into 20 sub-intervals, and the frequency of feature values ​​within each sub-interval was statistically analyzed to form a distribution histogram.

[0122] The intersection and union regions of the visible light and thermal infrared feature distributions are determined as follows: For each sub-interval of each feature dimension, the smaller value of the frequency of the visible light and thermal infrared features in that sub-interval is taken as the area contribution of the intersection region in that sub-interval, and the larger value is taken as the area contribution of the union region in that sub-interval. The intersection area contributions of all sub-intervals are summed to obtain the total intersection area; the union area contributions of all sub-intervals are summed to obtain the total union area. The ratio of the total intersection area to the total union area is calculated to obtain the initial overlap. For example, if the total intersection area is 350 and the total union area is 1000, the initial overlap is 0.35. The initial overlap is multiplied by an environmental correction factor to obtain the final feature distribution overlap. If the environmental correction factor is 0.81, the final feature distribution overlap is 0.35 × 0.81 = 0.2835.

[0123] A joint probability distribution of visible light and thermal infrared features is constructed by dividing both the visible light and thermal infrared feature spaces into 10 intervals, forming a 10×10 two-dimensional grid. The number of samples in each grid is counted and normalized to obtain a joint probability distribution table. The joint entropy of this joint probability distribution is calculated by multiplying the probability value of each grid by its logarithm (negative value) and then summing the results. For example, if the calculated joint entropy is 4.2...

[0124] Calculate the edge entropy of the visible light characteristic distribution and the thermal infrared characteristic distribution separately. The edge entropy is calculated by summing the joint probability distribution along the corresponding dimensions to obtain the edge probability distribution, and then calculating the entropy value of the edge probability distribution. Assuming the calculated edge entropy of the visible light characteristic distribution is 2.3 and the edge entropy of the thermal infrared characteristic distribution is 2.5, the sum of the entropies is 2.3 + 2.5 = 4.8.

[0125] Calculate the difference between the sum of entropies and the joint entropy to obtain the initial mutual information, which in this example is 4.8 - 4.2 = 0.6. Multiply the initial mutual information by the environmental correction factor to obtain the final mutual information. If the environmental correction factor is 0.81, then the final mutual information is 0.6 × 0.81 = 0.486.

[0126] This implementation modifies the feature distribution overlap and mutual information by using an environmental correction factor, which can adapt to different lighting conditions and scene changes, improve the robustness and adaptability of fusion, and is particularly suitable for visible light and infrared image fusion tasks in complex outdoor environments.

[0127] This invention relates to a dual-spectral day-night radiometric imaging fusion and switching system, the system comprising:

[0128] The first unit is used to extract the brightness distribution features of visible light radiation images and the temperature gradient distribution features of thermal infrared radiation images.

[0129] The second unit is used to determine the day / night time type and the degree of scene dynamic change of the current imaging environment based on the brightness distribution characteristics and the temperature gradient distribution characteristics, through spatial consistency analysis of radiation intensity and gradient direction, and to obtain the day / night environment discrimination result.

[0130] The third unit is used to perform feature domain transformation on the visible light radiation image and the thermal infrared radiation image respectively based on the day and night environment discrimination result, so as to obtain the visible light feature domain characterization and the thermal infrared feature domain characterization.

[0131] The fourth unit is used to project the visible light feature domain representation and the thermal infrared feature domain representation onto a unified fusion feature space by constructing a cross-spectral feature alignment mapping relationship based on the day and night time period type, and to perform spatial registration and semantic correspondence in the fusion feature space, and then reconstruct a fusion radiometric image through inverse transformation;

[0132] The fifth unit is used to calculate the overlap and mutual information of the feature distributions of the visible light feature domain representation and the thermal infrared feature domain representation in the fused feature space based on the day / night time period type and the degree of scene dynamic change, to obtain a coordination index, and to determine the mode switching and image output based on the coordination index.

[0133] A third aspect of the present invention provides an electronic device, comprising:

[0134] processor;

[0135] Memory used to store processor-executable instructions;

[0136] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0137] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0138] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A day-night radiance imaging fusion and switching method based on dual-spectrum, characterized in that, The method comprises the following steps: extracting the brightness distribution feature of the visible light radiation image and the temperature gradient distribution feature of the thermal infrared radiation image; based on the brightness distribution feature and the temperature gradient distribution feature, determining the day and night period type and the scene dynamic change degree of the current imaging environment through the spatial consistency analysis of the radiation intensity and the gradient direction, obtaining the day and night environment discrimination result, including: extracting the global brightness mean and the local brightness variance from the brightness distribution feature, determining the illumination intensity level based on the comparison relationship between the global brightness mean and the preset brightness classification interval, and determining the illumination distribution uniformity index based on the spatial distribution uniformity of the local brightness variance; extracting the gradient direction field from the temperature gradient distribution feature, dividing the gradient direction angle of each pixel point in the gradient direction field into intervals, counting the number of pixels in each interval to obtain the direction distribution histogram, and performing main direction clustering analysis on the direction distribution histogram to obtain a temperature gradient main direction set; performing pixel-level matching on the spatial position of the local brightness variance and the spatial position of the gradient amplitude field, calculating the angle deviation between the brightness change direction and the gradient direction at the matching position, and counting the proportion of pixels with an angle deviation less than a preset consistency angle threshold to obtain a spatial consistency index; based on the illumination intensity level and the illumination distribution uniformity index, determining the day and night period type, and based on the spatial consistency index and the number of main directions in the temperature gradient main direction set, determining the scene dynamic change degree to obtain the day and night environment discrimination result; According to the day and night environment discrimination result, the visible light radiation image and the thermal infrared radiation image are respectively subjected to feature domain transformation to obtain visible light feature domain representation and thermal infrared feature domain representation; According to the day and night period type, the visible light feature domain representation and the thermal infrared feature domain representation are projected to a unified fusion feature space by constructing a cross-spectrum feature alignment mapping relationship, and spatial registration and semantic correspondence are performed in the fusion feature space, and then reconstructed into a fusion radiation image through inverse transformation; based on the day and night period type and the scene dynamic change degree, calculating the feature distribution overlap and mutual information amount of the visible light feature domain representation and the thermal infrared feature domain representation in the fusion feature space to obtain a collaboration index, and taking the collaboration index as a criterion to determine the switching of modes and the output of images.

2. The method of claim 1, wherein, The method comprises the following steps: performing color space transformation on the visible light radiation image, converting the visible light radiation image from the original color space to the target color space containing the brightness component, separating the brightness component from the target color space to obtain the brightness channel image, performing multi-scale spatial decomposition on the brightness channel image to obtain a plurality of scale-level brightness sub-images, calculating the statistical distribution parameters of each brightness sub-image to obtain the brightness distribution feature; The thermal infrared radiation image is subjected to a nonlinear mapping transformation of radiation intensity and temperature, pixel gray values of the thermal infrared radiation image are converted into temperature physical quantities based on the nonlinear mapping transformation, a temperature field distribution is obtained, gradient calculation is performed on the temperature field distribution in horizontal and vertical directions respectively, a horizontal gradient component and a vertical gradient component are obtained, a gradient amplitude field and a gradient direction field are constructed based on the horizontal gradient component and the vertical gradient component, and the temperature gradient distribution feature is obtained.

3. The method of claim 1, wherein, According to the day and night environment discrimination result, the visible light radiation image and the thermal infrared radiation image are respectively subjected to feature domain transformation, to obtain visible light feature domain representation and thermal infrared feature domain representation, including: The visible light radiation image is subjected to spectral analysis, energy proportion of each frequency component is counted to obtain frequency energy distribution concentration, and the frequency energy distribution concentration is fused with the illumination intensity level to obtain a frequency domain decomposition level parameter; a global gradient amplitude distribution of the thermal infrared radiation image is calculated, pixel density of gradient amplitude exceeding a preset response threshold is counted to obtain a spatial gradient response activity index, and the spatial gradient response activity index is subjected to nonlinear mapping operation with the scene dynamic change degree to obtain a spatial filtering scale parameter; According to the frequency domain decomposition level parameter, the visible light radiation image is subjected to multi-level spectral transformation, image signals are decomposed into a plurality of coefficient components at each decomposition level to obtain visible light frequency domain coefficients of a plurality of frequency subbands, and texture descriptors are extracted from the visible light frequency domain coefficients to obtain the visible light feature domain representation; According to the spatial filtering scale parameter, a plurality of spatial filtering kernels of different scales are constructed, each spatial filtering kernel is subjected to convolution operation with the thermal infrared radiation image to obtain thermal infrared filtering response images of a plurality of scale levels, and edge response elements are extracted from the thermal infrared filtering response images to obtain the thermal infrared feature domain representation.

4. The method of claim 1, wherein, According to the day and night period type, the visible light feature domain representation and the thermal infrared feature domain representation are projected to a unified fusion feature space by constructing a cross-spectrum feature alignment mapping relationship, including: A dominant spectrum type is determined from the day and night period type, a corresponding feature space reference coordinate system is selected based on the dominant spectrum type, the visible light feature domain representation is projected to the feature space reference coordinate system to obtain a visible light feature domain representation with aligned dimensions; Euclidean distances between each edge response element and each texture descriptor are calculated, edge response elements and texture descriptors with Euclidean distances less than a preset alignment distance threshold are selected as spatial corresponding feature pairs, intensity difference values are calculated for response intensity values of edge response elements and texture intensity values of texture descriptors in each spatial corresponding feature pair, an intensity mapping coefficient is constructed according to the intensity difference values, response intensity values of each edge response element in the thermal infrared feature domain representation are subjected to amplitude scaling transformation by using the intensity mapping coefficient to obtain an amplitude-aligned thermal infrared feature domain representation; The visible light feature domain representation with aligned dimensions and the amplitude-aligned thermal infrared feature domain representation are subjected to coordinate mapping in the feature space reference coordinate system.

5. The method of claim 4, wherein, And in the fusion feature space is registered and semantic correspondence, and then reconstructed by inverse transformation into a fusion radiation image, comprising: Extracting the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation and the spatial coordinates of each feature point in the amplitude-aligned thermal infrared feature domain representation, and calculating the position deviation amount, constructing a spatial transformation matrix according to the position deviation amount, and performing coordinate correction transformation on the spatial coordinates of each feature point in the dimension-aligned visible light feature domain representation using the spatial transformation matrix to obtain a spatially registered fusion feature set; Semantically labeling each feature point in the spatially registered fusion feature set, dividing feature points with the same semantic category into the same semantic region, counting the number of feature points from the dimension-aligned visible light feature domain representation and the number of feature points from the amplitude-aligned thermal infrared feature domain representation in each semantic region, calculating the ratio of the two types of feature point numbers, determining the spectral contribution weight of each semantic region according to the ratio, and fusing the feature points in each semantic region according to the spectral contribution weight to obtain a semantically corresponding fusion feature representation; Inverse feature domain transformation is performed on the semantically corresponding fusion feature representation to map back to the image pixel space to obtain the fusion radiation image.

6. The method of claim 1, wherein, Based on the diurnal period type and the scene dynamic change degree, the feature distribution overlap and mutual information amount of the visible light feature domain representation and the thermal infrared feature domain representation in the fusion feature space are calculated, comprising: Determine the illumination stability coefficient according to the diurnal period type, determine the motion disturbance coefficient according to the scene dynamic change degree, and combine the illumination stability coefficient and the motion disturbance coefficient to obtain an environment correction factor; Respectively count the occurrence frequency of each feature value in the visible light feature domain representation and the thermal infrared feature domain representation in the fusion feature space, construct the visible light feature distribution and the thermal infrared feature distribution, calculate the intersection area and the union area of the visible light feature distribution and the thermal infrared feature distribution and the ratio, obtain the initial overlap degree, and modify the initial overlap degree using the environment correction factor to obtain the feature distribution overlap degree; Calculate the joint probability distribution of the visible light feature distribution and the thermal infrared feature distribution to obtain a dual-spectrum joint distribution, and calculate the joint entropy of the dual-spectrum joint distribution; calculate the marginal entropy of the visible light feature distribution and the thermal infrared feature distribution and sum them to obtain an entropy sum value, calculate the difference between the entropy sum value and the joint entropy to obtain an initial mutual information amount, and modify the initial mutual information amount using the environment correction factor to obtain the mutual information amount.

7. A dual-spectrum based day-night radiance imaging fusion and switching system for implementing the method of any one of claims 1-6, characterized in that, Comprising: A first unit for extracting the brightness distribution feature of a visible light radiation image and the temperature gradient distribution feature of a thermal infrared radiation image; A second unit for determining the diurnal period type and the scene dynamic change degree of the current imaging environment based on the brightness distribution feature and the temperature gradient distribution feature through spatial consistency analysis of radiation intensity and gradient direction to obtain a diurnal environment discrimination result. a third unit configured to perform feature domain transformation on the visible light radiation image and the thermal infrared radiation image respectively according to the diurnal environment discrimination result, to obtain visible light feature domain representation and thermal infrared feature domain representation; a fourth unit configured to project the visible light feature domain representation and the thermal infrared feature domain representation to a unified fusion feature space by constructing a cross-spectrum feature alignment mapping relationship according to the diurnal period type, to perform spatial registration and semantic correspondence in the fusion feature space, and to reconstruct a fusion radiation image by inverse transformation; a fifth unit configured to calculate feature distribution overlap and mutual information amount of the visible light feature domain representation and the thermal infrared feature domain representation in the fusion feature space based on the diurnal period type and the scene dynamic change degree, to obtain a synergy index, and to determine mode switching and image output according to the synergy index.

8. An electronic device, comprising: comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1 to 6.

Images