Infrared and visible light image fusion method and system based on CRITIC weight

Abstract

The invention relates to an infrared and visible light image fusion method and system based on CRITIC weight. The method comprises the following steps: decomposing an input visible light image and an infrared image into a feature image and an original image based on a Gaussian filter; respectively calculating corresponding variable weight coefficients based on the visible light original image and the visible light characteristic image; fusing the visible light original image and the infrared original image based on the variable weight coefficient, and obtaining a visible light fusion image; fusing the visible feature map and the infrared feature map based on a variable weight coefficient, and obtaining an infrared fusion map; and fusing the visible light fusion image and the infrared fusion image to obtain a final fusion image. According to the method, when the light is too bright or too dark, the variable weight coefficient is matched with the weight which can best reflect the features of the fused image at present for fusion, so that the fusion effect of the fused image is improved.

Description

Technical Field

[0001] The present application relates to the technical field of target detection and tracking, and in particular to a method and system for fusing infrared and visible light images based on CRITIC weights. Background Art

[0002] Infrared and visible light signals come from different modalities, thus providing scene information from different aspects. Visible light images capture reflected light, while infrared images capture thermal radiation; infrared and visible light images present features inherent to almost all objects; visible light images usually have high spatial resolution and considerable details and light-dark contrast, and visible light images are easily affected by adverse conditions, such as insufficient lighting, fog, and other bad weather; infrared images can resist these interferences, but usually have low resolution and poor texture. Therefore, due to the universality and complementarity of the images utilized, the fusion technology of visible and infrared images can be used in more fields than other fusion schemes.

[0003] As shown in Chinese patent CN116309211A, a method for fusion of infrared and visible light images focusing on the inherent characteristics of multimodal images is disclosed; the method includes obtaining an infrared image and a visible light image of a target image; inputting the infrared image and the visible light image of the target image into a feature extractor, extracting a feature map of the infrared image and a feature map of the visible light image; processing the feature map of the infrared image and the feature map of the visible light image using a channel attention module improved based on the characteristics of infrared and visible light image fusion, and obtaining the initial weight of the channel attention using a global gradient method. The features of the infrared image and the features of the visible light image will be sent to the feature discriminator to enhance the expressive ability of the extracted features for fixed modal information; the final features of the infrared image and the final features of the visible light image are input into the reconstruction network to generate an image fusion image of the target image. The present invention can better focus on the detail information of infrared and visible light images; the generated fusion image is rich in details and conforms to human visual perception.

[0004] However, in the process of image fusion, the corresponding features of images at different times are constantly changing, and the process of fusing images is also constantly changing. Fusion of images based on each frame through fixed weight coefficients results in unstable quality of the fused image. Summary of the invention

[0005] 1. Problem to be solved Based on this, it is necessary to provide a fusion method and system for infrared and visible light images based on CRITIC weights that can change based on the corresponding fused images at different times to address the above technical problems.

[0006] 2. Technical solution In a first aspect, the present application provides a method for fusing infrared and visible light images based on CRITIC weights. The method comprises: Decomposing the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; Calculate the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; The visible light original image and the infrared original image are fused based on the variable weight coefficient to obtain a visible light fusion image; The visible feature map and the infrared feature map are fused based on the variable weight coefficient to obtain an infrared fusion map; The visible light fusion image and the infrared fusion image are fused to obtain the final fusion image.

[0007] In one embodiment, calculating the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively includes: Construct a situation assessment matrix based on real-time assessment status and assessment indicators; normalizing the situation assessment matrix; Calculating the comprehensive information between the evaluation indicators; Determine a constant weight vector between evaluation indicators based on the comprehensive information amount; Obtain a variable weight vector based on element-by-element multiplication of the state variable weight vector and the constant weight vector; Based on the comprehensive information, the equilibrium function is constructed and solved to obtain the variable weight coefficient of the corresponding evaluation index.

[0008] In one embodiment, constructing a situation assessment matrix based on real-time assessment status and assessment indicators includes: Obtaining the real-time evaluation status at a preset time and forming a real-time evaluation status information set; Obtain a preset number of evaluation indicators and form an indicator set; Construct a situation assessment matrix based on the real-time assessment status information set and indicator set.

[0009] In one embodiment, normalizing the situation assessment matrix includes: When the evaluation index is a benefit index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a cost-based index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a fixed index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, It is the best value for the fixed indicator.

[0010] In one embodiment, calculating the comprehensive information volume between the evaluation indicators includes: The comprehensive information volume is calculated based on the correlation coefficient and standard deviation. The formula is as follows: ; in, For the The comprehensive information content of the indicators, For the The standard deviation of the index, For the The correlation coefficient of the indicators; The formula for standard deviation is as follows: ; in, For the The standard deviation of the index, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The average value of the indicators; The formula for the correlation coefficient is as follows: ; in, For the The correlation coefficient of the indicators, For the Indicators and The covariance of the indicators, For the The standard deviation of the index, For the The standard deviation of the indicator.

[0011] In one embodiment, constructing a balance function based on the comprehensive information amount and solving to obtain a variable weight coefficient of a corresponding evaluation index includes: Based on the comprehensive information, the equilibrium function is constructed and the state variable weight vector is calculated by derivation of the equilibrium function. The formula is as follows: ; in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator changes its weight; The equilibrium function formula is as follows: ; in, For the The equilibrium function of the index, For the The variable weight factor of each indicator, For the The evaluation index of each indicator changes its weight; The state of each indicator becomes a weight vector, and the formula is as follows: ; in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator becomes the weight. For the Variable weight factor of each indicator.

[0012] In the second aspect, the present application also provides a fusion system of infrared and visible light images based on CRITIC weights. The system includes: A feature original image decomposition module is used to decompose the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; A variable weight coefficient calculation module, used to calculate corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; An infrared and visible light original image fusion module is used to fuse the visible light original image and the infrared original image based on a variable weight coefficient and obtain a visible light fusion image; An infrared-visible light feature map fusion module is used to fuse the visible feature map and the infrared feature map based on a variable weight coefficient and obtain an infrared fusion map; The visible light infrared fusion image fusion module is used to fuse the visible light fusion image and the infrared fusion image to obtain the final fusion image.

[0013] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program: Decomposing the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; Calculate the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; The visible light original image and the infrared original image are fused based on the variable weight coefficient to obtain a visible light fusion image; The visible feature map and the infrared feature map are fused based on the variable weight coefficient to obtain an infrared fusion map; The visible light fusion image and the infrared fusion image are fused to obtain the final fusion image.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented: Decomposing the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; Calculate the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; The visible light original image and the infrared original image are fused based on the variable weight coefficient to obtain a visible light fusion image; The visible feature map and the infrared feature map are fused based on the variable weight coefficient to obtain an infrared fusion map; The visible light fusion image and the infrared fusion image are fused to obtain the final fusion image.

[0015] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented: Decomposing the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; Calculate the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; The visible light original image and the infrared original image are fused based on the variable weight coefficient to obtain a visible light fusion image; The visible feature map and the infrared feature map are fused based on the variable weight coefficient to obtain an infrared fusion map; The visible light fusion image and the infrared fusion image are fused to obtain the final fusion image.

[0016] 3. Beneficial effects The present application adopts the above method, decomposes the acquired visible light and infrared light images through Gaussian filters to obtain feature maps and original images, calculates variable weight coefficients for the visible light original image and the feature map respectively, matches the changing variable weight coefficients at different times and fuses the feature maps of visible light and infrared light respectively, fuses the original images of visible light and infrared light, and finally fuses the fused feature map and the original image to finally obtain a fused image. At different times, especially when the light is too bright or too dark, the variable weight coefficient is matched to the weight that best reflects the characteristics of the fused image at present, thereby improving the fusion effect of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 FIG. 1 is an application environment diagram of a method for fusing infrared and visible light images based on CRITIC weights in one embodiment; Figure 2 is a flow chart of a method for fusing infrared and visible light images based on CRITIC weights in one embodiment; Figure 3 is a structural block diagram of a fusion system of infrared and visible light images based on CRITIC weights in one embodiment; Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] The infrared and visible light image fusion method based on CRITIC weights provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The terminal communicates with the server through the network. The visible light image sensor and the infrared image sensor are used to collect visible light images and infrared images respectively and send the collected visible light images and infrared images to the server for processing. The server fuses the infrared image and the visible light image and sends the final fused image to the user's terminal through the network. The terminal can be but not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0020] In one embodiment, Figure 2 As shown, this method is applied to Figure 1 Taking the terminal in as an example, in this embodiment, the method includes the following steps: Step 202: Decompose the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map based on a Gaussian filter.

[0021] Among them, the visible light image and the infrared image are respectively acquired through the existing visible light image sensor and the infrared image sensor, or the visible light image and the infrared image corresponding to the sliding window can be directly acquired through the image sensor; the input infrared image and the visible light image are decomposed into the original image and the feature image by using a Gaussian filter based on the sliding window.

[0022] It is worth mentioning that Gaussian filter is a commonly used image processing technique for smoothing or denoising images. Gaussian filter is implemented by performing convolution operations on images; the process of decomposing an image into an original image and a feature image involves dividing the image into two parts: low-frequency information and high-frequency information.

[0023] Original image (Base Layer), the original image is the low-frequency component of the image, which represents the overall structure or smooth area of ​​the image. After the image is convolved with a Gaussian filter, the result is the original image. This part of information usually contains the overall outline, gradient and large-scale changes of the image; Feature map (Detail Layer), the feature map is the high-frequency component of the image, which represents the details and texture features in the image; the feature map is obtained by subtracting the original image from the original image, and contains edge, texture and fast-changing detail information; the Gaussian filter removes high-frequency components (such as noise or fast-changing edges) by convolving with the image, and only retains low-frequency components (smooth parts).

[0024] Step 204 , respectively calculating corresponding variable weight coefficients based on the visible light original image and the visible light feature image.

[0025] Among them, CTITIC weights use a network that fuses information and confronts each other for feature screening and reconstruction. It has strong robustness and generalization capabilities. In the fused input image, feature information of different depths is extracted, including edges, contours, positional relationships, rotations and other information of different scales. This information is reconstructed into a true color image as output through environmental weights.

[0026] It is worth mentioning that in Deep Reinforcement Learning (DRL), CRITIC weights refer to the parameter weights of the Critic network in the policy optimization framework. This is related to the Actor-Critic algorithm, which divides the learning process into two parts: Actor: responsible for selecting actions, optimizing strategies, and interacting directly with the environment; Critic: responsible for evaluating the quality of the Actor's selected actions, usually by estimating the value function. The weights of the Critic network define its ability to estimate the value function. These weights are learned by minimizing some form of loss function, common loss functions include mean squared error (MSE) or error based on time difference (TD).

[0027] Step 206: fuse the visible light original image and the infrared original image based on the variable weight coefficient to obtain a visible light fused image.

[0028] Step 208: fuse the visible feature map and the infrared feature map based on the variable weight coefficient and obtain an infrared fusion map.

[0029] Step 210: fuse the visible light fusion image and the infrared fusion image to obtain a final fusion image.

[0030] In the above-mentioned fusion method of infrared and visible light images based on CRITIC weights, the acquired visible light and infrared light images are decomposed by Gaussian filters to obtain feature maps and original images, and variable weight coefficients are calculated for the visible light original image and the feature map respectively. The feature maps of visible light and infrared light are fused respectively by matching the changing variable weight coefficients at different times, and the original images of visible light and infrared light are fused. Finally, the fused feature map and the original image are fused to finally obtain a fused image. At different times, especially when the light is too bright or too dark, the variable weight coefficient is matched to the weight that best reflects the characteristics of the fused image at present, thereby improving the fusion effect of the fused image.

[0031] In one embodiment, respectively calculating corresponding variable weight coefficients based on the visible light original image and the visible light feature image specifically includes the following steps: Step 302: construct a situation assessment matrix based on the real-time assessment status and assessment indicators.

[0032] Among them, situation assessment is the process of analyzing key factors and their interactions in a system or environment. Its purpose is to provide a comprehensive and dynamic situation awareness and prediction, and provide a basis for further actions or decisions. In this application, a situation assessment matrix is ​​constructed based on real-time assessment status information and evaluation indicators. The evaluation indicators include image brightness, color, texture, color features, shape features, and spatial position relationship features.

[0033] Step 304, normalize the situation assessment matrix.

[0034] Among them, different indicators are normalized to eliminate the influence of indicator type and dimension.

[0035] Step 306, calculating the comprehensive information volume between the evaluation indicators.

[0036] Among them, the conflict between indicators and the contrast intensity between data are expressed by correlation coefficient and standard deviation.

[0037] Step 308: Determine a constant weight vector between evaluation indicators based on the comprehensive information amount.

[0038] Step 310 , obtaining a variable weight vector based on the element-by-element multiplication of the state variable weight vector and the constant weight vector.

[0039] Among them, the main method of variable weight is to construct a state variable weight vector to achieve weighted adjustment of the state and avoid state imbalance.

[0040] Step 312: construct a balance function based on the comprehensive information and solve to obtain the variable weight coefficient of the corresponding evaluation index Among them, the core of variable weight solution is to construct an equilibrium function according to the influence of brightness, color, texture, color features, shape features, and spatial position relationship features, and calculate the state variable weight vector by derivation of the equilibrium function.

[0041] In this embodiment, a situation assessment matrix is ​​constructed and normalized, and after calculating the comprehensive information amount, a constant weight vector is determined by the comprehensive information amount, and a variable weight vector is determined based on the constant weight vector and the state variable weight vector.

[0042] In one embodiment, constructing a situation assessment matrix based on real-time assessment status and assessment indicators specifically includes: The real-time evaluation status at a preset time is obtained and a real-time evaluation status information set is formed; a preset number of evaluation indicators are obtained and an indicator set is formed; and a situation evaluation matrix is ​​constructed based on the real-time evaluation status information set and the indicator set.

[0043] The evaluation matrix is ​​composed of The real-time evaluation status information set And p evaluation indicators constitute the indicator set Composition, The moment The index value is , then the situation assessment matrix is ​​expressed as: ; Among them, the situation assessment matrix is ​​usually a two-dimensional table, the rows and columns in the table represent different influencing factors, indicators or variables respectively, and each unit in the matrix represents the relationship or state between these factors. In this embodiment, the two-dimensional sets are real-time assessment status and assessment indicators respectively.

[0044] In this embodiment, a specific process of constructing a situation assessment matrix through real-time assessment status and assessment indicators is disclosed.

[0045] In one embodiment, the normalized situation assessment matrix specifically includes benefit indicators, cost indicators, and fixed indicators. Different normalized indicators are selected based on different assessment indicators to perform normalized operations on the situation assessment matrix, specifically including: When the evaluation index is a benefit index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a cost-based index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a fixed index, the formula is as follows: ; in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, It is the best value for the fixed indicator.

[0046] In this embodiment, the normalization process of the evaluation matrix is ​​specifically disclosed by specifically disclosing the normalized benefit indicators, cost indicators and fixed indicators. It should be noted that different normalization matrices are preset based on different evaluation indicators.

[0047] In one embodiment, calculating the comprehensive information amount between evaluation indicators specifically includes: The comprehensive information volume is calculated based on the correlation coefficient and standard deviation. The formula is as follows: ; in, For the The comprehensive information content of the indicators, For the The standard deviation of the index, For the The correlation coefficient of the indicators; The formula for standard deviation is as follows: ; in, For the The standard deviation of the index, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The average value of the indicators; The formula for the correlation coefficient is as follows: ; in, For the The correlation coefficient of the indicators, For the Indicators and The covariance of the indicators, For the The standard deviation of the index, For the The standard deviation of the indicator.

[0048] In this embodiment, the specific calculation of the comprehensive information amount is disclosed to facilitate determination of a constant weight vector based on the comprehensive information amount.

[0049] In one embodiment, constructing a balance function based on the comprehensive information amount and solving to obtain a variable weight coefficient of a corresponding evaluation index includes: Based on the comprehensive information, the equilibrium function is constructed and the state variable weight vector is calculated by derivation of the equilibrium function. The formula is as follows: ; in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator changes its weight; The equilibrium function formula is as follows: ; in, For the The equilibrium function of the index, For the The variable weight factor of each indicator, For the The evaluation index of each indicator changes its weight; The state of each indicator becomes a weight vector, and the formula is as follows: ; in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator becomes the weight. For the Variable weight factor of each indicator.

[0050] Among them, the constant weight vector between indicators can be determined by evaluating the comprehensive information of the indicators: ; In the formula, ; The corresponding variable weight vector It can be expressed as: ; in, For the The evaluation index of each indicator changes its weight.

[0051] The main method of changing weights is to construct a state-changing weight vector to achieve weighted adjustment of the state to avoid state imbalance. The state-changing weight vector can be expressed as a mapping relationship: ; Variable weight vector The state-variable weight vector With constant weight vector The Hadamard (element-wise) product is: ; The core of variable weight solution is to construct a balance function according to the influence of brightness, color, texture, color features, shape features, and spatial position relationship features, and calculate the state variable weight vector by derivation of the balance function. The calculation formula is as follows: ; in, Situation Assessment Indicator As the brightness, color, texture and other indicators increase, the influence on the information contained in the image becomes greater and greater. The positive correlation nonlinear function is used for incentive variable weighting. The equalization function is expressed as: ; in, For the The variable weight factor of each indicator, =1, 2, ..., 5, >0 is incentive variable weight, <0 is a penalty variable weight, The values ​​of are various threat factors, among which 1 is brightness, 2 is color, 3 is texture, 4 is color feature, 5 is shape feature, and 6 is spatial position relationship.

[0052] The state variable weight vector of each indicator is expressed as: ; In summary, the CRITIC variable weights of each indicator are expressed as: ; In this embodiment, it is specifically disclosed how to solve the variable weight vector based on the equilibrium equation, and it is also disclosed how to obtain the constant weight vector and the relationship between the constant weight vector and the variable weight vector.

[0053] This embodiment adopts the above method, decomposes the acquired visible light and infrared light images through Gaussian filters to obtain feature maps and original images, calculates variable weight coefficients for the visible light original image and the feature map respectively, matches the changing variable weight coefficients at different times and fuses the feature maps of visible light and infrared light respectively, fuses the original images of visible light and infrared light, and finally fuses the fused feature map and the original image to finally obtain a fused image. At different times, especially when the light is too bright or too dark, the variable weight coefficient is matched to the weight that best reflects the characteristics of the fused image at present, thereby improving the fusion effect of the fused image.

[0054] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0055] Based on the same inventive concept, the embodiment of the present application also provides a fusion device of infrared and visible light images based on CRITIC weights for implementing the above-mentioned fusion method of infrared and visible light images based on CRITIC weights. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more fusion devices of infrared and visible light images based on CRITIC weights provided below can be referred to the limitations of the fusion method of infrared and visible light images based on CRITIC weights above, and will not be repeated here.

[0056] In one embodiment, Figure 3 As shown, a fusion system of infrared and visible light images based on CRITIC weights is provided, including: a feature original image decomposition module, a variable weight coefficient calculation module, an infrared visible light original image fusion module, an infrared visible light feature image fusion module and a visible light infrared fusion image fusion module, wherein: A feature original image decomposition module is used to decompose the input visible light image and infrared image into a visible light feature image, a visible light original image, an infrared feature image and an infrared original image based on a Gaussian filter; A variable weight coefficient calculation module, used to calculate corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; An infrared and visible light original image fusion module is used to fuse the visible light original image and the infrared original image based on a variable weight coefficient and obtain a visible light fusion image; An infrared-visible light feature map fusion module is used to fuse the visible feature map and the infrared feature map based on a variable weight coefficient and obtain an infrared fusion map; The visible light infrared fusion image fusion module is used to fuse the visible light fusion image and the infrared fusion image to obtain the final fusion image.

[0057] In one embodiment, the variable weight coefficient calculation module is also used to construct a situation assessment matrix based on the real-time assessment state and the assessment indicators; normalize the situation assessment matrix; calculate the comprehensive information between the assessment indicators; determine the constant weight vector between the assessment indicators based on the comprehensive information; obtain the variable weight vector based on the element-by-element multiplication of the state variable weight vector and the constant weight vector; construct an equilibrium function based on the comprehensive information and solve it to obtain the variable weight coefficient of the corresponding assessment indicator.

[0058] In one embodiment, the variable weight coefficient calculation module is also used to obtain the real-time evaluation status at a preset time and form a real-time evaluation status information set; obtain a preset number of evaluation indicators and form an indicator set; and construct a situation assessment matrix based on the real-time evaluation status information set and the indicator set.

[0059] In one embodiment, the variable weight coefficient calculation module is also used when the evaluation index is a benefit index, and the formula is as follows: ;in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; when the evaluation index is a cost-type index, the formula is as follows: ;in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; when the evaluation index is a fixed index, the formula is as follows: ;in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, It is the best value for the fixed indicator.

[0060] In one embodiment, the variable weight coefficient calculation module is also used to calculate the comprehensive information based on the correlation coefficient and the standard deviation, and the formula is as follows: ;in, For the The comprehensive information content of the indicators, For the The standard deviation of the index, For the The correlation coefficient of the indicators is as follows: ;in, For the The standard deviation of the index, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The average value of the indicators; the formula of the correlation coefficient is as follows: ;in, For the The correlation coefficient of the indicators, For the Indicators and The covariance of the indicators, For the The standard deviation of the index, For the The standard deviation of the indicator.

[0061] In one embodiment, the variable weight coefficient calculation module is further used to: construct a balance function based on the comprehensive information and calculate the state variable weight vector by derivation of the balance function, the formula is as follows: ;in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator becomes the weight; the equilibrium function formula is as follows: ;in, For the The equilibrium function of the index, For the The variable weight factor of each indicator, For the The evaluation index of each indicator becomes the weight; the state of each indicator becomes the weight vector, and the formula is as follows: ;in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator becomes the weight. For the Variable weight factor of each indicator.

[0062] Each module in the above-mentioned CRITIC weight-based infrared and visible light image fusion device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0063] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for fusing infrared and visible light images based on CRITIC weights is implemented.

[0064] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for fusing infrared and visible light images based on CRITIC weights is implemented.

[0065] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0066] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0067] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0068] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0069] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0070] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0071] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for fusing infrared and visible light images based on CRITIC weights, characterized in that: The method comprises: Decomposing the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; Calculate the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; The visible light original image and the infrared original image are fused based on the variable weight coefficient to obtain a visible light fusion image; The visible feature map and the infrared feature map are fused based on the variable weight coefficient to obtain an infrared fusion map; The visible light fusion image and the infrared fusion image are fused to obtain the final fusion image.

2. The method for fusing infrared and visible light images based on CRITIC weights according to claim 1, characterized in that: The calculating of the corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively comprises: Construct a situation assessment matrix based on real-time assessment status and assessment indicators; normalizing the situation assessment matrix; Calculating the comprehensive information between the evaluation indicators; Determine a constant weight vector between evaluation indicators based on the comprehensive information amount; Obtain a variable weight vector based on element-by-element multiplication of the state variable weight vector and the constant weight vector; Based on the comprehensive information, the equilibrium function is constructed and solved to obtain the variable weight coefficient of the corresponding evaluation index.

3. The method for fusing infrared and visible light images based on CRITIC weights according to claim 2, characterized in that: The construction of a situation assessment matrix based on real-time assessment status and assessment indicators includes: Obtaining the real-time evaluation status at a preset time and forming a real-time evaluation status information set; Obtain a preset number of evaluation indicators and form an indicator set; Construct a situation assessment matrix based on the real-time assessment status information set and indicator set.

4. The method for fusing infrared and visible light images based on CRITIC weights according to claim 2, characterized in that: The normalized situation assessment matrix includes: When the evaluation index is a benefit index, the formula is as follows: ； in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a cost-based index, the formula is as follows: ； in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, For the The moment corresponding to The minimum value of the index value, For the The moment corresponding to The maximum value of the index value; When the evaluation index is a fixed index, the formula is as follows: ； in, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The moment corresponding to Item index value, It is the best value for the fixed indicator.

5. The method for fusing infrared and visible light images based on CRITIC weights according to claim 2, characterized in that: The calculating of the comprehensive information between the evaluation indicators comprises: The comprehensive information volume is calculated based on the correlation coefficient and standard deviation. The formula is as follows: ； in, For the The comprehensive information content of the indicators, For the The standard deviation of the index, For the The correlation coefficient of the indicators; The formula for standard deviation is as follows: ； in, For the The standard deviation of the index, For the The moment corresponding to The standardized evaluation matrix of the indicators, For the The average value of the indicators; The formula for the correlation coefficient is as follows: ； in, For the The correlation coefficient of the indicators, For the Indicators and The covariance of the indicators, For the The standard deviation of the index, For the The standard deviation of the indicator.

6. The method for fusing infrared and visible light images based on CRITIC weights according to claim 2, characterized in that: The method of constructing a balance function based on the comprehensive information amount and solving to obtain the variable weight coefficient of the corresponding evaluation index includes: Based on the comprehensive information, the equilibrium function is constructed and the state variable weight vector is calculated by derivation of the equilibrium function. The formula is as follows: ； in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator changes its weight; The equilibrium function formula is as follows: ； in, For the The equilibrium function of the index, For the The variable weight factor of each indicator, For the The evaluation index of each indicator changes its weight; The state of each indicator becomes a weight vector, and the formula is as follows: ； in, For the The state of the indicator becomes the weight vector, Situation Assessment Indicator The equilibrium function, For the The evaluation index of each indicator becomes the weight. For the Variable weight factor of each indicator.

7. A fusion system of infrared and visible light images based on CRITIC weights, characterized in that: The system comprises: A feature original image decomposition module is used to decompose the input visible light image and infrared image into a visible light feature map, a visible light original map, an infrared feature map and an infrared original map; A variable weight coefficient calculation module, used to calculate corresponding variable weight coefficients based on the visible light original image and the visible light feature image respectively; An infrared and visible light original image fusion module is used to fuse the visible light original image and the infrared original image based on a variable weight coefficient and obtain a visible light fusion image; An infrared-visible light feature map fusion module is used to fuse the visible feature map and the infrared feature map based on a variable weight coefficient and obtain an infrared fusion map; The visible light infrared fusion image fusion module is used to fuse the visible light fusion image and the infrared fusion image to obtain the final fusion image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images