Image processing method, device and equipment

Abstract

The present application provides an image processing method, device and equipment, the method comprising: determining a visible light brightness image based on a visible light image, determining an infrared light brightness image based on an infrared light image; determining a differential feature image between the visible light brightness image and the infrared light brightness image; determining an initial infrared weight image corresponding to the infrared light brightness image based on the differential feature image; determining an attenuation coefficient based on the visible light brightness image and the infrared light brightness image, performing attenuation processing on the initial infrared weight image based on the attenuation coefficient to obtain a target infrared weight image corresponding to the infrared light brightness image; generating a visible light weight image corresponding to the visible light image based on the target infrared weight image, and generating a target image based on the visible light image, the visible light weight image, the infrared light brightness image and the target infrared weight image. Through the technical solution of the present application, the clarity and transparency of low-light images can be effectively improved, and the image color cast problem can be avoided.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and equipment. Background Art

[0002] When front-end devices (such as analog cameras, network cameras, etc.) collect images of night scenes, in order to collect high-definition images, high-brightness fill lights are often required. However, these fill lights will produce relatively strong light pollution, which has a strong stimulation to the driver's eyes and poses a traffic safety hazard.

[0003] In order to improve the brightness and clarity of the image, a method based on infrared fusion enhancement was developed. It mainly uses two starlight-level sensors to collect images. One sensor obtains the image brightness through infrared fill light, and the other sensor collects environmental color information. Infrared fusion enhancement is performed based on the above two images to improve the image brightness and clarity. This solution greatly reduces the intensity of the fill light. However, when encountering objects made of infrared reflective materials, the method based on infrared fusion enhancement has certain disadvantages in color restoration, which affects the image quality and results in poor image effects. For example, when using the infrared fusion enhancement method, if the clothes are made of black infrared reflective material, the black may be restored to gray or white, resulting in image color cast problems. Summary of the invention

[0004] The present application provides an image processing method, the method comprising:

[0005] Acquire a visible light image and an infrared light image at the same acquisition time, determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image;

[0006] Determining a differential feature image between the visible light brightness image and the infrared light brightness image;

[0007] Determining an initial infrared weight image corresponding to the infrared brightness image based on the differential feature image;

[0008] Determine an attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image; wherein the initial infrared weighted image includes a plurality of initial weight values, and the target infrared weighted image includes a plurality of target weight values ​​corresponding to the plurality of initial weight values;

[0009] A visible light weighted image corresponding to the visible light image is generated based on the target infrared weighted image, and a target image is generated based on the visible light image, the visible light weighted image, the infrared brightness image and the target infrared weighted image.

[0010] The present application provides an image processing device, the device comprising:

[0011] An acquisition module, used to acquire a visible light image and an infrared light image at the same acquisition moment, determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image;

[0012] A determination module, used to determine a differential feature image between the visible light brightness image and the infrared light brightness image; determine an initial infrared weighted image corresponding to the infrared light brightness image based on the differential feature image; determine an attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image; wherein the initial infrared weighted image includes a plurality of initial weight values, and the target infrared weighted image includes a plurality of target weight values ​​corresponding to the plurality of initial weight values;

[0013] A generation module is used to generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image, and to generate a target image based on the visible light image, the visible light weighted image, the infrared brightness image and the target infrared weighted image.

[0014] The present application provides an image processing device, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the above-mentioned image processing method.

[0015] It can be seen from the above technical solutions that in the embodiment of the present application, the infrared brightness image is an image obtained based on infrared strobe (invisible to the human eye) fill light, and the infrared brightness image is very clear. By determining the differential feature image between the infrared brightness image and the visible light brightness image, and determining the infrared weight image based on the differential feature image, the visible light image (i.e., white light image) and the infrared brightness image are fused based on the infrared weight image to obtain the fused target image. When the images are fused, for the shadow (or black) area, the weight of the infrared brightness image determined based on the infrared weight image is still relatively large, and the weight of the shadow area of ​​the visible light image is adjusted to further limit the brightness enhancement ratio so that it does not deviate from its own hue.

[0016] In the above method, infrared information can be used to enhance the visible light image (i.e., low-light image), which can effectively improve the brightness, clarity, and transparency of the visible light image, and the enhanced target image has no color cast. The initial infrared weighted image is determined using a differential feature image (the differential feature image can well reflect the area that is prone to color cast in fusion, and the color cast problem of fusion can be avoided when the differential feature image is used), and the initial infrared weighted image is corrected using an attenuation coefficient to obtain a target infrared weighted image, so that the visible light image is fused based on the target infrared weighted image, which can effectively avoid the side effect of image color cast caused by fusion of infrared information, and the fused target image is more realistic and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0018] Figure 1 is a flowchart of an image processing method in one embodiment of the present application;

[0019] Figure 2 is a flowchart of an image processing method in one embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a differential weight mapping curve in one embodiment of the present application;

[0021] Figure 4 is a schematic diagram of an attenuation mapping relationship in one implementation of the present application;

[0022] Figure 5 is a flowchart of an image processing method in one embodiment of the present application;

[0023] Figure 6 is a schematic structural diagram of an image processing device in one embodiment of the present application;

[0024] Figure 7 It is a hardware structure diagram of an image processing device in one embodiment of the present application. DETAILED DESCRIPTION

[0025] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, rather than limiting the present application. The singular forms of "a", "said" and "the" used in the present application and claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0026] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, in addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination".

[0027] In the embodiment of the present application, an image processing method is proposed, which can be applied to front-end devices (such as network cameras, analog cameras, etc.), and can also be applied to back-end devices (such as servers, management devices, storage devices, etc.). If the method is applied to the front-end device, the front-end device collects visible light images and infrared images for the same target scene, and based on the visible light images and infrared images, the front-end device uses the embodiment of the present application to perform image processing. If the method is applied to the back-end device, the front-end device collects visible light images and infrared images for the same target scene, and sends the visible light images and infrared images to the back-end device, and the back-end device uses the embodiment of the present application to perform image processing based on the visible light images and infrared images.

[0028] The visible light image and the infrared light image are images of the same target scene, and the acquisition time of the visible light image and the infrared light image is the same, that is, the exposure time of the visible light image is the same as the exposure time of the infrared light image. For example, the exposure start time of the visible light image can be the same as the exposure start time of the infrared light image, and the exposure end time of the visible light image can be the same as the exposure end time of the infrared light image.

[0029] In an embodiment of the present application, a visible light image and an infrared light image can be fused to obtain a fused target image. When performing image fusion, a differential feature image between the infrared light image and the visible light image is determined, an infrared weighted image and a visible light weighted image are generated based on the differential feature image, and the weight value of the visible light image and the weight value of the infrared light image are determined based on the infrared weighted image and the visible light weighted image. This allows the high signal-to-noise ratio under infrared imaging to be utilized to effectively improve the brightness, clarity and transparency of low-illuminance images, effectively avoid the side effect of color cast of infrared images, and make the synthesized image more realistic and natural.

[0030] The technical solution of the embodiments of the present application is described below in conjunction with specific embodiments.

[0031] See also Figure 1 FIG. 1 is a flow chart of an image processing method, which may include:

[0032] Step 101: Acquire a visible light image and an infrared light image at the same acquisition time.

[0033] Exemplarily, the front-end device may be a dual-sensor front-end device, where the dual sensors refer to a visible light sensor and an infrared light sensor. The visible light sensor is a sensor for obtaining the visible light intensity of an object, and the image collected by the visible light sensor is called a visible light image. The infrared light sensor is a sensor for obtaining the infrared light intensity of an object, and the image collected by the infrared light sensor is called an infrared light image. Based on this, the visible light image and the infrared light image at the same collection time can be obtained through the dual sensors of the front-end device.

[0034] Step 102: determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image. Exemplarily, the infrared light image may include an infrared light brightness channel, based on which the infrared light brightness channel may be directly determined as the infrared light brightness image.

[0035] In order to determine the visible light brightness image, in a possible implementation, if the visible light image may include a visible light image in RGB (Red Green Blue) format, the visible light image in RGB format may be converted into a visible light image in YUV (brightness chrominance) format, and the visible light image in YUV format may include a brightness channel Y, a chroma channel U, and a chroma channel V. On this basis, the brightness channel Y in the visible light image in YUV format may be determined as the visible light brightness image.

[0036] In order to determine the visible light brightness image, in another possible implementation, if the visible light image may include a visible light image in RGB format, the R channel, G channel, and B channel corresponding to the visible light image in RGB format may be determined. On this basis, for each pixel, based on the pixel value of the pixel in the R channel, the pixel value of the pixel in the G channel, and the pixel value of the pixel in the B channel, the target pixel value of the pixel is determined; the visible light brightness image is determined based on the target pixel values ​​of all pixels, for example, the target pixel values ​​of all pixels are combined into the visible light brightness image.

[0037] Step 103: determine a differential feature image between the visible light brightness image and the infrared light brightness image, where the differential feature image includes a plurality of differential feature values ​​for indicating a brightness comparison of the brightness value.

[0038] In a possible implementation, the differential feature image between the visible light brightness image and the infrared light brightness image can be determined based on the absolute value of the difference between the visible light brightness image and the infrared light brightness image. For example, for a pixel point (x, y), x represents the horizontal coordinate, y represents the vertical coordinate, and the absolute value of the difference between the brightness value of the pixel point (x, y) in the visible light brightness image and the brightness value of the pixel point (x, y) in the infrared light brightness image is used as the differential feature value of the pixel point (x, y) in the differential feature image. Obviously, after performing the above processing on each pixel point, the differential feature value of each pixel point in the differential feature image can be obtained, and the differential feature values ​​of all pixels in the differential feature image are combined into a differential feature image.

[0039] In another possible implementation, the visible light brightness image can be filtered to obtain a visible light brightness characteristic image, and the infrared light brightness image can be filtered to obtain an infrared light brightness characteristic image. Then, the difference between the visible light brightness characteristic image and the infrared light brightness characteristic image is determined, and the differential characteristic image between the visible light brightness image and the infrared light brightness image is determined based on the absolute value of the difference. For example, for the pixel point (x, y), the difference between the brightness value of the pixel point (x, y) in the visible light brightness characteristic image and the brightness value of the pixel point (x, y) in the infrared light brightness characteristic image is calculated, and the absolute value of the difference is used as the differential characteristic value of the pixel point (x, y) in the differential characteristic image. Obviously, after performing the above processing on each pixel point, the differential characteristic value of each pixel point in the differential characteristic image can be obtained, and the differential characteristic values ​​of all pixels in the differential characteristic image are combined into a differential characteristic image.

[0040] In the above embodiment, since the brightness value range of the visible light brightness image is [0, 255], the brightness value range of the infrared light brightness image is [0, 255], the brightness value range of the visible light brightness feature image is [0, 255], and the brightness value range of the infrared light brightness feature image is [0, 255], therefore, when the absolute value of the difference in brightness values ​​is used as the differential feature value, the differential feature value can be in the range of [0, 255].

[0041] Based on the differential feature image, it can be seen that when the value of the differential feature value is larger, the difference in brightness values ​​between the visible light brightness image and the infrared light brightness image is larger, and when the value of the differential feature value is smaller, the difference in brightness values ​​between the visible light brightness image and the infrared light brightness image is smaller.

[0042] Step 104: Determine an initial infrared weight image corresponding to the infrared brightness image based on the differential feature image, and the initial infrared weight image may include multiple initial weight values. For example, if the differential feature image includes multiple differential feature values, the initial infrared weight image includes multiple initial weight values ​​corresponding to the multiple differential feature values. Exemplarily, when the differential feature value is the minimum, the initial weight value corresponding to the differential feature value is the maximum, and as the differential feature value increases, the initial weight value corresponding to the differential feature value becomes smaller.

[0043] In a possible implementation, for each differential eigenvalue in the differential feature image, the configured differential weight mapping relationship is queried through the differential eigenvalue to obtain an initial weight value corresponding to the differential eigenvalue; and the initial infrared weighted image is determined based on the initial weight values ​​corresponding to all differential eigenvalues.

[0044] For example, for the pixel point (x, y), the differential feature value of the pixel point (x, y) can be selected from the differential feature image, and the differential weight mapping relationship can be queried through the differential feature value of the pixel point (x, y) to obtain the initial weight value corresponding to the differential feature value of the pixel point (x, y), that is, the initial weight value of the pixel point (x, y). After performing the above processing on each pixel point in the differential feature image, the initial weight value of each pixel point in the differential feature image can be obtained, and the initial weight values ​​of these pixels are combined to form the initial infrared weight image.

[0045] Exemplarily, the differential weight mapping relationship includes a correspondence between differential eigenvalues ​​and initial weight values; in the differential weight mapping relationship, when the differential eigenvalue is the smallest, the initial weight value corresponding to the differential eigenvalue is the largest, and as the differential eigenvalue increases, the initial weight value corresponding to the differential eigenvalue becomes smaller.

[0046] Step 105: determine the attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image. Exemplarily, the initial infrared weighted image may include multiple initial weight values, and the target infrared weighted image may include multiple target weight values ​​corresponding to the multiple initial weight values.

[0047] In a possible implementation, for each pixel, if the first brightness value of the pixel in the visible light brightness image is greater than or equal to the second brightness value of the pixel in the infrared light brightness image, it can be determined that the attenuation coefficient corresponding to the pixel is 1; if the first brightness value is less than the second brightness value, the configured attenuation mapping relationship can be queried through the first brightness value to obtain the attenuation coefficient corresponding to the pixel. Exemplarily, the attenuation mapping relationship may include a corresponding relationship between brightness value and attenuation coefficient, when the brightness value is the minimum, the attenuation coefficient corresponding to the brightness value is the minimum, as the brightness value increases, the attenuation coefficient corresponding to the brightness value increases, and the attenuation coefficient corresponding to the brightness value is between 0 and 1.

[0048] To summarize, for each pixel in the initial infrared weighted image, the attenuation coefficient corresponding to the pixel can be obtained. On this basis, for each pixel in the initial infrared weighted image, the target weight value corresponding to the pixel can be determined based on the initial weight value corresponding to the pixel and the attenuation coefficient corresponding to the pixel; then, the target infrared weighted image is generated based on the target weight values ​​corresponding to all pixels in the initial infrared weighted image, that is, the target weight values ​​corresponding to these pixels form the target infrared weighted image.

[0049] For example, for the pixel point (x, y) in the initial infrared weighted image, if the first brightness value of the pixel point (x, y) in the visible light brightness image is greater than or equal to the second brightness value of the pixel point (x, y) in the infrared light brightness image, the attenuation coefficient corresponding to the pixel point (x, y) is determined to be 1; if the first brightness value is less than the second brightness value, the attenuation mapping relationship is queried through the first brightness value to obtain the attenuation coefficient corresponding to the pixel point (x, y). The target weight value corresponding to the pixel point (x, y) is determined based on the initial weight value corresponding to the pixel point (x, y) and the attenuation coefficient corresponding to the pixel point (x, y), such as the product of the initial weight value and the attenuation coefficient as the target weight value. After performing the above processing on each pixel point in the initial infrared weighted image, the target weight value of each pixel point in the initial infrared weighted image can be obtained, and the target weight values ​​of these pixels are combined into a target infrared weighted image.

[0050] Step 106: Generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image.

[0051] In a possible implementation, a visible light weight image can be generated based on the target infrared weight image. For example, for each pixel, the difference between the maximum weight value and the target weight value corresponding to the pixel is used as the visible light weight value corresponding to the pixel in the visible light weight image. Then, the visible light weight values ​​corresponding to all pixels in the visible light weight image are combined into a visible light weight image.

[0052] For example, assuming that the target weight value in the target infrared weighted image is in the range of [0,1], and the visible light weight value in the visible light weighted image is in the range of [0,1], that is, the maximum weight value is 1 and the minimum weight value is 0, then for the pixel point (x, y), assuming that the target weight value of the pixel point (x, y) in the target infrared weighted image is q1, then the corresponding visible light weight value of the pixel point (x, y) in the visible light weighted image is 1-q1.

[0053] Step 107: Generate a target image based on the visible light image, the visible light weighted image, the infrared light brightness image and the target infrared weighted image. For example, perform weighted fusion on the visible light image, the visible light weighted image, the infrared light brightness image and the target infrared weighted image to obtain the target image to be output.

[0054] In a possible implementation, the visible light image is a visible light image in RGB format, and the visible light image in RGB format can be converted into a visible light image in YUV format, and the visible light image in YUV format includes a visible light brightness image (such as brightness channel Y) and a visible light chromaticity image (such as chromaticity channel U and chromaticity channel V). On this basis, a fused brightness image can be determined based on the visible light brightness image, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared light detail layer image corresponding to the infrared light brightness image. A brightness gain value is determined based on the fused brightness image and the visible light brightness image; a fused chromaticity image is determined based on the visible light chromaticity image and the brightness gain value; and a target image is generated based on the fused brightness image and the fused chromaticity image.

[0055] In another possible implementation, the visible light image may be a visible light image in RGB format, and the visible light image in RGB format may include an R channel, a G channel, and a B channel. On this basis, the fused R channel may be determined based on the R channel, the visible light weight image, the infrared brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared detail layer image corresponding to the infrared brightness image. The fused G channel may be determined based on the G channel, the visible light weight image, the infrared brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared detail layer image corresponding to the infrared brightness image. The fused B channel may be determined based on the B channel, the visible light weight image, the infrared brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared detail layer image corresponding to the infrared brightness image. Then, the target image may be generated based on the fused R channel, the fused G channel, and the fused B channel.

[0056] Exemplarily, the method for determining the visible light detail layer image corresponding to the visible light brightness image may include, but is not limited to: low-pass filtering the visible light brightness image to obtain a visible light brightness low-pass image; and determining the visible light detail layer image based on the difference between the visible light brightness image and the visible light brightness low-pass image. The method for determining the infrared light detail layer image corresponding to the infrared light brightness image may include, but is not limited to: low-pass filtering the infrared light brightness image to obtain an infrared light brightness low-pass image; and determining the infrared light detail layer image based on the difference between the infrared light brightness image and the infrared light brightness low-pass image.

[0057] It can be seen from the above technical scheme that in the embodiment of the present application, the infrared brightness image is an image obtained based on the infrared flash (invisible to the human eye) fill light, and the infrared brightness image has a very high signal-to-noise ratio. Therefore, the infrared brightness image is very clear. By determining the differential feature image between the infrared brightness image and the visible light brightness image, and determining the infrared weight image based on the differential feature image, the visible light image (i.e., white light image) and the infrared brightness image are fused based on the infrared weight image to obtain the fused target image. When the image is fused, for the area of ​​interest, such as the vehicle, face area, etc., the weight of the infrared brightness image determined based on the infrared weight image is still large, so that the clear, bright and transparent characteristics under infrared imaging can be utilized to effectively improve the quality of low-light visible light images. For the infrared reflection abnormal area, the weight of the visible light image determined based on the infrared weight image is small, so that it will not affect the color of the image itself, that is, it will not affect the imaging effects such as brightness and body color, and will not cause color cast problems in the fusion of black areas (or shadow areas). And, the weight of the shadow area of ​​the visible light image is adjusted to further limit the brightness increase ratio so that it does not deviate from its own hue.

[0058] In the above method, infrared information can be used to enhance the visible light image (i.e., low-light image), which can effectively improve the brightness, clarity, and transparency of the visible light image, and the enhanced target image has no color cast. The initial infrared weighted image is determined using a differential feature image (the differential feature image can well reflect the area that is prone to color cast in fusion, and the color cast problem of fusion can be avoided when the differential feature image is used), and the initial infrared weighted image is corrected using an attenuation coefficient to obtain a target infrared weighted image, so that the visible light image is fused based on the target infrared weighted image, which can effectively avoid the side effect of image color cast caused by fusion of infrared information, and the fused target image is more realistic and natural.

[0059] The following describes the image processing method of the embodiment of the present application in combination with specific application scenarios.

[0060] In order to collect high-definition images at night, improve image brightness and clarity, and obtain key information such as faces and license plates, infrared fusion enhancement methods can be used to process images. For example, the front-end equipment deploys visible light sensors and infrared light sensors. The infrared light sensor collects infrared light images and obtains image brightness through infrared fill light. The visible light sensor collects visible light images and obtains environmental color information. By fusing infrared light images and visible light images, the image brightness and clarity can be improved. However, when encountering objects made of infrared reflective materials, when fusing infrared light images and visible light images, the fused image may have color cast problems, affecting the imaging quality and resulting in poor image effects.

[0061] In response to the above problems, an image processing method is proposed in an embodiment of the present application, which can generate an infrared weighted image and a visible light weighted image based on a differential feature image, and fuse the infrared light image and the visible light image based on the infrared weighted image and the visible light weighted image, thereby improving the image brightness and clarity. Even if encountering objects made of infrared reflective materials, the fused image will not have color cast problems.

[0062] See also Figure 2 FIG. 1 is a flow chart of an image processing method, which may include:

[0063] Step 201: Obtain a visible light image and an infrared light image at the same acquisition time. The visible light image may be a visible light image in RGB format. The infrared light image may include an infrared light brightness channel. The infrared light brightness channel may be determined as an infrared light brightness image, which is recorded as infrared light brightness image Y. nir .

[0064] Exemplarily, the parameters such as shutter, gain, and aperture value of the front-end device can be set. After the parameters are set, the visible light image and the infrared light image at the same acquisition time can be acquired through the dual sensors of the front-end device. For example, the front-end device can include a visible light sensor and an infrared light sensor. The visible light image I is acquired through the visible light sensor. vis , and collect infrared light images I through infrared light sensors nir .

[0065] Visible light image I vis The size and infrared image I nir The same size as the visible light image I vis The width of the infrared image I nir The width of the visible light image I vis The height and infrared image I nir The height is the same.

[0066] Exemplarily, the infrared light image is an image obtained based on infrared strobe fill light, and the brightness of the infrared light image is greater than the strong light of the vehicle's high beam, so the imaging around the vehicle's lights in the infrared light image is very clear.

[0067] Step 202: convert the visible light image in RGB format into a visible light image in YUV format. There is no restriction on the conversion method. The visible light image in YUV format may include a brightness channel Y, a chroma channel U, and a chroma channel V. The brightness channel Y in the visible light image in YUV format may be determined as the visible light brightness image Y. vis , the chrominance channel U in the visible light image in YUV format is determined as the visible light chrominance image U vis , the chroma channel V in the visible light image in YUV format is determined as the visible light chroma image V vis .

[0068] At this point, the infrared brightness image Y at the same acquisition time is obtained nir and visible light brightness image Y vis .

[0069] Step 203: filter the visible light brightness image to obtain a visible light brightness characteristic image, and filter the infrared light brightness image to obtain an infrared light brightness characteristic image.

[0070] For example, the visible light brightness image Y vis Perform filtering processing (such as low-pass edge-preserving filtering processing, etc.) to obtain the visible light brightness feature image F vis For example, using median filtering, bilateral filtering, or guided filtering algorithms, the visible light brightness image Y vis Perform filtering to obtain the visible light brightness feature image F vis , see the following formula, which is an example of filtering: In the above formula, K is a low-pass edge-preserving filter operator. vis Filtering can smooth the visible light brightness image Y vis The local part of the visible light brightness image Y is eliminated vis The effect of singular points (or noise).

[0071] For example, the infrared brightness image Y nir Perform filtering processing (such as low-pass edge-preserving filtering processing, etc.) to obtain the infrared light brightness feature image F nir For example, using median filtering, bilateral filtering, or guided filtering algorithms, the infrared brightness image Y nir Perform filtering to obtain the infrared brightness feature image F nir , see the following formula, which is an example of filtering: In the above formula, K is a low-pass edge-preserving filter operator. nir Filtering can smooth the infrared brightness image Y nir Part of the infrared brightness image Y is eliminated nir The effect of singular points (or noise).

[0072] Step 204: Determine a differential feature image between the visible light brightness feature image and the infrared light brightness feature image based on the absolute value of the difference between the visible light brightness feature image and the infrared light brightness feature image, wherein the differential feature image includes a plurality of differential feature values ​​for indicating a brightness comparison of the brightness value.

[0073] Exemplarily, both subtraction and quotient are used to compare the magnitude of image values. "Subtraction" can measure the difference in absolute values, and "quotient" can compare the difference in relative values. However, when the divisor is small, "quotient" causes the value to be extremely unstable. Therefore, in this embodiment, "subtraction" is selected to compare the magnitude of image values. In summary, the differential feature image between the visible light brightness feature image and the infrared light brightness feature image can be determined based on the absolute value of the difference between the visible light brightness feature image and the infrared light brightness feature image.

[0074] See the following formula for an example of calculating a differential feature image: dif (x) = abs(F nir (x)-F vis (x)). In the above formula, (x) represents any pixel in the image, denoted as pixel (x), that is, the xth pixel in the image. nir (x) represents the pixel point (x) in the infrared light brightness feature image F nir The brightness value in F vis (x) represents the pixel point (x) in the visible light brightness feature image F vis The brightness value in, abs is the absolute value, F dif (x) represents the differential feature value of the pixel point (x) in the differential feature image. In summary, the pixel point (x) in the infrared brightness feature image F nir The brightness value in the image and the pixel point (x) in the visible light brightness feature image F vis The absolute value of the difference in brightness values ​​in is taken as the differential feature value of the pixel (x) in the differential feature image. Obviously, after performing the above processing on each pixel, the differential feature value of each pixel in the differential feature image can be obtained, and the differential feature values ​​of all pixels in the differential feature image are combined into a differential feature image.

[0075] Visible light brightness characteristic image F visThe brightness value in the range is [0,255], and the infrared brightness feature image F nir The range of the brightness value in is [0,255]. When the absolute value of the brightness difference is used as the differential eigenvalue, the range of the differential eigenvalue is [0,255]. When the value of the differential eigenvalue is larger, the difference in brightness between the visible light brightness image and the infrared light brightness image is larger. When the value of the differential eigenvalue is smaller, the difference in brightness between the visible light brightness image and the infrared light brightness image is smaller.

[0076] Exemplarily, after determining the differential feature image based on the absolute value of the difference between the visible light brightness feature image and the infrared light brightness feature image, the differential feature image can well reflect the area where fusion is prone to color cast, that is, using the differential feature image can effectively avoid fusion color cast. For example, the brighter the area in the differential feature image (i.e., the area with a larger differential feature value), the greater the difference in reflection characteristics between the visible light brightness image and the infrared light brightness image, and the area is prone to color cast. Conversely, the darker the area in the differential feature image (i.e., the area with a smaller differential feature value), the smaller the difference in reflection characteristics between the visible light brightness image and the infrared light brightness image, and the area is not prone to color cast.

[0077] Step 205: Determine an initial infrared weight image corresponding to the infrared brightness image based on the differential feature image, and the initial infrared weight image may include multiple initial weight values ​​corresponding to multiple differential feature values ​​in the differential feature image. Exemplarily, when the differential feature value is the smallest, the initial weight value corresponding to the differential feature value is the largest, and as the differential feature value increases, the initial weight value corresponding to the differential feature value becomes smaller.

[0078] Exemplarily, referring to the above embodiment, the brighter the area in the differential feature image (that is, the larger the value of the differential feature value), the greater the difference in reflection characteristics between the visible light brightness image and the infrared light brightness image, and the area is prone to color cast. The use of infrared brightness should be reduced, that is, the initial weight value should be relatively small. The darker the area in the differential feature image (that is, the smaller the value of the differential feature value), the smaller the difference in reflection characteristics between the visible light brightness image and the infrared light brightness image, and the area is not prone to color cast. The use of infrared brightness should be increased, that is, the initial weight value should be relatively large. In summary, when the differential feature value is larger, the initial weight value should be smaller, and when the differential feature value is smaller, the initial weight value should be larger.

[0079] Based on the above principle, in the embodiment of the present application, a differential weight mapping relationship can be pre-configured, and the differential weight mapping relationship can include a corresponding relationship between a differential eigenvalue and an initial weight value. In the differential weight mapping relationship, when the differential eigenvalue is the smallest, the initial weight value corresponding to the differential eigenvalue is the largest, and as the differential eigenvalue increases, the initial weight value corresponding to the differential eigenvalue becomes smaller.

[0080] Exemplarily, the differential weight mapping relationship may be a differential weight mapping curve, see Figure 3 As shown in the figure, an example of the differential weight mapping curve is shown. The horizontal axis of the differential weight mapping curve represents the differential eigenvalue, and the value range of the differential eigenvalue is [0,255]. The vertical axis of the differential weight mapping curve represents the initial weight value, and the value range of the initial weight value is [0,1]. It can be seen from the differential weight mapping curve that the initial weight value is inversely proportional to the differential eigenvalue, that is, the larger the differential eigenvalue, the smaller the initial weight value.

[0081] Exemplarily, the differential weight mapping relationship may be a differential weight mapping function, as shown in the following formula: α(x)=LUT(F dif (x)), is an example of a differential weight mapping function. dif (x) represents the differential eigenvalue of the pixel point (x) in the differential feature image, α(x) represents the initial weight value of the pixel point (x) in the initial infrared weighted image, and LUT represents the differential weight mapping function. The differential weight mapping function is pre-configured and there is no restriction on the differential weight mapping function. As long as it is based on the differential weight mapping function and can satisfy the condition that "the initial weight value is inversely proportional to the differential eigenvalue, that is, the larger the differential eigenvalue, the smaller the initial weight value", it will be fine.

[0082] Exemplarily, the differential weight mapping relationship can be a differential weight mapping table, which includes 256 differential eigenvalues ​​(such as 0, 1,..., 255, etc.) and initial weight values ​​corresponding to these differential eigenvalues. The differential weight mapping table can be pre-configured as long as it can satisfy "the initial weight value is inversely proportional to the differential eigenvalue, that is, the larger the differential eigenvalue, the smaller the initial weight value".

[0083] Of course, the above is just an example of the differential weight mapping relationship and is not limited to this.

[0084] In a possible implementation, based on the configured differential weight mapping relationship, for each differential feature value in the differential feature image, the differential weight mapping relationship is queried through the differential feature value to obtain the initial weight value corresponding to the differential feature value; then, the initial weight values ​​corresponding to all differential feature values ​​are combined into an initial infrared weight image. For example, for pixel point (x), the differential feature value of pixel point (x) can be selected from the differential feature image, and the differential weight mapping relationship is queried through the differential feature value of pixel point (x) to obtain the initial weight value corresponding to the differential feature value of pixel point (x), that is, the initial weight value of pixel point (x). After performing the above processing on each pixel point in the differential feature image, the initial weight value of each pixel point in the differential feature image can be obtained, and the initial weight values ​​of these pixels are combined into the initial infrared weight image, and the initial infrared weight image includes multiple initial weight values.

[0085] Exemplarily, the initial weight value in the initial infrared weighted image is the weight value of the infrared light image. Therefore, if the initial weight value is smaller, it means that when the images are fused, the weight value of the infrared light image is smaller, and the weight value of the visible light image is larger (the sum of the weight value of the visible light image and the weight value of the infrared light image is a fixed value, the smaller one weight value is, the larger the other weight value is). If the initial weight value is larger, it means that when the images are fused, the weight value of the infrared light image is larger, and the weight value of the visible light image is smaller.

[0086] In summary, when the value of the differential eigenvalue is larger, it is easier to produce color cast, and the initial weight value is smaller. Therefore, the use of infrared brightness can be reduced, so that when the images are fused, the weight value of the infrared image is smaller, and the weight value of the visible light image is larger, and the fused image is closer to the visible light image, avoiding color cast. When the value of the differential eigenvalue is smaller, it is less likely to produce color cast, and the initial weight value is larger. Therefore, the use of infrared brightness can be increased, so that when the images are fused, the weight value of the infrared image is larger, and the weight value of the visible light image is smaller. The fused image refers to the infrared image, effectively improving the signal-to-noise ratio.

[0087] Step 206: Determine the attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image. Exemplarily, the initial infrared weighted image may include multiple initial weight values, and the target infrared weighted image may include multiple target weight values ​​corresponding to the multiple initial weight values.

[0088] For example, for infrared brightness images, the black and gray in the infrared brightness images do not appear the same. For example, the absolute value of the brightness of a black garment is 10, and the brightness after fusion is 30. The absolute value of the brightness of another gray garment is 80, and the brightness after fusion is 100. The brightness is also increased by 20 absolute values. However, the brightness of the black garment is three times that before fusion, and the brightness of 30 after fusion looks more like gray. The brightness of the gray garment is only 25% higher than before fusion, and the brightness of 100 after fusion still looks gray.

[0089] From the above analysis, it can be seen that it is necessary to limit the increase ratio of the brightness value in the infrared brightness image so that the brightness value in the infrared brightness image does not deviate from its own hue. Therefore, the initial weight value corresponding to the brightness value in the infrared brightness image can be attenuated to obtain the target weight value, and the target weight value is less than or equal to the initial weight value, thereby reducing the increase ratio of the brightness value in the infrared brightness image.

[0090] Based on the above findings, in the embodiment of the present application, for each initial weight value in the initial infrared weighted image, the attenuation coefficient corresponding to the initial weight value can also be determined, and the initial weight value is attenuated based on the attenuation coefficient to obtain the target weight value corresponding to the initial weight value. After obtaining the target weight values ​​corresponding to all the initial weight values, all the target weight values ​​can be combined into a target infrared weighted image.

[0091] In a possible implementation, for the pixel point (x), based on the initial weight value of the pixel point (x), the target weight value corresponding to the pixel point (x) may be determined using the following formula (1):

[0092]

[0093] In formula (1), α(x) represents the initial weight value of pixel (x), and w(x) represents the target weight value of pixel (x). It can be seen from formula (1) that if pixel (x) is in the visible light brightness image Y vis The brightness value Y in vis (x) is greater than or equal to the pixel (x) in the infrared brightness image Y nir The brightness value Y in nir (x), the attenuation coefficient corresponding to the pixel point (x) can be 1. On this basis, the initial weight value α(x) of the pixel point (x) can be attenuated based on the attenuation coefficient 1 to obtain the target weight value w(x) of the pixel point (x), that is, w(x) = α(x)*1, that is, the target weight value w(x) of the pixel point (x) is equal to the initial weight value α(x) of the pixel point (x).

[0094] From formula (1), we can see that if the pixel point (x) is in the visible light brightness image Yvis The brightness value Y in vis (x) is smaller than the pixel (x) in the infrared brightness image Y nir The brightness value Y in nir (x), then the attenuation coefficient corresponding to the pixel point (x) is β(x). On this basis, the initial weight value α(x) of the pixel point (x) is attenuated based on the attenuation coefficient β(x) to obtain the target weight value w(x) of the pixel point (x), that is, w(x) = α(x) · β(x), that is, the target weight value w(x) of the pixel point (x) is equal to the product of the initial weight value α(x) of the pixel point (x) and the attenuation coefficient β(x).

[0095] Exemplarily, the attenuation coefficient β(x) is an attenuation coefficient ranging from 0 to 1, that is, the attenuation coefficient β(x) is greater than or equal to 0 and less than or equal to 1. As the visible light brightness value (that is, the brightness value of the pixel in the visible light brightness image) increases, the value of the attenuation coefficient β(x) becomes larger and larger. When the value of the attenuation coefficient β(x) is larger, the target weight value w(x) is closer to the initial weight value α(x), and the infrared weight is correspondingly not subject to attenuation constraints. The above method can adjust the visible light shadow area to fuse the infrared weight so that it does not have color cast.

[0096] For example, an attenuation mapping relationship (such as an attenuation mapping curve) can be pre-configured. The attenuation mapping relationship may include a correspondence between brightness value and attenuation coefficient. When the brightness value is the minimum, the attenuation coefficient corresponding to the brightness value is the minimum. As the brightness value increases, the attenuation coefficient corresponding to the brightness value increases, and the attenuation coefficient corresponding to the brightness value is between 0 and 1. Figure 4 , which is an example of the attenuation mapping relationship, the horizontal axis represents the brightness value of the pixel in the visible light brightness image, and the vertical axis represents the attenuation coefficient.

[0097] In summary, for a pixel (x), if the pixel (x) is in the visible light brightness image Y vis The brightness value Y in vis (x) is greater than or equal to the pixel (x) in the infrared brightness image Y nir The brightness value Y in nir (x), then the attenuation coefficient is 1, and the target weight value w(x) of the pixel point (x) is equal to the initial weight value α(x) of the pixel point (x).

[0098] If the pixel (x) is in the visible light brightness image Y vis The brightness value Y in vis (x) is smaller than the pixel (x) in the infrared brightness image Y nir The brightness value Y in nir (x), then the pixel point (x) is used in the visible light brightness image Y visThe brightness value Y in vis (x) Query Figure 4 The attenuation mapping relationship shown in the figure obtains the attenuation coefficient β(x), and the target weight value w(x) of the pixel point (x) is equal to the product of the initial weight value α(x) of the pixel point (x) and the attenuation coefficient β(x).

[0099] After performing the above processing on each pixel in the initial infrared weighted image, a target weight value of each pixel can be obtained, and the target weight values ​​of these pixels are combined into a target infrared weighted image.

[0100] Step 207: Generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image.

[0101] Exemplarily, the target infrared weighted image may include multiple target weight values, and the visible light weighted image may include multiple visible light weight values ​​corresponding to the multiple target weight values. For example, for pixel (x), the difference between the maximum weight value and the target weight value corresponding to pixel (x) is used as the visible light weight value corresponding to pixel (x) in the visible light weighted image. Then, the visible light weight values ​​corresponding to all pixels in the visible light weighted image are combined to form a visible light weighted image. For example, assuming that the target weight value range is [0,1] and the visible light weight value range is [0,1], that is, the maximum weight value is 1 and the minimum weight value is 0, then for pixel (x), assuming that the target weight value of pixel (x) in the target infrared weighted image is w(x), the visible light weight value corresponding to pixel (x) in the visible light weighted image is 1-w(x).

[0102] Step 208: low-pass filter the visible light brightness image to obtain a visible light brightness low-pass image; and determine a visible light detail layer image based on the difference between the visible light brightness image and the visible light brightness low-pass image.

[0103] For example, the visible light brightness image Y vis Perform low-pass filtering to obtain a low-pass image of visible light brightness. For example, use algorithms such as mean filtering or Gaussian filtering to filter the visible light brightness image Y vis Perform low-pass filtering to obtain a low-pass image of visible light brightness. Then, based on the visible light brightness image Y vis The difference between the visible light brightness low-pass image and the visible light detail layer image is determined as shown in the following formula (2):

[0104]

[0105] In formula (2), Represents the low-pass image of visible light brightness, that is, the low-pass filter operator G is used to filter the visible light brightness image Y visThe image after low-pass filtering, D vis It is the visible light detail layer image.

[0106] Step 209: low-pass filter the infrared brightness image to obtain an infrared brightness low-pass image; and determine the infrared detail layer image based on the difference between the infrared brightness image and the infrared brightness low-pass image.

[0107] For example, the infrared brightness image Y nir Perform low-pass filtering to obtain an infrared brightness low-pass image. For example, use algorithms such as mean filtering or Gaussian filtering to filter the infrared brightness image Y nir Perform low-pass filtering to obtain an infrared brightness low-pass image. Then, based on the infrared brightness image Y nir The infrared light detail layer image is determined by the difference between the infrared light brightness low-pass image and the infrared light brightness low-pass image, as shown in the following formula (3):

[0108]

[0109] In formula (3), Represents the infrared brightness low-pass image, that is, the infrared brightness image Y is subjected to the low-pass filter operator G nir The image after low-pass filtering, D nir It is the infrared detail layer image.

[0110] Step 210: Based on the visible light brightness image and the visible light chromaticity image (such as the visible light chromaticity image U vis and the visible light chromaticity image V vis ), visible light weighted image, infrared light brightness image, target infrared weighted image, visible light detail layer image and infrared light detail layer image, to generate a target image, i.e., the image to be output.

[0111] In a possible implementation, the target image may be generated by taking the following steps:

[0112] Step S11: Based on the visible light brightness image Y vis , visible light weight image, infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fused brightness image, for example, perform weighted fusion on the above images to obtain the fused brightness image Y fus .

[0113] For example, for a pixel (x), the following formula (4) can be used to determine the pixel (x) in the fused brightness image Y fus Based on this, we get the brightness value of all pixels in the fused brightness image Yfus After obtaining the brightness values ​​in , the brightness values ​​of these pixels can be combined into a fused brightness image Y fus .

[0114] Y fus (x) = (1-w(x))Y vis (x)+w(x)Y nir (x)+(1-w(x))D vis (x)+w(x)D nir (x) Formula (4)

[0115] In formula (4), Y fus (x) represents the pixel point (x) in the fused brightness image Y fus The brightness value in , 1-w(x) represents the visible light weight value of the pixel (x) in the visible light weight image, Y vis (x) represents the pixel point (x) in the visible light brightness image Y vis The brightness value in the image, w(x) represents the target weight value of the pixel (x) in the target infrared weight image, Y nir (x) represents the pixel point (x) in the infrared brightness image Y nir The brightness value in D vis (x) represents the pixel point (x) in the visible light detail layer image D vis The brightness value in D nir (x) represents the pixel point (x) in the infrared detail layer image D nir In summary, it can be seen from formula (4) that the brightness value of the visible light image Y vis , visible light weight image, infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fused brightness image Y fus .

[0116] Step S12: Determine the brightness gain value based on the fused brightness image and the visible light brightness image. For example, for a pixel (x), the brightness gain value of the pixel (x) is determined using the following formula: fus (x) / Y vis (x).

[0117] Step S13: Based on the visible light chromaticity image U vis And the brightness gain value determines the fused chrominance image U fus .

[0118] For example, for a pixel (x), the following formula is used to determine the pixel (x) in the fused chromaticity image U fusChroma value in: U fus (x) = U vis (x)·Y fus (x) / Y vis (x), after obtaining all pixels in the fused chromaticity image U fus After the chromaticity values ​​in the image are obtained, the chromaticity values ​​of these pixels can be combined into a fused chromaticity image U fus .

[0119] U fus (x) represents the pixel (x) in the fused chromaticity image U fus The chromaticity value in U vis (x) represents the pixel point (x) in the visible light chromaticity image U vis Chroma value in, Y fus (x) / Y vis (x) represents the brightness gain value of pixel (x).

[0120] Step S14: Based on the visible light chromaticity image V vis and the brightness gain value determine the fused chrominance image V fus .

[0121] For example, for a pixel (x), the following formula is used to determine the pixel (x) in the fused chromaticity image V fus Chroma value in: V fus (x) = V vis (x)·Y fus (x) / Y vis (x), after obtaining all pixels in the fused chromaticity image V fus After the chromaticity values ​​in the image are obtained, the chromaticity values ​​of these pixels can be combined into a fused chromaticity image V fus .

[0122] V fus (x) represents the pixel (x) in the fused chromaticity image V fus Chroma value in V vis (x) represents the pixel point (x) in the visible light chromaticity image V vis Chroma value in, Y fus (x) / Y vis (x) represents the brightness gain value of pixel (x).

[0123] Step S15: Based on the fused brightness image Y fus , fused chrominance image U fus And the fused chrominance image V fus Generate the target image. For example, fuse the brightness image Y fus , fused chrominance image U fus And the fused chrominance image V fusCombine them to get the target image in YUV format. Or, combine the fused brightness image Y fus , fused chrominance image U fus And the fused chrominance image V fus Combine them to get an image in YUV format, then convert the YUV format image into an RGB format image, and use the RGB format image as the target image.

[0124] See also Figure 5 FIG. 2 is another flow chart of an image processing method, which may include:

[0125] Step 501: Obtain a visible light image and an infrared light image at the same acquisition time. Exemplarily, the visible light image may be a visible light image in RGB format, and the infrared light image may include an infrared light brightness channel. The infrared light brightness channel may be determined as an infrared light brightness image.

[0126] Step 502: Determine a visible light brightness image based on the visible light image in RGB format. For example, based on the R channel, G channel, and B channel corresponding to the visible light image, for each pixel, determine the target pixel value of the pixel based on the pixel value of the pixel in the R channel, the pixel value of the pixel in the G channel, and the pixel value of the pixel in the B channel; determine the visible light brightness image based on the target pixel values ​​of all pixels, such as combining the target pixel values ​​of all pixels into the visible light brightness image.

[0127] For example, based on the visible light image, the visible light brightness image can be determined using the following formula (5):

[0128]

[0129] Method 1: For pixel point (x), when c belongs to r, the pixel value of pixel point (x) in the R channel can be As the target pixel value Y of the pixel point (x) in the visible light brightness image vis (x). Alternatively, when c belongs to g, the pixel value of the pixel point (x) in the G channel can be As the target pixel value Y of the pixel point (x) in the visible light brightness image vis (x). Alternatively, when c belongs to b, the pixel value of the pixel point (x) in the B channel can be As the target pixel value Y of the pixel point (x) in the visible light brightness image vis (x).

[0130] Method 2: For pixel (x), based on the pixel value of pixel (x) in the R channel, the pixel value of pixel (x) in the G channel, and the pixel value of pixel (x) in the B channel, the maximum value of the above three pixel values ​​can be As the target pixel value Y of the pixel point (x) in the visible light brightness image vis (x).

[0131] Method 3: For pixel point (x), based on the pixel value of pixel point (x) in the R channel , pixel value of pixel point (x) in G channel , and the pixel value of the pixel point (x) in the B channel , the above three pixel values ​​can be weighted to obtain the target pixel value Y of the pixel point (x) in the visible light brightness image vis (x). When performing weighted calculations, c r Represents pixel value The corresponding weight value can be configured based on experience, c g Represents pixel value The corresponding weight value can be configured based on experience, c b Represents pixel value The corresponding weight value can be configured based on experience, and c r +c g +c b =1.

[0132] Step 503: filter the visible light brightness image to obtain a visible light brightness characteristic image, and filter the infrared light brightness image to obtain an infrared light brightness characteristic image.

[0133] Step 504 : Determine a differential feature image between the visible light brightness image and the infrared light brightness image based on the absolute value of the difference between the visible light brightness feature image and the infrared light brightness feature image.

[0134] Step 505: Determine an initial infrared weight image corresponding to the infrared brightness image based on the differential feature image. The initial infrared weight image may include a plurality of initial weight values ​​corresponding to a plurality of differential feature values.

[0135] Step 506: determine the attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image, wherein the target infrared weighted image includes multiple target weight values ​​corresponding to multiple initial weight values.

[0136] Step 507: Generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image.

[0137] Step 508: low-pass filter the visible light brightness image to obtain a visible light brightness low-pass image; and determine a visible light detail layer image based on the difference between the visible light brightness image and the visible light brightness low-pass image.

[0138] Step 509: low-pass filter the infrared light brightness image to obtain an infrared light brightness low-pass image; and determine the infrared light detail layer image based on the difference between the infrared light brightness image and the infrared light brightness low-pass image.

[0139] Step 510: Generate a target image based on the visible light image, the visible light weighted image, the infrared light brightness image, the target infrared weighted image, the visible light detail layer image, and the infrared light detail layer image.

[0140] In a possible implementation, the target image may be generated by taking the following steps:

[0141] Step S21: Based on the R channel of the visible light image, the visible light weight image, and the infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fused R channel. For example, weighted fusion is performed on the above images to obtain the fused R channel.

[0142] For example, for a pixel (x), the following formula (6) can be used to determine the pixel (x) in the fusion R channel I c fus Based on this, we get the pixel values ​​of all pixels in the fused R channel I c fus After the pixel values ​​in the R channel are obtained, the pixel values ​​of these pixels can be combined into the fused R channel I c fus , c belongs to r.

[0143]

[0144] In formula (6), I c fus (x) indicates the pixel point (x) in the fusion R channel I c fus The pixel value in (i.e., R value), 1-w(x) represents the visible light weight value of the pixel point (x) in the visible light weight image, represents the pixel value (i.e., R value) of the pixel point (x) in the R channel of the visible light image, w(x) represents the target weight value of the pixel point (x) in the target infrared weight image, and Y nir (x) represents the pixel point (x) in the infrared brightness image Y nir The brightness value in Dvis (x) represents the pixel point (x) in the visible light detail layer image D vis The brightness value in D nir (x) represents the pixel point (x) in the infrared detail layer image D nir In summary, we can use the R channel of the visible light image, the visible light weight image, and the infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fusion R channel I c fus .

[0145] Step S22: based on the G channel of the visible light image, the visible light weight image, and the infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fusion G channel I c fus , c belongs to g. Step S22 is similar to step S21, and the pixel value in the R channel of the visible light image (ie, the R value) is replaced with the pixel value in the G channel of the visible light image (ie, the G value).

[0146] Step S23: based on the B channel of the visible light image, the visible light weight image, and the infrared light brightness image Y nir , target infrared weighted image, visible light detail layer image D vis And infrared detail layer image D nir , determine the fusion B channel I c fus , c belongs to b. Step S23 is similar to step S21, and the pixel value in the R channel of the visible light image (ie, the R value) is replaced with the pixel value in the B channel of the visible light image (ie, the B value).

[0147] Step S24: Generate a target image based on the fused R channel, the fused G channel, and the fused B channel. For example, the fused R channel, the fused G channel, and the fused B channel are combined to obtain a target image.

[0148] It can be seen from the above technical solutions that in the embodiments of the present application, infrared information can be used to enhance the visible light image (i.e., low-light image), effectively improving the brightness, clarity, and transparency of the visible light image, and the enhanced target image has no color cast. The initial infrared weighted image is determined using a differential feature image (the differential feature image can well reflect the area that is prone to color cast in fusion, and the color cast problem of fusion can be avoided when the differential feature image is used), and the initial infrared weighted image is corrected using an attenuation coefficient to obtain a target infrared weighted image, thereby fusion of the visible light image based on the target infrared weighted image, effectively avoiding the side effect of image color cast caused by fusion of infrared information, and the fused target image is more realistic and natural.

[0149] For example, suppose there are two sets of visible light and infrared brightness data, the visible light brightness Vis1 of group 1 is 10, and the infrared brightness Nir1 is 30, the visible light brightness Vis2 of group 2 is 100, and the infrared brightness Nir2 is 150. The difference between the infrared and visible light brightness of the above two sets of data is 50. If they correspond to the same infrared fusion weight, such as 0.5, the fusion brightness of group 1 is: 10*0.5+60*0.5=35, and the fusion brightness of group 2 is: 100*0.5+150*0.5=125, that is to say, the brightness of group 1 is increased by (35-10) / 10=250%, and the brightness of group 2 is increased by (125-100) / 100=25%. It can be seen that the improvement gap between the two is very large. When 10 becomes 35, black may become gray, and when 100 becomes 125, gray is still gray.

[0150] In view of the above findings, in this embodiment, after using the differential feature image to determine the initial infrared weighted image, the attenuation coefficient can also be used to correct the initial infrared weighted image to obtain a target infrared weighted image, so as to fuse the visible light image based on the target infrared weighted image, effectively avoiding the side effect of image color cast caused by the fusion of infrared information, that is, avoiding the color cast problem. When collecting images of pedestrians and vehicles, if the black clothes worn by pedestrians and drivers in vehicles are particularly reflective of infrared, color cast will be caused after fusion. In this embodiment, the initial infrared weighted image is determined by using the differential feature image, and the attenuation coefficient can be used to correct the initial infrared weighted image to obtain the target infrared weighted image, which can avoid the color cast problem.

[0151] Based on the same application concept as the above method, an image processing device is proposed in the embodiment of the present application, see Figure 6 FIG. 1 is a schematic diagram of the structure of the image processing device, and the device may include:

[0152] The acquisition module 61 is used to acquire a visible light image and an infrared light image at the same acquisition time, determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image; the determination module 62 is used to determine a differential feature image between the visible light brightness image and the infrared light brightness image; determine an initial infrared weighted image corresponding to the infrared light brightness image based on the differential feature image; determine an attenuation coefficient based on the visible light brightness image and the infrared light brightness image, and perform attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared light brightness image; wherein the initial infrared weighted image includes multiple initial weight values, and the target infrared weighted image includes multiple target weight values ​​corresponding to the multiple initial weight values; the generation module 63 is used to generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image, and generate a target image based on the visible light image, the visible light weighted image, the infrared light brightness image and the target infrared weighted image.

[0153] In a possible implementation, the visible light image includes a visible light image in RGB format, and the acquisition module 61 is specifically used to determine the visible light brightness image: convert the visible light image in RGB format into a visible light image in YUV format, and determine the brightness channel in the visible light image in YUV format as the visible light brightness image; or, determine the R channel, G channel and B channel corresponding to the visible light image in RGB format; for each pixel, determine the target pixel value of the pixel based on the pixel value of the pixel in the R channel, the pixel value of the pixel in the G channel and the pixel value of the pixel in the B channel; and determine the visible light brightness image based on the target pixel values ​​of all pixels.

[0154] In a possible implementation, the determination module 62 is specifically used to determine the differential feature image between the visible light brightness image and the infrared light brightness image by: filtering the visible light brightness image to obtain a visible light brightness feature image; filtering the infrared light brightness image to obtain an infrared light brightness feature image; determining the difference between the visible light brightness feature image and the infrared light brightness feature image, and determining the differential feature image based on the absolute value of the difference.

[0155] In a possible implementation, the determination module 62 is specifically used to determine the initial infrared weight image corresponding to the infrared brightness image based on the differential feature image: for each differential feature value in the differential feature image, query the configured differential weight mapping relationship through the differential feature value to obtain the initial weight value corresponding to the differential feature value; determine the initial infrared weight image based on the initial weight values ​​corresponding to all differential feature values; wherein the differential weight mapping relationship includes the correspondence between the differential feature value and the initial weight value; when the differential feature value is the minimum, the initial weight value corresponding to the differential feature value is the maximum, and as the differential feature value increases, the initial weight value corresponding to the differential feature value becomes smaller.

[0156] In a possible implementation, the determination module 62 is specifically used to determine the attenuation coefficient based on the visible light brightness image and the infrared light brightness image: for each pixel point, if the first brightness value of the pixel point in the visible light brightness image is greater than or equal to the second brightness value of the pixel point in the infrared light brightness image, then the attenuation coefficient corresponding to the pixel point is determined to be 1; if the first brightness value is less than the second brightness value, the configured attenuation mapping relationship is queried through the first brightness value to obtain the attenuation coefficient corresponding to the pixel point; wherein the attenuation mapping relationship includes a corresponding relationship between the brightness value and the attenuation coefficient, when the brightness value is the minimum, the attenuation coefficient corresponding to the brightness value is the minimum, and as the brightness value increases, the attenuation coefficient corresponding to the brightness value becomes larger, and the attenuation coefficient corresponding to the brightness value is between 0 and 1.

[0157] In a possible implementation, the determination module 62 performs attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared brightness image, and is specifically used for: for each pixel point in the initial infrared weighted image, determining the target weight value corresponding to the pixel point based on the initial weight value corresponding to the pixel point and the attenuation coefficient corresponding to the pixel point; and generating the target infrared weighted image based on the target weight values ​​corresponding to all pixels in the initial infrared weighted image.

[0158] In a possible implementation, the visible light image is a visible light image in RGB format, and the visible light image in RGB format is converted into a visible light image in YUV format, wherein the visible light image in YUV format includes a visible light brightness image and a visible light chromaticity image; the generation module 63 generates a target image based on the visible light image, the visible light weight image, the infrared light brightness image and the target infrared weight image, and is specifically used to: determine a fused brightness image based on the visible light brightness image, the visible light weight image, the infrared light brightness image, the target infrared weight image, a visible light detail layer image corresponding to the visible light brightness image, and an infrared light detail layer image corresponding to the infrared light brightness image; determine a brightness gain value based on the fused brightness image and the visible light brightness image; determine a fused chromaticity image based on the visible light chromaticity image and the brightness gain value; and generate a target image based on the fused brightness image and the fused chromaticity image.

[0159] In a possible implementation, the visible light image is a visible light image in RGB format, and the visible light image in RGB format includes an R channel, a G channel and a B channel; the generation module 63 generates a target image based on the visible light image, the visible light weight image, the infrared light brightness image and the target infrared weight image, and is specifically used to: determine the fused R channel based on the R channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared light detail layer image corresponding to the infrared light brightness image; determine the fused G channel based on the G channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image and the infrared light detail layer image; determine the fused B channel based on the B channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image and the infrared light detail layer image; and generate the target image based on the fused R channel, the fused G channel and the fused B channel.

[0160] Based on the same application concept as the above method, an image processing device is proposed in the embodiment of the present application, see Figure 7 As shown, the image processing device may include: a processor 71 and a machine-readable storage medium 72, the machine-readable storage medium 72 storing machine-executable instructions that can be executed by the processor 71; the processor 71 is used to execute the machine-executable instructions to implement the image processing method disclosed in the above example of this application.

[0161] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the image processing method disclosed in the above example of the present application can be implemented.

[0162] The above-mentioned machine-readable storage medium may be any electronic, magnetic, optical or other physical storage device, which may contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium may be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as CD, DVD, etc.), or similar storage medium, or a combination thereof.

[0163] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0164] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a visible light image and an infrared light image at the same acquisition time, determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image; Determine a differential feature image between the visible light brightness image and the infrared light brightness image; wherein the differential feature image includes a plurality of differential feature values ​​for indicating a brightness comparison of brightness values; Determining an initial infrared weight image corresponding to the infrared brightness image based on the differential feature image; wherein the initial infrared weight image includes a plurality of initial weight values ​​corresponding to the plurality of differential feature values, and the initial weight values ​​corresponding to the differential feature values ​​are inversely proportional to the differential feature values; The attenuation coefficient is determined based on the visible light brightness image and the infrared light brightness image; wherein, for each pixel point, if the first brightness value of the pixel point in the visible light brightness image is greater than or equal to the second brightness value of the pixel point in the infrared light brightness image, the attenuation coefficient corresponding to the pixel point is 1; if the first brightness value is less than the second brightness value, the configured attenuation mapping relationship is queried through the first brightness value to obtain the attenuation coefficient corresponding to the pixel point; and performing attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared brightness image; wherein the initial infrared weighted image includes a plurality of initial weight values, and the target infrared weighted image includes a plurality of target weight values ​​corresponding to the plurality of initial weight values; A visible light weighted image corresponding to the visible light image is generated based on the target infrared weighted image, and a target image is generated based on the visible light image, the visible light weighted image, the infrared brightness image and the target infrared weighted image.

2. The method according to claim 1, characterized in that The visible light image includes a visible light image in RGB format, and determining the visible light brightness image based on the visible light image includes: converting the visible light image in RGB format into a visible light image in YUV format, and determining the brightness channel in the visible light image in YUV format as the visible light brightness image; or, Determine the R channel, G channel, and B channel corresponding to the visible light image in the RGB format; For each pixel, a target pixel value of the pixel is determined based on the pixel value of the pixel in the R channel, the pixel value of the pixel in the G channel, and the pixel value of the pixel in the B channel; and the visible light brightness image is determined based on the target pixel values ​​of all pixels.

3. The method according to claim 1, characterized in that The determining of the differential feature image between the visible light brightness image and the infrared light brightness image comprises: Performing filtering processing on the visible light brightness image to obtain a visible light brightness feature image; Performing filtering processing on the infrared brightness image to obtain an infrared brightness characteristic image; The difference between the visible light brightness characteristic image and the infrared light brightness characteristic image is determined, and the differential characteristic image is determined based on the absolute value of the difference.

4. The method according to claim 1, characterized in that: The determining, based on the differential feature image, an initial infrared weight image corresponding to the infrared brightness image comprises: For each differential feature value in the differential feature image, query the configured differential weight mapping relationship through the differential feature value to obtain an initial weight value corresponding to the differential feature value; Determining the initial infrared weighted image based on initial weight values ​​corresponding to all differential eigenvalues; Wherein, the differential weight mapping relationship includes a corresponding relationship between differential eigenvalues ​​and initial weight values; Among them, when the differential eigenvalue is the smallest, the initial weight value corresponding to the differential eigenvalue is the largest, and as the differential eigenvalue increases, the initial weight value corresponding to the differential eigenvalue becomes smaller.

5. The method according to claim 1, characterized in that The attenuation mapping relationship includes a correspondence between brightness value and attenuation coefficient. When the brightness value is minimum, the attenuation coefficient corresponding to the brightness value is minimum. As the brightness value increases, the attenuation coefficient corresponding to the brightness value increases, and the attenuation coefficient corresponding to the brightness value is between 0 and 1.

6. The method according to claim 1 or 5, characterized in that: The attenuation processing is performed on the initial infrared weighted image based on the attenuation coefficient to obtain a target infrared weighted image corresponding to the infrared brightness image, including: For each pixel in the initial infrared weighted image, a target weight value corresponding to the pixel is determined based on the initial weight value corresponding to the pixel and the attenuation coefficient corresponding to the pixel; and the target infrared weighted image is generated based on the target weight values ​​corresponding to all pixels in the initial infrared weighted image.

7. The method according to claim 1, characterized in that The visible light image is a visible light image in RGB format, and the visible light image in RGB format is converted into a visible light image in YUV format, wherein the visible light image in YUV format includes a visible light brightness image and a visible light chromaticity image; The generating a target image based on the visible light image, the visible light weighted image, the infrared light brightness image and the target infrared weighted image comprises: Determine a fused brightness image based on the visible light brightness image, the visible light weighted image, the infrared light brightness image, the target infrared weighted image, a visible light detail layer image corresponding to the visible light brightness image, and an infrared light detail layer image corresponding to the infrared light brightness image; Determine a brightness gain value based on the fused brightness image and the visible light brightness image; Determine a fused chromaticity image based on the visible light chromaticity image and the luminance gain value; A target image is generated based on the fused luminance image and the fused chrominance image.

8. The method according to claim 1, characterized in that The visible light image is a visible light image in RGB format, and the visible light image in RGB format includes an R channel, a G channel, and a B channel; The generating a target image based on the visible light image, the visible light weighted image, the infrared light brightness image and the target infrared weighted image comprises: Determine a fused R channel based on the R channel, the visible light weighted image, the infrared light brightness image, the target infrared weighted image, a visible light detail layer image corresponding to the visible light brightness image, and an infrared light detail layer image corresponding to the infrared light brightness image; Determine a fused G channel based on the G channel, the visible light weighted image, the infrared light brightness image, the target infrared weighted image, the visible light detail layer image, and the infrared light detail layer image; Determine a fused B channel based on the B channel, the visible light weighted image, the infrared light brightness image, the target infrared weighted image, the visible light detail layer image, and the infrared light detail layer image; A target image is generated based on the fused R channel, the fused G channel and the fused B channel.

9. The method according to claim 7 or 8, characterized in that: The method for determining the visible light detail layer image corresponding to the visible light brightness image comprises: performing low-pass filtering on the visible light brightness image to obtain a visible light brightness low-pass image; and determining the visible light detail layer image based on a difference between the visible light brightness image and the visible light brightness low-pass image; The method for determining the infrared light detail layer image corresponding to the infrared light brightness image includes: performing low-pass filtering on the infrared light brightness image to obtain an infrared light brightness low-pass image; and determining the infrared light detail layer image based on the difference between the infrared light brightness image and the infrared light brightness low-pass image.

10. An image processing device, characterized in that: The device comprises: An acquisition module, used to acquire a visible light image and an infrared light image at the same acquisition moment, determine a visible light brightness image based on the visible light image, and determine an infrared light brightness image based on the infrared light image; A determination module is used to determine a differential feature image between the visible light brightness image and the infrared light brightness image, wherein the differential feature image includes a plurality of differential feature values ​​for indicating a brightness comparison of the brightness value; based on the differential feature image, an initial infrared weight image corresponding to the infrared light brightness image is determined, wherein the initial infrared weight image includes a plurality of initial weight values ​​corresponding to the plurality of differential feature values, and the initial weight values ​​corresponding to the differential feature values ​​are inversely proportional to the differential feature values; based on the visible light brightness image and the infrared light brightness image, an attenuation coefficient is determined, and based on the attenuation coefficient, an attenuation process is performed on the initial infrared weight image to obtain a target infrared weighted image corresponding to the infrared brightness image; wherein the initial infrared weighted image includes a plurality of initial weight values, and the target infrared weighted image includes a plurality of target weight values ​​corresponding to the plurality of initial weight values; wherein, when determining the attenuation coefficient, for each pixel point, if the first brightness value of the pixel point in the visible light brightness image is greater than or equal to the second brightness value of the pixel point in the infrared brightness image, then the attenuation coefficient corresponding to the pixel point is 1; if the first brightness value is less than the second brightness value, then the configured attenuation mapping relationship is queried through the first brightness value to obtain the attenuation coefficient corresponding to the pixel point; A generation module is used to generate a visible light weighted image corresponding to the visible light image based on the target infrared weighted image, and to generate a target image based on the visible light image, the visible light weighted image, the infrared brightness image and the target infrared weighted image.

11. The device according to claim 10, characterized in that in, The visible light image includes a visible light image in RGB format, and the acquisition module is specifically used to determine the visible light brightness image based on the visible light image: convert the visible light image in RGB format into a visible light image in YUV format, and determine the brightness channel in the visible light image in YUV format as the visible light brightness image; or determine the R channel, G channel and B channel corresponding to the visible light image in RGB format; For each pixel, determine a target pixel value of the pixel based on the pixel value of the pixel in the R channel, the pixel value of the pixel in the G channel, and the pixel value of the pixel in the B channel; Determine the visible light brightness image based on target pixel values ​​of all pixels; Wherein, when the determination module determines the differential feature image between the visible light brightness image and the infrared light brightness image, it is specifically used to: filter the visible light brightness image to obtain a visible light brightness feature image; filter the infrared light brightness image to obtain an infrared light brightness feature image; determine the difference between the visible light brightness feature image and the infrared light brightness feature image, and determine the differential feature image based on the absolute value of the difference; Wherein, when the determination module determines the initial infrared weight image corresponding to the infrared brightness image based on the differential feature image, it is specifically used to: for each differential feature value in the differential feature image, query the configured differential weight mapping relationship through the differential feature value to obtain the initial weight value corresponding to the differential feature value; determine the initial infrared weight image based on the initial weight values ​​corresponding to all differential feature values; wherein the differential weight mapping relationship includes the corresponding relationship between the differential feature value and the initial weight value; wherein, when the differential feature value is the minimum, the initial weight value corresponding to the differential feature value is the maximum, and as the differential feature value increases, the initial weight value corresponding to the differential feature value becomes smaller; The attenuation mapping relationship includes a corresponding relationship between a brightness value and an attenuation coefficient. When the brightness value is the minimum, the attenuation coefficient corresponding to the brightness value is the minimum. As the brightness value increases, the attenuation coefficient corresponding to the brightness value increases, and the attenuation coefficient corresponding to the brightness value is between 0 and 1. Wherein, the determination module performs attenuation processing on the initial infrared weighted image based on the attenuation coefficient to obtain the target infrared weighted image corresponding to the infrared brightness image, and is specifically used for: for each pixel point in the initial infrared weighted image, determining the target weight value corresponding to the pixel point based on the initial weight value corresponding to the pixel point and the attenuation coefficient corresponding to the pixel point; generating the target infrared weighted image based on the target weight values ​​corresponding to all pixels in the initial infrared weighted image; Wherein, the visible light image is a visible light image in RGB format, and the visible light image in RGB format is converted into a visible light image in YUV format, and the visible light image in YUV format includes a visible light brightness image and a visible light chromaticity image; the generation module is based on the visible light image, the visible light weight image, the infrared light brightness image and the target infrared weight image, and when generating the target image, is specifically used to: determine a fused brightness image based on the visible light brightness image, the visible light weight image, the infrared light brightness image, the target infrared weight image, a visible light detail layer image corresponding to the visible light brightness image, and an infrared light detail layer image corresponding to the infrared light brightness image; determine a brightness gain value based on the fused brightness image and the visible light brightness image; determine a fused chromaticity image based on the visible light chromaticity image and the brightness gain value; and generate a target image based on the fused brightness image and the fused chromaticity image; Wherein, the visible light image is a visible light image in RGB format, and the visible light image in RGB format includes an R channel, a G channel and a B channel; the generation module is based on the visible light image, the visible light weight image, the infrared light brightness image and the target infrared weight image, and is specifically used to: determine the fused R channel based on the R channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image corresponding to the visible light brightness image, and the infrared light detail layer image corresponding to the infrared light brightness image; determine the fused G channel based on the G channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image and the infrared light detail layer image; determine the fused B channel based on the B channel, the visible light weight image, the infrared light brightness image, the target infrared weight image, the visible light detail layer image and the infrared light detail layer image; and generate the target image based on the fused R channel, the fused G channel and the fused B channel.

12. An image processing device, characterized in that: include: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor; The processor is used to execute machine executable instructions to implement the method steps described in any one of claims 1-9.

Images