Full-color imaging method and device, electronic equipment and storage medium

Abstract

The invention provides a full-color imaging method and device, electronic equipment and a storage medium. The full-color imaging method comprises the following steps: acquiring a visible light image and a near-infrared image, wherein image data of the visible light image and image data of the near-infrared image are original data shot by a dual-camera module; performing sub-channel fusion on the visible light image and the near-infrared image to obtain channel fusion images corresponding to each channel; obtaining a dynamic and static judgment result obtained based on the channel fusion image and the near-infrared image; and acquiring a full-color image based on the dynamic and static judgment result and the channel fusion image. The method is suitable for fusion of a high-noise visible light image and a near-infrared image in a low-illumination environment, and can improve the signal-to-noise ratio of original image data and effectively enhance key information in a low-illumination image sequence before time domain noise reduction on the premise of keeping a visible light color relationship.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a full-color imaging method and device, an electronic device, and a storage medium. Background Art

[0002] Low-light imaging environment is an important application scenario in the field of nighttime security monitoring. However, the signal-to-noise ratio of visible light imaging in low-light environment is poor, and high-level noise may seriously affect the display of real signals in the image, making it difficult to improve the image quality of the visible light imaging to an ideal state through subsequent algorithms. Visible light-near infrared image fusion technology can enhance low-light visible light images using the rich details in near-infrared images, effectively improving the signal-to-noise ratio of visible light images without adding additional visible light sources.

[0003] For continuous image sequences of security monitoring, multiple frames of image data are often superimposed for temporal noise reduction in low-light imaging at night, which can effectively improve the signal-to-noise ratio of static areas. However, the moving areas may have obvious artifacts (such as motion smear, etc.) or excessive denoising due to poor signal-to-noise ratio, and the clarity of the moving areas is still not ideal. There are several fusion technologies for high-noise visible light images, and their fusion results have a certain degree of noise reduction, but they are all applied to single-frame images in RGB format that have been transformed by image processing, which is not suitable for continuous image sequences of security monitoring.

[0004] In addition, due to the different depths of the moving foreground and the static background, even after the near-infrared images are registered, there may still be a certain degree of parallax in local areas. Summary of the invention

[0005] The present disclosure provides a full-color imaging method and device, an electronic device, and a storage medium, which can improve the image quality of visible light imaging in a low-light environment.

[0006] In a first aspect, the purpose of the embodiments of the present disclosure is to provide a full-color imaging method. The full-color imaging method includes: acquiring a visible light image and a near-infrared image, wherein the image data of the visible light image and the image data of the near-infrared image are original data captured by a dual-camera module; performing channel fusion on the visible light image and the near-infrared image to obtain a channel fusion image corresponding to each channel; acquiring a motion and stillness judgment result based on the channel fusion image and the near-infrared image; and acquiring a full-color image based on the motion and stillness judgment result and the channel fusion image.

[0007] In an implementation of the first aspect, performing channel-by-channel fusion on the visible light image and the near-infrared image to obtain channel fusion images corresponding to each channel includes: fusing a first color channel of the visible light image of a current frame with the near-infrared image of the current frame to obtain a first color channel fusion image; based on the first color channel fusion image, fusing a second color channel of the visible light image of the current frame with the near-infrared image of the current frame to obtain a second color channel fusion image, and fusing a third color channel of the visible light image of the current frame with the near-infrared image of the current frame to obtain a third channel fusion image; and using the first color channel fusion image, the second color channel fusion image, and the third channel fusion image as the channel fusion image of the current frame, respectively.

[0008] In an implementation of the first aspect, obtaining a full-color image based on the motion-stillness judgment result and the channel fusion image includes: performing channel-by-channel time domain denoising on multiple frames of the channel fusion image to obtain each channel inter-frame fusion image by combining the motion-stillness judgment result obtained by combining the channel fusion image and the near-infrared image; and performing image signal processing on the channel inter-frame fusion image to obtain the full-color image.

[0009] In an implementation manner of the first aspect, the first color channel of the visible light image includes at least one channel associated with the G color, and fusing the first color channel of the visible light image with the near-infrared image to obtain a first color channel fused image includes: when the first color channel of the visible light image includes only one channel, fusing the first color channel of the visible light image with the near-infrared image to obtain a first color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, fusing each sub-channel of the first color channel of the visible light image with the near-infrared image to obtain each sub-channel fused image, wherein each sub-channel of the first color channel of the visible light image is fused in the same manner as the near-infrared image, and is the same as the fusion manner of the first color channel of the visible light image and the near-infrared image when the first color channel of the visible light image includes only one channel, and this fusion manner is the first fusion manner.

[0010] In an implementation of the first aspect, fusing the second color channel of the visible light image with the near-infrared image to acquire a second color channel fused image includes: when the first color channel of the visible light image includes only one channel, based on the first color channel fused image of the visible light image, fusing the second color channel of the visible light image with the near-infrared image to acquire the second color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, based on the weighted mean of all sub-channel fused images of the first color channel of the visible light image, fusing the second color channel of the visible light image with the near-infrared image to acquire the second color channel fused image.

[0011] In an implementation of the first aspect, fusing the third color channel of the visible light image with the near-infrared image to acquire a third color channel fused image includes: when the first color channel of the visible light image includes only one channel, based on the first color channel fused image of the visible light image, fusing the third color channel of the visible light image with the near-infrared image to acquire the third color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, based on the weighted mean of all sub-channel fused images of the first color channel of the visible light image, fusing the third color channel of the visible light image with the near-infrared image to acquire the third color channel fused image.

[0012] In an implementation of the first aspect, the first fusion method for fusing the first color channel of the visible light image with the near-infrared image includes: obtaining a first gradient relationship diagram between a first-order gradient of the first color channel of the visible light image and a first-order gradient of the near-infrared image; obtaining an initial value of a first visual edge preservation coefficient based on the first-order gradient calculation of the first color channel of the visible light image, wherein the first visual edge preservation coefficient is a two-dimensional weight coefficient; and obtaining the first color channel fused image based on the first-order gradient of the near-infrared image, the first gradient relationship diagram, and the initial value of the first visual edge preservation coefficient.

[0013] In an implementation of the first aspect, acquiring the first color channel fused image based on the first-order gradient of the near-infrared image, the first relationship gradient map, and the initial value of the first visual edge preservation coefficient includes using the following objective function formula:

[0014]

[0015] Where I1 represents the optimal first color channel fusion image to be solved, represents the first-order gradient of the optimal first color channel fusion image to be solved, s represents the gradient relationship diagram between the optimal first color channel fusion image to be solved and the near-infrared image, represents the first-order gradient of the near-infrared image, I0 represents the image of the first color channel of the visible light image, is the control term of s, α is represented by the first visual edge preservation coefficient, λ and β are constant terms; the objective function is calculated alternately and iteratively to obtain the optimal first color channel fused image.

[0016] In an implementation of the first aspect, obtaining the first visual edge preservation coefficient includes using the following formula:

[0017]

[0018] Among them, α k represents the first visual edge-preserving coefficient corresponding to the k-th iteration of calculating the objective function, I k-1 represents the first color channel fused image calculated at the k-1th iteration, th is a constant coefficient, represents the first-order gradient of the first color channel fused image calculated at the k-1th iteration after denoising, I0 represents the original image of the first color channel of the visible light image at the first iteration, and the first visual edge preservation coefficient obtained based on I0 is the initial value of the first visual edge preservation coefficient.

[0019] In an implementation of the first aspect, fusing the second color channel of the visible light image with the near-infrared image to obtain a second color channel fused image includes: obtaining a second gradient relationship diagram between a first-order gradient of the second color channel of the visible light image and a first-order gradient of the near-infrared image; obtaining an initial value of a second visual edge preservation coefficient based on the first-order gradient calculation of the second color channel of the visible light image; obtaining a second channel priori item based on the first color channel fused image; and obtaining the second color channel fused image based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship diagram, and the second channel priori item.

[0020] In an implementation manner of the first aspect, the third color channel of the visible light image is fused with the near-infrared image to obtain a third color channel fused image. The third color channel fused image includes: obtaining a third gradient relationship diagram between the first-order gradient of the third color channel of the visible light image and the first-order gradient of the near-infrared image; obtaining an initial value of a third visual edge preservation coefficient based on the first-order gradient calculation of the third color channel of the visible light image; obtaining a third channel priori item based on the first color channel fused image; and obtaining the third color channel fused image based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship diagram, and the third channel priori item.

[0021] In an implementation of the first aspect, acquiring the second color channel fused image based on an initial value of a second visual edge preservation coefficient, a first-order gradient of the near-infrared image, the second gradient relationship map, and the second channel priori item, or acquiring the third color channel fused image based on an initial value of a third visual edge preservation coefficient, a first-order gradient of the near-infrared image, the third gradient relationship map, and the third channel priori item includes using the following objective function formula:

[0022]

[0023] Wherein, I represents the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved, s represents the gradient relationship diagram between the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved and the near infrared image, and R IG represents the ratio of the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved to the first color channel fusion image, R Lf The objective function is calculated by alternately iteratively to obtain an optimal fused image of the second color channel or an optimal fused image of the third color channel.

[0024] In an implementation of the first aspect, obtaining a full-color image based on the motion-stillness judgment result and the channel fusion image includes: for the acquired channel fusion images of continuous frames, superimposing the channel fusion images frame by frame based on the motion-stillness judgment result to obtain channel inter-frame fusion images of continuous frames, and performing image signal processing on the channel inter-frame fusion image to obtain the full-color image.

[0025] In an implementation of the first aspect, obtaining a motion / stillness judgment result based on the channel fusion image and the near-infrared image includes: respectively calculating and obtaining a motion / stillness judgment weight distribution map of the channel fusion image and a motion / stillness judgment weight distribution map of the near-infrared image; and fusing the motion / stillness judgment weight distribution map of the channel fusion image and the motion / stillness judgment weight distribution map of the near-infrared image to obtain a motion / stillness judgment result.

[0026] In an implementation of the first aspect, before the visible light image and the near-infrared image are fused channel by channel, it also includes: performing parallax correction on the near-infrared image to align the near-infrared image with the visible light image; and preprocessing the visible light image and the near-infrared image after parallax correction, respectively.

[0027] In a second aspect, the embodiments of the present disclosure provide a full-color imaging device. The full-color imaging device includes: a memory; and at least one processor, which is coupled to the memory and configured to: obtain a visible light image and a near-infrared image, wherein the image data of the visible light image and the image data of the near-infrared image are original data captured by a dual camera module; perform channel-by-channel fusion on the visible light image and the near-infrared image to obtain a channel-fused image; obtain a motion and stillness judgment result based on the channel-fused image and the near-infrared image; and obtain a full-color image based on the motion and stillness judgment result and the channel-fused image.

[0028] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory configured to store an executable program; and a processor configured to execute the program so that the electronic device performs the full-color imaging method according to any one of the first aspects of the present disclosure.

[0029] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the full-color imaging method according to any one of the first aspects of the present disclosure.

[0030] As described above, the full-color imaging method and device, electronic device and storage medium described in the embodiments of the present disclosure have the following features:

[0031] Beneficial effects:

[0032] The disclosed embodiments are applicable to the fusion of high-noise visible light images and near-infrared images in low-light environments. Data fusion is performed based on the visible light original image data and the near-infrared original image data. In the process of multi-channel data fusion, first, a visual edge preservation coefficient is introduced to supplement constraint items for the near-infrared texture-free area, thereby further preserving the visible light texture information. Second, a priori items between color channels are introduced to further preserve the visible light color information.

[0033] The disclosed embodiments can improve the signal-to-noise ratio of the original image data and effectively enhance the key information in the low-illumination image sequence before time-domain noise reduction, while maintaining the visible light color relationship. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Shown is a schematic diagram of an application scenario of an embodiment of the present disclosure.

[0035] Figure 2 A flow chart of a full-color imaging method provided by an embodiment of the present disclosure is shown.

[0036] Figure 3 Shown is a flowchart of the image fusion method provided by an embodiment of the present disclosure.

[0037] Figure 4 Shown is a flowchart of an image fusion method based on a visual edge-preserving coefficient provided by an embodiment of the present disclosure.

[0038] Figure 5 Shown is a flowchart of an image fusion method based on color prior terms provided by an embodiment of the present disclosure.

[0039] Figure 6 Shown is a flowchart of a method for obtaining motion and stillness judgment results provided by an embodiment of the present disclosure.

[0040] Figures 7a to 7d Shown is a rendering of the full-color imaging method provided by an embodiment of the present disclosure.

[0041] Figures 8a to 8d Shown is a rendering of the full-color imaging method provided by an embodiment of the present disclosure.

[0042] Fig. 9 Shown is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0044] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0045] In addition, in the present disclosure, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present disclosure.

[0046] For continuous image sequences of security monitoring, multi-frame superposition of image data is often used for time domain noise reduction in low-light imaging at night, which can effectively improve the signal-to-noise ratio of static areas. However, the moving area may have obvious artifacts (such as motion smear, etc.) or excessive denoising due to poor signal-to-noise ratio, and the clarity of the moving area is still not ideal.

[0047] Some technical solutions use fusion technology based on noise-free images or fusion technology based on single-frame visible light format images. The fusion results have a certain degree of noise reduction, but they are all applied to single-frame images in RGB format that have been transformed by image processing, and are not suitable for continuous image sequences of security monitoring. Moreover, for the image data collected by the dual-camera module composed of two camera modules, due to the different depths of the moving foreground and the static background, there is still a certain degree of parallax in the local area even after registration, which leads to the misalignment of different spectral information after image fusion.

[0048] Some technical solutions use the scale map algorithm to fuse near-infrared images and visible light images, but in practical applications, there are still the following defects:

[0049] (1) Due to the different absorption spectra of materials, there are areas where the visible light image has texture but the near-infrared image has no texture. This area lacks the information guidance of the near-infrared image, which causes the scale map algorithm to cause obvious visible light information attenuation in this area when performing image fusion, especially for high-noise visible light images.

[0050] (2) When the scale map algorithm fuses visible light images and near-infrared images, each color channel is processed independently without considering the a priori relationship between color channels. For high-noise visible light images, this may lead to obvious color information loss in the fusion result.

[0051] At least in view of the above problems, the embodiments of the present disclosure provide a full-color imaging method. The full-color imaging method comprises: obtaining a visible light image and a near-infrared image taken by a dual camera module; the image data of the visible light image and the image data of the near-infrared image are the original data taken by the dual camera module; performing channel-by-channel fusion on the visible light image and the near-infrared image to obtain a fused image; obtaining a motion and stillness judgment result based on the fused image and the near-infrared image; and obtaining a full-color image based on the motion and stillness judgment result and the fused image.

[0052] Figure 1 The full-color imaging device can be used to implement the full-color imaging method provided by the embodiment of the present disclosure, but the application scenario of the full-color imaging method provided by the embodiment of the present disclosure is not limited to Figure 1 The full-color imaging device. Figure 1 As shown, the full-color imaging device 1 includes: a collection device 11, a processor 12 and a display terminal 13. The full-color imaging method provided in the embodiment of the present disclosure can be applied to the processor 12.

[0053] Figure 1 The acquisition device is an image or video acquisition device including a dual camera module, which can acquire visible light images and near-infrared images in low-light environments.

[0054] Figure 1 The processor 12 in the embodiment may be a single processor or a processor cluster or a cloud computing center composed of multiple processors, and the specific details are not limited here. Figure 1 Only one acquisition device 11, one processor 12 and one display terminal 13 are shown, but it should be understood that Figure 1 The examples are only used to understand this solution, and the specific number of display terminals and processors should be flexibly determined based on actual conditions.

[0055] In some other implementations, the full-color imaging device 1 may not include the display terminal 13, but only include the acquisition device 11 and the processor 12 with display function. The processor 12 with display function may include a tablet computer, a laptop computer, a PDA, a mobile phone, a personal computer (PC), and a voice interaction device, or a monitoring device, a face recognition device, etc., which is not limited here.

[0056] In the disclosed embodiment, the obtained visible light images and near-infrared images are original image data taken by a dual camera module. After preprocessing the original data, the fusion of the visible light images and the near-infrared images is achieved by introducing a visual fidelity coefficient and a channel-by-channel fusion method of prior terms between color channels. While ensuring the visible light texture information and color information, the signal-to-noise ratio of the original image data of the visible light image is improved, and at the same time, the key information in the image sequence under low illumination environment can be enhanced.

[0057] The technical solutions in the embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present disclosure.

[0058] The following embodiments of the present disclosure provide a full-color imaging method. Figure 1 The flowchart of the full-color imaging method according to the embodiment of the present disclosure is shown as follows: Figure 2 As shown, the full-color imaging method includes steps S1 to S4.

[0059] Step S1, obtaining a visible light image and a near infrared image, wherein the image data of the visible light image and the image data of the near infrared image are original data captured by a dual camera module.

[0060] In this embodiment, the original data is image data in the RAW domain, which is unprocessed data taken by the dual camera module.

[0061] Step S2: performing channel fusion on the visible light image and the near infrared image to obtain channel fusion images corresponding to each channel.

[0062] Step S3: Obtaining a motion / stillness judgment result based on the channel fusion image and the near infrared image.

[0063] Step S4: acquiring a full-color image based on the motion / stillness judgment result and the channel fusion image.

[0064] It should be noted that the imaging results obtained by the dual camera module have parallax. In order to reduce the parallax and achieve preliminary image registration, the parallax of the dual camera module can be calibrated before actual imaging. In addition, since the image data taken by the dual camera module is raw data, it is necessary to pre-process the obtained visible light image first to suppress the medium and high frequency noise of the visible light image.

[0065] In some possible implementations, the full-color imaging method of the embodiment of the present disclosure also includes: performing parallax correction on the near-infrared image to align the near-infrared image with the visible light image; and preprocessing the visible light image and the near-infrared image after parallax correction respectively.

[0066] Performing parallax correction on a near-infrared image to align the near-infrared image with the visible light image includes: obtaining parameter data such as the depth required for imaging, placing a calibration plate at the required imaging depth and imaging, obtaining a parallax distribution map between the near-infrared image and the visible light image through a block search, and achieving parallax correction on the near-infrared image based on the obtained parallax distribution map.

[0067] The texture and brightness distribution of the fusion results of the near-infrared image and the visible light image are largely related to the near-infrared image. The preprocessing of the visible light image and the near-infrared image after parallax correction includes: the near-infrared image after parallax correction is also adjusted by gamma transformation and other methods to enhance the area of ​​interest in the near-infrared image.

[0068] The process of fusing the near-infrared image and the visible light image provided by the embodiment of the present disclosure is a nonlinear process. In order to avoid obvious color cast in the image fusion result, the visible light image is firstly white balanced, and the visible light image is pre-denoised by edge-preserving filtering algorithms such as bilateral filtering to suppress the mid- and high-frequency noise of the visible light image.

[0069] In the disclosed embodiment, the obtained visible light images and near-infrared images are the original image data taken by the dual camera module. After preprocessing the original data, the visible light images and near-infrared images are fused channel by channel. On the basis of ensuring the visible light texture information and color information, the signal-to-noise ratio of the original image data of the visible light image is improved, and at the same time, the key information in the image sequence in the low-light environment can be enhanced.

[0070] Figure 3 The flowchart of the image fusion method described in the embodiment of the present disclosure is shown as follows: Figure 3 As shown, step S2 includes step S21 to step S22.

[0071] Step S21 : fusing the first color channel of the visible light image of the current frame with the near infrared image of the current frame to obtain a first color channel fused image.

[0072] Step S22: Based on the first color channel fused image, the second color channel of the visible light image of the current frame is fused with the near-infrared image of the current frame to obtain a second color channel fused image, and the third color channel of the visible light image of the current frame is fused with the near-infrared image of the current frame to obtain a third color channel fused image.

[0073] In this embodiment, the first color channel fusion image, the second color channel fusion image, and the third color channel fusion image are channel fusion images of the current frame respectively.

[0074] In some possible implementations, the GR channel, GB channel, R channel and B channel of the visible light image and the near-infrared image channel fusion images are obtained respectively, and the fusion image of the visible light image and the near-infrared image is obtained based on the above-mentioned channel fusion images.

[0075] In this embodiment, fusing the first color channel of the visible light image with the near-infrared image to obtain a first color channel fused image includes: fusing the GR channel of the visible light image with the near-infrared image to obtain a first sub-channel fused image, and fusing the GB channel of the visible light image with the near-infrared image to obtain a second sub-channel fused image. Fusing the second color channel of the visible light image with the near-infrared image to obtain a second color channel fused image includes: based on the weighted average of the first sub-channel fused image and the second sub-channel fused image, fusing the R channel of the visible light image with the near-infrared image to obtain the second color channel fused image. Fusing the third color channel of the visible light image with the near-infrared image to obtain a third color channel fused image includes: based on the weighted average of the first sub-channel fused image and the second sub-channel fused image, fusing the B channel of the visible light image with the near-infrared image to obtain the third color channel fused image.

[0076] In the fusion method for visible light images and near-infrared images provided in the embodiments of the present disclosure, in order to enable the different structural regions where the visible light image has texture and the near-infrared image has no texture to maintain the texture of the visible light image, a visual edge preservation coefficient is specially proposed and set to preserve the visible light edge of the textureless region of the near-infrared image, so that during the fusion process, even if the textureless region of the near-infrared image cannot provide fusion guidance with the textured region of the visible light, the information attenuation of the textureless region of the near-infrared image can be reduced.

[0077] The following will introduce the image fusion method that introduces the visual edge-preserving coefficient.

[0078] Figure 4 The flowchart of the image fusion method based on the visual edge preservation coefficient according to the embodiment of the present disclosure is shown as follows: Figure 4 As shown, the method includes steps S211 to S213.

[0079] Step S211 : obtaining a first gradient relationship diagram between a first-order gradient of a first color channel of a visible light image and a first-order gradient of a near infrared image.

[0080] Step S212: Obtain an initial value of a first visual edge preservation coefficient based on a first-order gradient calculation of a first color channel of the visible light image; the first visual edge preservation coefficient is a two-dimensional weight coefficient.

[0081] Step S213: acquiring a first color channel fused image based on the first-order gradient of the near-infrared image, the first gradient relationship graph, and the initial value of the first visual edge preservation coefficient.

[0082] In some possible implementations, obtaining the first visual edge preservation coefficient includes using the following formula:

[0083]

[0084] In formula (1), α k It represents the first visual edge-preserving coefficient corresponding to the kth iteration calculation to obtain the first color channel fusion image, I k-1 represents the first color channel fused image calculated by the k-1th iteration, th is a constant coefficient, represents the first-order gradient of the first color channel fused image calculated at the k-1th iteration after denoising, I0 represents the original image of the first color channel of the visible light image at the first iteration, and the first visual edge preservation coefficient obtained based on I0 is the initial value of the first visual edge preservation coefficient.

[0085] In formula (1), the first-order gradient of the first color channel fusion image calculated by the k-1th iteration after denoising is obtained by a simple denoising algorithm such as Gaussian filtering or mean filtering. The degree of overall edge preservation is regulated by the constant coefficient th, and the noise contained in the visible light image is further suppressed by g(·). Generally, the degree of filtering can be reduced as the number of iterations increases.

[0086] In the above implementation, when the visible light image corresponding to the texture-free area of ​​the near-infrared image has a significant gradient, the position The value of is large, which makes the value of the first visual edge preservation coefficient small, so that the first-order gradient of the first color channel fused image accounts for a small proportion in the image fusion process, and the smoothing effect on the position is reduced, thereby improving the edge preservation degree of the visible light information at this position; and for the textured area of ​​the near-infrared image, after combining the above-mentioned first visual edge preservation coefficient, the first color channel fused image, on the basis of including the visible light texture information and the near-infrared texture information, improves the visible light information component corresponding to the textured area of ​​the near-infrared image.

[0087] In some possible implementations, obtaining a first color channel fused image based on the first-order gradient of the near-infrared image, the first gradient relationship graph, and the initial value of the first visual edge preservation coefficient includes: obtaining an objective function based on the first-order gradient of the near-infrared image and the initial value of the first visual edge preservation coefficient; and alternately iteratively calculating the objective function to obtain an optimal first color channel fused image.

[0088] In some possible implementations, obtaining the first color channel fused image based on the first-order gradient of the near-infrared image and the initial value of the first visual edge-preserving coefficient includes using the following objective function formula:

[0089]

[0090] In formula (2), I1 represents the optimal first color channel fusion image to be solved, represents the first-order gradient of the optimal first color channel fusion image to be solved, s represents the first gradient relationship diagram of the optimal first color channel fusion image to be solved and the near-infrared image, represents the first-order gradient of the near-infrared image, I0 represents the image of the first color channel of the visible light image, is the control term for the smoothness and continuity of s, α is the first visual edge-preserving coefficient, and λ and β are constant terms.

[0091] In equation (2), the constant term λ controls the confidence of the noisy image I0, and its value can usually be set empirically to [1,10]. β corresponds to the smoothness of the relationship graph s, and its value can usually be set empirically to [0.2,0.8].

[0092] In the above objective function (2), the optimal first gradient relationship map s of the first color channel fusion image and the near-infrared image to be solved includes the inhomogeneous information that may exist in the two spectral imaging (such as the inconsistency of the gradient amplitude and direction in the local area, etc.). After the objective function is introduced, the image fusion anomaly caused by the above inhomogeneity can be suppressed. At the same time, the ability of the first gradient relationship map s to express the above inhomogeneous information also reduces the parallax of image fusion to a certain extent.

[0093] The above objective function (2) is implemented based on the classic fusion algorithm Scale map. For the classic fusion algorithm Scalemap, it includes the visual gradient fidelity term in the above objective function (2) Brightness fusion degree control term λ||I1-I0|| and regularization term The relationship between the target gradient map and the near-infrared image is represented by an s-map, which includes the possible different configuration information in the two spectral images. Introducing the s-map into the objective function of the classic fusion algorithm Scale map can suppress the image fusion anomaly caused by the different configurations. At the same time, the ability of the s-map to express the different configuration information is also the key to the algorithm's ability to tolerate parallax to a certain extent.

[0094] It should be noted that the s-map has the following properties: negative values ​​in the s-map: indicates that there is an edge at that position in the two images, but the gradient direction of the edge is opposite; zero values ​​in the s-map: structures unique to near-infrared images, such as shadows caused by infrared lighting; when the visible light image contains texture areas that do not exist in the near-infrared image, When it is 0, the value of the s map has little effect on the result. At this time, due to the lack of information guidance from the near-infrared image, the classic fusion algorithm Scale map will cause obvious visible light information attenuation in the fusion result in this area, especially for high-noise visible light images.

[0095] In the objective function (2) of the present disclosure, the first visual edge-preserving coefficient α is introduced to maintain the visible light texture guided by the near-infrared texture-free region. When the visible light image has a significant gradient, The smaller the proportion coefficient in the objective function, the less smoothing effect on the position, and the higher the edge preservation degree of the visible light information at that position.

[0096] In some possible implementations, when obtaining the first color channel fused image based on the objective function (2), the objective function (2) is vectorized and written in the form of a matrix equation, and the preconditioned conjugate gradient method is used to alternately iterate and solve the first color channel fused image I1 and the first gradient relationship graph s. Generally, after 3 to 6 iterations, the fused image result, i.e., the optimal first color channel fused image, can be obtained.

[0097] In the method for fusing visible light images and near-infrared images provided in the embodiments of the present disclosure, in order to enable the fused image to maintain the color of the visible light image and avoid the same numerical regions of each color channel due to the separate fusion of each color channel and the further weakening of some areas with weaker color information after fusion, making the color gray, a priori term between color channels is specially proposed and set, and the color information between different color channels is associated and introduced into the above-mentioned objective function to optimize the performance of maintaining the visible light color.

[0098] The following will introduce the image fusion method that introduces color prior terms.

[0099] Figure 5 The flowchart of the image fusion method based on color prior items described in the embodiment of the present disclosure is shown as follows: Figure 5 As shown, the method includes steps S221 to S224.

[0100] Step S221, obtaining a second gradient relationship diagram between the first-order gradient of the second color channel of the visible light image and the first-order gradient of the near-infrared image, and obtaining a third gradient relationship diagram between the first-order gradient of the third color channel of the visible light image and the first-order gradient of the near-infrared image.

[0101] Step S222: Obtain an initial value of a second visual edge preservation coefficient based on a first-order gradient calculation of a second color channel of the visible light image, and obtain an initial value of a third visual edge preservation coefficient based on a first-order gradient calculation of a third color channel of the visible light image.

[0102] Step S223: obtaining a second channel priori item based on the first color channel fused image, and obtaining a third channel priori item based on the first color channel fused image.

[0103] Step S224, obtaining a second color channel fused image based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship map and the second channel prior item, and obtaining a third color channel fused image based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship map and the third channel prior item.

[0104] The method of obtaining the second visual edge preservation coefficient and the method of obtaining the third visual edge preservation coefficient are the same as the method of obtaining the first visual edge preservation coefficient, so they will not be described in detail here.

[0105] In some possible implementations, obtaining a second color channel fused image based on an initial value of a second visual edge preservation coefficient, a first-order gradient of a near-infrared image, a second gradient relationship graph, and a second channel prior item, or obtaining a third color channel fused image based on an initial value of a third visual edge preservation coefficient, a first-order gradient of a near-infrared image, a third gradient relationship graph, and a third channel prior item includes: obtaining an objective function based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship graph, and the second channel prior item, or obtaining an objective function based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship graph, and the third channel prior item; and alternately iteratively calculating the objective function to obtain an optimal second color channel fused image or a third color channel fused image.

[0106] In some implementations, the objective function based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship map, and the second channel prior item, or the objective function based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship map, and the third channel prior item is:

[0107]

[0108] In formula (3), I represents the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved, s represents the gradient relationship diagram between the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved and the near-infrared image, and R IGR represents the ratio of the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved to the first color channel fusion image. Lf The ratio of the second color channel or the third color channel to the first color channel representing the low-frequency component in the visible light image.

[0109] In equation (3), the constant term λ controls the confidence of the noisy image I0, and its value can usually be set empirically to [1,10]. β corresponds to the smoothness of the relationship graph s, and its value can usually be set empirically to [0.2,0.8].

[0110] It should be noted that the low-frequency component of the visible light image in the objective function (3) above is obtained by bilateral filtering, and the near-infrared gradient map can be optionally used as the filtering guide map. Lf Introducing the low-frequency components of the visible light image can suppress the influence of color noise on color restoration. Lf When the denominator is set to a minimum value, the denominator should be set to avoid the situation where the denominator is 0. Preferably, it is set to 10. -5 or other decimal value.

[0111] The color prior term ||R in the above objective function (3) IG -R Lf ||, based on the prior assumption: near-infrared images do not supplement color information, so the numerical differences between the color channels of visible light images and the numerical differences between the fused images of each color channel should be similar. If there are numerical differences between the color channels in a certain area of ​​the visible light image, it means that the area contains (non-gray) color information. The color prior items designed based on the above prior assumptions can maintain the numerical difference level between the color channels, and to a certain extent suppress the obvious weakening or color cast of the channel fused image.

[0112] In some possible implementations, when obtaining the second color channel fusion image or the third color channel fusion image based on the objective function (3), the objective function (3) is vectorized and written in the form of a matrix equation. The channel fusion image result can be obtained by using the preconditioned conjugate gradient method to alternately iterate and solve 3 to 6 times, that is, the optimal second color channel fusion image or the optimal third color channel fusion image.

[0113] In some possible implementations, in order to avoid the appearance of smearing, transparency and blur in the moving area, it is necessary to divide the moving and static areas in the imaging results. Compared with the original visible light image, the overall signal-to-noise ratio of the channel fusion image has been significantly improved, and the information of the channel fusion image sequence can be directly extracted for motion and static judgment. It is worth noting that when the noise level of the visible light original image is very high, the channel fusion image may still have a certain degree of low-frequency noise, which affects the accuracy of motion and static judgment. In addition, due to the differences in lighting direction and material absorption spectrum during imaging, there may be partial differences between the near-infrared image and the channel fusion image. Using only near-infrared information for motion and static judgment may cause errors such as misalignment of the moving area. Therefore, in the embodiment of the present disclosure, the motion and static judgment results are obtained by combining the channel fusion image and the near-infrared image to further improve the accuracy of the motion and static judgment and optimize the image quality of the moving area. The following describes a method for obtaining motion and static judgment results based on channel fusion images and near-infrared images.

[0114] Figure 6 The flowchart of the method for obtaining the dynamic and static judgment result according to the embodiment of the present disclosure is shown as follows: Figure 6 As shown, the method includes steps S31 to S32.

[0115] Step S31, respectively calculating and acquiring a motion / stillness decision weight distribution map of a channel fusion image and a motion / stillness decision weight distribution map of a near infrared image.

[0116] Step S32, fusion channel fuses the motion and stillness judgment weight distribution map of the image and the motion and stillness judgment weight distribution map of the near infrared image to obtain a motion and stillness judgment result.

[0117] In some implementations, respectively calculating and acquiring a motion / stillness judgment weight distribution map of a channel-fused image and a motion / stillness judgment weight distribution map of a near-infrared image includes but is not limited to: performing motion / stillness judgment by respectively setting corresponding thresholds for the inter-frame differences of the channel-fused image and the near-infrared image; respectively calculating and acquiring feature maps of the channel-fused image and the near-infrared image, performing motion / stillness judgment based on the feature maps, etc.

[0118] In some implementations, the motion and stillness decision weight distribution map of the fusion channel fusion image and the motion and stillness decision weight distribution map of the near-infrared image include but are not limited to: calculating the smaller value of the two weight distribution maps point by point to suppress the misidentification of motion areas caused by the visible light image being affected by high noise.

[0119] In some implementations, for the acquired channel fusion images of continuous frames, the channel fusion images are superimposed frame by frame based on the motion and stillness judgment results to obtain the channel inter-frame fusion images of continuous frames, including: having a larger superposition weight in the static area to complete the noise reduction in the static area; having a smaller superposition weight in the moving area to avoid the ghosting and transparency phenomenon in the moving area.

[0120] In some implementations, image signal processing is performed on the acquired inter-channel frame fusion image to obtain a full-color image, including demosaicing the acquired inter-channel frame fusion image using a bilinear interpolation algorithm or an adaptive interpolation algorithm, and contrast adjustment of the acquired inter-channel frame fusion image using methods such as histogram equalization, gamma correction, or local contrast enhancement.

[0121] The following example illustrates the effect of the full-color imaging method provided by the embodiment of the present disclosure.

[0122] like Figures 7a to 7d As shown, the full-color imaging result of a still scene is displayed.

[0123] Figure 7a For low-light visible light images, Figure 7b For Figure 7a The corresponding near-infrared image, Figure 7c is a full-color image obtained based on the Scalemap algorithm. Figure 7d A full-color image obtained by the method provided in an embodiment of the present disclosure.

[0124] Due to the particularity of the material, some areas (such as the number card at the bottom of the image) have no obvious texture in the near-infrared imaging results, and some areas (such as the weak texture on the rectangular object pointed by the arrow on the right) can be clearly displayed in near-infrared imaging but are submerged by severe noise in the visible light image. Figure 7c In the example, the weak texture on the rectangular object pointed by the arrow can be restored, but the texture edge of the number card is not clear and the color information is obviously lost. Figure 7d In the figure, on the basis of restoring the weak texture on the brown rectangular object, the texture edge and color information of the number card area are further maintained.

[0125] like Figures 8a to 8d Shown is a full-color imaging result including the motion area.

[0126] Figure 8a For low-light visible light images, Figure 8b For Figure 8a The corresponding near-infrared image, Figure 8c is a full-color image obtained based on the Scalemap algorithm. Figure 8d A full-color image obtained by the method provided in an embodiment of the present disclosure.

[0127] The low-light environment (high noise in visible light) contains moving areas. For the scale map algorithm without near-infrared fusion, when performing temporal noise reduction, the motion information between frames may be submerged by severe noise, resulting in abnormal motion and stillness judgment, and then obvious motion smear, such as Figure 8cAs shown. The full-color imaging method provided by the present disclosure can fuse a near-infrared image with a high signal-to-noise ratio and a visible light image with a low signal-to-noise ratio before temporal noise reduction to obtain a fused image with a high overall signal-to-noise ratio. When subsequent temporal noise reduction is performed based on the fused image, the motion information between frames can be effectively identified, and abnormal phenomena such as motion smear can be suppressed, such as Figure 8d shown.

[0128] It should be noted that the protection scope of the full-color imaging method of the embodiment of the present disclosure is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present disclosure are included in the protection scope of the present disclosure.

[0129] The embodiment of the present disclosure also provides a full-color imaging device. The full-color imaging device can implement the full-color imaging method provided by the embodiment of the present disclosure, but the sight device of the full-color imaging method provided by the embodiment of the present disclosure includes but is not limited to the structure of the full-color imaging device listed in this embodiment, and all structural deformations and replacements of the prior art made according to the principles of the present disclosure are included in the protection scope of the present disclosure.

[0130] In some implementations, a full-color imaging device includes a memory and at least one processor, wherein the at least one processor is coupled to the memory.

[0131] The at least one processor is configured to acquire a visible light image and a near infrared image, wherein the image data of the visible light image and the image data of the near infrared image are the original data captured by the dual camera module. The at least one processor is configured to perform channel fusion on the visible light image and the near infrared image to acquire a channel fusion image. The at least one processor is configured to acquire a motion and stillness judgment result based on the channel fusion image and the near infrared image. In addition, the at least one processor is configured to acquire a full-color image based on the motion and stillness judgment result and the channel fusion image.

[0132] It should be understood that the at least one processor may be further configured to execute each corresponding step or action in the method described in the above embodiment, which will not be described in detail here.

[0133] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0134] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure. For example, the functional modules / units in the various embodiments of the present disclosure may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0135] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0136] The embodiment of the present disclosure also provides an electronic device. Fig. 9 The structure diagram of the electronic device described in the embodiment of the present disclosure is shown. Fig. 9 As shown, in this embodiment, the electronic device 900 includes a memory 910 and a processor 920 .

[0137] The memory 910 is used to store computer programs and includes: ROM, RAM, disk, USB flash drive, memory card, optical disk, and other media that can store program codes.

[0138] Specifically, the memory 910 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 910 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0139] The processor 120 is connected to the memory 910 and is used to execute the computer program stored in the memory 910 so that the electronic device 900 executes the full-color imaging method described in any embodiment of the present disclosure.

[0140] In some embodiments, the processor 920 may be a general-purpose processor, including a central processing unit (CPU) and a network processor (NP). In other embodiments, the processor 320 may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0141] In this embodiment, the electronic device 900 may further include a display 930. The display 930 is communicatively connected with the memory 910 and the processor 920, and is used to display a graphical user interface (GUI) interaction interface related to the full-color imaging method described in the embodiment of the present disclosure.

[0142] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0143] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. A full-color imaging method, characterized in that: include: Acquire a visible light image and a near infrared image, wherein the image data of the visible light image and the image data of the near infrared image are original data captured by a dual camera module; The visible light image and the near infrared image are fused channel by channel to obtain channel fused images corresponding to each channel; Obtaining a motion / stillness judgment result based on the channel fusion image and the near infrared image; as well as A full-color image is acquired based on the motion / stillness judgment result and the channel fusion image.

2. The full-color imaging method according to claim 1, characterized in that: The step of fusing the visible light image and the near infrared image by channel to obtain channel fused images corresponding to the respective channels includes: fusing a first color channel of the visible light image of a current frame with the near infrared image of the current frame to obtain a first color channel fused image; Based on the first color channel fused image, a second color channel of the visible light image of the current frame is fused with the near infrared image of the current frame to obtain a second color channel fused image, and a third color channel of the visible light image of the current frame is fused with the near infrared image of the current frame to obtain a third channel fused image; and The first color channel fusion image, the second color channel fusion image and the third channel fusion image are respectively used as the channel fusion images of the current frame.

3. The full-color imaging method according to claim 2, characterized in that: Acquiring a full-color image based on the motion / stillness judgment result and the channel fusion image includes: By combining the motion / stillness judgment result obtained by combining the channel fusion image and the near-infrared image, performing channel-by-channel time-domain noise reduction on multiple frames of the channel fusion image to obtain fusion images between frames of each channel respectively; and Image signal processing is performed on the channel inter-frame fusion image to obtain the full-color image.

4. The full-color imaging method according to claim 2, characterized in that: The first color channel of the visible light image includes at least one channel associated with a G color; The fusing of the first color channel of the visible light image with the near-infrared image to obtain a first color channel fused image comprises: when the first color channel of the visible light image includes only one channel, fusing the first color channel of the visible light image with the near-infrared image to obtain a first color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, fusing each sub-channel of the first color channel of the visible light image with the near-infrared image to obtain each sub-channel fused image, wherein each sub-channel of the first color channel of the visible light image is fused in the same manner as the near-infrared image, and is the same as the fusion manner of the first color channel of the visible light image and the near-infrared image when the first color channel of the visible light image includes only one channel, and this fusion manner is the first fusion manner; The fusing of the second color channel of the visible light image with the near-infrared image to obtain a second color channel fused image comprises: when the first color channel of the visible light image includes only one channel, based on the first color channel fused image of the visible light image, fusing the second color channel of the visible light image with the near-infrared image to obtain the second color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, based on the weighted average of all sub-channel fused images of the first color channel of the visible light image, fusing the second color channel of the visible light image with the near-infrared image to obtain the second color channel fused image; The fusing of the third color channel of the visible light image with the near-infrared image to obtain a third color channel fused image comprises: when the first color channel of the visible light image includes only one channel, based on the first color channel fused image of the visible light image, fusing the third color channel of the visible light image with the near-infrared image to obtain the third color channel fused image; when the first color channel of the visible light image includes two or more sub-channels, based on the weighted mean of all sub-channel fused images of the first color channel of the visible light image, fusing the third color channel of the visible light image with the near-infrared image to obtain the third color channel fused image.

5. The full-color imaging method according to claim 4, characterized in that: The first fusion method for fusing the first color channel of the visible light image with the near infrared image includes: Acquire a first gradient relationship diagram between a first-order gradient of a first color channel of the visible light image and a first-order gradient of the near-infrared image; Obtaining an initial value of a first visual edge preservation coefficient based on a first-order gradient calculation of a first color channel of the visible light image, where the first visual edge preservation coefficient is a two-dimensional weight coefficient; The first color channel fused image is acquired based on the first-order gradient of the near-infrared image, the first gradient relationship map, and the initial value of the first visual edge preservation coefficient.

6. The full-color imaging method according to claim 5, characterized in that: Acquiring the first color channel fused image based on the first-order gradient of the near-infrared image, the first gradient relationship graph, and the initial value of the first visual edge preservation coefficient includes using the following objective function formula: Where I1 represents the optimal first color channel fusion image to be solved, represents the first-order gradient of the optimal first color channel fusion image to be solved, s represents the gradient relationship diagram between the optimal first color channel fusion image to be solved and the near-infrared image, represents the first-order gradient of the near-infrared image, I0 represents the image of the first color channel of the visible light image, is the control term of s, α is the first visual edge-preserving coefficient, λ and β are constant terms; The objective function is calculated alternately and iteratively to obtain the optimal first color channel fused image.

7. The full-color imaging method according to claim 6, characterized in that: Obtaining the first visual edge preservation coefficient includes using the following formula: Among them, α k represents the first visual edge-preserving coefficient corresponding to the k-th iteration calculation of the objective function, I k-1 represents the first color channel fused image calculated at the k-1th iteration, th is a constant coefficient, represents the first-order gradient of the first color channel fused image calculated at the k-1th iteration after denoising, I0 represents the original image of the first color channel of the visible light image at the first iteration, and the first visual edge preservation coefficient obtained based on I0 is the initial value of the first visual edge preservation coefficient.

8. The full-color imaging method according to claim 2, characterized in that: Fusing the second color channel of the visible light image with the near infrared image to obtain a second color channel fused image includes: Acquire a second gradient relationship diagram between a first-order gradient of a second color channel of the visible light image and a first-order gradient of the near-infrared image; Obtaining an initial value of a second visual edge preservation coefficient based on a first-order gradient calculation of a second color channel of the visible light image; Acquire a second channel priori item based on the first color channel fused image; The second color channel fused image is acquired based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship map and the second channel priori item.

9. The full-color imaging method according to claim 2, characterized in that: Fusing the third color channel of the visible light image with the near infrared image to obtain a third color channel fused image. The third color channel fused image includes: Acquire a third gradient relationship diagram between a first-order gradient of a third color channel of the visible light image and a first-order gradient of the near-infrared image; Obtaining an initial value of a third visual edge preservation coefficient based on a first-order gradient calculation of a third color channel of the visible light image; Acquire a third channel priori item based on the first color channel fused image; The third color channel fused image is acquired based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship graph and the third channel priori item.

10. The full-color imaging method according to claim 8 or 9, characterized in that: The second color channel fused image is obtained based on the initial value of the second visual edge preservation coefficient, the first-order gradient of the near-infrared image, the second gradient relationship map and the second channel priori item, or the third color channel fused image is obtained based on the initial value of the third visual edge preservation coefficient, the first-order gradient of the near-infrared image, the third gradient relationship map and the third channel priori item, including using the following objective function formula: Wherein, I represents the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved, s represents the gradient relationship diagram between the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved and the near infrared image, and R IG represents the ratio of the optimal second color channel fusion image to be solved or the optimal third color channel fusion image to be solved to the first color channel fusion image, R Lf A ratio of the second color channel or the third color channel to the first color channel representing a low-frequency component in the visible light image; The objective function is calculated alternately and iteratively to obtain an optimal second color channel fused image or an optimal third color channel fused image.

11. The full-color imaging method according to claim 10, characterized in that: Acquiring a full-color image based on the motion / stillness judgment result and the channel fusion image includes: For the acquired channel fusion images of continuous frames, the channel fusion images are superimposed frame by frame based on the motion and stillness judgment results to obtain channel inter-frame fusion images of continuous frames, and image signal processing is performed on the channel inter-frame fusion images to obtain the full-color image.

12. The full-color imaging method according to claim 1, characterized in that: Obtaining a motion / stillness judgment result based on the channel fusion image and the near infrared image includes: Respectively calculating and obtaining a motion and stillness decision weight distribution map of the channel fusion image and a motion and stillness decision weight distribution map of the near infrared image; The motion / stillness judgment weight distribution map of the channel fusion image and the motion / stillness judgment weight distribution map of the near infrared image are fused to obtain a motion / stillness judgment result.

13. The full-color imaging method according to claim 1, characterized in that: Before performing channel-by-channel fusion on the visible light image and the near infrared image, the method further includes: performing parallax correction on the near infrared image to align the near infrared image with the visible light image; and The visible light image and the near infrared image after parallax correction are preprocessed respectively.

14. A full-color imaging device, characterized in that: include: Memory; as well as at least one processor coupled to the memory and configured to: Acquire a visible light image and a near infrared image, wherein the image data of the visible light image and the image data of the near infrared image are original data captured by a dual camera module; Performing channel-by-channel fusion on the visible light image and the near infrared image to obtain a channel-fused image; Obtaining a motion / stillness judgment result based on the channel fusion image and the near infrared image; as well as A full-color image is acquired based on the motion / stillness judgment result and the channel fusion image.

15. An electronic device, characterized in that: include: a memory configured to store an executable program; as well as The processor is configured to execute the program so that the electronic device performs the full-color imaging method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed, the full-color imaging method according to any one of claims 1 to 13 is implemented.

Images