A method for achieving color enhancement imaging based on spectral transfer

Abstract

This invention provides a method for achieving color enhancement imaging based on spectral migration. It collects imaging information including ultraviolet and / or near-infrared and visible light intensity using a visible light imaging lens that includes ultraviolet and / or near-infrared spectra; and calculates the imaging intensity In of the ultraviolet and / or near-infrared color channels for each pixel. UV and / or In NIR Calculate the hue H of each pixel in the HSaIn color space for visible light. VL Saturation Sa VL and intensity value In VL ; via In IN＝ In VL +In UV +In NIR In N =In OUT =LUT(In IN ), Sa IN =Sa VL Sa N =Sa OUT =LUT(Sa IN ), H N =H VI Calculate the output color data H N Sa N In N Then convert it to the new XYZ color space X N Y N Z N Value. This invention utilizes novel technical theories and borrows imaging spectral energy from the near-infrared and / or ultraviolet bands to shift imaging spectral energy according to the method of this invention, thereby improving the signal-to-noise ratio of nighttime imaging, significantly enhancing full-color imaging effects, and achieving clearer and more visible images.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and in particular to a method for achieving color enhancement imaging based on spectral migration. Background Technology

[0002] With societal development, the demand for video cameras capable of color imaging in extremely low-light environments is increasing, and the application areas are expanding. From mobile phone shooting and smart cars to outdoor shooting, security monitoring, and various large-scale engineering projects, color imaging equipment for extremely low-light environments can play a significant role.

[0003] Currently, ordinary surveillance equipment can hardly achieve effective monitoring in low-light environments at night, and some scenarios cannot or are not suitable for large-area light sources, which makes ordinary cameras ineffective in low-light environments.

[0004] While infrared cameras can shoot at night, they still have certain limitations. Infrared cameras cannot accurately reflect the surrounding environment; furthermore, the image color is limited, typically black and white or red and blue, failing to reflect the true colors of objects. Due to their imaging principle and manufacturing process, infrared cameras have relatively low resolution, which cannot meet the requirements for high-definition viewing. Summary of the Invention

[0005] To address the technical problems mentioned in the background, this invention provides a method for color enhancement imaging based on spectral migration. This method offers a novel solution to the current problems in video surveillance under low-light conditions at night, effectively improving the color imaging performance of cameras in various low-light environments and achieving full-color imaging in ultra-low light. Through a novel technical theory and by utilizing the imaging spectral energy of the near-infrared and / or ultraviolet bands, the imaging spectral energy is migrated according to the method of this invention, improving the signal-to-noise ratio of nighttime imaging, significantly enhancing the full-color imaging effect, and achieving a clearer and more visible image.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A method for achieving color enhancement imaging based on spectral transfer includes the following steps:

[0008] Step 1: Collect imaging information including ultraviolet and / or near-infrared and visible light intensity through a visible light imaging lens that includes ultraviolet and / or near-infrared spectra;

[0009] Step 2: Acquire multispectral image information R, G, B through a CMOS image sensor capable of receiving imaging intensity information including ultraviolet and / or near-infrared and visible light. R, G, B are the original RGB image information. In engineering, R, G, B of RGB image information approximate X, Y, Z in the XYZ color space.

[0010] Step 3: Calculate the imaging intensity of the ultraviolet and / or near-infrared color channels: Intensity of ultraviolet light UV and / or the intensity of near-infrared light In NIR ;

[0011] Extract visible light information X from the XYZ color space VL Y VL Z VL Z VL That is, the original XYZ color space information X O Y O Z O ;

[0012] Step 4: Calculate the H of each pixel in the visible light spectrum using the HSaIn color space and the (X, Y, Z) conversion model. VL Sa VL In VL H VL Sa VL In VL These are the hue, saturation, and intensity values ​​for the HSaIn color space, respectively.

[0013] Step 5: Via In N =In OUT =LUT(In IN = LUT(In) VL +In UV +In NIR ) or In N =In OUT =LUT(In IN = LUT(In) VL +In UV ) or In N =In OUT =LUT(In IN = LUT(In) VL +In NIR ),

[0014] Sa N =Sa OUT =LUT(Sa IN ) = LUT(Sa VL ), H N =H VL Calculate the output color data H N Sa N In N ;

[0015] Among them: In NSa represents the light intensity value in the new HSaIn color space. N For the new HSaIn color space, H N The hue values ​​for the new HSaIn color space;

[0016] LUT is a computational function in the field of color computing. It means that after each color information is "repositioned" by the LUT, a new color information can be obtained.

[0017] In OUT For intensity output; Sa OUT Output saturation;

[0018] In IN For intensity input; Sa IN Input for saturation;

[0019] In VL The intensity of visible light;

[0020] H VL Visible light hue value;

[0021] Sa VL This represents the visible light saturation value.

[0022] Step 6: Using the HSaIn color space and (X, Y, Z) mutual conversion model, convert the color data H in the new HSaIn color space. N Sa N In N Transformed into the new XYZ color space N Y N Z N value.

[0023] Furthermore, the imaging intensity In of the ultraviolet and / or near-infrared color channels UV and / or In NIR as follows:

[0024] Ultraviolet and near-infrared light lack hue and chroma; their hue value H and chroma value Cl are both 0, i.e., H = 0, Cl = 0. Using the formula In = Gl + Cl, the light intensity of ultraviolet and near-infrared light is equal to their gray value Gl, i.e.:

[0025] In UV =Gl UV In NIR =Gl NIR

[0026] Gl UV This represents the gray value of ultraviolet light;

[0027] Gl NIRThis represents the gray value of near-infrared light.

[0028] Furthermore, the saturation input value Sa IN and light intensity input value In IN The calculation is as follows:

[0029] Sa IN =Sa VL =Cl VL / In VL ;

[0030] In IN =In VL +In UV +In NIR ;

[0031] Cl VL This represents the color intensity value of visible light.

[0032] Furthermore, in step 5, the LUT operation is as follows:

[0033] Determine the intensity mapping relationship and saturation mapping relationship under the isotone surface based on the color gamut of the input and output devices, the color distribution range of the image, and the color rendering intention;

[0034] Image input device-side color data is processed based on intensity and saturation mapping relationships. IN Sa IN Color data to output device side In OUT Sa OUT The mapping yields the output device-side HSaIn format color data Sa. N In N .

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] This invention provides a method for color enhancement imaging based on spectral migration, offering a novel solution to the problems existing in current video surveillance under low-light conditions at night. It effectively improves the color imaging performance of cameras in various low-light environments, achieving full-color imaging in ultra-low light. Through a novel technical theory and by utilizing the imaging spectral energy of the near-infrared and / or ultraviolet bands, the imaging spectral energy is migrated according to the method of this invention, improving the signal-to-noise ratio of nighttime imaging, significantly enhancing the full-color imaging effect, and achieving a clearer and more visible image. Attached Figure Description

[0037] Figure 1 This is a flowchart of a method for achieving color enhancement imaging based on spectral transfer according to the present invention;

[0038] Figure 2 This is the intensity mapping diagram of the present invention;

[0039] Figure 3 This is the saturation mapping diagram of the present invention;

[0040] Figure 4 This is a typical image taken in low-light conditions at night.

[0041] Figure 5 These are images showing the shooting effect of the method of this invention in a low-light environment at night. Detailed Implementation

[0042] The specific embodiments provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0043] like Figure 1 As shown, a method for achieving color enhancement imaging based on spectral migration according to the present invention includes the following steps:

[0044] Step 1: Collect imaging information including ultraviolet and / or near-infrared and visible light intensity through a visible light imaging lens that includes ultraviolet and / or near-infrared spectra;

[0045] Step 2: Acquire multispectral image information R, G, B through a CMOS image sensor capable of receiving imaging intensity information including ultraviolet and / or near-infrared and visible light. R, G, B are the original RGB image information. In engineering, R, G, B of RGB image information approximate X, Y, Z in the XYZ color space.

[0046] Step 3: Calculate the imaging intensity of the ultraviolet and / or near-infrared color channels: Intensity of ultraviolet light UV and / or the intensity of near-infrared light In NIR ;

[0047] Extract visible light information X from the XYZ color space VL Y VL Z VL ;X VL Y VL Z VL That is, the original XYZ color space information X O Y O Z O ;

[0048] Step 4: Calculate the H of each pixel in the visible light spectrum using the HSaIn color space and the (X, Y, Z) conversion model. VL Sa VL In VL H VL Sa VL In VLThese are the hue, saturation, and intensity values ​​for the HSaIn color space, respectively.

[0049] Step 5: Via In N =In OUT =LUT(In IN = LUT(In) VL +In UV +In NIR ) or In N =In OUT =LUT(In IN = LUT(In) VL +In UV ) or In N =In OUT =LUT(In IN = LUT(In) VL +In NIR ),

[0050] Sa N =Sa OUT =LUT(Sa IN ) = LUT(Sa VL ), H N =H VL Calculate the output color data H N Sa N In N ;

[0051] Among them: In N Sa represents the light intensity value in the new HSaIn color space. N For the new HSaIn color space, H N The hue values ​​for the new HSaIn color space;

[0052] LUT is a computational function in the field of color computing. It means that after each color information is "repositioned" by the LUT, a new color information can be obtained.

[0053] In OUT For intensity output; Sa OUT Output saturation;

[0054] In IN For intensity input; Sa IN Input for saturation;

[0055] In VL The intensity of visible light;

[0056] H VL Visible light hue value;

[0057] SaVL This represents the visible light saturation value.

[0058] Step 6: Using the HSaIn color space and (X, Y, Z) mutual conversion model, convert the color data H in the new HSaIn color space. N Sa N In N Transformed into the new XYZ color space N Y N Z N value.

[0059] Furthermore, the imaging intensity In of the ultraviolet and / or near-infrared color channels UV and / or In NIR as follows:

[0060] Ultraviolet and near-infrared light lack hue and chroma; their hue value H and chroma value Cl are both 0, i.e., H = 0, Cl = 0. Using the formula In = Gl + Cl, the light intensity of ultraviolet and near-infrared light is equal to their gray value Gl, i.e.:

[0061] In UV =Gl UV In NIR =Gl NIR

[0062] Gl UV This represents the gray value of ultraviolet light;

[0063] Gl NIR This represents the gray value of near-infrared light.

[0064] Intensity of ultraviolet light UV That is, the ash content value Gl UV and the light intensity of near-infrared light In NIR That is, the ash content value Gl NIR Image data is acquired using one or more combined image sensors based on existing technologies, or derived using algorithms based on existing technologies.

[0065] Furthermore, the saturation input value Sa IN and light intensity input value In IN The calculation is as follows:

[0066] Sa IN =Sa VL =Cl VL / In VL ;

[0067] In IN =In VL +In UV +In NIR ;

[0068] Cl VL This represents the color intensity value of visible light.

[0069] Furthermore, in step 5, the LUT operation is as follows:

[0070] Determine the intensity mapping relationship and saturation mapping relationship under the isotone surface based on the color gamut of the input and output devices, the color distribution range of the image, and the color rendering intention;

[0071] Image input device-side color data is processed based on intensity and saturation mapping relationships. IN Sa IN Color data to output device side In OUT Sa OUT The mapping yields the output device-side HSaIn format color data Sa. N In N .

[0072] See mapping relationship Figure 2-3 The mapping diagram.

[0073] In the above description, the variables and subscripts are explained as follows:

[0074] UV: Ultraviolet light;

[0075] NIR: Near-infrared light;

[0076] VL: Visible light;

[0077] N: A new function generated after a system operation;

[0078] O: An old function input before system operation;

[0079] IN: A function input in a system operation;

[0080] OUT: A function output during a system operation.

[0081] See Figure 4-5 The image is enhanced based on the calculated new image color data, enabling the acquisition of video images in low-light environments at night.

[0082] Specific Implementation Example 1: Imaging Lens and CMOS Image Sensor

[0083] Visible light, the light waves that are perceptible to the human eye, has a wavelength between approximately 780 nm and 400 nm. Light waves with wavelengths higher or lower than this range are imperceptible to the human eye. Light waves with wavelengths higher than 780 nm but lower than microwaves are called infrared light, which is divided into near-infrared, mid-infrared, and far-infrared rays. Light waves with wavelengths lower than 400 nm but higher than X-rays are called ultraviolet light.

[0084] In step 1, the imaging lens of the present invention uses a camera lens with a coating treatment to achieve imaging energy acquisition in the near-infrared band and / or ultraviolet band; and to achieve confocal imaging of ultraviolet, near-infrared and visible light.

[0085] In step 2, the present invention employs a CMOS image sensor capable of receiving ultraviolet, near-infrared, and visible light bands to achieve multispectral image data acquisition.

[0086] Ultraviolet, near-infrared, and visible light all exhibit linear superposition of imaging light intensities. When a camera collects light signals (including ultraviolet, near-infrared, and visible light) in a low-light environment, the intensity of its monochromatic light signal is approximately 11.5 times that of the monochromatic light signal in visible light (1.5 times the intensity of ultraviolet light + 9 times the intensity of near-infrared light + 1 time the intensity of monochromatic light).

[0087] The camera used in this invention only collects ultraviolet, visible, and near-infrared light for the following reasons:

[0088] 1) The intensity of light reflected by an object from ultraviolet, visible and near-infrared light is directly proportional to the intensity of the irradiating light. That is, under a certain absorption rate, the higher the intensity of the irradiating light, the higher the intensity of the reflected light, and vice versa.

[0089] 2) Lens glass has high compatibility with ultraviolet, visible, and near-infrared light. Lens glass that can transmit ultraviolet, visible, and near-infrared light simultaneously is readily available.

[0090] 3) By selecting suitable optical glass with wide transmittance and changing the lens coating spectrum, imaging information including ultraviolet, visible and near-infrared light spectra can be obtained.

[0091] 4) It is necessary to acquire ultraviolet and near-infrared light imaging signals, as well as visible light spectral CMOS imaging signals, for use in color enhancement algorithms.

[0092] Specific Implementation Example 2: HSaIn Color Space Model

[0093] Color space is a mathematical expression that describes the laws governing human color perception.

[0094] 1. The HSaIn color space and its relationship with the XYZ color space.

[0095] 1) HSaIn color space

[0096] The HSaIn color space includes hue (H), saturation (Sa), and intensity (In).

[0097] 2) The relationship between HSaIn color space and XYZ color space

[0098] In engineering, the approximation is (R, G, B) = (X, Y, Z).

[0099] 3) Definition of relevant parameters for the HSaIn color space

[0100] Achromatic light is called gray, described by the gray quantity defined below. When light has a hue, it is called chroma, described by the chroma quantity defined below.

[0101] Gray Level (Gl): Defined as the luminance stimulus value of gray light. The effective range of gray level is 0 to white light saturation value.

[0102] ChromaticLevel Defined as the stimulus value of the brightness of a pure color light under a single hue, it is a vector. (Chroma vector) It contains dual orthogonal information: chroma stimulus value (the magnitude Cl of the chroma vector, or simply chroma) and chroma hue (direction, or simply hue H). The effective range of the chroma stimulus value is 0 to the chroma saturation value.

[0103] Hue (H): The visual stimulus value of the color attribute of light perceived by the human eye. It is a component of the chroma vector, described by the polar angle or hue angle of the hue vector, which ranges from [0, 360°].

[0104] The intensity of colored light, In, is the sum of the gray intensity and chroma intensity of the colored light. In = Gl + Cl.

[0105] Saturation (Sa): The proportion of the chromaticity of a light source to its intensity. Sa = Cl / In.

[0106] 2. Convert XYZ color space (X,Y,Z) values ​​to HSaLn color space values.

[0107] The hue H in the HSaIn format color data is obtained using the following formula.

[0108]

[0109] Where X, Y, and Z are XYZ format color data, which are the tristimulus values ​​of color data in the XYZ color space, representing the values ​​on the X, Y, and Z axes of the XYZ color space, respectively.

[0110] The saturation Sa and intensity In in the HSaIn format color data are obtained according to the following formula and based on the XYZ format color data, that is, the Sa and In values ​​are obtained through the X, Y, and Z values.

[0111]

[0112] Km ,K M Let be positive real numbers, ln≥Gl≥0, A≥0, B≥0, and p and q be non-zero real numbers.

[0113] or

[0114]

[0115] Km and KM are positive real numbers, KM>Km, ln≥Gl≥0, A≥0, B≥0, p and q are non-zero real numbers.

[0116] or

[0117]

[0118] Km and KM are positive real numbers, KM>Km, ln≥Gl≥0, A≥0, B≥0, and p and q are non-zero real numbers.

[0119] or

[0120]

[0121] Km and KM are positive real numbers, ln≥Gl≥0, A≥0, B≥0, and p and m are non-zero real numbers.

[0122] or

[0123]

[0124] Km and KM are positive real numbers, KM>Km>0, In≥Gl≥0, A≥0, B≥0, and p, q, and r are non-zero real numbers.

[0125] or

[0126]

[0127] Km and KM are positive real numbers, ln ≥ Gl ≥ 0, A ≥ 0, B ≥ 0, and p, q, r are non-zero real numbers.

[0128] 3. Convert HSaLn color space values ​​to XYZ color space (X,Y,Z) values.

[0129] For example, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula:

[0130]

[0131] K m K M Let be positive real numbers, ln ≥ Gl ≥ 0, A ≥ 0, B ≥ 0, and p and q be non-zero real numbers.

[0132] The XYZ format color data from the output device side is obtained using the following formula:

[0133]

[0134] Alternatively, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula:

[0135]

[0136] Km and KM are positive real numbers, KM>Km, ln≥Gl≥0, A≥0, B≥0, and p and q are non-zero real numbers.

[0137] The XYZ format color data from the output device side is obtained using the following formula:

[0138] When 0°≤H<120°

[0139]

[0140]

[0141] When 120°≤H<240°

[0142]

[0143]

[0144]

[0145] When 240°≤H<360°

[0146]

[0147]

[0148]

[0149] Alternatively, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula:

[0150]

[0151] Km and KM are positive real numbers, KM>Km, ln≥Gl≥0, A≥0, B≥0, and p and q are non-zero real numbers.

[0152] The XYZ format color data from the output device side is obtained using the following formula:

[0153]

[0154] Alternatively, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula:

[0155]

[0156] Km and KM are positive real numbers, ln≥Gl≥0, A≥0, B≥0, and p and m are non-zero real numbers.

[0157] The XYZ format color data from the output device side is obtained using the following formula:

[0158] [·] is the integer operator for ·, where H ∈ [0°, 360°), h = 0, 1, 2

[0159] If h = 0

[0160]

[0161] If h = 1

[0162]

[0163] If h = 2

[0164]

[0165] Alternatively, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula.

[0166]

[0167] Km and KM are positive real numbers, KM>Km>0, In≥Gl≥0, A≥0, B≥0, and p, q, and r are non-zero real numbers.

[0168] The XYZ format color data from the output device side is obtained using the following formula:

[0169] [·] is the integer operator for ·, where H ∈ [0°, 360°), h = 0, 1, 2

[0170] If h = 0

[0171]

[0172]

[0173]

[0174] Based on the specific values ​​of p, q, and r, the values ​​of X and Y, represented by In, Sa, H, p, q, and r, are derived, where X>Z≥0, Y>Z≥0, and Z takes a value that conforms to the actual physical situation.

[0175] If h = 1

[0176]

[0177]

[0178]

[0179] Based on the specific p, q, r values, the X and Y values ​​represented by In, Sa, H, p, q, r are obtained, where X>Z≥0, Y>Z≥0, and Z takes a value that conforms to the actual physical situation.

[0180] If h = 2

[0181]

[0182]

[0183]

[0184] Based on the specific p, q, r values, the X and Y values ​​represented by In, Sa, H, p, q, r are derived, where X>Z≥0, Y>Z≥0, and Z takes a value that conforms to the actual physical situation.

[0185] Alternatively, the saturation Sa and intensity In in the HSaIn format color data on the input device side are obtained according to the following formula:

[0186]

[0187] Km and KM are positive real numbers, In≥Gl≥0, A≥0, B≥0, and p,q,r are non-zero real numbers.

[0188] The XYZ format color data from the output device side is obtained using the following formula:

[0189] [·] is the integer operator for ·, where H ∈ [0°, 360°), h = 0, 1, 2

[0190] If h = 0

[0191]

[0192] If h = 1

[0193]

[0194] If h = 2

[0195]

[0196] Variables in the above formulas that are not explicitly defined are all common knowledge.

[0197] The above embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the above embodiments. Unless otherwise specified, the methods used in the above embodiments are conventional methods.

Claims

1. A method for achieving color enhancement imaging based on spectral transfer, characterized in that, Includes the following steps: Step 1: Collect imaging information including ultraviolet and / or near-infrared and visible light intensity through a visible light imaging lens that includes ultraviolet and / or near-infrared spectra; Step 2: Acquire multispectral image information R, G, B through a CMOS image sensor capable of receiving imaging intensity information including ultraviolet and / or near-infrared and visible light. R, G, B are the original RGB image information. In engineering, R, G, B of RGB image information approximate X, Y, Z in the XYZ color space. Step 3: Calculate the imaging intensity of the ultraviolet and / or near-infrared color channels: Intensity of ultraviolet light UV and / or the intensity of near-infrared light In NIR ; Extract visible light information X from the XYZ color space VL Y VL Z VL ;X VL Y VL Z VL That is, the original XYZ color space information X O Y O Z O ; Step 4: Calculate the H value for each pixel of visible light using the HSaIn color space and XYZ color space conversion model. VL Sa VL In VL H VL Sa VL In VL These represent the hue, saturation, and intensity values ​​of visible light in the HSaIn color space. Step 5: Through In N = In OUT = LUT(In IN ) = LUT(In VL + In UV + In NIR ) or In N = In OUT = LUT(In IN ) = LUT(In VL + In UV ) or In N = In OUT = LUT(In IN ) = LUT(In VL + In NIR ), Sa N =Sa OUT =LUT(Sa IN ) = LUT(Sa VL ), H N =H VL Calculate the output color data H N Sa N In N ; Among them: In N Sa represents the light intensity value in the new HSaIn color space. N For the new HSaIn color space, H N The hue values ​​for the new HSaIn color space; LUT is a computational function in the field of color computing. It means that after each color information is "repositioned" by the LUT, a new color information can be obtained. In OUT For intensity output; Sa OUT Output saturation; In IN For intensity input; Sa IN Input for saturation; In VL The intensity of visible light; H VL Visible light hue value; Sa VL This represents the visible light saturation value. Step 6: Using the HSaIn color space and (X, Y, Z) mutual conversion model, convert the color data H in the new HSaIn color space. N Sa N In N Transformed into the new XYZ color space N Y N Z N value.

2. The method for achieving color enhancement imaging based on spectral transfer according to claim 1, characterized in that, The ultraviolet and / or near-infrared color channel imaging intensity In UV and / or In NIR as follows: Ultraviolet and near-infrared light lack hue and chroma; their hue value H and chroma value Cl are both 0, i.e., H = 0, Cl = 0. Using the formula In = Gl + Cl, the light intensity of ultraviolet and near-infrared light is equal to their gray value Gl, i.e.: In UV =Gl UV ,In NIR =Gl NIR Gl UV This represents the gray value of ultraviolet light; Gl NIR This represents the gray value of near-infrared light.

3. The method for achieving color enhancement imaging based on spectral transfer according to claim 1, characterized in that, Saturation input value Sa IN and light intensity input value In IN The calculation is as follows: In IN =In VL =Cl VL / In VL ; In IN =In VL +In UV +In NIR ; Cl VL This represents the color intensity value of visible light.

4. The method for achieving color enhancement imaging based on spectral transfer according to claim 1, characterized in that, In step 5, the LUT operation is as follows: Determine the intensity mapping relationship and saturation mapping relationship under the isotone surface based on the color gamut of the input and output devices, the color distribution range of the image, and the color rendering intention; Image input device-side color data is processed based on intensity and saturation mapping relationships. IN Sa IN Color data to the output device side In OUT Sa OUT The mapping yields the output device-side HSaIn format color data Sa. N In N .

Images