An image registration method

Abstract

The embodiment of the present application provides a kind of image registration method, method includes: obtaining infrared image and color image for the same monitoring area;For each first pixel in infrared image, according to the difference between the luminance component of the first pixel and the luminance component of each corresponding second pixel, and the corresponding relationship between the preset luminance difference value and the weight value, determine the first weight value of each fourth pixel corresponding to third pixel in color image;According to the first weight value of each fourth pixel corresponding to each third pixel, the chroma component of each fourth pixel corresponding to each third pixel is weighted average, obtains the first chroma component after each third pixel disparity removal;Based on the first chroma component of each third pixel, generate target image.Application the technical scheme provided in the embodiment of the present application, the disparity of the two images collected by double camera camera can be removed, and the image quality in low-illumination scene is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image registration method. Background Technology

[0002] To acquire high-quality images in low-light scenes, a method for image fusion using dual cameras has been introduced. Specifically, a color image and a black-and-white image are acquired using dual cameras, and the chroma of the color image and the luminance of the black-and-white image are fused to obtain a high-quality target image.

[0003] The image fusion method described above can improve image quality. However, due to the difference in position between the two cameras, i.e., a certain parallax between them, the images captured by the two cameras cannot be perfectly fused together, resulting in a poor-quality fused image. Summary of the Invention

[0004] The purpose of this application is to provide an image registration method to remove the parallax between two images captured by dual cameras and improve image quality in low-light scenes. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an image registration method, the method comprising:

[0006] Acquire infrared and color images of the same monitored area;

[0007] For each first pixel in the infrared image, based on the difference between the brightness component of the first pixel and the brightness component of each corresponding second pixel, and the preset correspondence between the brightness difference and the weight value, a first weight value is determined for each fourth pixel corresponding to the third pixel in the color image. The second pixel is a pixel in the infrared image whose distance from the first pixel is less than or equal to a first preset length, the third pixel is a pixel whose coordinates match the coordinates of the first pixel, and the fourth pixel is a pixel whose coordinates match the coordinates of the second pixel.

[0008] Based on the first weight value of each fourth pixel corresponding to each third pixel, the chromaticity components of each fourth pixel corresponding to each third pixel are weighted and averaged to obtain the first chromaticity component of each third pixel after parallax removal.

[0009] The target image is generated based on the first chromaticity component of each third pixel.

[0010] Secondly, embodiments of this application provide an image registration method, the method comprising:

[0011] Acquire a color image and an infrared image, wherein the color image and the infrared image cover the same monitoring area;

[0012] Generate a first offset, wherein the first offset is the offset that minimizes the sum of the brightness differences between all pixels in the infrared image after the offset and all pixels in the color image.

[0013] Based on the first offset, all the first pixels of the color image are offset to generate the offset color image;

[0014] Based on the luminance component of each pixel in the infrared image, a preset table is consulted to determine the first weight value of each pixel in the infrared image, wherein the preset table stores the correspondence between luminance difference and weight.

[0015] When the illuminance in the monitored area is lower than a first threshold, the chromaticity components of each pixel in the offset color image are weighted and filtered using the first weight value of each pixel in the infrared image to obtain the first target image.

[0016] Beneficial effects of the embodiments in this application:

[0017] In the technical solution provided in this application, the weights of pixels in a color image are determined using the luminance components of pixels in an infrared image. Then, the chrominance components of pixels in the color image are weighted and filtered using these weights to obtain a color image registered with the infrared image, i.e., the target image. This is because the probability of chrominance change is very high when luminance changes, and vice versa. Therefore, the luminance distribution can reflect the chrominance distribution to a certain extent. Furthermore, in low-light scenes, infrared images have low noise and exhibit luminance variations. Therefore, in the technical solution provided in this application, using the luminance components of pixels in an infrared image to weight and filter the chrominance components of pixels in a color image allows the chrominance distribution of pixels in the obtained target image to closely approximate the luminance distribution of pixels in the infrared image. In other words, it makes the chrominance of pixels in the target image closer to the true chrominance distribution, minimizing parallax between the infrared and color images and improving image quality in low-light scenes.

[0018] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating the optical principle of dual-camera imaging.

[0021] Figure 2 This is a schematic diagram of the first flowchart of the image registration method provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of a second process for the image registration method provided in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram of a third process for the image registration method provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of the fourth process of the image registration method provided in the embodiments of this application;

[0025] Figure 6 A fifth flowchart illustrating the image registration method provided in this application embodiment;

[0026] Figure 7 This is a sixth flowchart illustrating the image registration method provided in the embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0028] To acquire high-quality images in low-light scenes, a method for image fusion using dual cameras has been introduced. Specifically, a color image and a black-and-white image are acquired using dual cameras, and the chroma of the color image and the luminance of the black-and-white image are fused to obtain the target image.

[0029] Since black and white images have better light sensitivity than color images, and their signal-to-noise ratio is also better, the target image has a higher signal-to-noise ratio than the original color image. Therefore, the image fusion method using dual cameras described above can improve image quality.

[0030] However, in low-light scenes, natural light is almost zero. Both the color and black-and-white images are formed using natural light. These two images themselves have a low signal-to-noise ratio and poor image quality, resulting in a poor-quality target image after fusion. Furthermore, there is a certain parallax between the two cameras that captured the color and black-and-white images.

[0031] like Figure 1The diagram shows a schematic representation of the optical principle of dual-camera imaging. Figure 1 In the middle, O R and O T Let x represent the optical centers of the two cameras, P represent the object located within the monitored area, and P' and P” represent the imaging points in the image. R O R The coordinates of P' in the image captured by the camera, x T O T The coordinates of P in the image captured by the camera, O R and O T The focal lengths of the two cameras are f and O, respectively. R and O T The distance between the two cameras is b, and the distance between object P and the cameras (i.e., object distance) is z. Based on the triangle theorem, we know that:

[0032]

[0033] Based on the above formula (1), it can be seen that the disparity x for object P can be determined. T -x R for:

[0034]

[0035] As can be seen from the above formula (2), objects at different distances from the camera will have different coordinates in the images captured by the two cameras. When the dual-camera structure is fixed, that is, when the focal length f and distance b are unchanged, the coordinate difference of the same object in the images captured by the two cameras is inversely proportional to the object distance, that is, the parallax between the two cameras is inversely proportional to the object distance.

[0036] With a constant object distance z, the parallax between two cameras is proportional to the focal length f and the distance b. In some surveillance scenarios, cameras require telephoto lenses with large apertures, resulting in larger focal lengths f and distances b, which in turn leads to a larger parallax even with a constant object distance z. Combined with the fact that objects at different distances have different parallax values, the images captured by the two cameras cannot be perfectly fused together, resulting in a poor-quality fused image.

[0037] To improve image quality in low-light scenes, this application provides an image registration method. This image registration method can be applied to dual-camera systems or to electronic devices connected to dual-camera systems. The dual-camera system includes a color camera and an infrared camera. The color camera captures color images, and the infrared camera captures infrared images. The electronic device can be a tablet computer, mobile phone, personal computer, server, or other device with image processing capabilities.

[0038] In this image registration method, the weights of pixels in the color image are determined by the luminance components of pixels in the infrared image. Then, the weights are used to perform weighted filtering on the chrominance components of pixels in the color image to obtain a color image registered with the infrared image, i.e., the target image.

[0039] When brightness changes, the probability of chromaticity change is very high, and vice versa. Therefore, brightness distribution can reflect chromaticity distribution to a certain extent. Furthermore, in low-light scenes, infrared images have low noise and exhibit brightness variations. Therefore, the technical solution provided in this application uses the luminance component of pixels in the infrared image to perform weighted filtering on the chromaticity component of pixels in the color image. This makes the chromaticity distribution of pixels in the obtained target image closer to the luminance distribution of pixels in the infrared image; that is, it makes the chromaticity of pixels in the target image closer to the true chromaticity distribution, minimizing the parallax effect between the infrared and color images and improving image quality in low-light scenes.

[0040] The image registration method provided in this application will be described in detail below through specific embodiments. For ease of understanding, the following description uses an electronic device as the execution subject and is not intended to be limiting.

[0041] See Figure 2 , Figure 2 This is a schematic flowchart of an image registration method provided in an embodiment of this application. The method includes the following steps.

[0042] Step S21: Acquire infrared and color images of the same monitored area.

[0043] In this embodiment, the dual-camera system uses infrared illumination. One camera captures a color image, while the other captures an infrared image. The fields of view of the two cameras may or may not completely overlap. The color image contains only visible light information, while the infrared image contains both visible light and infrared light information.

[0044] A dual-camera system can send captured color and infrared images to an electronic device. The electronic device then receives and processes these images.

[0045] The dual-camera system can also store the captured color and infrared images in a preset database. The electronic device then retrieves the color and infrared images from this database.

[0046] In one embodiment of this application, after acquiring infrared and color images, the electronic device can further smooth the infrared and color images to reduce noise. Subsequently, the electronic device uses the smoothed infrared and color images for image registration to further improve the quality of the target image obtained after registration.

[0047] In this embodiment, the electronic device may use median filtering, Gaussian filtering, or other methods to smooth the infrared and color images. No limitation is imposed.

[0048] Step S22: For each first pixel in the infrared image, based on the difference between the brightness component of the first pixel and the brightness component of each corresponding second pixel, and the preset correspondence between the brightness difference and the weight value, determine the first weight value of each fourth pixel corresponding to the third pixel in the color image. The second pixel is a pixel in the infrared image whose distance from the first pixel is less than or equal to the first preset length, the third pixel is a pixel whose coordinates match the coordinates of the first pixel, and the fourth pixel is a pixel whose coordinates match the coordinates of the second pixel.

[0049] In this embodiment, after acquiring infrared and color images, the electronic device converts the infrared and color images into color spaces where color and chromaticity are relatively independent. The color space may include, but is not limited to, YUV, HSV, or LAB spaces. Here, Y, H, and L represent the luminance components, and U, V, S, V, A, and B represent the chromaticity components. Based on the color spaces obtained after converting the infrared and color images, the electronic device performs step S22 as described above.

[0050] Infrared images consist of multiple pixels, each of which is a first pixel, denoted by Pex. 11 Let's take an example to illustrate. The first pixel, Pex... 11 Pex, the third pixel in a color image 21 The coordinates match. Here, "coordinates match" can be understood as the coordinates being identical. Electronic devices have a pre-defined correspondence between brightness differences and weight values.

[0051] In the infrared image, the first pixel Pex 11 Multiple pixels whose distance is less than or equal to the first preset length are the second pixel Pex. 12 In the color image, the third pixel Pex 21 The distance is less than or equal to the fourth pixel Pex of the first preset length 22 Let's take an example to illustrate. As you can understand, the second pixel Pex... 12 With the fourth pixel Pex 22Coordinate matching.

[0052] Electronic devices calculate the first pixel Pex 11 The luminance component and each second pixel Pex 12 The brightness difference L of the brightness component 差1 For each second pixel Pex 12 The electronic device searches for the correspondence between preset brightness differences and weight values ​​to determine the Pex of the second pixel. 12 The corresponding brightness difference L 差1 The corresponding first weight value w 11 The second pixel Pex 12 The corresponding first weight value w 11 As, with the second pixel Pex 12 The fourth pixel Pex with matching coordinates 22 The first weight value.

[0053] Based on the above method, the electronic device can obtain the first weight value of each fourth pixel corresponding to the third pixel.

[0054] The aforementioned first preset length can be set according to actual needs, and this application embodiment does not limit it.

[0055] Step S23: Based on the first weight value of each fourth pixel corresponding to each third pixel, perform a weighted average of the chromaticity components of each fourth pixel corresponding to each third pixel to obtain the first chromaticity component of each third pixel after parallax removal.

[0056] In this embodiment of the application, for each third pixel, the electronic device performs a weighted average of the chromaticity components of each fourth pixel corresponding to the third pixel according to the weight of each fourth pixel corresponding to the third pixel, to obtain the chromaticity component of the third pixel after parallax removal, namely the first chromaticity component.

[0057] For example, the electronic device converts infrared and color images to the YUV space respectively, with a first preset length of r. The electronic device obtains the chromaticity component U of pixel (i, j) in the color image. c1 (i, j) and V c1 (i, j), and obtain the brightness component Y of pixel (i, j) in the infrared image. m1 (i, j). The coordinates of pixel (i, j) in the color image are matched with those of pixel (i, j) in the infrared image. Pixel (i, j) represents the pixel at coordinates (i, j).

[0058] In an infrared image, the luminance component of each pixel within a radius r centered at pixel (i, j) is Y. m1 (m, n). Electronic device calculates Y m1(m, n) and Y m1 The brightness difference between (i, j) is used to find the pre-defined correspondence between the brightness difference and the weight value, and the weight value of each pixel (m, n) in the infrared image is determined, which is then used as the weight value w of the corresponding pixel (m, n) in the color image. m,n .

[0059] The electronic device obtains the weight values ​​w of the corresponding pixels (m, n) in the color image. m,n Then, the first chromaticity component U of the chromaticity component U of pixel (i, j) in the color image can be determined using the following formula. c2 :

[0060]

[0061] In formula (3), U c2 (i, j) represents the first chromaticity component U of pixel (i, j). c2 w m,n U represents the weight value of pixel (m, n) in a color image. c1(m，n) U represents the chromaticity component of a pixel (m, n) in a color image.

[0062] Similarly, electronic devices can determine the first degree component V of the chromaticity component V of pixel (i, j) in a color image. c2 Further explanation is not provided here.

[0063] The above U c2 (i, j) and V c2 (i, j) are collectively referred to as the first chromaticity component of pixel (i, j).

[0064] Step S24: Generate the target image based on the first chromaticity component of each third pixel.

[0065] In this embodiment, after obtaining the first chromaticity component of each third pixel, the electronic device generates a target image based on the first chromaticity component of each third pixel. For example, the electronic device uses the first chromaticity component of each third pixel as the chromaticity component of the pixel at the corresponding coordinate in the target image. That is, it uses the first chromaticity component of pixel (i, j) as the chromaticity component of pixel (i, j) in the target image, and combines it with the luminance component of pixel (i, j) in the color image, that is, it uses the luminance component of pixel (i, j) in the color image as the luminance component of pixel (i, j) in the target image to obtain the target image.

[0066] In the technical solution provided in this application embodiment, the luminance component of the pixel in the infrared image is used to perform weighted filtering on the chrominance component of the pixel in the color image. This makes the chrominance distribution of the pixel in the obtained target image close to the luminance distribution of the pixel in the infrared image. In other words, it makes the chrominance of the pixel in the target image close to the true chrominance distribution, thereby eliminating the parallax between the infrared image and the color image as much as possible and improving the image quality in low-light scenes.

[0067] To meet the needs of image registration in scenes with different lighting intensities, embodiments of this application also provide an image registration method, such as... Figure 3 As shown, the image registration method may also include steps S25 and S26. In this case, step S24 can be refined into step S241.

[0068] Step S25: For each first pixel in the infrared image, determine the second weight value of each fourth pixel corresponding to the third pixel in the color image based on the difference between the chromaticity component of the first pixel and the chromaticity component of the corresponding second pixel, and the preset correspondence between the chromaticity difference and the weight.

[0069] Let's continue with the example from step S22 above. Taking the infrared image with the first pixel Pex... 11 Multiple pixels whose distance is less than or equal to the first preset length are the second pixel Pex. 12 In the color image, the third pixel Pex 21 The distance is less than or equal to the fourth pixel Pex of the first preset length 22 Let's take an example to illustrate.

[0070] Electronic devices calculate the first pixel Pex 11 The chromaticity components and each second pixel Pex 12 The chromaticity difference L of the chromaticity components 差2 For each second pixel Pex 12 The electronic device searches for the preset correspondence between chromaticity difference values ​​and weight values ​​to determine the Pex of the second pixel. 12 The corresponding chromaticity difference L 差2 The corresponding second weight value w 22 The second pixel Pex 12 The second weight value w 22 As, with the second pixel Pex 12 The fourth pixel Pex with matching coordinates 22 The second weight value.

[0071] In one embodiment of this application, after acquiring an infrared image, the electronic device can perform white balance processing on the infrared image to obtain a processed infrared image. The colors in this processed infrared image are not entirely accurate, but the human eye can distinguish the color differences. Based on the chromaticity components of the first pixel in the processed infrared image, the electronic device can execute steps S25, S26, and S241 to obtain a target image, further improving the quality of the final image.

[0072] The method for white balance processing is shown below.

[0073]

[0074]

[0075] R' ij =R ij *R gain ;

[0076] B' ij =B ij *B gain ;

[0077] G' ij =G ij .

[0078] Among them, G ij R represents the green component of pixel (i, j) in the original infrared image. ij B represents the red component of pixel (i, j) in the original infrared image. ij G' represents the blue component of pixel (i, j) in the original infrared image. ij R' represents the green component of pixel (i, j) in the infrared image after white balance processing. ij B' represents the red component of pixel (i, j) in the infrared image after white balance processing. ij R represents the blue component of pixel (i, j) in the infrared image after white balance processing. gain B represents the gain of the red component. gain This represents the gain of the blue component.

[0079] Step S26: Based on the second weight value of each fourth pixel corresponding to each third pixel, perform a weighted average of the chromaticity components of each fourth pixel corresponding to each third pixel to obtain the second chromaticity component of each third pixel after parallax removal.

[0080] In this embodiment of the application, for each third pixel, the electronic device performs a weighted average of the chromaticity components of each fourth pixel corresponding to the third pixel according to the second weight value of each fourth pixel corresponding to the third pixel, to obtain the chromaticity component of the third pixel after parallax removal, that is, the second chromaticity component.

[0081] For example, the electronic device converts infrared and color images to the YUV space respectively, with a first preset length of r. The electronic device obtains the chromaticity component U of pixel (i, j) in the color image. c1 (i, j) and V c1 (i, j), and obtain the chromaticity component of pixel (i, j) in the infrared image as U. m1 (i, j) and V m1 (i, j). The coordinates of pixel (i, j) in the color image are matched with those of pixel (i, j) in the infrared image. Pixel (i, j) represents the pixel at coordinates (i, j).

[0082] In an infrared image, the chromaticity component of each pixel within a radius r centered at pixel (i, j) is U. m1 (m, n). Electronic device calculation U m1 (m, n) and U m1 The chromaticity difference of (i, j) is used to find the pre-defined correspondence between chromaticity difference and weight value, and the weight value of each pixel (m, n) in the infrared image is determined, which is then used as the weight value w' of the corresponding pixel (m, n) in the color image. m,n .

[0083] The electronic device obtains the weight values ​​w' of the corresponding pixels (m, n) in the color image. m,n Then, the second chromaticity component U of the chromaticity component U of pixel (i, j) in the color image can be determined using the following formula. c3 :

[0084]

[0085] In formula (4), U c3 (i, j) represents the second chromaticity component U of pixel (i, j). c3 w' m,n U represents the weight value of pixel (m, n) in a color image. c1(m，n) This represents the chromaticity component of a pixel (m, n) in a color image.

[0086] Similarly, electronic devices can determine the second chromaticity component V of the chromaticity component V of pixel (i, j) in a color image. c3 Further explanation is not provided here.

[0087] The above U c3 (i, j) and V c3 (i, j) are collectively referred to as the second chromaticity component of pixel (i, j).

[0088] Step S241: Generate the target image based on the first chromaticity component and the second chromaticity component of each third pixel.

[0089] In this embodiment of the application, after obtaining the first chromaticity component and the second chromaticity component of each third pixel, the electronic device generates a target image based on the first chromaticity component and the second chromaticity component of each third pixel.

[0090] For example, an electronic device can use the first chromaticity component of each third pixel as the chromaticity component of the pixel at the corresponding coordinate in the target image to obtain the target image.

[0091] Electronic devices can also use the second chromaticity component of each third pixel as the chromaticity component of the pixel at the corresponding coordinate in the target image to obtain the target image.

[0092] The electronic device can also acquire the illuminance of the image and generate the target image based on the illuminance and the first and second chromaticity components of each third pixel. This will be explained in detail below and will not be elaborated upon here.

[0093] The above illuminance can be measured using exposure gain. Exposure gain can be estimated from the exposure information of a color image. The above illuminance can also be measured using other parameters, which are not limited here.

[0094] In this embodiment, the infrared camera can simultaneously sense both natural light and infrared light. Therefore, in environments where natural light is present (i.e., high-illuminance scenes), infrared light is relatively weak, and the infrared image is close to the color image. The infrared image has richer colors, lower distortion, and color differences that are perceptible to the human eye. By filtering the chromaticity components of pixels in the color image using the chromaticity components of pixels in the infrared image, the chromaticity distribution of pixels in the obtained target image can be made closer to the chromaticity distribution of pixels in the infrared image. In other words, the chromaticity of pixels in the target image is made closer to the true chromaticity distribution, minimizing the parallax effect between the infrared and color images and improving image quality in high-illuminance scenes.

[0095] The first chromaticity component is suitable for image registration in low-light scenes, as described above. Figure 2 The second chromaticity component is described in part as being suitable for image registration in high-light scenes, as mentioned above. Figure 3 Partial description. In the technical solution provided by the embodiments of this application, a target image is generated based on the first chromaticity component and the second chromaticity component of each third pixel, which can adapt to the needs of image registration in different lighting intensity scenes and improve the image quality in different lighting intensity scenes.

[0096] To meet the needs of image registration in scenes with different lighting intensities and to obtain high-quality registered images, embodiments of this application also provide an image registration method, such as... Figure 4 As shown, in this image registration method, step S241 can be further refined into steps S241a, S241b and S241c.

[0097] In step S241a, if the illuminance is less than the first preset illuminance threshold, the first target image is generated from the first chromaticity component of each third pixel.

[0098] The aforementioned first preset illuminance threshold can be set according to actual needs.

[0099] In this embodiment, when the obtained illuminance is less than a first preset illuminance threshold, the electronic device can determine that the illuminance of the monitored area is low, the infrared image is close to a black and white image, the chromaticity of the infrared image is not guiding, while the luminance of the infrared image is relatively guiding. The electronic device uses the luminance component of the pixels in the infrared image to perform weighted filtering on the chromaticity component of the pixels in the color image, so as to eliminate the parallax effect between the infrared image and the color image as much as possible, and improve the image quality in low-illuminance scenes. Therefore, the electronic device generates a first target image from the first chromaticity component of each third pixel, that is, the first chromaticity component of each third pixel is used as the chromaticity component of the pixel at the corresponding coordinate in the target image, combined with the corresponding luminance component, to obtain the first target image.

[0100] In step S241b, if the illuminance is greater than the second preset illuminance threshold, a second target image is generated from the second chromaticity component of each third pixel.

[0101] The aforementioned second preset illuminance threshold can be set according to actual needs. The first preset illuminance threshold is less than the second preset illuminance threshold.

[0102] In this embodiment, when the obtained illuminance is greater than a second preset illuminance threshold, the electronic device can determine that the illuminance of the monitored area is high. At this time, the infrared light is relatively weak, the infrared image is close to the color image, the infrared image has richer colors and less distortion, the luminance component of the infrared image is not guiding, while the chrominance component of the infrared image has better guiding properties. The electronic device uses the chrominance component of the pixels in the infrared image to perform weighted filtering on the chrominance component of the pixels in the color image, so as to eliminate the parallax effect between the infrared image and the color image as much as possible, and improve the image quality in high-illuminance scenes. Therefore, the electronic device generates a second target image from the second chrominance component of each third pixel, that is, the second chrominance component of each third pixel is used as the chrominance of the pixel at the corresponding coordinate in the target image, combined with the corresponding luminance component, to obtain the second target image.

[0103] Step S241c: If the illuminance is greater than or equal to the first preset illuminance threshold and the illuminance is less than or equal to the second preset illuminance threshold, then based on the difference between the illuminance and the first preset illuminance threshold, and the difference between the illuminance and the second preset illuminance threshold, the first chromaticity component and the second chromaticity component of each third pixel are weighted and summed to obtain the third chromaticity component of each third pixel; and the third target image is generated from the third chromaticity component of each third pixel.

[0104] In an optional embodiment, for each third pixel, the third chromaticity component LL3 of the third pixel is determined using the following formula:

[0105] LL3=[(S-th1)*LL2+(th2-S)*LL1] / (th2-th1) (5)

[0106] In the above formula (5), LL1 represents the first chromaticity component of the third pixel, LL2 represents the second chromaticity component of the third pixel, th1 represents the first preset illuminance threshold, th2 represents the second preset illuminance threshold, and S represents the illuminance obtained by the electronic device.

[0107] LL3 can be divided into the third chromaticity component of chromaticity component U and the third chromaticity component of chromaticity component V. The third chromaticity component of chromaticity component U and the third chromaticity component of chromaticity component V can be calculated based on formula (5). The difference is that the calculation of the third chromaticity component of chromaticity component U uses the corresponding parameters of chromaticity component U, and the calculation of the third chromaticity component of chromaticity component V uses the corresponding parameters of chromaticity component V.

[0108] In this embodiment, illuminance can be measured by exposure gain. The greater the exposure gain, the smaller the illuminance; conversely, the smaller the exposure gain, the greater the illuminance.

[0109] In this case, the electronic device can convert exposure gain into illuminance, for example, by taking the reciprocal of the exposure gain of the color image as the illuminance. Then, the electronic device uses the converted illuminance and combines it with the above formula (5) to obtain the third chromaticity component of each third pixel and generate the third target image.

[0110] Electronic devices can also directly utilize exposure gain to generate a third target image.

[0111] For example, an electronic device may have a first preset gain threshold and a second preset gain threshold. The first preset gain threshold is less than the second preset gain threshold.

[0112] When the exposure gain of the color image is greater than the second preset gain threshold, the first target image is generated from the first chromaticity component of each third pixel.

[0113] When the exposure gain of the color image is less than the first preset gain threshold, the second target image is generated from the second chromaticity component of each third pixel.

[0114] When the exposure gain of the color image is greater than or equal to the first preset gain threshold and the exposure gain of the color image is less than or equal to the second preset gain threshold, the first chromaticity component and the second chromaticity component of each third pixel are weighted and summed based on the difference between the exposure gain and the first preset gain threshold, and the difference between the exposure gain and the second preset gain threshold, to obtain the third chromaticity component of each third pixel; and the third target image is generated from the third chromaticity component of each third pixel.

[0115] For example, for each third pixel, the electronic device can determine the third chromaticity component U of the third pixel's chromaticity component U using the following formula. c4 :

[0116] U c4 =[(db-th'1)*U c2 +(th'2-db)*U c3 ] / (th'2-th'1) (6)

[0117] In the above formula (6), U c2 This represents the first chromaticity component U of the second pixel. c3 The second chromaticity component of the chromaticity component U of the third pixel is represented by th'1, the first preset gain threshold is represented by th'2, the second preset gain threshold is represented by db, and the exposure gain of the color image acquired by the electronic device is represented by db.

[0118] Similarly, electronic devices can determine the third chromaticity component V of the chromaticity component V of a pixel in a color image. c4 Further explanation is not provided here.

[0119] The above U c4 and V c4 These are collectively referred to as the third chromaticity component of a pixel.

[0120] In one embodiment of this application, an image registration method is also provided, such as... Figure 5 As shown, the image registration method may also include the following steps.

[0121] Step S27: Based on the luminance component of each pixel in the infrared image and the color image, determine the initial offset of the color image, where the sum of the differences between the luminance components of all pixels in the color image and the luminance components of all pixels in the infrared image with the initial offset is minimized.

[0122] Step S28: Update the chromaticity component of each pixel in the color image to the chromaticity component of the pixel whose coordinates are offset by the initial offset.

[0123] In this embodiment, the initial offset can be roughly estimated based on the typical scene in which the camera is used. The initial offset can also be calculated using the SAD (Sum of Absolute Differences) algorithm.

[0124] For example, the estimated offset is (k, p), where k represents the offset in the x-direction and p represents the offset in the y-direction.

[0125]

[0126] In formula (7), SAD(k, p) represents the sum of the differences between the luminance component of each third pixel and the luminance component of the first pixel with the initial coordinate offset, CHOR i,j MONO represents the luminance component of pixel (i, j) in a color image. i+k,j+p This represents the luminance component of pixel (i+k, j+p) in an infrared image. abs() represents the absolute value. M represents the total number of pixels in the x-direction and N represents the total number of pixels in the y-direction.

[0127] The electronic device uses the above formula (7) to determine the estimated offset (k, p) when SAD(k, p) is minimized, and uses the minimum estimated offset (k, p) as the initial offset.

[0128] The electronic device uses an initial offset to perform coordinate offset on the chromaticity component of the third pixel in the color image, as follows:

[0129] U c1 (i, j) = U c (i+k,j+p) (8)

[0130] V c1 (i, j) = V c (i+k,j+p) (9)

[0131] In formulas (8) and (9) above, U c1 (i, j) represents the chromaticity components U and V of the third pixel (i, j) after coordinate offset. c1 (i, j) represents the chromaticity components V and U of the third pixel (i, j) after coordinate offset. c (i+k, j+p) represents the chromaticity components U and V of the third pixel (i+k, j+p) before the coordinate offset. c (i+k, j+p) represents the chromaticity component V of the third pixel (i+k, j+p) before the coordinate offset.

[0132] Subsequently, the electronic device uses the chromaticity components after coordinate offset to determine the first chromaticity component and the second chromaticity component, thereby generating the target image.

[0133] In this embodiment, the electronic device sets the initial offset to a global offset, meaning there is only one offset for the entire image. This initial offset offers the strongest compatibility across the entire image, minimizing the parallax between the color and infrared images. By using this initial offset to shift the chromaticity components of pixels, the electronic device effectively reduces the difficulty of subsequent parallax reduction.

[0134] The following is combined Figure 6 The image registration method shown herein will be described in detail according to the embodiments of this application. This image registration method may include the following steps.

[0135] Step S61: The dual cameras activate infrared fill light to capture infrared and color images.

[0136] In step S62, the electronic device performs global white balance processing and smoothing processing on the infrared image to obtain the processed infrared image.

[0137] For details, please refer to the descriptions in steps S21 and S25.

[0138] In step S63, the electronic device converts the processed infrared image and color image into color spaces where color and chromaticity are relatively independent.

[0139] Step S64: The electronic device matches the processed infrared image and color image to obtain the initial offset.

[0140] For details, please refer to the descriptions in steps S27 and S28.

[0141] In step S65, the electronic device uses the luminance component of the processed infrared image to guide the color image to perform chroma filtering, and uses the chroma component of the processed infrared image to guide the color image to perform chroma filtering.

[0142] For details, please refer to the descriptions in steps S22, S23, S25, and S26.

[0143] In step S66, the electronic device obtains two chroma-filtered images using the methods described in step S65, thus obtaining the target image. See steps S241a-c for details.

[0144] The technical solution provided in this application can adapt to the image requirements in scenes with different lighting intensities and obtain images of higher quality. This image is a color image. Using this color image, electronic devices can perform image fusion to obtain a higher quality fused image, facilitating subsequent processing.

[0145] Corresponding to the image registration method described above, this application also provides an image registration method, such as... Figure 7 As shown, it includes the following steps:

[0146] Step S71: Acquire a color image and an infrared image, wherein the color image and the infrared image cover the same monitoring area;

[0147] Step S72, generate a first offset, wherein the first offset is the offset that minimizes the sum of the brightness differences between all pixels in the infrared image and all pixels in the color image after the offset.

[0148] Step S73: Based on the first offset, offset processing is performed on all first pixels of the color image to generate the offset color image;

[0149] Step S74: Based on the luminance component of each pixel in the infrared image, look up a preset table to determine the first weight value of each pixel in the infrared image. The preset table stores the correspondence between luminance difference and weight.

[0150] Step S75: When the illuminance in the monitored area is lower than the first threshold, the first weight value of each pixel in the infrared image is used to perform weighted filtering on the chromaticity components of each pixel in the offset color image to obtain the first target image.

[0151] In an optional embodiment, step S74 above may specifically be:

[0152] For each pixel in the infrared image, determine the difference between the brightness component of that pixel and the brightness components of each neighboring pixel, wherein the distance between the neighboring pixel and the pixel is less than or equal to a first preset length; look up a preset table to determine the first weight value corresponding to the difference in brightness components of each neighboring pixel.

[0153] In an optional embodiment, step S75 above may specifically be:

[0154] For each pixel in the offset color image, the chromaticity components of the neighboring pixels of the pixel are weighted and averaged using the first weight value of the neighboring pixels of the pixel to obtain the first chromaticity component of the pixel after parallax removal. The first weight value of the neighboring pixels of the pixel is the same as that of the neighboring pixels of the position-matching pixel in the infrared image.

[0155] The first target image is generated based on the first chromaticity component after parallax removal from each pixel of the offset color image.

[0156] In an optional embodiment, the preset table also stores the correspondence between chromaticity differences and weights.

[0157] In this case, the above image registration method may also include:

[0158] Based on the chromaticity components of each pixel in the infrared image, a preset table is consulted to determine the second weight value of each pixel in the infrared image;

[0159] When the illuminance in the monitored area is greater than the second threshold, the second weight value of each pixel in the infrared image is used to perform weighted filtering on the chromaticity components of each pixel in the offset color image to obtain the second target image.

[0160] In an optional embodiment, the step of determining the second weight value of each pixel of the infrared image by looking up a preset table based on the chromaticity components of each pixel of the infrared image may include:

[0161] For each pixel in the infrared image, determine the difference between the chromaticity component of that pixel and the chromaticity component of each neighboring pixel, wherein the distance between the neighboring pixel and the pixel is less than or equal to a second preset length; look up a preset table to determine the second weight value corresponding to the difference of the chromaticity components of each neighboring pixel.

[0162] In an optional embodiment, the step of using the second weight value of each pixel in the infrared image to perform weighted filtering on the chromaticity components of each pixel in the offset color image to obtain the second target image may include:

[0163] For each pixel in the offset color image, the chromaticity components of the neighboring pixels of the pixel are weighted and averaged using the second weight value of the neighboring pixels of the pixel to obtain the second chromaticity component of the pixel after disparity removal. The second weight value of the neighboring pixels of the pixel is the same as that of the neighboring pixels of the position-matching pixel in the infrared image.

[0164] A second target image is generated based on the second chromaticity component after parallax removal from each pixel of the offset color image.

[0165] In an optional embodiment, the image registration method described above may further include:

[0166] When the illuminance in the monitored area is between the first threshold and the second threshold, the chromaticity components of each pixel in the offset color image are weighted and filtered using the first weight value and the second weight value of each pixel in the infrared image to obtain the third target image.

[0167] In an optional embodiment, the step of using the first and second weight values ​​of each pixel in the infrared image to perform weighted filtering on the chromaticity components of each pixel in the offset color image to obtain the third target image may include:

[0168] For each pixel in the offset color image, the chromaticity components of the neighboring pixels of the pixel are weighted and averaged using the first weight value of the neighboring pixels of the pixel to obtain the first chromaticity component of the pixel after disparity removal. Then, the chromaticity components of the neighboring pixels of the pixel are weighted and averaged using the second weight value of the neighboring pixels of the pixel to obtain the second chromaticity component of the pixel after disparity removal. The first weight value and the second weight value of the neighboring pixels of the pixel are the same as those of the neighboring pixels of the pixel whose position is matched in the infrared image.

[0169] Based on the difference between illuminance and the first threshold, and the difference between illuminance and the second threshold, the first chromaticity component and the second chromaticity component of each pixel of the offset color image are weighted and summed to obtain the third chromaticity component of each pixel of the offset color image after parallax removal.

[0170] A third target image is generated based on the third chromaticity component after parallax removal from each pixel of the offset color image.

[0171] The technical solution provided in this application embodiment utilizes the luminance component of pixels in an infrared image.

[0172] The weights of pixels in the color image are determined, and then the chromaticity components of the pixels in the color image are weighted and filtered using these weights to obtain a color image registered with the infrared image, i.e., the target image. This is because the probability of chromaticity change is very high when brightness changes, and vice versa. Therefore, the brightness distribution can reflect the chromaticity distribution to a certain extent. Furthermore, in low-light scenes, infrared images have low noise and exhibit brightness variations. Therefore, the technical solution provided in this application, which uses the luminance components of pixels in the infrared image to weight and filter the chromaticity components of pixels in the color image, can make the chromaticity distribution of pixels in the obtained target image closer to the luminance distribution of pixels in the infrared image. In other words, it makes the chromaticity of pixels in the target image closer to the true chromaticity distribution, minimizing the parallax between the infrared and color images and improving image quality in low-light scenes.

[0173] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0174] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0175] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for... Figure 7 As for the image registration method embodiment shown, since it is basically similar to Figure 2-6 The image registration method embodiment shown is described in a relatively simple manner; for relevant details, please refer to the description of the method embodiment.

[0176] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image registration method, characterized in that, The method includes: Acquire infrared and color images of the same monitored area; For each first pixel in the infrared image, based on the difference between the brightness component of the first pixel and the brightness component of each corresponding second pixel, and the preset correspondence between the brightness difference and the weight value, a first weight value is determined for each fourth pixel corresponding to the third pixel in the color image. The second pixel is a pixel in the infrared image whose distance from the first pixel is less than or equal to a first preset length, the third pixel is a pixel whose coordinates match the coordinates of the first pixel, and the fourth pixel is a pixel whose coordinates match the coordinates of the second pixel. Based on the first weight value of each fourth pixel corresponding to each third pixel, the chromaticity components of each fourth pixel corresponding to each third pixel are weighted and averaged to obtain the first chromaticity component of each third pixel after parallax removal. The target image is generated based on the first chromaticity component of each third pixel.

2. The method according to claim 1, characterized in that, The method further includes: For each first pixel in the infrared image, the second weight value of each fourth pixel corresponding to the third pixel in the color image is determined based on the difference between the chromaticity component of the first pixel and the chromaticity component of the corresponding second pixel, and the preset correspondence between the chromaticity difference and the weight. Based on the second weight value of each fourth pixel corresponding to each third pixel, the chromaticity components of each fourth pixel corresponding to each third pixel are weighted and averaged to obtain the second chromaticity component of each third pixel after parallax removal. The step of generating the target image based on the first chromaticity component of each third pixel includes: The target image is generated based on the first chromaticity component and the second chromaticity component of each third pixel.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the illuminance of the monitored area; The step of generating a target image based on the first chromaticity component and the second chromaticity component of each third pixel includes: A target image is generated based on the illuminance, the first chromaticity component and the second chromaticity component of each third pixel.

4. The method according to claim 3, characterized in that, The step of generating a target image based on the illuminance, the first chromaticity component of each third pixel, and the second chromaticity component includes: If the illuminance is less than the first preset illuminance threshold, then the first target image is generated from the first chromaticity component of each third pixel; If the illuminance is greater than the second preset illuminance threshold, then the second target image is generated from the second chromaticity component of each third pixel; If the illuminance is greater than or equal to the first preset illuminance threshold and the illuminance is less than or equal to the second preset illuminance threshold, then based on the difference between the illuminance and the first preset illuminance threshold, and the difference between the illuminance and the second preset illuminance threshold, the first chromaticity component and the second chromaticity component of each third pixel are weighted and summed to obtain the third chromaticity component of each third pixel; and a third target image is generated from the third chromaticity component of each third pixel.

5. The method according to claim 4, characterized in that, The first chromaticity component and the second chromaticity component of each third pixel are weighted and summed based on the difference between the illuminance and the first preset illuminance threshold, and the difference between the illuminance and the second preset illuminance threshold, to obtain the third chromaticity component of each third pixel. The step of generating a third target image from the third chromaticity component of each third pixel includes: For each third pixel, the third chromaticity component LL3 of that third pixel is determined using the following formula: LL3=[(S-th1)*LL2+(th2-S)*LL1] / (th2-th1); Wherein, LL1 represents the first chromaticity component of the third pixel, LL2 represents the second chromaticity component of the third pixel, th1 represents the first preset illuminance threshold, th2 represents the second preset illuminance threshold, and S represents the illuminance.

6. The method according to any one of claims 1-5, characterized in that, After acquiring the infrared image and the color image, and before determining the first weight value of each fourth pixel corresponding to the third pixel in the color image, the method further includes: Based on the luminance component of each pixel in the infrared image and the color image, an initial offset of the color image is determined, such that the sum of the differences between the luminance components of all pixels in the color image and the coordinates of all pixels in the infrared image offset by the initial offset is minimized. The chromaticity component of each pixel in the color image is updated to the chromaticity component of the pixel whose coordinates are offset by the initial offset.

7. An image registration method, characterized in that, The method includes: Acquire a color image and an infrared image, wherein the color image and the infrared image cover the same monitoring area; Generate a first offset, wherein the first offset is the offset that minimizes the sum of the brightness differences between all pixels in the infrared image after the offset and all pixels in the color image. Based on the first offset, all first pixels of the color image are offset to generate the offset color image; Based on the luminance component of each pixel in the infrared image, a preset table is consulted to determine the first weight value of each pixel in the infrared image, wherein the preset table stores the correspondence between luminance difference and weight. When the illuminance in the monitored area is lower than a first threshold, the chromaticity components of each pixel in the offset color image are weighted and filtered using the first weight value of each pixel in the infrared image to obtain the first target image.

8. The method according to claim 7, characterized in that, The step of determining a first weight value for each pixel of the infrared image by looking up a preset table based on the luminance component of each pixel includes: For each pixel in the infrared image, the difference between the brightness component of the pixel and the brightness component of each neighboring pixel is determined, wherein the distance between the neighboring pixel and the pixel is less than or equal to a first preset length; a preset table is looked up to determine the first weight value corresponding to the difference in brightness components of each neighboring pixel.

9. The method according to claim 7 or 8, characterized in that, The step of using the first weight value of each pixel in the infrared image to perform weighted filtering on the chromaticity components of each pixel in the offset color image to obtain the first target image includes: For each pixel in the offset color image, the chromaticity components of the neighboring pixels of the pixel are weighted and averaged using the first weight value of the neighboring pixels of the pixel to obtain the first chromaticity component of the pixel after parallax removal. The first weight value of the neighboring pixels of the pixel is the same as that of the neighboring pixels of the position-matching pixel in the infrared image. A first target image is generated based on the first chromaticity component after parallax removal from each pixel of the offset color image.

10. The method according to claim 7 or 8, characterized in that, The preset table also stores the correspondence between chromaticity differences and weights; the method further includes: Based on the chromaticity components of each pixel in the infrared image, the preset table is consulted to determine the second weight value of each pixel in the infrared image; When the illuminance in the monitored area is greater than the second threshold, the chromaticity components of each pixel in the offset color image are weighted and filtered using the second weight value of each pixel in the infrared image to obtain the second target image.

Images