Night image shooting method and device based on infrared assistance

The camera position and infrared light source position are determined through the ambient brightness data, visible light and infrared images are generated and fused, and the imaging parameters are adjusted, which solves the problem of low sharpness at night image and achieves higher quality night image shooting.

CN119052606AActive Publication Date: 2024-11-29SHENZHEN LINJING INTELLIGENT CO LTD

Patent Information

Application Number
CN202411010985.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-29
Estimated Expiration
2044-07-26

AI Technical Summary

Technical Problem

Existing night image shooting technology has low image clarity in low light conditions, resulting in the loss of important details.

Method used

By obtaining the environmental brightness data of the target area, determining the imaging position and infrared light source position, generating visible light images and infrared images, and using exposure weight algorithms and image sharpness algorithms to fusion images, adjusting imaging parameters to improve image clarity.

Benefits of technology

Improves the clarity and quality of night images, enhances the visualization and recognition capabilities of target areas, and ensures more accurate and clearer images under night conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052606A_ABST
    Figure CN119052606A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of infrared assistance, and discloses a night image shooting method and device based on infrared assistance, and the method comprises the steps: extracting the environment brightness data of a target region shot at night, and determining the shooting position and the infrared light source position of the target region according to the environment brightness data; generating a visible light image and an infrared image of the target area according to the initial camera shooting parameters and the camera shooting position of the target area; calculating a first image weight of the visible light image, calculating a second image weight of the infrared image, and performing image fusion on the visible light image and the infrared image according to the first image weight and the second image weight; calculating the definition of the night fusion image, and generating a shooting parameter control factor of the target area according to the definition; and adjusting the initial shooting parameter through the shooting parameter control factor and the parameter circulation condition, and carrying out position fusion shooting on the target area by using the optimal shooting parameter to obtain a night shooting image. According to the invention, the definition of night image shooting can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of infrared assistance technology, and in particular to a nighttime image shooting method and device based on infrared assistance. Background Art

[0002] With the development of infrared-assisted technology and low-light camera technology, night shooting is becoming more and more common and important in various applications. The advancement of these technologies makes it possible to obtain high-quality images and videos at night or in low-light conditions, thus playing an important role in security monitoring, search and rescue, military reconnaissance and other fields. However, in order to improve the clarity of images taken at night, it is necessary to optimize the parameters of night shooting to achieve high-definition night shooting.

[0003] The existing night image capture technology uses the thermal energy information in the infrared spectrum to enhance the capture capability of night images. It can work in complete darkness or extremely low light conditions and is not limited by insufficient visible light. In practical applications, only collecting infrared images to determine the images taken at night may cause the images taken at night to be too monotonous and lose important details in the images, resulting in low clarity when taking night images. Summary of the invention

[0004] The present invention provides a method and device for nighttime image shooting based on infrared assistance, the main purpose of which is to solve the problem of low definition when shooting nighttime images.

[0005] To achieve the above object, the present invention provides a night image shooting method based on infrared assistance, comprising:

[0006] Acquire a target area for night photography, extract ambient brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the ambient brightness data;

[0007] Extracting initial imaging parameters of the target area, generating a visible light image of the target area according to the initial imaging parameters and the imaging position, and generating an infrared image of the target area according to the initial imaging parameters and the infrared light source position;

[0008] Calculating a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fused image;

[0009] The definition of the night fusion image is calculated using a preset image definition algorithm, including: extracting a highlight image region according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of ​​the highlight image region and identifying the center position of the highlight image region; determining a first clear region according to the center position and a preset region radius; filtering invalid pixels in the first clear region according to a preset second brightness threshold to obtain a second clear region; performing a difference operation between the first clear region and the second clear region to obtain a target clear region; and calculating the target definition of the target clear region using the following preset image definition algorithm:

[0010]

[0011] Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point;

[0012] Determine the definition of the nighttime fused image using the target definition, and generate a camera parameter control factor of the target area according to the definition;

[0013] The initial imaging parameters are adjusted by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters, and the target area is photographed by position fusion using the optimal imaging parameters to obtain a nighttime photographic image.

[0014] Optionally, determining the camera position and the infrared light source position of the target area according to the ambient brightness data includes:

[0015] Identifying a first area center position of the target area, and determining a camera position of the target area according to a preset camera distance and the first area center position;

[0016] Identifying a dark area of ​​the target area according to the ambient brightness data, and identifying a second area center position of the dark area;

[0017] Determine the infrared distance between the infrared light source and the center position of the second area according to the preset infrared light source properties, and determine the infrared out-of-area point according to the infrared distance and the camera distance;

[0018] The infrared light source position of the target area is determined as the infrared light source position of the target area.

[0019] Optionally, extracting initial imaging parameters of the target area includes:

[0020] Determining the camera lens angle of view according to the camera position of the target area;

[0021] Extracting exposure parameters, focal length parameters and sensitivity parameters of the camera corresponding to the target area;

[0022] Extracting the irradiation angle and illumination intensity of the infrared light source corresponding to the target area;

[0023] The camera lens viewing angle, the exposure parameter, the focal length parameter, the sensitivity parameter, the illumination angle and the illumination intensity are determined as initial camera parameters of the target area.

[0024] Optionally, generating the visible light image of the target area according to the initial imaging parameters and the imaging position includes:

[0025] Focusing the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters;

[0026] The focused light is converted into an electronic signal through the photosensitive unit in the photosensitive element;

[0027] Performing multiple signal enhancement processing on the electronic signal, and encoding the electronic signal after the signal enhancement processing into a digital image format;

[0028] A visible light image of the target area is generated according to the digital image format.

[0029] Optionally, calculating the first image weight of the visible light image by using a preset exposure weight algorithm includes:

[0030] Extracting camera parameters corresponding to the visible light image;

[0031] Counting the exposure of the visible light image, and determining an image enhancement factor of the visible light image according to the exposure;

[0032] The first image weight of the visible light image is calculated according to the camera parameters, the exposure level and the image enhancement factor by using the following preset exposure weight algorithm:

[0033]

[0034] Wherein, ω is the first image weight, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, K i is the exposure degree corresponding to the visible light image i.

[0035] Optionally, fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fused image includes:

[0036] The visible light image is subjected to a convolution kernel operation through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and the infrared image is subjected to a convolution kernel operation to obtain a Gaussian pyramid infrared image;

[0037] Performing a difference operation between two adjacent layers of the Gaussian pyramid visible light image to obtain a Gaussian visible light difference image, and performing a difference operation between two adjacent layers of the Gaussian pyramid infrared image to obtain a Gaussian infrared difference image;

[0038] Counting the visible light feature points of the Gaussian visible light difference image, and counting the infrared feature points of the Gaussian infrared difference image;

[0039] Registering the visible light feature point with the infrared feature point to obtain a registered feature point;

[0040] Image fusion is performed according to the first image weight, the second image weight and the registration feature points to obtain a night fusion image, wherein the image fusion calculation formula is:

[0041] R=ω1×I(x,y)+ω2×g(I(f(x,y)))

[0042] Among them, R is the grayscale value of the night fusion image, ω1 is the weight of the first image, ω2 is the weight of the second image, I(x, y) is the grayscale value of the registration feature point at the position (x, y), f(x, y) is the coordinate transformation of the position (x, y) in the two-dimensional space of the image, and g is the radiation transformation function.

[0043] Optionally, generating the imaging parameter control factor of the target area according to the clarity includes:

[0044] When the clarity is less than a preset clarity threshold, determining the camera parameter control factor of the target area as a first factor;

[0045] When the clarity is greater than or equal to a preset clarity threshold, the camera parameter control factor of the target area is determined as a second factor.

[0046] Optionally, the adjusting the initial imaging parameters by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters includes:

[0047] When the camera parameter control factor is the first factor, adjusting the initial camera parameter according to a preset camera parameter adjustment window;

[0048] Calculating the clarity of the nighttime fused image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition;

[0049] When the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition, the adjusted initial imaging parameters are used as optimal imaging parameters.

[0050] When the imaging parameter control factor is the second factor, the initial imaging parameter is determined as the optimal imaging parameter.

[0051] Optionally, the using the optimal camera parameters to perform position fusion shooting on the target area to obtain a night shot image includes:

[0052] Generating an optimal visible light image at the imaging position using the optimal imaging parameters;

[0053] Generating an optimal infrared image using the optimal imaging parameters at the position of the infrared light source;

[0054] The best visible light image and the best infrared image are fused to obtain a nighttime image.

[0055] In order to solve the above problems, the present invention further provides a night image shooting device based on infrared assistance, the device comprising:

[0056] A position determination module, used to obtain a target area for night photography, extract ambient brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the ambient brightness data;

[0057] An image generation module, used to extract initial imaging parameters of the target area, generate a visible light image of the target area according to the initial imaging parameters and the imaging position, and generate an infrared image of the target area according to the initial imaging parameters and the infrared light source position;

[0058] an image fusion module, configured to calculate a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fuse the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image;

[0059] A camera parameter control factor generation module, used to calculate the clarity of the night fusion image using a preset image clarity algorithm, and generate a camera parameter control factor of the target area according to the clarity;

[0060] The night-time image generation module is used to adjust the initial image parameters through the image parameter control factor and the preset parameter cycle condition to obtain the optimal image parameters, and use the optimal image parameters to perform position fusion shooting on the target area to obtain the night-time image.

[0061] The embodiment of the present invention can accurately determine the camera position and the infrared light source position by extracting the ambient brightness data of the target area, ensuring that the camera can capture the image of the target area at the best angle and distance in the night environment; using the initial camera parameters, respectively generate visible light images and infrared images of the target area. This helps to obtain detailed information under different spectra, the visible light image provides visual information, and the infrared image reveals the heat distribution and other hidden details; through the exposure weight algorithm and the image clarity algorithm, the visible light image and the infrared image are fused, and the advantages of the two spectral images are utilized to produce a richer and clearer night fusion image, which helps to improve the visualization and recognition ability of the target area; based on the generated night fusion image, by adjusting the initial camera parameters and the preset parameter cycle conditions, the optimal camera parameters are calculated, and the image of the target area can be captured under the most optimized conditions, thereby improving the image quality and clarity; the optimal camera parameters are used to perform position fusion shooting to obtain the final night shooting image. The integration and optimization of these steps enable more accurate and clearer images to be obtained under night conditions. Therefore, the infrared-assisted night image shooting method, device, electronic device and computer-readable storage medium proposed by the present invention can solve the problem of low clarity when shooting night images. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of a process of an infrared-assisted nighttime image capturing method provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of a process for extracting initial camera parameters provided by an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a process for calculating a first image weight according to an embodiment of the present invention;

[0065] Figure 4 This is a functional module diagram of an infrared-assisted nighttime image shooting device provided by an embodiment of the present invention.

[0066] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0068] The embodiment of the present application provides a night image shooting method based on infrared assistance. The execution subject of the infrared-assisted night image shooting method includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided in the embodiment of the present application. In other words, the infrared-assisted night image shooting method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0069] Reference Figure 1 FIG. 1 is a flow chart of a method for nighttime image shooting based on infrared assistance provided by an embodiment of the present invention. In this embodiment, the method for nighttime image shooting based on infrared assistance includes:

[0070] S1. Acquire a target area for night photography, extract ambient brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the ambient brightness data.

[0071] In an embodiment of the present invention, the target area refers to the area range where shooting is required. The target area for night shooting can be determined according to demand. When the brightness of the surrounding environment of the target area is detected, the ambient light level around the target area can be directly measured by an environmental sensor (such as a photometer or an illuminance meter), providing more direct and accurate ambient brightness data, wherein the ambient brightness data refers to data describing the light level of the surrounding environment, that is, the intensity of light received in a certain area, thereby reflecting the brightness level of the environment.

[0072] Furthermore, the optimal position of the camera is selected based on the illumination and photometric data of the environment to ensure that objects or scenes in the target area can be clearly seen. In addition, in order to provide sufficient infrared lighting in low-light or completely dark environments so that the camera can capture clear infrared images without being disturbed by ambient light conditions, it is also necessary to determine the optimal placement of the infrared light source.

[0073] In the embodiment of the present invention, the camera position refers to the installation position of the camera, that is, where the camera is placed to best capture the target area; the infrared light source position refers to the installation position of the infrared lighting device, which emits infrared light to enhance the monitoring effect at night or in low light conditions. The position of the infrared light source is selected based on the dark areas in the environment to ensure that these areas can be adequately illuminated in the monitoring system so that the camera can capture clear images.

[0074] In the embodiment of the present invention, determining the camera position and the infrared light source position of the target area according to the ambient brightness data includes:

[0075] Identifying a first area center position of the target area, and determining a camera position of the target area according to a preset camera distance and the first area center position;

[0076] Identifying a dark area of ​​the target area according to the ambient brightness data, and identifying a second area center position of the dark area;

[0077] Determine the infrared distance between the infrared light source and the center position of the second area according to the preset infrared light source properties, and determine the infrared out-of-area point according to the infrared distance and the camera distance;

[0078] The infrared light source position of the target area is determined as the infrared light source position of the target area.

[0079] In detail, the target area is planarized and the planarized target area is gridded, so that the center position of the target area is selected based on the number of grids, that is, the center position of the first area, and then the position with a distance of the camera distance in the horizontal direction of the center position of the first area is determined as the camera position of the target area, wherein the camera distance is determined according to the characteristics of the camera, that is, the closest distance at which the camera can clearly capture the target area, and the camera distance is pre-set. For example, the focal length of the lens determines the field of view angle and imaging magnification of the camera, and a longer focal length usually means that the camera can capture clear images at a farther distance.

[0080] Specifically, according to the distribution of each brightness level in the target area displayed by the ambient brightness data, the dark area usually has a lower brightness level, and the area below the brightness threshold in the ambient brightness data is identified as the dark area, and the dark area is planarized, and the planarized dark area is gridded, so as to select the center position of the dark area based on the number of grids in the dark area, that is, the center position of the second area, and determine the propagation distance of the infrared light source according to the wavelength in the infrared light source properties, and then determine the infrared distance between the infrared light source and the center position of the second area as the propagation distance of the infrared light source, but the position of the infrared light source must be close not only to the dark area, but also to the camera position of the camera. Therefore, a circular area is obtained with the camera position as the center and the camera distance as the radius, and the point where the propagation distance straight line intersects the camera distance straight line on the boundary of the circular area is found, that is, the infrared outer point, and this infrared outer point is determined as the infrared light source position corresponding to the target area.

[0081] Furthermore, after determining the camera position and the infrared light source position of the target area, before photographing the target area, it is also necessary to determine the camera parameters for photographing. Therefore, the camera parameters need to be analyzed.

[0082] S2. Extracting initial imaging parameters of the target area, generating a visible light image of the target area according to the initial imaging parameters and the imaging position, and generating an infrared image of the target area according to the initial imaging parameters and the infrared light source position.

[0083] In the embodiment of the present invention, the initial camera parameters refer to basic parameters required when configuring the camera system, and are parameters set initially for photographing the target area.

[0084] In the embodiment of the present invention, refer to Figure 2 As shown, the extracting of initial imaging parameters of the target area includes:

[0085] S21, determining the camera lens angle of view according to the camera position of the target area;

[0086] S22, extracting exposure parameters, focal length parameters and sensitivity parameters of the camera corresponding to the target area;

[0087] S23, extracting the irradiation angle and illumination intensity of the infrared light source corresponding to the target area;

[0088] S24: Determine the camera lens angle of view, the exposure parameter, the focal length parameter, the sensitivity parameter, the illumination angle, and the illumination intensity as initial camera parameters of the target area.

[0089] In detail, the horizontal and vertical viewing angles of the camera position are determined as the viewing angles of the camera lens, or the field of view of the camera can be determined by actual measurement or by looking up the camera specification table; and the exposure parameters, focal length parameters and sensitivity parameters of the camera are determined according to the specifications of the camera, wherein the exposure parameters include shutter speed, aperture size and ISO sensitivity, and the parameters determine the exposure amount of the camera under specific lighting conditions, that is, the shutter speed controls the length of the exposure time, affecting the motion blur and brightness of the image; the aperture size determines the amount of light entering the camera, directly affecting the clarity and depth of field of the image; ISO sensitivity specifies the sensitivity of the camera sensor under given lighting conditions, and a high ISO is usually used in low-light conditions, but may increase image noise; the focal length parameter refers to the focal length of the lens, affecting the size and viewing angle of objects in the image; the sensitivity parameter, that is, the ISO value, indicates the photosensitivity performance of the camera under specific lighting conditions.

[0090] Specifically, the illumination angle and illumination intensity of the infrared light source corresponding to the target area are measured, wherein the illumination angle describes the angular range of the infrared light source illumination, usually referring to the diffusion angle of the light beam; the illumination intensity, i.e. the radiation intensity or luminous flux of the infrared light source, is used to measure the illumination effect of the light source on the target area, and then the extracted exposure parameters, focal length parameters, sensitivity parameters, illumination angle and illumination intensity are determined as the initial camera parameters of the target area. The camera parameters are very important when configuring and adjusting the camera system to ensure that the camera system can obtain clear and accurate images under various environmental conditions. In addition, when determining the camera parameters, it is usually necessary to refer to the technical manual or specification sheet of the camera and the technical manual of the infrared light source or the data provided by the supplier.

[0091] Furthermore, the target area can be photographed at the photographing position according to the configured initial photographing parameters to ensure that the content of the target area can be completely covered, thereby obtaining a visible light image corresponding to the target area.

[0092] In the embodiment of the present invention, the visible light image refers to an image captured by a camera within the visible light range, which can be seen by an ordinary camera or the naked eye. It is characterized by rich colors and details, and can reflect the real appearance of the object and the lighting conditions of the environment.

[0093] In the embodiment of the present invention, generating the visible light image of the target area according to the initial imaging parameters and the imaging position includes:

[0094] Focusing the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters;

[0095] The focused light is converted into an electronic signal through the photosensitive unit in the photosensitive element;

[0096] Performing multiple signal enhancement processing on the electronic signal, and encoding the electronic signal after the signal enhancement processing into a digital image format;

[0097] A visible light image of the target area is generated according to the digital image format.

[0098] In detail, at the corresponding camera position of the camera, the light is focused on the photosensitive element through the optical system of the camera based on the initial camera parameters. The lens usually includes components such as a convex lens and a reflector, and the light in the scene is focused on the photosensitive element through these components. The photosensitive element is composed of many tiny photosensitive units, each of which measures the intensity of light and converts it into an electrical signal. That is, when light shines on the surface of the photosensitive element, photons excite electrons inside the photosensitive unit, and the number of these electrons is proportional to the intensity of the light. Therefore, the number of electrons measured by each photosensitive unit reflects the intensity of light received by the unit, and the electrons generated inside the photosensitive unit are collected and converted into electrical signals.

[0099] Specifically, once the photosensitive element converts the optical image into an electronic signal, the electrical signal generated by the photosensitive element is transmitted to the electronic signal processing unit of the camera. The electronic signal is vertically transmitted to the edge of the sensor and then processed by a dedicated circuit. Each photosensitive unit on the photosensitive element corresponds to an image pixel, and the electronic signal is amplified, filtered, digitized, etc. to optimize image quality and reduce noise. Among them, the electrical signal output from the photosensitive element may be very weak initially, so it needs to be amplified to enhance the signal strength to ensure that the camera can capture sufficient electrical signals even in dim light conditions; the electronic signal may contain noise from uneven light, electronic noise or other interference sources, and the noise is removed by a filter to ensure that the final image is clear and accurate; the electronic signal will be converted into digital form for subsequent digital signal processing, including converting analog electrical signals into digital formats for storage, transmission and further processing, and then the processed electronic signals are encoded into digital image formats, such as JPEG or RAW formats. Therefore, the image data corresponding to the digital image format is a visible light image of the target area.

[0100] Furthermore, when the target area is monitored under low light or night conditions, the infrared light source can provide fill light so that the camera can capture scenes in the dark. The irradiation angle and lighting intensity of the infrared light source affect the working effect of the camera, that is, under the condition of infrared light source fill light, an infrared image of the target area is generated according to the initial camera parameters and the position of the infrared light source, wherein with the position of the infrared light source supplemented, the infrared image of the target area is generated by the camera according to the initial camera parameters and the camera position, wherein the infrared image refers to an image captured by infrared optical technology, which shows a heat energy distribution different from that of visible light. Infrared light is a type of electromagnetic wave with a wavelength longer than visible light, ranging from approximately 0.75 microns to 1000 microns. The infrared image provides an information perspective different from that of the visible light image.

[0101] Furthermore, in order to ensure the importance of distinguishing between visible light images and infrared images, it is necessary to determine the weights of visible light images and infrared images respectively to ensure that the first frame image obtained from the camera can provide a clear and bright image under different conditions, thereby facilitating subsequent image processing.

[0102] S3. Calculate the first image weight of the visible light image and the second image weight of the infrared image through a preset exposure weight algorithm, and fuse the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fusion image.

[0103] In an embodiment of the present invention, the first image weight is used to measure the importance of the visible light image, that is, the weight value calculated based on the camera parameters and the exposure of the image, which is used to reflect the relative importance of each pixel in the image or its contribution to the overall image quality.

[0104] In the embodiment of the present invention, refer to Figure 3 As shown, the first image weight of the visible light image is calculated by a preset exposure weight algorithm, including:

[0105] S31, extracting camera parameters corresponding to the visible light image;

[0106] S32, collecting statistics on the exposure of the visible light image, and determining an image enhancement factor of the visible light image according to the exposure;

[0107] S33, calculating a first image weight of the visible light image according to the camera parameters, the exposure level and the image enhancement factor by using the following preset exposure weight algorithm:

[0108]

[0109] Wherein, ω is the first image weight, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, K i is the exposure degree corresponding to the visible light image i.

[0110] In detail, the camera parameters are generally fixed parameters of most cameras, so a=-0.329, b=1.1258 are set, then the first fixed parameter a and the second fixed parameter b in the camera parameters are pre-configured, and then the exposure of the visible light image is counted, then the exposure refers to the measurement and analysis of the light brightness or brightness level of each pixel in the image, then firstly, the gray value of each pixel in the visible light image is calculated, and the gray value is usually between 0 and 255, indicating the brightness of the pixel, where 0 represents black and 255 represents white, and then the number of pixels at different gray levels is displayed according to the gray histogram, then the shape of the histogram is It reflects the overall exposure distribution of the image, such as whether there is overexposure or underexposure, and then calculates the average grayscale value and the variance of the grayscale value of the image, thereby reflecting the information of the overall brightness and contrast of the image, and obtaining the exposure of the visible light image. Then, the image enhancement factor is determined according to the exposure of the visible light image to adjust the visual effect of the image. When the exposure of each pixel in the image is relatively uniform, the image enhancement factor μ=0. When the exposure of each pixel in the image is overexposed or underexposed, the image enhancement factor μ=1. The weight of the visible light image is calculated based on the fixed camera parameters, exposure and image enhancement factor.

[0111] Specifically, when shooting, usually multiple visible light images are taken, then the exposure of each image is calculated respectively among the multiple visible light images, and then the image with the best exposure is selected to be determined as the visible light image, and then the weight of the visible light image is calculated according to the image with the best exposure, that is, the image with the best exposure is selected from the many images taken as the final visible light image, and the weight corresponding to the image is calculated. Then, in the exposure weight algorithm, the first image weight of the visible light image is determined based on the image enhancement factor by calculating the best image exposure. When the image enhancement factor μ=0, it means that the exposure of the visible light image is relatively uniform, so the weight of the visible light image at this time is 1; and when the image enhancement factor μ=1, it means that the visible light image is overexposed or underexposed, then the weight of the image corresponding to the best exposure among all visible light images is calculated, where P iis the grayscale value or color value of each pixel in the visible light image i. By using nonlinear functions and maximum functions, the effect of exposure on image pixel values ​​can be more accurately reflected, which in turn affects the final image weight calculation result. In order to improve calculation efficiency, when solving the optimal exposure, the nearest neighbor interpolation algorithm is used to reduce the input image size to 60×60.

[0112] Furthermore, the step of calculating the second image weight of the infrared image is consistent with the step of calculating the first image weight of the visible light image by means of a preset exposure weight algorithm, which will not be repeated here, wherein the second image weight is used to measure the importance of the infrared image, that is, the weight value of the infrared image calculated according to the camera parameters and the exposure of the image, which is used to reflect the relative importance of each pixel in the infrared image or its contribution to the overall infrared image quality, that is, to count the exposure of the infrared image, determine the image enhancement factor of the infrared image according to the exposure, and then calculate the second image weight of the infrared image according to the image enhancement factor, camera parameters and exposure.

[0113] Furthermore, according to the image weight of the visible light image and the image weight of the infrared image, image fusion can be performed according to the size of the weight, which can determine the contribution of the visible light image and the infrared image in the final fused image. If the weight of the first image is higher, the visible light image will have a greater impact on the final fused image. If the weight of the second image is higher, the infrared image will have a greater impact on the final fused image. For example, in night image fusion, if the goal is to better capture temperature distribution and target detection, the weight of the infrared image may be increased; and if more details and color information need to be retained, the weight of the visible light image may be increased.

[0114] In an embodiment of the present invention, the night fusion image refers to the synthesis or fusion of visible light images and infrared images to provide more comprehensive and clearer nighttime visual information, which is used to enhance the performance of visible light photography under nighttime conditions, and to supplement the deficiencies of visible light images under low-light conditions through thermal imaging enhancement of infrared images.

[0115] In the embodiment of the present invention, the step of fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image includes:

[0116] The visible light image is subjected to a convolution kernel operation through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and the infrared image is subjected to a convolution kernel operation to obtain a Gaussian pyramid infrared image;

[0117] Performing a difference operation between two adjacent layers of the Gaussian pyramid visible light image to obtain a Gaussian visible light difference image, and performing a difference operation between two adjacent layers of the Gaussian pyramid infrared image to obtain a Gaussian infrared difference image;

[0118] Counting the visible light feature points of the Gaussian visible light difference image, and counting the infrared feature points of the Gaussian infrared difference image;

[0119] Registering the visible light feature point with the infrared feature point to obtain a registered feature point;

[0120] Image fusion is performed according to the first image weight, the second image weight and the registration feature points to obtain a night fusion image, wherein the image fusion calculation formula is:

[0121] R=ω1×I(x,y)+ω2×g(I(f(x,y)))

[0122] Among them, R is the grayscale value of the night fusion image, ω1 is the weight of the first image, ω2 is the weight of the second image, I(x, y) is the grayscale value of the registration feature point at the position (x, y), f(x, y) is the coordinate transformation of the position (x, y) in the two-dimensional space of the image, and g is the radiation transformation function.

[0123] In detail, first, the original image is convolved with Gaussian filters of different scales. Gaussian filters are usually used to smooth images and reduce noise while retaining structural information in the image. By performing multiple downsampling and Gaussian smoothing operations on the original image, a series of images of different scales can be constructed, which is the Gaussian pyramid. Each layer of the Gaussian pyramid image is generated by downsampling and Gaussian filtering, so as to obtain the Gaussian pyramid image corresponding to the visible light image and the Gaussian pyramid image corresponding to the infrared image, and subtraction operations are performed between different layers of the Gaussian pyramid to obtain the Gaussian difference image of the visible light image and the Gaussian difference image of the infrared image. In the Gaussian difference image, for each pixel and its surrounding 8 adjacent pixels and all pixels in the same degree layer and adjacent two layers, if the pixel is greater than or less than the value of all compared pixels, the pixel is considered to be a local extreme point, and the local extreme point is determined as a feature point. The feature point represents the prominent features in the image, thereby counting the visible light feature points and infrared feature points.

[0124] Specifically, feature points are extracted from the processed image, and the feature points in the visible light image are registered with the feature points in the infrared image according to their positions in the image to ensure the spatial consistency of the visible light image and the infrared image. The mapping relationship between the visible light image and the infrared image is I(x,y)=g(I(f(x,y))), where f represents the coordinate transformation of the image in two-dimensional space, i.e., f(x,y)=(x′ ,y ′ ), describes how to convert from the coordinate system of one image to the coordinate system of another image for pixel-level registration and alignment, and g represents a one-dimensional grayscale or radiation transformation. In image processing, radiation transformation usually refers to the process of transforming or adjusting the radiation value of each pixel in the image (such as the thermal radiation value in an infrared image). The simplest radiation transformation is a linear transformation, which can be used to adjust the brightness and contrast of the image by scaling and offsetting the radiation value through the linear function g(x)=ax+b. The radiation transformation function is used to adjust or enhance the radiation value distribution of the second image so that it can be better integrated into the night fusion image to obtain a fused image. The night fusion image will combine the information of the visible light and infrared bands to provide richer and clearer night vision effects.

[0125] Furthermore, it is necessary to evaluate the overall clarity level of the fused nighttime images, so as to adjust the camera parameters based on the clarity level of the images to achieve the highest clarity nighttime images.

[0126] S4. Calculate the clarity of the nighttime fused image using a preset image clarity algorithm, and generate a camera parameter control factor of the target area according to the clarity.

[0127] In the embodiment of the present invention, the clarity refers to the clarity of edges and details of the captured image, and measures the clarity or resolution of the details in the captured image.

[0128] In the embodiment of the present invention, the method of calculating the clarity of the nighttime fused image by using a preset image clarity algorithm includes:

[0129] Extracting a highlight image area according to the brightness of the nighttime fused image and a preset first brightness threshold;

[0130] Calculating the area of ​​the highlighted image region and identifying the center position of the highlighted image region;

[0131] Determine a first clear area according to the central position and a preset area radius;

[0132] Filter invalid pixels in the first clear area according to a preset second brightness threshold to obtain a second clear area;

[0133] Performing a difference operation on the first clear area and the second clear area to obtain a target clear area;

[0134] The target clarity of the target clear area is calculated using the following preset image clarity algorithm:

[0135]

[0136] Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point;

[0137] The target definition is determined as the definition of the nighttime fused image.

[0138] In detail, the highlighted part of the night fusion image is screened by a custom brightness threshold, that is, the brightness of each pixel in the night fusion image is compared with the preset first brightness threshold. If the brightness of each pixel is greater than the preset first brightness threshold, the area corresponding to the pixel is determined as the highlighted area, and the area area of ​​the highlighted image area is counted, the total number of pixels in the highlighted image area, that is, the area area, is calculated, and the center position of the highlighted image area is identified. The center position can be determined by the center of gravity of the area, that is, the center of gravity refers to the weighted average position of all pixels in the area. The step of calculating the center of gravity is to multiply the position (l, d) of each pixel by its gray value or weight for a two-dimensional image, and then divide the weighted positions of all pixels by the total weight to obtain the center of gravity coordinates, thereby determining the center position of the highlighted image area, and according to the center position and the preset area radius, determine the first clear area, that is, the area within the preset radius with the center of the highlighted area as the center, and the preset area radius is an area of ​​5 pixels.

[0139] Specifically, a second threshold segmentation is performed in the first clear area to remove invalid pixels with too low brightness in the image, that is, the brightness of each pixel in the first clear area image is compared with the preset second brightness threshold. If the pixel of each pixel is less than the preset second brightness threshold, the area corresponding to this pixel is determined as an invalid pixel, and the image obtained by the second brightness threshold segmentation is subtracted from the image obtained by the first brightness threshold segmentation, and the part of the highlight area is removed to obtain a new area, that is, the target clear area. Because the area difference value of the highlight part is too small, it will affect the overall clarity evaluation of the image when calculating the image clarity. Therefore, after two brightness threshold segmentations, the most critical and effective part of the night fusion image is highlighted and processed separately. The part left by the two brightness threshold segmentations, that is, the pixel point between the two thresholds, is used to calculate the clarity of the night fusion image by using the absolute variance method, and the calculated value is used as another evaluation factor of the final clarity. Finally, all the obtained values ​​are image-averaged to obtain the target clarity of the target clear area, and the target clarity of the target clear area is determined as the clarity of the night fusion image.

[0140] Furthermore, the clarity information can be used to adjust the focal length of the lens or the focus of the camera to ensure that the captured target area always remains clear. Therefore, by determining the parameter control factor according to the clarity, the camera parameters can be optimized, unnecessary image processing and calculation can be reduced, and the system efficiency and response speed can be improved.

[0141] In the embodiment of the present invention, the camera parameter control factor refers to a factor for optimizing the camera control parameter, so that the camera parameter control factor is determined based on the definition, and then the camera control parameter is adjusted to obtain the best camera control parameter.

[0142] In the embodiment of the present invention, the step of generating the imaging parameter control factor of the target area according to the definition includes:

[0143] When the clarity is less than a preset clarity threshold, determining the camera parameter control factor of the target area as a first factor;

[0144] When the clarity is greater than or equal to a preset clarity threshold, the camera parameter control factor of the target area is determined as a second factor.

[0145] In detail, when the clarity of the target area is less than a preset clarity threshold, it means that the clarity of the target area has not reached the image clarity target, and the camera parameter control factor corresponding to the target area is determined as a first factor, wherein the first factor refers to a factor used to control the camera parameters so that they can be optimized, and the first factor is configured to a value of 1; and when the clarity is greater than or equal to the preset clarity threshold, it means that the clarity of the target area has reached the image clarity target, and the camera parameter control factor corresponding to the target area is determined as a second factor, wherein the second factor refers to a factor used to control the camera parameters that do not need to be optimized, and the second factor is configured to a value of zero.

[0146] Furthermore, clarity, as a feedback signal, can reflect the quality of the current captured image. By adjusting the camera parameters in real time, the system can quickly respond to changes in different scenes and lighting conditions to ensure that the best image clarity is always maintained. By adjusting the camera parameters, the quality of the captured image can be optimized. Using clarity as a control factor, it is possible to find and ensure the best combination of camera parameters under different conditions to achieve the best shooting level and obtain the best night-time images.

[0147] S5. Adjust the initial imaging parameters by using the imaging parameter control factor and the preset parameter cycle condition to obtain optimal imaging parameters, and use the optimal imaging parameters to perform position fusion shooting on the target area to obtain a night-time shooting image.

[0148] In an embodiment of the present invention, the optimal camera parameters refer to the parameters obtained by optimizing the basic parameters required for configuring the camera system, and the optimal parameters obtained by optimizing the parameters initially set for photographing the target area, including exposure parameters, focal length parameters, sensitivity parameters, illumination angle, and illumination intensity.

[0149] In the embodiment of the present invention, the adjusting the initial imaging parameters by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters includes:

[0150] When the camera parameter control factor is the first factor, adjusting the initial camera parameter according to a preset camera parameter adjustment window;

[0151] Calculating the clarity of the nighttime fused image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition;

[0152] When the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition, the adjusted initial imaging parameters are used as optimal imaging parameters.

[0153] When the imaging parameter control factor is the second factor, the initial imaging parameter is determined as the optimal imaging parameter.

[0154] In detail, when the camera parameter control factor is the first factor, it means that the camera parameters need to be optimized, and the initial camera parameters are adjusted according to the pre-customized camera parameter adjustment window. For example, if the camera parameter adjustment window is set to 2, when the camera parameters need to be adjusted, the parameter values ​​corresponding to the exposure parameters, focal length parameters, sensitivity parameters, illumination angles and illumination intensity are added by 2 in turn, and the shooting and image fusion are performed according to the adjusted camera parameters, and the clarity of the night fusion image is calculated. The camera parameters are adjusted in a continuous and iterative cycle, and the clarity of the night fusion image after shooting is calculated until the clarity is greater than or equal to the preset image clarity threshold. When the clarity is greater than or equal to the preset image clarity threshold, the adjusted initial camera parameters are used as the optimal camera parameters. When the camera parameter control factor is the second factor, it means that the camera parameters do not need to be optimized, and the initial camera parameters are used as the optimal camera parameters.

[0155] Furthermore, the target area can be photographed according to the optimized best camera parameters, thereby obtaining a photographed image with the best clarity.

[0156] In the embodiment of the present invention, the night-time image refers to a night-time image obtained by fusing a visible light image and an infrared image captured based on optimal imaging parameters.

[0157] In the embodiment of the present invention, the step of performing position fusion shooting of the target area using the optimal shooting parameters to obtain a night-time shooting image includes:

[0158] Generating an optimal visible light image at the imaging position using the optimal imaging parameters;

[0159] Generating an optimal infrared image using the optimal imaging parameters at the position of the infrared light source;

[0160] The best visible light image and the best infrared image are fused to obtain a nighttime image.

[0161] In detail, at the camera position of the target area, the preset optimal camera parameters are used to shoot, and at the infrared light source position, the same optimal camera parameters are used to shoot, and the obtained optimal visible light image and the optimal infrared image are fused to provide richer and more detailed night shooting results. By utilizing the optimal camera parameters and light source configuration, clearer and more detailed images can be obtained under night conditions, while combining the advantages of visible light and infrared images to enhance the visual analysis and recognition capabilities of the target area.

[0162] Specifically, the step of fusing the best visible light image with the best infrared image to obtain a night-time shot image is consistent with the step of fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night-time fused image in S3, and will not be repeated here.

[0163] The embodiment of the present invention can accurately determine the camera position and the infrared light source position by extracting the ambient brightness data of the target area, ensuring that the camera can capture the image of the target area at the best angle and distance in the night environment; using the initial camera parameters, respectively generate visible light images and infrared images of the target area. This helps to obtain detailed information under different spectra, the visible light image provides visual information, and the infrared image reveals the heat distribution and other hidden details; through the exposure weight algorithm and the image clarity algorithm, the visible light image and the infrared image are fused, and the advantages of the two spectral images are utilized to produce a richer and clearer night fusion image, which helps to improve the visualization and recognition ability of the target area; based on the generated night fusion image, by adjusting the initial camera parameters and the preset parameter cycle conditions, the optimal camera parameters are calculated, and the image of the target area can be captured under the most optimized conditions, thereby improving the image quality and clarity; the optimal camera parameters are used to perform position fusion shooting to obtain the final night shooting image. The integration and optimization of these steps enable more accurate and clearer images to be obtained under night conditions. Therefore, the infrared-assisted night image shooting method, device, electronic device and computer-readable storage medium proposed by the present invention can solve the problem of low clarity when shooting night images.

[0164] like Figure 4 , is a functional module diagram of an infrared-assisted nighttime image shooting device provided by an embodiment of the present invention.

[0165] The infrared-assisted night image shooting device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the infrared-assisted night image shooting device 100 can include a position determination module 101, an image generation module 102, an image fusion module 103, a camera parameter control factor generation module 104, and a night image generation module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0166] In this embodiment, the functions of each module / unit are as follows:

[0167] The position determination module 101 is used to obtain a target area for night photography, extract environmental brightness data of the target area, and determine the camera position and infrared light source position of the target area according to the environmental brightness data;

[0168] The image generation module 102 is used to extract initial imaging parameters of the target area, generate a visible light image of the target area according to the initial imaging parameters and the imaging position, and generate an infrared image of the target area according to the initial imaging parameters and the infrared light source position;

[0169] The image fusion module 103 is used to calculate the first image weight of the visible light image and the second image weight of the infrared image by using a preset exposure weight algorithm, and fuse the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fusion image;

[0170] The imaging parameter control factor generating module 104 is used to calculate the clarity of the nighttime fused image using a preset image clarity algorithm, and generate the imaging parameter control factor of the target area according to the clarity;

[0171] The night-time image generation module 105 is used to adjust the initial imaging parameters through the imaging parameter control factors and preset parameter cycle conditions to obtain optimal imaging parameters, and use the optimal imaging parameters to perform position fusion photography on the target area to obtain night-time images.

[0172] In detail, each module described in the infrared-assisted nighttime image shooting device 100 in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The same technical means are used as the infrared-assisted night image shooting method described in , and can produce the same technical effects, so they will not be repeated here.

[0173] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0174] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0176] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0177] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is not limited only according to the above description, and it is intended that all changes within the meaning and scope of equivalent elements within the scope of protection are included in the present invention.

[0178] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application device that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0179] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in a device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for nighttime image capture based on infrared assistance, characterized in that: The method comprises: Acquire a target area for night photography, extract ambient brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the ambient brightness data; Extracting initial imaging parameters of the target area, generating a visible light image of the target area according to the initial imaging parameters and the imaging position, and generating an infrared image of the target area according to the initial imaging parameters and the infrared light source position; Calculating a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fused image; The definition of the night fusion image is calculated using a preset image definition algorithm, including: extracting a highlight image region according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of ​​the highlight image region and identifying the center position of the highlight image region; determining a first clear region according to the center position and a preset region radius; filtering invalid pixels in the first clear region according to a preset second brightness threshold to obtain a second clear region; performing a difference operation between the first clear region and the second clear region to obtain a target clear region; and calculating the target definition of the target clear region using the following preset image definition algorithm: Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point; Determine the target definition as the definition of the nighttime fused image, and generate a camera parameter control factor of the target area according to the definition; The initial imaging parameters are adjusted by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters, and the target area is photographed by position fusion using the optimal imaging parameters to obtain a nighttime photographic image.

2. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of determining the camera position and the infrared light source position of the target area according to the ambient brightness data includes: Identifying a first area center position of the target area, and determining a camera position of the target area according to a preset camera distance and the first area center position; Identifying a dark area of ​​the target area according to the ambient brightness data, and identifying a second area center position of the dark area; Determine the infrared distance between the infrared light source and the center position of the second area according to the preset infrared light source properties, and determine the infrared out-of-area point according to the infrared distance and the camera distance; The infrared light source position of the target area is determined as the infrared light source position of the target area.

3. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The extracting initial imaging parameters of the target area includes: Determining the camera lens angle of view according to the camera position of the target area; Extracting exposure parameters, focal length parameters and sensitivity parameters of the camera corresponding to the target area; Extracting the irradiation angle and illumination intensity of the infrared light source corresponding to the target area; The camera lens viewing angle, the exposure parameter, the focal length parameter, the sensitivity parameter, the illumination angle and the illumination intensity are determined as initial camera parameters of the target area.

4. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: Generating the visible light image of the target area according to the initial imaging parameters and the imaging position includes: Focusing the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters; The focused light is converted into an electronic signal through the photosensitive unit in the photosensitive element; Performing multiple signal enhancement processing on the electronic signal, and encoding the electronic signal after the signal enhancement processing into a digital image format; A visible light image of the target area is generated according to the digital image format.

5. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The calculating the first image weight of the visible light image by using a preset exposure weight algorithm includes: Extracting camera parameters corresponding to the visible light image; Counting the exposure of the visible light image, and determining an image enhancement factor of the visible light image according to the exposure; The first image weight of the visible light image is calculated according to the camera parameters, the exposure level and the image enhancement factor by using the following preset exposure weight algorithm: Wherein, ω is the first image weight, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, K i is the exposure degree corresponding to the visible light image i.

6. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image includes: The visible light image is subjected to a convolution kernel operation through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and the infrared image is subjected to a convolution kernel operation to obtain a Gaussian pyramid infrared image; Performing a difference operation between two adjacent layers of the Gaussian pyramid visible light image to obtain a Gaussian visible light difference image, and performing a difference operation between two adjacent layers of the Gaussian pyramid infrared image to obtain a Gaussian infrared difference image; Counting the visible light feature points of the Gaussian visible light difference image, and counting the infrared feature points of the Gaussian infrared difference image; Registering the visible light feature point with the infrared feature point to obtain a registered feature point; Image fusion is performed according to the first image weight, the second image weight and the registration feature points to obtain a night fusion image, wherein the image fusion calculation formula is: R=ω1×I(x,y)+ω2×g(I(f(x,y))) Among them, R is the grayscale value of the night fusion image, ω1 is the weight of the first image, ω2 is the weight of the second image, I(x, y) is the grayscale value of the registration feature point at the position (x, y), f(x, y) is the coordinate transformation of the position (x, y) in the two-dimensional space of the image, and g is the radiation transformation function.

7. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of generating the imaging parameter control factor of the target area according to the definition includes: When the clarity is less than a preset clarity threshold, determining the camera parameter control factor of the target area as a first factor; When the clarity is greater than or equal to a preset clarity threshold, the camera parameter control factor of the target area is determined as a second factor.

8. The infrared-assisted nighttime image shooting method according to claim 6, characterized in that: The step of adjusting the initial imaging parameters by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters includes: When the camera parameter control factor is the first factor, adjusting the initial camera parameter according to a preset camera parameter adjustment window; Calculating the clarity of the nighttime fused image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition; When the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition, the adjusted initial imaging parameters are used as optimal imaging parameters. When the imaging parameter control factor is the second factor, the initial imaging parameter is determined as the optimal imaging parameter.

9. The infrared-assisted nighttime image shooting method according to claim 2, characterized in that: The step of performing position fusion shooting of the target area using the optimal shooting parameters to obtain a nighttime shooting image includes: Generating an optimal visible light image at the imaging position using the optimal imaging parameters; Generating an optimal infrared image using the optimal imaging parameters at the position of the infrared light source; The best visible light image and the best infrared image are fused to obtain a nighttime image.

10. A night image shooting device based on infrared assistance, characterized in that: Used to execute the infrared-assisted nighttime image shooting method according to any one of claims 1 to 9, the device comprising: A position determination module, used to obtain a target area for night photography, extract ambient brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the ambient brightness data; An image generation module, used to extract initial imaging parameters of the target area, generate a visible light image of the target area according to the initial imaging parameters and the imaging position, and generate an infrared image of the target area according to the initial imaging parameters and the infrared light source position; an image fusion module, configured to calculate a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fuse the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image; The camera parameter control factor generation module is used to calculate the clarity of the night fusion image using a preset image clarity algorithm, including: extracting a highlight image area according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of ​​the highlight image area and identifying the center position of the highlight image area; determining a first clear area according to the center position and a preset area radius; filtering invalid pixels in the first clear area according to a preset second brightness threshold to obtain a second clear area; performing a difference operation between the first clear area and the second clear area to obtain a target clear area; and calculating the target clarity of the target clear area using the following preset image clarity algorithm: Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point; Determine the target definition as the definition of the nighttime fused image, and generate a camera parameter control factor of the target area according to the definition; The night-time image generation module is used to adjust the initial image parameters through the image parameter control factor and the preset parameter cycle condition to obtain the optimal image parameters, and use the optimal image parameters to perform position fusion shooting on the target area to obtain the night-time image.

Citation Information

Patent Citations

  • Adjusting method and adjusting device for imaging precision of motion sensing camera

    CN108234874A

  • Method and device for improving image definition of specific area and electronic equipment

    CN114339062A

  • Camera definition detection method

    CN116091613A

  • Image processing method and electronic device

    US20240185389A1

  • Photographing method, storage medium, and electronic device

    WO2020215720A1

Cited By

  • Night vision device, night vision method and computer readable storage medium

    CN119254932A

  • An image processing system based on the Internet of Things

    CN119788941A