An infrared-assisted method and device for night-time image shooting

Abstract

The present invention relates to the field of infrared assistance technology, and discloses a method and device for night image shooting based on infrared assistance, including: extracting environmental brightness data of a target area captured at night, and determining the camera position and infrared light source position of the target area according to the environmental brightness data; generating a visible light image and an infrared image of the target area according to the initial camera parameters and camera position of the target area; calculating a first image weight of the visible light image, calculating a second image weight of the infrared image, and performing image fusion on the visible light image and the infrared image according to the first image weight and the second image weight; calculating the clarity of the night fusion image, and generating a camera parameter control factor for the target area according to the clarity; adjusting the initial camera parameters through the camera parameter control factor and the parameter loop condition, and performing position fusion shooting on the target area with the optimal camera parameters to obtain a night captured image. The present invention can improve the clarity of night image shooting.

Description

Technical Field

[0001] The present invention relates to the field of infrared assistance technology, and particularly to a method and device for night image shooting based on infrared assistance. Background Art

[0002] With the development of infrared assistance technology and low-light imaging technology, night shooting has become increasingly common and important in various applications. The progress of these technologies has made it possible to obtain high-quality images and videos at night or in low-light conditions, thus playing an important role in fields such as security monitoring, search and rescue, etc. However, in order to improve the clarity of night-shot images, it is necessary to optimize the parameters for night shooting, so as to perform night shooting with high clarity.

[0003] Existing night image shooting technologies use the thermal energy information in the infrared spectrum to enhance the capture ability of night images. It can work in complete darkness or extremely low-light conditions and is not restricted by insufficient visible light illumination. In practical applications, simply collecting infrared images to determine the night-shot images may result in overly single night-shot images, losing important details in the images, thus resulting in relatively low clarity when performing night image shooting. Summary of the Invention

[0004] The present invention provides a method and device for night image shooting based on infrared assistance, and its main purpose is to solve the problem of relatively low clarity when performing night image shooting.

[0005] To achieve the above object, a method for night image shooting based on infrared assistance provided by the present invention includes:

[0006] Obtain the target area for night shooting, extract the environmental brightness data of the target area, and determine the camera position and infrared light source position of the target area according to the environmental brightness data;

[0007] Extract the initial camera parameters of the target area, generate a visible light image of the target area according to the initial camera parameters and the camera position, and generate an infrared image of the target area according to the initial camera parameters and the infrared light source position;

[0008] Calculate the first image weight of the visible light image through a preset exposure weight algorithm, calculate the second image weight of the infrared image, and perform image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image;

[0009] Calculating the sharpness of the night fusion image using a preset image sharpness algorithm, including: extracting a highlighted image area according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of the highlighted image area and identifying the central position of the highlighted image area; determining a first clear area according to the central position and a preset area radius; screening out invalid pixel points in the first clear area according to a preset second brightness threshold to obtain a second clear area; performing a difference operation on the first clear area and the second clear area to obtain a target clear area; calculating the target sharpness of the target clear area using the following preset image sharpness algorithm:

[0010]

[0011] Where Q is the target sharpness, s is the area, u is the average gray value of the target clear area, f(l, d) is the gray value corresponding to each pixel point (l, d) in the target clear area, n is the width of the target clear area, m is the length of the target clear area, l is the abscissa position of the pixel point, and d is the ordinate position of the pixel point;

[0012] Determining the sharpness of the night fusion image based on the target sharpness, and generating a camera parameter control factor for the target area according to the sharpness;

[0013] Adjusting the initial camera parameters through the camera parameter control factor and a preset parameter loop condition to obtain the optimal camera parameters, and performing position fusion shooting on the target area using the optimal camera parameters to obtain a night shooting image.

[0014] Optionally, the determining the camera position and the infrared light source position of the target area according to the environmental brightness data includes:

[0015] Identifying the first area center position of the target area, and determining the camera position of the target area according to a preset camera distance and the first area center position;

[0016] Identifying the dark area of the target area according to the environmental brightness data, and identifying the second area center position of the dark area;

[0017] Determining the infrared distance between the infrared light source and the second area center position according to a preset infrared light source attribute, and determining an infrared intersection point through the infrared distance and the camera distance;

[0018] Determining the infrared intersection point as the infrared light source position of the target area.

[0019] Optionally, the extracting the initial camera parameters of the target area includes:

[0020] Determine the camera lens viewing angle according to the camera position of the target area;

[0021] Extract the exposure parameter, focal length parameter and sensitivity parameter of the camera corresponding to the target area;

[0022] Extract the irradiation angle and illumination intensity of the infrared light source corresponding to the target area;

[0023] Determine the above-mentioned camera lens viewing angle, exposure parameter, focal length parameter, sensitivity parameter, irradiation angle and illumination intensity as the initial camera parameters of the target area.

[0024] Optionally, generating the visible light image of the target area according to the initial camera parameters and the camera position includes:

[0025] Focus the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters;

[0026] Convert the focused light into an electronic signal through the photosensitive units in the photosensitive element;

[0027] Perform multiple signal enhancement processing on the electronic signal, and encode the enhanced electronic signal into a digital image format;

[0028] Generate the visible light image of the target area according to the digital image format.

[0029] Optionally, calculating the first image weight of the visible light image through a preset exposure weight algorithm includes:

[0030] Extract the camera parameters corresponding to the visible light image;

[0031] Statistically analyze the exposure of the visible light image, and determine the image enhancement factor of the visible light image according to the exposure;

[0032] Calculate the first image weight of the visible light image according to the camera parameters, the exposure and the image enhancement factor through the following preset exposure weight algorithm:

[0033]

[0034] where ω is the first image weight, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, K i is the exposure corresponding to the visible light image i.

[0035] Optionally, the step of performing image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image includes:

[0036] Performing convolution kernel operation on the visible light image through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and performing convolution kernel operation on the infrared image to obtain a Gaussian pyramid infrared image;

[0037] Performing differential operation between two adjacent layers of the Gaussian pyramid visible light image to obtain a Gaussian visible light difference image, and performing differential operation between two adjacent layers of the Gaussian pyramid infrared image to obtain a Gaussian infrared difference image;

[0038] Counting visible light feature points of the Gaussian visible light difference image, and counting infrared feature points of the Gaussian infrared difference image;

[0039] Registering the visible light feature points and the infrared feature points to obtain registered feature points;

[0040] Performing image fusion according to the first image weight, the second image weight and the registered feature points to obtain a night fusion image, where the image fusion calculation formula is:

[0041] R = ω1×I(x,y)+ω2×g(I(f(x,y)))

[0042] where R is the gray value of the night fusion image, ω1 is the first image weight, ω2 is the second image weight, I(x,y) is the gray value of the registered feature points at position (x,y), f(x,y) is the coordinate transformation of position (x,y) in the two-dimensional image space, and g is a radiation transformation function.

[0043] Optionally, the step of generating a camera parameter control factor for the target area according to the sharpness includes:

[0044] When the sharpness is less than a preset sharpness threshold, determining the camera parameter control factor for the target area as a first factor;

[0045] When the sharpness is greater than or equal to the preset sharpness threshold, determining the camera parameter control factor for the target area as a second factor.

[0046] Optionally, the step of adjusting the initial camera parameters through the camera parameter control factor and a preset parameter loop condition to obtain optimal camera parameters includes:

[0047] When the camera parameter control factor is the first factor, adjusting the initial camera parameters according to a preset camera parameter adjustment window;

[0048] Calculate the clarity of the night fusion image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter loop condition;

[0049] When the clarity is greater than or equal to the image clarity threshold in the parameter loop condition, use the adjusted initial camera parameters as the optimal camera parameters.

[0050] When the camera parameter control factor is the second factor, determine the initial camera parameters as the optimal camera parameters.

[0051] Optionally, the using the optimal camera parameters to perform position fusion shooting on the target area to obtain a night shooting image includes:

[0052] Generate an optimal visible light image using the optimal camera parameters at the camera position;

[0053] Generate an optimal infrared image using the optimal camera parameters at the infrared light source position;

[0054] Perform image fusion on the optimal visible light image and the optimal infrared image to obtain a night shooting image.

[0055] To solve the above problems, the present invention also provides a night image shooting device based on infrared assistance, and the device includes:

[0056] A position determination module, configured to obtain a target area for night shooting, extract environmental brightness data of the target area, and determine the camera position and infrared light source position of the target area according to the environmental brightness data;

[0057] An image generation module, configured to extract the initial camera parameters of the target area, generate a visible light image of the target area according to the initial camera parameters and the camera position, and generate an infrared image of the target area according to the initial camera parameters and the infrared light source position;

[0058] An image fusion module, configured to calculate a first image weight of the visible light image through a preset exposure weight algorithm, calculate a second image weight of the infrared image, and perform image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image;

[0059] A camera parameter control factor generation module, configured to calculate the clarity of the night fusion image using a preset image clarity algorithm, and generate a camera parameter control factor of the target area according to the clarity;

[0060] The night-time shooting image generation module is used to adjust the initial shooting parameters through the shooting parameter control factor and the preset parameter loop condition to obtain the optimal shooting parameters, and use the optimal shooting parameters to perform position fusion shooting on the target area to obtain night-time shooting images.

[0061] In the embodiment of the present invention, by extracting the environmental brightness data of the target area, the shooting position and the infrared light source position can be accurately determined, ensuring that the camera can capture the image of the target area at the best angle and distance in the night environment; using the initial shooting parameters, a visible light image and an infrared image of the target area are respectively generated. This helps to obtain detailed information under different spectra. The visible light image provides visual information, while the infrared image reveals the heat distribution and other hidden details; through the exposure weight algorithm and the image sharpness algorithm, the visible light image and the infrared image are fused, taking advantage of the two spectral images to produce a richer and clearer night-time fused image, which helps to improve the visualization and recognition ability of the target area; based on the generated night-time fused image, by adjusting the initial shooting parameters and the preset parameter loop condition, the optimal shooting parameters are calculated to ensure that the image of the target area can be captured under the most optimized conditions, improving the quality and sharpness of the image; using the optimal shooting parameters for position fusion shooting to obtain the final night-time shooting image. The integration and optimization of these steps enable more accurate and clearer images to be obtained under night conditions. Therefore, the infrared-assisted night-time image shooting method, device, electronic device and computer-readable storage medium proposed by the present invention can solve the problem of low clarity when shooting night-time images. Description of the Drawings

[0062] Figure 1 It is a schematic flowchart of the infrared-assisted night-time image shooting method provided by an embodiment of the present invention;

[0063] Figure 2 It is a schematic flowchart of extracting the initial shooting parameters provided by an embodiment of the present invention;

[0064] Figure 3 It is a schematic flowchart of calculating the first image weight provided by an embodiment of the present invention;

[0065] Figure 4 It is a functional module diagram of the infrared-assisted night-time image shooting device provided by an embodiment of the present invention.

[0066] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed Embodiments

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0068] An embodiment of the present application provides a method for taking night images assisted by infrared. The execution subject of the method for taking night images assisted by infrared includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for taking night images assisted by infrared can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0069] Refer to Figure 1 As shown, it is a schematic flowchart of a method for taking night images assisted by infrared provided by an embodiment of the present invention. In this embodiment, the method for taking night images assisted by infrared includes:

[0070] S1. Obtain the target area taken at night, extract the environmental brightness data of the target area, and determine the camera position and infrared light source position of the target area according to the environmental brightness data.

[0071] In an embodiment of the present invention, the target area refers to the area range that needs to be photographed. The target area taken at night can be determined according to requirements. By detecting the brightness of the surrounding environment of the target area, the environmental light level around the target area can be directly measured through an environmental sensor (such as a photometer or illuminometer), providing more direct and accurate environmental brightness data. The environmental brightness data refers to the data describing the light level of the surrounding environment, that is, the light intensity received in a certain area, thereby reflecting the brightness level of the environment.

[0072] Furthermore, according to the illuminance and photometric data of the environment, select the best position of the camera to ensure that the objects or scenes in the target area can be clearly visible. In addition, in order to provide sufficient infrared illumination in a low-light or completely dark environment so that the camera can capture clear infrared images without being interfered by environmental light conditions, it is also necessary to determine the best placement position of the infrared light source.

[0073] In the embodiments of the present invention, the camera position refers to the installation position of the camera, that is, where the camera is placed to best capture the target area; the infrared light source position refers to the installation position of the infrared lighting device, which emits infrared light to enhance the monitoring effect at night or in low-light conditions. The position of the infrared light source is selected based on the dark areas in the environment to ensure that these areas can be fully illuminated in the monitoring system, so that the camera can capture clear images.

[0074] In the embodiments of the present invention, determining the camera position and the infrared light source position of the target area according to the environmental brightness data includes:

[0075] Identifying the first area center position of the target area, and determining the camera position of the target area according to the preset camera distance and the first area center position;

[0076] Identifying the dark areas of the target area according to the environmental brightness data, and identifying the second area center position of the dark areas;

[0077] Determining the infrared distance between the infrared light source and the second area center position according to the preset infrared light source attributes, and determining the infrared intersection point through the infrared distance and the camera distance;

[0078] Determining the infrared intersection point as the infrared light source position of the target area.

[0079] Specifically, the target area is flattened, and the flattened target area is meshed. Then, based on the number of grids, the center position of the target area, that is, the first area center position, is selected. According to the horizontal direction of the first area center position, the position at a distance of the camera distance is determined as the camera position of the target area. The camera distance is determined according to the characteristics of the camera, that is, the closest distance at which the camera can clearly capture the target area. The camera distance is preset. For example, the focal length of the lens determines the field of view angle and imaging magnification of the camera. A longer focal length usually means that the camera can capture clear images at a farther distance.

[0080] Specifically, according to the ambient brightness data, the distribution of each brightness level in the target area is displayed. Usually, the dark area has a lower brightness level. Then, the area in the ambient brightness data that is lower than the brightness threshold is identified as the dark area, and the dark area is flattened. The flattened dark area is meshed, and based on the number of grids in the dark area, the central position of the dark area, that is, the central position of the second area, is selected. According to the wavelength in the infrared light source attributes, the propagation distance of the infrared light source is determined. Furthermore, the infrared distance between the infrared light source and the central position of the second area is determined as the propagation distance of the infrared light source. However, the position of the infrared light source should not only be close to the dark area but also close to the imaging position of the camera. Therefore, with the imaging position as the center and the imaging distance as the radius, a circular area is obtained. On the boundary of the circular area, the intersection point of the propagation distance line and the imaging distance line is found, that is, the infrared intersection point, and this infrared intersection point is determined as the position of the infrared light source corresponding to the target area.

[0081] Furthermore, after determining the imaging position and the infrared light source position of the target area, before shooting the target area, it is also necessary to determine the shooting imaging parameters. Therefore, the imaging parameters need to be analyzed.

[0082] S2. Extract the initial imaging parameters of the target area, generate a visible light image of the target area according to the initial imaging parameters and the imaging position, and generate an infrared image of the target area according to the initial imaging parameters and the infrared light source position.

[0083] In the embodiment of the present invention, the initial imaging parameters refer to the basic parameters required for configuring the imaging system and the parameters for shooting the target area set initially.

[0084] In the embodiment of the present invention, referring to Figure 2 as shown, the extraction of the initial imaging parameters of the target area includes:

[0085] S21. Determine the imaging lens angle of view according to the imaging position of the target area;

[0086] S22. Extract the exposure parameter, focal length parameter, and sensitivity parameter of the camera corresponding to the target area;

[0087] S23. Extract the irradiation angle and illumination intensity of the infrared light source corresponding to the target area;

[0088] S24. Determine the imaging lens angle of view, the exposure parameter, the focal length parameter, the sensitivity parameter, the irradiation angle, and the illumination intensity as the initial imaging parameters of the target area.

[0089] Specifically, determine the horizontal and vertical viewing angles of the camera position as the viewing angle of the camera lens, or the field of view angle of the camera can be determined by actual measurement or by referring to the camera specification sheet; and determine the exposure parameters, focal length parameters, and sensitivity parameters of the camera according to the specifications of the camera. The exposure parameters include shutter speed, aperture size, and ISO sensitivity. These parameters determine the exposure amount of the camera under specific lighting conditions, that is, the shutter speed controls the length of the exposure time, affecting the motion blur and brightness of the image; the aperture size determines the amount of light entering the camera, directly affecting the clarity and depth of field of the image; the ISO sensitivity specifies the sensitivity of the camera sensor under a given lighting condition. High ISO is usually used in low-light conditions but may increase image noise; the focal length parameter refers to the focal length of the lens, affecting the size and viewing angle of the object in the image; the sensitivity parameter, that is, the ISO value, indicates the photosensitive performance of the camera under specific lighting conditions.

[0090] Specifically, measure the irradiation angle and illumination intensity of the infrared light source corresponding to the target area. The irradiation angle describes the angular range of the infrared light source irradiation, usually referring to the divergence angle of the light beam; the illumination intensity, that is, the radiation intensity or luminous flux of the infrared light source, is used to measure the illumination effect of the light source on the target area. Furthermore, determine the extracted exposure parameters, focal length parameters, sensitivity parameters, irradiation angle, and illumination intensity as the initial camera parameters of the target area. Camera parameters are very important when configuring and adjusting the camera system to ensure that the camera system can obtain clear and accurate images under various environmental conditions. In addition, when determining the camera parameters, it is usually necessary to refer to the technical manual or specification sheet of the camera and the technical specification or data provided by the supplier of the infrared light source.

[0091] Furthermore, according to the configured initial camera parameters, the target area can be photographed at the camera position to ensure that the content of the target area is completely covered, thereby obtaining a visible light image corresponding to the target area.

[0092] In the embodiments of the present invention, the visible light image refers to an image captured by the camera with light within the visible light range, an image that can be seen through an ordinary camera or the naked eye. Its characteristics are rich in color and details, and can reflect the true appearance of the object and the lighting conditions of the environment.

[0093] In the embodiments of the present invention, generating the visible light image of the target area according to the initial camera parameters and the camera position includes:

[0094] Focus the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters;

[0095] Convert the focused light into an electrical signal through the photosensitive units in the photosensitive element;

[0096] Perform multiple signal enhancement processing on the electronic signal and encode the enhanced electronic signal into a digital image format;

[0097] Generate a visible light image of the target area according to the digital image format.

[0098] Specifically, at the imaging position corresponding to the camera, based on the initial imaging parameters, the light is focused on the photosensitive element through the optical system of the camera. The lens usually includes components such as convex lenses and mirrors, and the light in the scene is focused on the photosensitive element through these components. The photosensitive element consists of many tiny photosensitive units, each unit measures the intensity of the light and converts it into an electrical signal. That is, when the light irradiates the surface of the photosensitive element, photons will excite the electrons inside the photosensitive unit, and the number of these electrons is proportional to the intensity of the light. Therefore, the number of electrons measured by each photosensitive unit reflects the intensity of the light received by that unit, and the electrons generated inside the photosensitive unit are collected and converted into an electrical signal.

[0099] Specifically, once the photosensitive element converts the optical image into an electrical signal, the electrical signal generated by the photosensitive element is transmitted to the electronic signal processing unit of the camera. The electrical signal is vertically transmitted to the edge of the sensor and then processed by a dedicated circuit. Each photosensitive unit on the photosensitive element corresponds to an image pixel, and the electrical signal is processed through amplification, filtering, digitization, etc. to optimize the image quality and reduce noise. Among them, the electrical signal initially output from the photosensitive element may be very weak, so it needs to be amplified to enhance the signal intensity to ensure that the camera can capture sufficient electrical signals even in low-light conditions; the electrical signal may contain noise from uneven light, electronic noise, or other interference sources, and the noise is removed through a filter to ensure that the final image is clear and accurate; the electrical signal is converted into a digital form for subsequent digital signal processing, including converting the analog electrical signal into a digital format for easy storage, transmission, and further processing. Then, the processed electrical signal is encoded into a digital image format, such as JPEG or RAW format. Therefore, the image data corresponding to the digital image format is the visible light image of the target area.

[0100] Furthermore, when the target area is monitored under low light or night conditions, an infrared light source can provide supplementary lighting so that the camera can capture scenes in the dark. The irradiation angle and illumination intensity of the infrared light source affect the working effect of the camera. That is, under the condition of infrared light source supplementary lighting, an infrared image of the target area is generated according to the initial camera parameters and the position of the infrared light source. Among them, with the supplement of the position of the infrared light source, the camera generates an infrared image of the target area according to the initial camera parameters and the camera position. An infrared image refers to an image captured through infrared optical technology, showing a thermal energy distribution different from that of visible light. Infrared light is a type of electromagnetic wave with a wavelength longer than that of visible light, ranging from approximately 0.75 micrometers to 1000 micrometers. The infrared image provides an information perspective different from that of visible light images.

[0101] Furthermore, to ensure the importance of differentiating between visible light images and infrared images, it is necessary to determine the weights of visible light images and infrared images separately to ensure that the first frame of images obtained from the camera can provide clear and bright images under different conditions, thus contributing to subsequent image processing.

[0102] S3. Calculate the first image weight of the visible light image through a preset exposure weight algorithm, calculate the second image weight of the infrared image, and fuse the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image.

[0103] In the embodiment of the present invention, the first image weight is used to measure the importance of the visible light image, that is, a weight value calculated according to camera parameters and the exposure of the image, used to reflect the relative importance of each pixel in the image or the contribution degree to the overall image quality.

[0104] In the embodiment of the present invention, referring to Figure 3 as shown, calculating the first image weight of the visible light image through a preset exposure weight algorithm includes:

[0105] S31. Extract the camera parameters corresponding to the visible light image;

[0106] S32. Statistically analyze the exposure of the visible light image, and determine the image enhancement factor of the visible light image according to the exposure;

[0107] S33. Calculate the first image weight of the visible light image through the following preset exposure weight algorithm according to the camera parameters, the exposure, and the image enhancement factor:

[0108]

[0109] where ω is the weight of the first image, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, and K i is the exposure corresponding to the visible light image i.

[0110] Specifically, camera parameters are generally fixed parameters of most cameras. Set a = -0.329 and b = 1.1258. Then, the first fixed parameter a and the second fixed parameter b in the camera parameters are pre-configured. Furthermore, the exposure of the visible light image is statistically calculated. The exposure refers to measuring and analyzing the illumination brightness or luminance level of each pixel in the image. First, calculate the grayscale value of each pixel in the visible light image. The grayscale value usually ranges from 0 to 255, representing the brightness of the pixel, where 0 represents black and 255 represents white. Then, display the number of pixels at different grayscale levels according to the grayscale histogram. The shape of the histogram reflects the overall exposure distribution of the image, such as whether there is overexposure or underexposure. Next, calculate the average grayscale value and the variance of the grayscale values of the image to reflect the overall brightness and contrast information of the image, and obtain the exposure of the visible light image. Furthermore, determine the image enhancement factor according to the exposure of the visible light image to adjust the visual effect of the image. When the exposure of each pixel in the image is relatively uniform, the image enhancement factor μ = 0. When there is overexposure or underexposure in the exposure of each pixel in the image, the image enhancement factor μ = 1. Thus, calculate the weight of the visible light image according to the fixed camera parameters, exposure, and image enhancement factor.

[0111] Specifically, during shooting, multiple visible light images are usually taken. Then, calculate the exposure of each image among the multiple visible light images. Furthermore, select the image with the best exposure as the visible light image. Then, calculate the weight of the visible light image according to the image with the best exposure. That is, select the image with the best exposure among the numerous images taken as the final visible light image and calculate the weight corresponding to its image. In the exposure weight algorithm, determine the first image weight of the visible light image by calculating the best image exposure and then based on the image enhancement factor. When the image enhancement factor μ = 0, it indicates that the exposure of the visible light image is relatively uniform. Therefore, the weight of the visible light image at this time is 1. When the image enhancement factor μ = 1, it indicates that there is overexposure or underexposure in the visible light image. Then, calculate the weight of the image corresponding to the best exposure among all visible light images, where P iis the gray value or color value of each pixel of the visible light image. By using a non-linear function and a maximum value function, it can more accurately reflect the influence of exposure on the image pixel value, thereby affecting the final calculation result of the image weight. In order to improve the calculation efficiency, when solving the optimal exposure, the input image size is reduced to 60×60 using the nearest neighbor interpolation algorithm.

[0112] Further, the step of calculating the second image weight of the infrared image is the same as the step of calculating the first image weight of the visible light image by a preset exposure weight algorithm, which will not be elaborated here. The second image weight is used to measure the importance of the infrared image, that is, the weight value of the infrared image calculated according to the camera parameters and the exposure of the image, which is used to reflect the relative importance of each pixel in the infrared image or the contribution degree to the overall infrared image quality, that is, to statistically calculate the exposure of the infrared image, determine the image enhancement factor of the infrared image according to the exposure, and then calculate the second image weight of the infrared image according to the image enhancement factor, camera parameters and exposure.

[0113] Furthermore, according to the image weights of the visible light image and the infrared image, image fusion can be performed according to the size of the weights, which can determine the contribution degree of the visible light image and the infrared image in the final fused image. For example, if the first image weight is higher, the influence of the visible light image in the final fused image will be greater; if the second image weight is higher, the influence of the infrared image in the final fused image will be greater. For example, in night image fusion, if the goal is to better capture the temperature distribution and target detection, the weight of the infrared image may be increased; if more detail and color information need to be retained, the weight of the visible light image can be increased.

[0114] In the embodiment of the present invention, the night fused image refers to synthesizing or fusing the visible light image and the infrared image to provide more comprehensive and clearer night visual information, which is used to enhance the performance of visible light photography under night conditions. The thermal image enhancement of the infrared image is used to supplement the deficiency of the visible light image under low light conditions.

[0115] In the embodiment of the present invention, the step of performing image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fused image includes:

[0116] Performing convolution kernel operation on the visible light image through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and performing convolution kernel operation on the infrared image to obtain a Gaussian pyramid infrared image;

[0117] Perform a difference operation between two adjacent layers of the Gaussian pyramid visible light images to obtain a Gaussian visible light difference image, and perform a difference operation between two adjacent layers of the Gaussian pyramid infrared images to obtain a Gaussian infrared difference image;

[0118] Count the visible light feature points of the Gaussian visible light difference image and count the infrared feature points of the Gaussian infrared difference image;

[0119] Register the visible light feature points with the infrared feature points to obtain registered feature points;

[0120] Perform image fusion based on the first image weight, the second image weight, and the registered feature points to obtain a night fusion image, where the image fusion calculation formula is:

[0121] R = ω1 × I(x, y) + ω2 × g(I(f(x, y)))

[0122] where R is the gray value of the night fusion image, ω1 is the first image weight, ω2 is the second image weight, I(x, y) is the gray value of the registered feature point at position (x, y), f(x, y) is the coordinate transformation of position (x, y) in the two-dimensional image space, and g is the radiation transformation function.

[0123] Specifically, first perform convolution of the original image with Gaussian filters of different scales. The Gaussian filter is usually used to smooth the image and reduce noise while retaining the structural information in the image. By performing multiple downsampling and Gaussian smoothing operations on the original image, a series of images with different scales can be constructed, which is the Gaussian pyramid. Each layer of the Gaussian pyramid image is generated by downsampling and Gaussian filtering, so as to obtain the Gaussian pyramid image corresponding to the visible light image and the Gaussian pyramid image corresponding to the infrared image, and perform subtraction operations between different layers of the Gaussian pyramid to obtain the Gaussian difference image of the visible light image and the Gaussian difference image of the infrared image. In the Gaussian difference image, for each pixel and its surrounding 8 adjacent pixels, as well as all pixels in the same degree layer and adjacent two layers, if the pixel is greater than or less than the values of all comparison pixels, then this pixel is considered a local extreme point, and the local extreme point is determined as a feature point. The feature point represents the prominent features in the image, so as to count the visible light feature points and infrared feature points.

[0124] Specifically, feature points are extracted from the processed images, and the feature points in the visible light image are registered with the feature points in the infrared image according to the positions of the feature points in the images to ensure the spatial consistency between the visible light image and the infrared image. Then, the mapping relationship between the visible light image and the infrared image is I(x, y) = g(I(f(x, y))), where f represents the coordinate transformation in the two-dimensional image space, that is, f(x, y) = (x', y'), which describes how to transform from the coordinate system of one image to that of another image for pixel-level registration and alignment. And g represents a one-dimensional gray or radiation transformation. The radiation transformation generally refers to the process of transforming or adjusting the radiation value of each pixel in the image (such as the thermal radiation value in the infrared image) in image processing. The simplest radiation transformation is a linear transformation, and the radiation value can be scaled and offset through the linear function g(x) = ax + b, which can be used to adjust the brightness and contrast of the image. The radiation transformation function is used to adjust or enhance the radiation value distribution of the second image so as to better fuse it into the night fusion image, thereby obtaining the fused image. The night fusion image will combine the information of the visible light and infrared bands, providing a richer and clearer night vision effect.

[0125] Furthermore, the overall clarity level of the image needs to be evaluated for the fused night image, and then the camera parameters are adjusted based on the clarity level of the image to achieve the night shooting image with the highest clarity.

[0126] S4. Calculate the clarity of the night fusion image using a preset image clarity algorithm, and generate a camera parameter control factor for the target area according to the clarity.

[0127] In the embodiment of the present invention, the clarity refers to the clarity of the edges and details of the captured image, which measures the clarity degree or resolution of the details in the captured image.

[0128] In the embodiment of the present invention, the calculating the clarity of the night fusion image using a preset image clarity algorithm includes:

[0129] Extract a highlighted image area according to the brightness of the night fusion image and a preset first brightness threshold;

[0130] Calculate the area of the highlighted image area and identify the central position of the highlighted image area;

[0131] Determine a first clear area according to the central position and a preset area radius;

[0132] Filter out invalid pixel points in the first clear area according to a preset second brightness threshold to obtain a second clear area;

[0133] Perform a subtraction operation on the first clear region and the second clear region to obtain a target clear region;

[0134] Use the following preset image sharpness algorithm to calculate the target sharpness of the target clear region:

[0135]

[0136] where Q is the target sharpness, s is the area of the region, u is the average gray value of the target clear region, f(l, d) is the gray value corresponding to each pixel point (l, d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the abscissa position of the pixel point, and d is the ordinate position of the pixel point;

[0137] Determine the target sharpness as the sharpness of the night fusion image.

[0138] Specifically, filter out the highlighted parts of the night fusion image through a custom brightness threshold, that is, compare the brightness of each pixel in the night fusion image with a preset first brightness threshold. If the pixel value of each pixel is greater than the preset first brightness threshold, determine the region corresponding to this pixel as the highlighted region, and count the area of the highlighted image region, calculate the total number of pixels in the highlighted image region, that is, the area of the region, identify the central position of the highlighted image region, and the center position can be determined by the centroid of the region, that is, the centroid refers to the weighted average position of all pixels in the region. The steps to calculate the centroid are for a two-dimensional image, multiply the position (l, d) of each pixel by its gray value or weight, and then divide the weighted positions of all pixels by the total weight to obtain the centroid coordinates, so as to determine the central position of the highlighted image region, and determine the first clear region according to the central position and a preset region radius, that is, a region within a preset radius centered on the center of the highlighted region, and the preset region radius is a region with a size of 5 pixels.

[0139] Specifically, in the first clear area, a second threshold segmentation is performed to remove invalid pixel points with too low brightness in the picture. That is, the brightness of each pixel in the first clear area image is compared with a preset second brightness threshold. If the pixel value of each pixel is less than the preset second brightness threshold, the area corresponding to this pixel is determined as an invalid pixel point, and the image obtained by the second brightness threshold segmentation is subtracted from the image obtained by the first brightness threshold segmentation to remove the part with a high-brightness area, resulting in a new area, that is, the target clear area. Because the area difference value of the high-brightness part is too small, it will affect the overall clarity evaluation of the picture when calculating the image clarity. Therefore, the most critical and effective part of the night fusion image is highlighted and processed separately through two brightness threshold segmentations. For the part left after the two brightness threshold segmentations, that is, the pixel points between the two thresholds, the absolute variance method is used to calculate the clarity of the night fusion image, and the calculated value is used as another evaluation factor for the final clarity. Finally, all the obtained values are averaged graphically to obtain the target clarity of the target clear area, and the target clarity of the target clear area is determined as the clarity of the night fusion image.

[0140] Furthermore, the clarity information can be used to adjust the focal length of the lens or the focus of the camera to ensure that the target area captured is always kept clear. Therefore, by determining the parameter control factor according to the clarity, the camera parameters can be optimized, unnecessary image processing and calculations can be reduced, and the system efficiency and response speed can be improved.

[0141] In the embodiment of the present invention, the camera parameter control factor refers to a factor for optimizing and adjusting whether the camera control parameters are optimized, so as to determine the camera parameter control factor based on the clarity, and then adjust the camera control parameters to obtain the best camera control parameters.

[0142] In the embodiment of the present invention, the generating the camera parameter control factor of the target area according to the clarity includes:

[0143] When the clarity is less than the preset clarity threshold, the camera parameter control factor of the target area is determined as the first factor;

[0144] When the clarity is greater than or equal to the preset clarity threshold, the camera parameter control factor of the target area is determined as the second factor.

[0145] Specifically, when the clarity of the target area is less than the preset clarity threshold, it indicates that the clarity of the target area does not meet the image clarity target. Then, the camera parameter control factor corresponding to the target area is determined as the first factor, where the first factor refers to the factor used to control the camera parameters for optimization, and the first factor is configured as the value 1. When the clarity is greater than or equal to the preset clarity threshold, it indicates that the clarity of the target area has reached the image clarity target. Then, the camera parameter control factor corresponding to the target area is determined as the second factor, where the second factor refers to the factor used to control the camera parameters without the need for optimization, and the second factor is configured as the value zero.

[0146] Furthermore, clarity, as a feedback signal, can reflect the quality of the currently captured image. By adjusting the camera parameters in real time, the system can quickly respond to changes in different scenarios and lighting conditions, ensuring that the best image clarity is always maintained. By adjusting the camera parameters, the quality of the captured image is optimized. Using clarity as a control factor can find and ensure the best combination of camera parameters under different conditions to achieve the best shooting level and obtain the best night-time shooting image.

[0147] S5. Adjust the initial camera parameters through the camera parameter control factor and the preset parameter loop condition to obtain the best camera parameters, and use the best camera parameters to perform position fusion shooting on the target area to obtain a night-time shooting image.

[0148] In the embodiment of the present invention, the best camera parameters refer to the parameters obtained by optimizing the basic parameters required for configuring the camera system, and the best parameters obtained by optimizing the parameters initially set for shooting the target area, including exposure parameters, focal length parameters, sensitivity parameters, irradiation angle, and illumination intensity.

[0149] In the embodiment of the present invention, the adjusting the initial camera parameters through the camera parameter control factor and the preset parameter loop condition to obtain the best camera parameters includes:

[0150] When the camera parameter control factor is the first factor, adjust the initial camera parameters according to the preset camera parameter adjustment window;

[0151] Calculate the clarity of the night-time fusion image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter loop condition;

[0152] When the clarity is greater than or equal to the image clarity threshold in the parameter loop condition, regard the adjusted initial camera parameters as the best camera parameters.

[0153] When the camera parameter control factor is the second factor, determine the initial camera parameters as the best camera parameters.

[0154] Specifically, when the camera parameter control factor is the first factor, it indicates that the camera parameters need to be optimized. Then, the initial camera parameters are adjusted according to the pre-customized camera parameter adjustment window. For example, if the camera parameter adjustment window is set to 2, when the camera parameters need to be adjusted, the parameter values corresponding to the exposure parameter, focal length parameter, sensitivity parameter, irradiation angle, and illumination intensity are incremented by 2 in sequence. Then, shooting and image fusion are performed according to the adjusted camera parameters, and the clarity of the night fusion image is calculated. The camera parameters are continuously adjusted through iterative cycles, and the clarity of the captured night fusion image is calculated until the clarity is greater than or equal to the preset image clarity threshold. When the clarity is greater than or equal to the preset image clarity threshold, the adjusted initial camera parameters are used as the optimal camera parameters. When the camera parameter control factor is the second factor, it indicates that the camera parameters do not need to be optimized, and the initial camera parameters are used as the optimal camera parameters.

[0155] Furthermore, the target area can be photographed according to the optimized optimal camera parameters, thereby obtaining a captured image with the best clarity.

[0156] In the embodiment of the present invention, the night captured image refers to a night image obtained by fusing a visible light image and an infrared image captured based on the optimal camera parameters.

[0157] In the embodiment of the present invention, the position fusion shooting of the target area using the optimal camera parameters to obtain a night captured image includes:

[0158] Generating an optimal visible light image using the optimal camera parameters at the camera position;

[0159] Generating an optimal infrared image using the optimal camera parameters at the infrared light source position;

[0160] Fusing the optimal visible light image and the optimal infrared image to obtain a night captured image.

[0161] Specifically, at the camera position of the target area, shooting is performed using the preset optimal camera parameters. At the infrared light source position, the same optimal camera parameters are also used for shooting. The obtained optimal visible light image and optimal infrared image are fused to provide a richer and more detailed night shooting result. By using the optimal camera parameters and light source configuration, clearer and more detailed images can be obtained under night conditions, while combining the advantages of visible light and infrared images to enhance the visual analysis ability and recognition ability of the target area.

[0162] Specifically, the step of performing image fusion on the best visible light image and the best infrared image to obtain a night-shot image is the same as the step in S3 of performing image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image, which will not be elaborated here.

[0163] In the embodiment of the present invention, by extracting the environmental brightness data of the target area, the camera position and the infrared light source position can be accurately determined, ensuring that the camera can capture the image of the target area at the best angle and distance in the night environment; using the initial camera parameters, the visible light image and the infrared image of the target area are respectively generated. This helps to obtain detailed information under different spectra, where the visible light image provides visual information, while the infrared image reveals the heat distribution and other hidden details; by the exposure weight algorithm and the image sharpness algorithm, the visible light image and the infrared image are fused, taking advantage of the two spectral images to produce a richer and clearer night fusion image, which helps to improve the visualization and recognition ability of the target area; based on the generated night fusion image, by adjusting the initial camera parameters and the preset parameter loop condition, the best camera parameters are calculated, ensuring that the image of the target area can be captured under the most optimized conditions, improving the quality and sharpness of the image; using the best camera parameters for position fusion shooting to obtain the final night-shot image. The integration and optimization of these steps enable more accurate and clearer images to be obtained under night conditions. Therefore, the night image shooting method, device, electronic device and computer-readable storage medium based on infrared assistance proposed by the present invention can solve the problem of low clarity when taking night images.

[0164] As Figure 4 shown, it is a functional module diagram of a night image shooting device based on infrared assistance provided by an embodiment of the present invention.

[0165] The night image shooting device 100 based on infrared assistance described in the present invention can be installed in an electronic device. According to the functions achieved, the night image shooting device 100 based on infrared assistance may include a position determination module 101, an image generation module 102, an image fusion module 103, a camera parameter control factor generation module 104 and a night-shot image generation module 105. The modules described in the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0166] In this embodiment, the functions of each module / unit are as follows:

[0167] The position determination module 101 is configured to obtain a target area captured at night, extract ambient brightness data of the target area, and determine the camera position and the infrared light source position of the target area according to the ambient brightness data;

[0168] The image generation module 102 is configured to extract initial camera parameters of the target area, generate a visible light image of the target area according to the initial camera parameters and the camera position, and generate an infrared image of the target area according to the initial camera parameters and the infrared light source position;

[0169] The image fusion module 103 is configured to calculate a first image weight of the visible light image through a preset exposure weight algorithm, calculate a second image weight of the infrared image, and perform image fusion on the visible light image and the infrared image according to the first image weight and the second image weight to obtain a night fusion image;

[0170] The camera parameter control factor generation module 104 is configured to calculate the sharpness of the night fusion image by using a preset image sharpness algorithm, and generate a camera parameter control factor of the target area according to the sharpness;

[0171] The night captured image generation module 105 is configured to adjust the initial camera parameters through the camera parameter control factor and a preset parameter loop condition to obtain optimal camera parameters, and perform position fusion shooting on the target area by using the optimal camera parameters to obtain a night captured image.

[0172] Specifically, each module in the infrared-assisted night image capturing device 100 in the embodiments of the present invention adopts the same technical means as those in the Figures 1 to 3 above-described infrared-assisted night image capturing method, and can produce the same technical effects, which will not be elaborated here.

[0173] In several embodiments provided by the present invention, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0174] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, in each embodiment of the present invention, each functional module can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0176] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms.

[0177] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is not limited only by the above description. Therefore, it is intended to include all changes within the meaning and scope of equivalent elements that fall within the scope of protection of the present invention.

[0178] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology and application device that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results.

[0179] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described in the apparatus claims can also be implemented by one unit or device through software or hardware. The terms such as "first" and "second" are used to represent names and do not represent any specific order.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for nighttime image capture based on infrared assistance, characterized in that: The method comprises: Acquire a target area for night photography, extract environmental brightness data of the target area, and determine a camera position and an infrared light source position of the target area according to the environmental brightness data, including: identifying a first area center position of the target area, and determining the camera position of the target area according to a preset camera distance and the first area center position; identifying a dark area of ​​the target area according to the environmental brightness data, and identifying a second area center position of the dark area; determining an infrared distance between an infrared light source and the second area center position according to preset infrared light source properties, and determining an infrared out-point through the infrared distance and the camera distance; and determining the infrared out-point as the infrared light source position of the target area; Extracting initial imaging parameters of the target area, generating a visible light image of the target area according to the initial imaging parameters and the imaging position, and generating an infrared image of the target area according to the initial imaging parameters and the infrared light source position; Calculating a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a night fused image; The definition of the night fusion image is calculated using a preset image definition algorithm, including: extracting a highlight image region according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of ​​the highlight image region and identifying the center position of the highlight image region; determining a first clear region according to the center position and a preset region radius; filtering invalid pixels in the first clear region according to a preset second brightness threshold to obtain a second clear region; performing a difference operation between the first clear region and the second clear region to obtain a target clear region; and calculating the target definition of the target clear region using the following preset image definition algorithm: Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point; Determine the target definition as the definition of the nighttime fused image, and generate a camera parameter control factor of the target area according to the definition; The initial imaging parameters are adjusted by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters, and the target area is photographed by position fusion using the optimal imaging parameters to obtain a nighttime photographic image.

2. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The extracting initial imaging parameters of the target area includes: Determining the camera lens angle of view according to the camera position of the target area; Extracting exposure parameters, focal length parameters and sensitivity parameters of the camera corresponding to the target area; Extracting the irradiation angle and illumination intensity of the infrared light source corresponding to the target area; The camera lens viewing angle, the exposure parameter, the focal length parameter, the sensitivity parameter, the illumination angle and the illumination intensity are determined as initial camera parameters of the target area.

3. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: Generating the visible light image of the target area according to the initial imaging parameters and the imaging position includes: Focusing the light corresponding to the target area on the photosensitive element of the camera according to the camera position and the initial camera parameters; The focused light is converted into an electronic signal through the photosensitive unit in the photosensitive element; Performing multiple signal enhancement processing on the electronic signal, and encoding the electronic signal after the signal enhancement processing into a digital image format; A visible light image of the target area is generated according to the digital image format.

4. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The calculating the first image weight of the visible light image by using a preset exposure weight algorithm includes: Extracting camera parameters corresponding to the visible light image; Counting the exposure of the visible light image, and determining an image enhancement factor of the visible light image according to the exposure; The first image weight of the visible light image is calculated according to the camera parameters, the exposure level and the image enhancement factor by using the following preset exposure weight algorithm: Wherein, ω is the first image weight, μ is the image enhancement factor, argmax is the maximum value function, e is a constant, b is the first fixed parameter in the camera parameters, a is the second fixed parameter in the camera parameters, P i is the pixel value of the visible light image i, K i is the exposure degree corresponding to the visible light image i.

5. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of fusing the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image includes: The visible light image is subjected to a convolution kernel operation through a preset Gaussian filter to obtain a Gaussian pyramid visible light image, and the infrared image is subjected to a convolution kernel operation to obtain a Gaussian pyramid infrared image; Performing a difference operation between two adjacent layers of the Gaussian pyramid visible light image to obtain a Gaussian visible light difference image, and performing a difference operation between two adjacent layers of the Gaussian pyramid infrared image to obtain a Gaussian infrared difference image; Counting the visible light feature points of the Gaussian visible light difference image, and counting the infrared feature points of the Gaussian infrared difference image; Registering the visible light feature point with the infrared feature point to obtain a registered feature point; Image fusion is performed according to the first image weight, the second image weight and the registration feature points to obtain a night fusion image, wherein the image fusion calculation formula is: R=ω1×I(x,y)+ω2×g(I(f(x,y))) Among them, R is the grayscale value of the night fusion image, ω1 is the weight of the first image, ω2 is the weight of the second image, I(x, y) is the grayscale value of the registration feature point at the position (x, y), f(x, y) is the coordinate transformation of the position (x, y) in the two-dimensional space of the image, and g is the radiation transformation function.

6. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of generating the imaging parameter control factor of the target area according to the definition includes: When the clarity is less than a preset clarity threshold, determining the camera parameter control factor of the target area as a first factor; When the clarity is greater than or equal to a preset clarity threshold, the camera parameter control factor of the target area is determined as a second factor.

7. The infrared-assisted nighttime image shooting method according to claim 5, characterized in that: The step of adjusting the initial imaging parameters by the imaging parameter control factor and the preset parameter cycle condition to obtain the optimal imaging parameters includes: When the camera parameter control factor is the first factor, adjusting the initial camera parameter according to a preset camera parameter adjustment window; Calculating the clarity of the nighttime fused image according to the adjusted initial camera parameters until the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition; When the clarity is greater than or equal to the image clarity threshold in the parameter cycle condition, the adjusted initial imaging parameters are used as optimal imaging parameters; When the imaging parameter control factor is the second factor, the initial imaging parameter is determined as the optimal imaging parameter.

8. The infrared-assisted nighttime image shooting method according to claim 1, characterized in that: The step of performing position fusion shooting of the target area using the optimal shooting parameters to obtain a nighttime shooting image includes: Generating an optimal visible light image at the imaging position using the optimal imaging parameters; Generating an optimal infrared image using the optimal imaging parameters at the position of the infrared light source; The best visible light image and the best infrared image are fused to obtain a nighttime image.

9. A night image shooting device based on infrared assistance, characterized in that: Used to execute the infrared-assisted nighttime image shooting method according to any one of claims 1 to 8, the device comprising: A position determination module, used for acquiring a target area for night photography, extracting ambient brightness data of the target area, and determining a camera position and an infrared light source position of the target area according to the ambient brightness data, including: identifying a first area center position of the target area, and determining the camera position of the target area according to a preset camera distance and the first area center position; identifying a dark area of ​​the target area according to the ambient brightness data, and identifying a second area center position of the dark area; determining an infrared distance between an infrared light source and the second area center position according to preset infrared light source properties, and determining an infrared out-point through the infrared distance and the camera distance; and determining the infrared out-point as the infrared light source position of the target area; An image generation module, used to extract initial imaging parameters of the target area, generate a visible light image of the target area according to the initial imaging parameters and the imaging position, and generate an infrared image of the target area according to the initial imaging parameters and the infrared light source position; an image fusion module, configured to calculate a first image weight of the visible light image and a second image weight of the infrared image by using a preset exposure weight algorithm, and fuse the visible light image with the infrared image according to the first image weight and the second image weight to obtain a nighttime fused image; The camera parameter control factor generation module is used to calculate the clarity of the night fusion image using a preset image clarity algorithm, including: extracting a highlight image area according to the brightness of the night fusion image and a preset first brightness threshold; calculating the area of ​​the highlight image area and identifying the center position of the highlight image area; determining a first clear area according to the center position and a preset area radius; filtering invalid pixels in the first clear area according to a preset second brightness threshold to obtain a second clear area; performing a difference operation between the first clear area and the second clear area to obtain a target clear area; and calculating the target clarity of the target clear area using the following preset image clarity algorithm: Wherein, Q is the target clarity, s is the area of ​​the region, u is the average grayscale value of the target clear region, f(l,d) is the grayscale value corresponding to each pixel point (l,d) in the target clear region, n is the width of the target clear region, m is the length of the target clear region, l is the horizontal coordinate position of the pixel point, and d is the vertical coordinate position of the pixel point; Determine the target definition as the definition of the nighttime fused image, and generate a camera parameter control factor of the target area according to the definition; The night-time image generation module is used to adjust the initial image parameters through the image parameter control factor and the preset parameter cycle condition to obtain the optimal image parameters, and use the optimal image parameters to perform position fusion shooting on the target area to obtain the night-time image.

Images