Image fusion method, terminal and storage medium

Abstract

The embodiment of the present application relates to the field of image processing technology, and discloses an image fusion method, terminal and storage medium. In the present application, a plurality of exposure parameters are used to continuously shoot the target scene; the exposure parameters corresponding to the first picture frame shot for the first time and the second picture frame shot for the last time are the same; since the first picture frame shot for the first time and the second picture frame shot for the last time can cover the motion area of ​​the intermediate picture frame with a high probability, determining the motion area according to the first picture frame and the second picture frame reduces the workload of calculation, so that the pixels at the same position in the motion area of ​​multiple picture frames are subsequently fused to obtain fused pixel values; the weight of the motion area in each picture frame is set according to the pixel value of each pixel point in the motion area of ​​each picture frame and the fused pixel value corresponding to each pixel point; the speed of fusing the motion area according to the weight of the motion area in each picture frame is faster, that is, the speed of removing ghost images is faster, thereby meeting the real-time requirements.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular to an image fusion method, terminal, and storage medium. Background Art

[0002] High-Dynamic Range (HDR) imaging is a technique that obtains low-dynamic range (LDR) images of different exposures and fuses them together to create a final HDR image. However, due to camera shake, object motion in the scene, and other factors, ghosting may occur in the HDR image obtained by direct fusion. Currently, in the process of fusing LDR images of different exposures to obtain HDR images, gradient domain-based correlation algorithms or irradiance domain-based correlation algorithms are mainly used to eliminate ghosting. However, both methods have the problem of high computational complexity and cannot meet real-time requirements. Summary of the Invention

[0003] The embodiments of the present application provide an image fusion method, a terminal, and a storage medium, which can reduce the computational workload and make ghost removal faster, thereby meeting real-time requirements.

[0004] To achieve the above-mentioned purpose, an embodiment of the present application provides an image fusion method, including: continuously shooting a target scene using multiple exposure parameters to obtain multiple picture frames; wherein the exposure parameters corresponding to the first picture frame shot for the first time and the second picture frame shot for the last time are the same; determining the motion area in the target scene based on the first picture frame and the second picture frame; fusing pixel points at the same position in the motion area of ​​each of the multiple picture frames to obtain fused pixel values ​​corresponding to the pixel points at the same position; setting the weight of the motion area in each of the picture frames based on the pixel value of each pixel point in the motion area of ​​each of the picture frames and the fused pixel value corresponding to each pixel point; and fusing the motion areas in the multiple picture frames based on the weight of the motion area in each of the picture frames.

[0005] To achieve the above-mentioned purpose, an embodiment of the present application also provides a terminal, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image fusion method described above.

[0006] To achieve the above-mentioned purpose, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned image fusion method when executed by a processor.

[0007] The image fusion method, terminal, and storage medium proposed in the present application use multiple exposure parameters to continuously shoot a target scene to obtain multiple picture frames; wherein, the exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are the same; based on the first picture frame and the second picture frame taken for the last time, the motion area in the target scene is determined; since the first picture frame obtained for the first time and the second picture frame obtained for the last time can cover the motion area in the intermediate picture frame with a high probability, the motion area in the target scene can be determined only based on the first picture frame obtained for the first time and the second picture frame obtained for the last time, that is, there is no need to determine the motion area based on all picture frames, which reduces the computational workload and enables the subsequent fusion of pixels at the same position in the motion areas of multiple picture frames to obtain the fused pixel values ​​corresponding to the pixels at the same position; the weight of the motion area in each picture frame is set based on the pixel value of each pixel in the motion area of ​​each picture frame and the fused pixel value corresponding to each pixel; based on the weight of the motion area in each picture frame, the speed of fusion of the motion areas in multiple picture frames is faster, that is, the speed of removing ghost images is faster, thereby meeting the real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a flowchart of the image fusion method according to the first embodiment of the present application;

[0009] Figure 2 is a flowchart of a specific implementation method of step 103 in the first embodiment of the present application;

[0010] Figure 3 is a flowchart of a specific implementation method of step 104 in the first embodiment of the present application;

[0011] Figure 4 is a flowchart of the image fusion method according to the second embodiment of the present application;

[0012] Figure 5 is a flowchart of a specific implementation of step 202 in the second embodiment of the present application;

[0013] Figure 6 is a flowchart of a specific implementation of step 204 in the second embodiment of the present application;

[0014] Figure 7 is a flowchart of the image fusion method according to the third embodiment of the present application;

[0015] Figure 8 It is a structural diagram of a terminal according to the fourth embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0017] The first embodiment of the present application relates to an image fusion method, which is applied to a terminal, such as a mobile phone, a tablet computer, etc. The specific process is as follows: Figure 1 As shown, including:

[0018] Step 101 : continuously photographing a target scene using a plurality of exposure parameters to obtain a plurality of picture frames; wherein the exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are the same.

[0019] Specifically, within a short shooting time, the terminal uses a camera to continuously shoot a target scene using multiple exposure parameters to obtain multiple image frames. The exposure parameters used in the first and last shots of the continuous shooting are the same, that is, the exposure parameters corresponding to the first image frame of the first shooting and the second image frame of the last shooting are the same; wherein the exposure parameters include one of the following or any combination thereof: exposure time, exposure level. The clear positions of the image frames shot under different exposure parameters are different. For example, when the exposure time is short, the highlight position of the target scene is clearer; when the exposure time is long, the dark position of the target scene is clearer.

[0020] In one example, the exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are medium exposure parameters. Specifically, exposure parameters can be divided into high exposure parameters, medium exposure parameters, and low exposure parameters based on exposure time and / or exposure level. The exposure parameters used for the first and last pictures are medium exposure parameters, that is, the exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are medium exposure parameters. Because the medium exposure parameter is a relatively appropriate exposure parameter, setting the exposure parameters for the first and last pictures to the medium exposure parameter can more accurately determine the motion area.

[0021] Step 102: Determine a motion region in the target scene according to the first image frame and the second image frame.

[0022] Specifically, since the target scene contains moving objects such as pedestrians, vehicles, and animals, the area where the moving objects move during the capture time is the moving area. Since the first image frame obtained by the first capture and the second image frame obtained by the last capture are likely to cover the moving area in the intermediate image frame, the moving area in the target scene can be determined based solely on the first image frame obtained by the first capture and the second image frame obtained by the last capture. In this embodiment, the terminal subtracts the pixel values ​​of the pixels at the same position in the first image frame obtained by the first capture and the second image frame obtained by the last capture, then takes the absolute value of the difference, and determines a differential image based on the absolute value of the difference, wherein the absolute value of the difference is used as the pixel value of the pixel in the differential image. Then, from the pixels in the differential image, target pixels whose pixel values ​​are greater than a preset threshold are selected. Based on connectivity, the areas where the connected target pixels are located are connected as a moving area, and the areas where the target pixels are located that cannot be connected are treated as another area, thereby obtaining the moving area in the target scene. The preset threshold can be set according to actual needs and is not specifically limited in this embodiment.

[0023] In one example, determining a motion region in a target scene based on a first image frame and a second image frame includes: performing registration processing on the first image frame and the second image frame to obtain a registered first image frame and a registered second image frame; and determining the motion region in the target scene based on the registered first image frame and the registered second image frame. Specifically, registration processing refers to matching the geographic coordinates of different image graphics obtained by different imaging methods within the same area. Therefore, the registered first image frame and the registered second image frame obtained after the registration processing can eliminate the impact of the position difference between the first image frame and the second image frame caused by reasons such as hand shaking. That is, the terminal first performs registration processing on the first picture frame obtained by the first shooting and the second picture frame obtained by the last shooting, respectively, to obtain the first picture frame after registration and the second picture frame after registration, and then subtracts the pixel values ​​of the pixel points at the same position in the first picture frame after registration and the second picture frame after registration, and then takes the absolute value of the difference, and determines the differential image according to the absolute value of the difference, wherein the absolute value of the difference is used as the pixel value of the pixel point of the differential image; then selects the target pixel points whose pixel values ​​are greater than the preset threshold from the pixel points of the differential image, and connects the areas where the connected target pixel points are located as a motion area according to the connectivity, and regards the areas where the target pixel points are located that cannot be connected as another area to obtain the motion area in the target scene; wherein the preset threshold can be set according to actual needs, and this embodiment does not make a specific limitation.

[0024] Step 103 : fusion of pixels at the same position in the motion regions of the plurality of picture frames to obtain fused pixel values ​​corresponding to the pixels at the same position.

[0025] In one example, the specific flow chart of fusing pixels at the same position in the motion regions of multiple image frames to obtain the fused pixel values ​​corresponding to the pixels at the same position is as follows: Figure 2 As shown, including:

[0026] Step 1031, determining the weights of the pixel points at the same position in the motion regions of the multiple picture frames based on the attribute information of the pixel points at the same position in the motion regions of the multiple picture frames; wherein the attribute information includes one of the following or any combination thereof: brightness, saturation, and contrast.

[0027] Step 1032 , performing weighted fusion on the pixels at the same position in the motion regions of the multiple picture frames according to the weights of the pixels at the same position in the motion regions of the multiple picture frames, and obtaining fused pixel values ​​corresponding to the pixels at the same position.

[0028] Specifically, according to the algorithm pre-set in the terminal, the preset algorithm is used to determine the weights of the pixels at the same position in the motion area of ​​each picture frame based on the attribute information of the pixels at the same position in the motion area of ​​each picture frame. The sum of the weights of the pixels at the same position in the motion area of ​​each picture frame is 1, and the attribute information includes one of the following or any combination thereof: brightness, saturation, contrast. Then, based on the weights of the pixels at the same position in the motion area of ​​each picture frame, the pixel values ​​of the pixels at the same position in each picture frame are weighted and the result of the weighted calculation is used as the fused pixel value corresponding to the pixel at the same position. The attribute information of the pixel point can be used to more accurately determine the weight of the pixel point, thereby more accurately obtaining the corresponding fused pixel value. The following will be explained with an example: If there are 4 picture frames, there are 4 pixels in the motion area K,

[0029] Corresponding to the position of pixel 1, the pixels in the four picture frames are P1-1, P2-1, P3-1, and P4-1.

[0030] Corresponding to the position of pixel 2, the pixels in the four picture frames are P1-2, P2-2, P3-2, and P4-2.

[0031] Corresponding to the position of pixel 3, the pixels in the four picture frames are P1-3, P2-3, P3-3, and P4-3.

[0032] Corresponding to the position of pixel 4, the pixels in the four picture frames are P1-4, P2-4, P3-4, and P4-4 respectively.

[0033] The pixel values ​​of pixel 1 in the four image frames are A1, A2, A3, and A4 respectively, and the weights are a1, a2, a3, and a4 respectively. The fused pixel value is: M1 = A1*a1+A2*a2+A3*a3+A4*a4,

[0034] The pixel values ​​of pixel 2 in the four image frames are B1, B2, B3, and B4 respectively, and the weights are b1, b2, b3, and b4 respectively. The fused pixel value is: M2 = B1*b1+B2*b2+B3*b3+B4*b4,

[0035] The pixel values ​​of pixel 3 in the four frames are C1, C2, C3, and C4 respectively, and the weights are c1, c2, c3, and c4 respectively. The fused pixel value is: M3 = C1*c1+C2*c2+C3*c3+C4*c4,

[0036] The pixel values ​​of pixel 4 in the 4 frames are D1, D2, D3, and D4 respectively, and the weights are d1, d2, d3, and d4 respectively. The fused pixel value is: M4 = D1*d1+D2*d2+D3*d3+D4*d4.

[0037] Step 104 : setting a weight of the motion region in each picture frame according to the pixel value of each pixel point in the motion region of each picture frame and the fused pixel value corresponding to each pixel point.

[0038] In one example, when there are multiple motion regions, the specific flow chart for setting the weight of the motion region in each picture frame according to the pixel value of each pixel point in the motion region of each picture frame and the fusion pixel value corresponding to each pixel point is as follows: Figure 3 As shown, including:

[0039] Step 1041 traverses each motion region, calculates the absolute value of the difference between the pixel value of each pixel point in the motion region currently traversed in each picture frame and the fused pixel value corresponding to each pixel point, and calculates the sum of the absolute values ​​of the differences in each picture frame.

[0040] Step 1042 : setting the weight of the currently traversed motion region in each picture frame according to the sum of the absolute values ​​of the differences.

[0041] Specifically, when there are multiple motion areas, the weights of each motion area in the same picture frame may be different, and the weights of each motion area in each picture frame need to be determined separately. For example: there are two motion areas, K and L, and the weight of motion area K in the first frame is 1, and the weight in the other three frames is 0. The weight of motion area L in the second frame is 1, and the weight in the other three frames is 0. The following will take the motion area K as an example to illustrate how to set the weight of the motion area K in each picture frame. For example: if there are 4 picture frames, there are 4 pixels in the motion area K, the pixel values ​​of pixel 1 in the 4 picture frames are A1, A2, A3, and A4 respectively, and the fused pixel value corresponding to pixel 1 is M1, the pixel values ​​of pixel 2 in the 4 picture frames are B1, B2, B3, and B4 respectively, and the fused pixel value corresponding to pixel 2 is M2, the pixel values ​​of pixel 3 in the 3 frames of pictures are C1, C2, C3, and C4 respectively, and the fused pixel value corresponding to pixel 3 is M3, the pixel values ​​of pixel 4 in the 4 frames of pictures are D1, D2, D3, and D4 respectively, and the fused pixel value corresponding to pixel 4 is M3, calculate the absolute value of the difference between the pixel value of each pixel in the motion area K of each picture frame and the fused pixel value corresponding to each pixel, and calculate the sum of the absolute values ​​of the differences in each picture frame, that is, the LI norm:

[0042] The L1 norm of the motion region K in the first frame is: F1 = |A1-M1|+|B1-M2|+|C1-M3|+|D1-M4|,

[0043] The L1 norm of the motion region K in the second frame is: F2 = |A2-M1|+|B2-M2|+|C2-M3|+|D2-M4|,

[0044] The L1 norm of the motion region K in the third frame is: F3 = |A3-M1|+|B3-M2|+|C3-M3|+|D3-M4|,

[0045] The L1 norm of the motion region K in the fourth frame is: F4 = |A4-M1|+|B4-M2|+|C4-M3|+|D4-M4|,

[0046] By comparing the L1 norms F1, F2, F3, and F4, the motion region in the image frame corresponding to the smallest L1 norm can be selected and its weight set to 1, while the weights of the motion regions in the remaining image frames are set to 0. Alternatively, the motion region in the image frame corresponding to the smallest L1 norm can be selected and its weight set to 0.7, while the weights of the motion regions in the remaining image frames are set to 0.1. That is, the setting rules can be pre-set according to actual needs and are not specifically limited in this embodiment. The sum of the absolute values ​​of the differences calculated in this way can reflect the quality of the motion regions in each image frame, that is, setting the weights of the motion regions in each image frame based on the absolute values ​​of the differences is more accurate.

[0047] Step 105 : Fusing the motion regions in the plurality of picture frames according to the weights of the motion regions in the respective picture frames.

[0048] Specifically, in the process of image fusion of each picture frame, for the motion area, the motion area is fused according to the weight of the motion area in each picture frame, that is, the pixel points at the same position in each picture frame are weightedly fused to obtain the pixel values ​​of the pixel points at the same position in the high dynamic range image, that is, the motion area in the high dynamic range image is completed. For example, if the fusion of the motion area in the picture frame corresponding to the smallest L1 norm is selected, its weight is set to 1, and the weights of the motion areas in the remaining picture frames are set to 0. If F1 in the first frame is the smallest, then in the process of image fusion of the motion areas of each picture frame, the motion area K directly uses the image block of the motion area in the first frame. For non-motion areas, which are areas other than the motion area in the picture frame, the weights of the pixels at the same position in the non-motion areas of each picture frame are determined based on the attribute information of the pixels at the same position in the non-motion areas of each picture frame. Based on the weights of the pixels at the same position in the non-motion areas of each picture frame, the pixels at the same position in each picture frame are weightedly fused to obtain the pixel values ​​of the pixels at the same position in the high dynamic range image, thereby completing the fusion of the non-motion areas in the high dynamic range image and obtaining the high dynamic range image.

[0049] In this embodiment, a target scene is continuously photographed using multiple exposure parameters to obtain multiple image frames. The exposure parameters corresponding to the first image frame and the second image frame taken last are the same. A motion region in the target scene is determined based on the first and second image frames. Since the first image frame taken first and the second image frame taken last are likely to cover the motion region in the intermediate image frames, the motion region in the target scene can be determined based solely on the first image frame taken first and the second image frame taken last. This means that the motion region does not need to be determined based on all image frames, which reduces the computational workload and enables subsequent fusion of pixels at the same position in the motion regions of multiple image frames to obtain fused pixel values ​​corresponding to the pixels at the same position. A weight of the motion region in each image frame is set based on the pixel value of each pixel in the motion region of each image frame and the fused pixel value corresponding to each pixel. Based on the weight of the motion region in each image frame, the motion regions in the multiple image frames are fused more quickly, i.e., ghosting is removed more quickly, thereby meeting real-time requirements.

[0050] The second embodiment of the present application relates to an image fusion method. The second embodiment is substantially the same as the first embodiment, except that: a first differential image is first calculated, and then the first differential image is normalized to obtain a first normalized image, and the first normalized image is divided into blocks and then calculated. The specific flow chart is as follows: Figure 4 As shown, including:

[0051] In step 201, a target scene is continuously photographed using a variety of exposure parameters to obtain multiple image frames. The exposure parameters corresponding to the first image frame taken initially and the second image frame taken last are the same. Step 201 is similar to step 101 in the first embodiment and will not be further described here.

[0052] Step 202: Perform differential processing on the first picture frame and the second picture frame to obtain a first differential image.

[0053] Specifically, the pixel points at the same position in the first picture frame obtained by the first shooting and the second picture frame obtained by the last shooting are subtracted, and then the absolute value of the difference is taken, and the first differential image is determined based on the absolute value of the difference, wherein the absolute value of the difference is used as the value of the pixel point of the first differential image.

[0054] In one example, a specific flow chart of performing differential processing on the first picture frame and the second picture frame to obtain a first differential image is as follows: Figure 5 As shown, including:

[0055] Step 2021 : Perform Gaussian downsampling on the first picture frame and the second picture frame respectively to obtain a Gaussian downsampled first picture frame and a Gaussian downsampled second picture frame.

[0056] Step 2022 : performing difference processing on the first image frame after Gaussian downsampling and the second image frame after Gaussian downsampling to obtain a first difference image.

[0057] Specifically, Gaussian downsampling is used to downsample image frames. The image frames are first Gaussian blurred, and then the even-numbered rows and columns of the image frames are deleted, thereby obtaining the subsampled image frames. The first image frame, obtained by the initial capture, is Gaussian blurred, and then the even-numbered rows and columns of the image frame are deleted, thereby obtaining the Gaussian-downsampled first image frame. The second image frame, obtained by the final capture, is then Gaussian blurred, and then the even-numbered rows and columns of the image frame are deleted, thereby obtaining the Gaussian-downsampled second image frame. Pixels at the same position in the Gaussian-downsampled first and second frames are then subtracted, and the absolute value of the difference is taken. A first differential image is determined based on the absolute value of the difference. This method can reduce the impact of noise in the first and second frames, as well as the positional differences between the first and second frames. Furthermore, performing differential processing on the Gaussian-downsampled first and second frames can further reduce computational workload.

[0058] In one example, a first image frame and a second image frame are first registered to obtain a first image frame and a second image frame after the registration process; Gaussian downsampling is then performed on the first image frame and the second image frame after the registration process to obtain a first downsampled Gaussian image frame and a second downsampled Gaussian image frame; and a difference process is then performed on the first downsampled Gaussian image frame and the second downsampled Gaussian image frame to obtain a first difference image. By performing the registration process on the first image frame obtained by the first capture and the second image frame obtained by the last capture, the impact of the positional difference between the first image frame and the second image frame can be reduced.

[0059] Step 203 : performing normalization processing on the first differential image according to each pixel point of the first differential image to obtain a first normalized image.

[0060] Specifically, the average value of the pixel values ​​of each pixel in the first differential image is calculated, and the variance is calculated based on the average value. For each pixel, the absolute value of the difference between the pixel value of the pixel and the average value is calculated, and then the ratio of the absolute value to the variance is calculated. The ratio is used as the pixel value of the corresponding pixel in the first standardized image, so that the pixel values ​​of the obtained first standardized image satisfy the normal distribution with a mean of 0 and a standard deviation of 1, that is, the pixels of the first standardized image obey the standard normal distribution. For example: the pixel values ​​of the first differential image are E1, E2, E3, and E4 respectively, then the average value of these pixels is calculated as F = (E1+E2+E3+E4) / 4, and then the variance is calculated as I = [(E1-F) 2 +(E2-F) 2 +(E3-F) 2 +E4-F)2 ] / 4, and then perform normalization processing to obtain the pixel values ​​corresponding to the first standardized image as follows: (|E1-F|) / I, (|E2-F|) / I, (|E3-F|) / I, (|E4-F|) / I.

[0061] Step 204, entering the determination step, which includes: dividing the first standardized image into multiple image blocks, and calculating the sum of the squares of the pixels in each image block; selecting an image block that meets a first preset condition, and determining the motion area in the target scene based on the selected image block; wherein the first preset condition is that the sum of the squares of the pixels in the image block is greater than or equal to a preset threshold.

[0062] Specifically, the terminal can divide the first standard image into multiple image blocks according to a preset ratio. The preset ratio can be set according to actual needs. For example, the first standard image can be divided into 16*16 blocks or 64*64 blocks. For each image block, the sum of the squares of the pixel values ​​of the pixels in each image block is calculated to obtain the sum of the squares of the pixel values ​​of each image block. Since each pixel follows a standard normal distribution, the sum of the squares of the pixel values ​​follows a chi-square distribution. Then, an image block is selected in which the sum of squares of pixel values ​​is greater than or equal to a preset threshold, where the preset threshold may be the upper quantile of Φ=0.99; the connected image blocks are connected according to connectivity as one motion region, and the image blocks that cannot be connected are regarded as another motion region, to obtain the motion region in the target scene; for example, the first standard image is divided into blocks according to a 16*16 ratio to obtain multiple image blocks, the sum of squares of pixels in each image block is calculated respectively, and the sum of squares of pixels in the second image block in the first row, the third image block in the first row, the second image block in the second row, the fifth image block in the fourth row, and the sixth image block in the eighth row are obtained to meet the first preset condition, and according to connectivity, the second image block in the first row, the third image block in the first row, and the second image block in the second row are connected, then the region is regarded as one motion region, the fifth image block in the fourth row is regarded as another region, and the sixth image block in the eighth row is regarded as another region, to obtain the motion regions in the target scene.

[0063] In one example, a specific flow chart of dividing the first standardized image into blocks to obtain multiple image blocks is as follows: Figure 6 As shown, including:

[0064] Step 2041 : Perform mean blur processing on the first standardized image to obtain a standardized image after mean blur processing.

[0065] Step 2042 : performing block processing on the normalized image after the mean blurring process to obtain a plurality of image blocks.

[0066] Specifically, for each pixel, a preset number of pixels above, below, to the left, and to the right of the pixel are selected. The sum of the pixel values ​​of the selected pixels is calculated and used as the pixel value of the pixel. After traversing each pixel, a normalized image after mean blurring is obtained. The normalized image after mean blurring is then divided into multiple image blocks. This method can reduce the impact of positional differences between target image frames.

[0067] Step 205 : fusion of pixels at the same position in the motion regions of the plurality of picture frames to obtain fused pixel values ​​corresponding to the pixels at the same position.

[0068] Step 206 , setting the weight of the motion region in each picture frame according to the pixel value of each pixel point in the motion region of each picture frame and the fused pixel value corresponding to each pixel point.

[0069] Step 207 : Fusing the motion regions in the plurality of picture frames according to the weights of the motion regions in the respective picture frames.

[0070] Steps 205-207 are similar to steps 103-105 and will not be repeated here.

[0071] In this embodiment, the first differential image is first calculated, and then the first differential image is standardized to obtain a first standardized image, and the first standardized image is divided into blocks and then calculated. This can further reduce the workload of calculation, thereby further improving the speed of removing ghost images. Since the pixels in the first standardized image satisfy the standard normal distribution, the sum of the squares of the pixels satisfying the standard normal distribution will satisfy the chi-square distribution. The motion area in the target scene is then determined based on the sum of the squares of the pixels, which can improve the robustness of the algorithm.

[0072] The third embodiment of the present application relates to an image fusion method. The third embodiment is substantially the same as the second embodiment, except that: each motion region in the target scene needs to be reacquired. The specific flow chart is as follows: Figure 7 As shown, including:

[0073] Step 301 : Continuously photograph a target scene using a variety of exposure parameters to obtain a plurality of picture frames; wherein the exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are the same.

[0074] Step 302: Perform differential processing on the first picture frame and the second picture frame to obtain a first differential image.

[0075] Step 303 : performing normalization processing on the first differential image according to each pixel point of the first differential image to obtain a first normalized image.

[0076] Step 304, entering the determination step, which includes: dividing the first standardized image into multiple image blocks, and calculating the sum of the squares of the pixels in each image block; selecting an image block that meets a first preset condition, and determining the motion area in the target scene based on the selected image block; wherein the first preset condition is that the sum of the squares of the pixels in the image block is greater than or equal to a preset threshold.

[0077] Steps 301-304 are similar to steps 201-204 and will not be repeated here.

[0078] Step 305, judge whether the number of times the determination step is executed reaches the preset number. If it reaches the preset number, go to step 3010. If it does not reach the preset number, go to step 306. The preset number can be set according to actual needs, and the preset number is at least 2 times.

[0079] In step 306, local brightness correction is performed on the non-moving areas in the first picture frame and the second picture frame, respectively, and overall brightness correction is performed on the target picture frame that meets the second preset condition, to obtain a corrected first picture frame and a corrected second picture frame; wherein the non-moving area is the other area in the picture frame except the moving area, and the target picture frame includes the first picture frame after local brightness correction and / or the second picture frame after local brightness correction.

[0080] Specifically, local brightness correction is performed on the non-motion areas of the first and second picture frames, respectively. For example, a histogram matching algorithm can be used to perform local brightness correction, which can reduce the difference in pixel brightness between the first and second picture frames caused by hardware and other reasons. The overall brightness of the first and second picture frames after local brightness correction is then obtained, and it is determined whether the overall brightness meets a second preset condition. For example, if the overall brightness of the first and second picture frames after local brightness correction is less than a preset threshold, the target picture frame is the first picture frame after local brightness correction. If the overall brightness of the second picture frame after local brightness correction is less than the preset threshold, the target picture frame is the second picture frame after local brightness correction. If the overall brightness of the first and second picture frames after local brightness correction is less than the preset threshold, the target picture frame is the second picture frame after local brightness correction. If the overall brightness of both the first and second picture frames after local brightness correction is less than the preset threshold, the target picture frames are the first and second picture frames after local brightness correction. The target picture frames are then subjected to overall brightness correction. For example, a logarithmic method can be used for brightening. In this way, the corrected first and second picture frames can be obtained.

[0081] Step 307 : performing difference processing on the corrected first image frame and the corrected second image frame to obtain a second difference image.

[0082] Specifically, the pixel values ​​of the pixel points at the same position in the corrected first image frame and the corrected second image frame are subtracted, and then the absolute value of the difference is taken, and the second differential image is determined based on the absolute value of the difference, wherein the absolute value of the difference is used as the pixel value of the pixel point of the second differential image.

[0083] Step 308 : normalize the first differential image according to the pixels in the non-motion area of ​​the second differential image to obtain a second normalized image.

[0084] Specifically, due to the existence of the motion area, the result of the normalization processing of the differential image may not be very accurate, so the average value of the pixel values ​​of the pixel points in the non-motion area of ​​the second differential image is calculated, and the variance is calculated based on the average value. For each pixel point, the absolute value of the difference between the pixel value of the pixel point and the average value is calculated, and then the ratio of the absolute value to the variance is calculated. The ratio is used as the pixel value of the corresponding pixel point of the second standardized image, so that the value of the pixel point of the obtained second standardized image satisfies the normal distribution with a mean of 0 and a standard deviation of 1, that is, the pixel points of the second standardized image obey the standard normal distribution.

[0085] Step 309: Update the first standardized image to a second standardized image.

[0086] Step 3010: fusion of pixels at the same position in the motion regions of multiple image frames to obtain fused pixel values ​​corresponding to the pixels at the same position.

[0087] Step 3011 , according to the pixel value of each pixel point in the motion region of each picture frame and the fused pixel value corresponding to each pixel point, the weight of the motion region in each picture frame is set.

[0088] Step 3012: Fusing the motion regions in the plurality of picture frames according to the weights of the motion regions in the respective picture frames.

[0089] Steps 3010-3012 are similar to steps 103-105 and will not be repeated here.

[0090] In this embodiment, due to hardware reasons, there may be a slight difference in brightness between the first picture frame and the second picture frame, and the existence of motion areas in the first picture frame and the second picture frame may cause pixels of the first picture frame or the second picture frame to be retained in the non-motion area of ​​the first differential image. This will cause the result of the standardization processing of the first differential image to be not very accurate, thereby causing the determined motion area to be not very accurate. Through this method, the obtained second standardized image can be made more accurate, so that the motion area can be determined again, making the final motion area more accurate.

[0091] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0092] A fourth embodiment of the present invention relates to an electronic device, such as Figure 8 As shown, it includes at least one processor 402; and a memory 401 that is communicatively connected to the at least one processor; wherein the memory 401 stores instructions that can be executed by the at least one processor 402, and the instructions are executed by the at least one processor 402 to enable the at least one processor 402 to execute an embodiment of the above-mentioned road safety assessment method.

[0093] Memory 401 and processor 402 are connected using a bus. The bus may include any number of interconnected buses and bridges, connecting one or more processors 402 and various circuits of memory 401. The bus may also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and, therefore, will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 402 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to processor 402.

[0094] The processor 402 is responsible for managing the bus and general processing, and may also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 401 may be used to store data used by the processor 402 when performing operations.

[0095] A fifth embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0096] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0097] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. An image fusion method, characterized in that: include: Continuously photographing a target scene using a variety of exposure parameters to obtain a plurality of picture frames; wherein the exposure parameters corresponding to a first picture frame taken for the first time and a second picture frame taken for the last time are the same; determining a motion region in the target scene according to the first picture frame and the second picture frame; Performing local brightness correction on the non-moving areas in the first picture frame and the second picture frame, respectively, and performing overall brightness correction on the target picture frame that meets the second preset condition, to obtain a corrected first picture frame and a corrected second picture frame; wherein the non-moving area is an area of ​​the picture frame other than the moving area, and the target picture frame includes the first picture frame after the local brightness correction and / or the second picture frame after the local brightness correction; performing a step of determining a motion region in the target scene according to the corrected first image frame and the corrected second image frame until the step of determining a motion region in the target scene is performed a preset number of times; Fusing pixel points at the same position in the motion regions of the multiple image frames to obtain fused pixel values ​​corresponding to the pixel points at the same position; Setting a weight of the motion region in each picture frame according to the pixel value of each pixel point in the motion region of each picture frame and the fused pixel value corresponding to each pixel point; The motion regions in the multiple picture frames are fused according to the weights of the motion regions in the respective picture frames.

2. The image fusion method according to claim 1, characterized in that: The determining the motion area in the target scene according to the first picture frame and the second picture frame includes: performing differential processing on the first picture frame and the second picture frame to obtain a first differential image; performing normalization processing on the first differential image according to each pixel point of the first differential image to obtain a first normalized image; Entering the determination step, the determination step includes: dividing the first standardized image into blocks to obtain multiple image blocks, and calculating the sum of squares of pixels in each of the image blocks; selecting an image block that meets a first preset condition, and determining the motion area in the target scene based on the selected image block; wherein the first preset condition is that the sum of squares of pixels in the image block is greater than or equal to a preset threshold.

3. The image fusion method according to claim 2, characterized in that: The step of determining the motion area in the target scene according to the corrected first picture frame and the corrected second picture frame includes: performing differential processing on the corrected first image frame and the corrected second image frame to obtain a second differential image; performing normalization processing on the first differential image according to the pixel points in the non-motion area of ​​the second differential image to obtain a second normalized image; The first standardized image is updated to the second standardized image, and the determination step is re-entered.

4. The image fusion method according to claim 1, wherein: When the motion region includes multiple motion regions, setting the weight of the motion region in each picture frame according to the pixel value of each pixel point in the motion region of each picture frame and the fused pixel value corresponding to each pixel point includes: Traversing each motion region, calculating the absolute value of the difference between the pixel value of each pixel point in the motion region currently traversed in each picture frame and the fused pixel value corresponding to each pixel point, and calculating the sum of the absolute values ​​of the differences in each picture frame; The weight of the currently traversed motion region in each of the picture frames is set according to the sum of the absolute values ​​of the differences.

5. The image fusion method according to claim 1, characterized in that: The fusing the pixel points at the same position in the motion regions of the multiple picture frames to obtain fused pixel values ​​corresponding to the pixel points at the same position includes: Determining weights of the pixels at the same position in the motion regions of the multiple picture frames based on attribute information of the pixels at the same position in the motion regions of the multiple picture frames; wherein the attribute information includes one or any combination of the following: brightness, saturation, and contrast; According to the weights of the pixels at the same position in the motion areas of the multiple picture frames, weighted fusion is performed on the pixels at the same position in the multiple picture frames, and fused pixel values ​​corresponding to the pixels at the same position are obtained.

6. The image fusion method according to claim 2, characterized in that: The step of performing block processing on the first standardized image to obtain a plurality of image blocks includes: Performing mean blur processing on the first standardized image to obtain a first standardized image after mean blur processing; The first standardized image after the mean blurring process is divided into blocks to obtain a plurality of image blocks.

7. The image fusion method according to claim 2, characterized in that: The performing differential processing on the first picture frame and the second picture frame to obtain a first differential image includes: Performing Gaussian downsampling on the first picture frame and the second picture frame respectively to obtain a Gaussian downsampled first picture frame and a Gaussian downsampled second picture frame; Performing difference processing on the first image frame after Gaussian downsampling and the second image frame after Gaussian downsampling to obtain a first difference image.

8. The image fusion method according to claim 1, characterized in that: The determining the motion area in the target scene according to the first picture frame and the second picture frame includes: Performing registration processing on the first picture frame and the second picture frame respectively to obtain a first picture frame after the registration processing and a second picture frame after the registration processing; A motion region in the target scene is determined according to the first image frame after the registration processing and the second image frame after the registration processing.

9. The image fusion method according to claim 1, characterized in that: The exposure parameters corresponding to the first picture frame taken for the first time and the second picture frame taken for the last time are medium exposure parameters.

10. A terminal, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image fusion method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image fusion method according to any one of claims 1 to 9 is implemented.

Images