Image processing method and device, electronic device and computer-readable storage medium

Abstract

The present application discloses an image processing method, an image processing device, an electronic device, and a non-volatile computer-readable storage medium. The image processing method includes acquiring a first image and a second image, and calculating a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; determining a first weight of the first pixel and a second weight of the second pixel based on the first similarity; acquiring a first target area containing a target object in the first image and a second target area containing the target object in the second image; obtaining a third weight based on the first weight of each first pixel within a preset range of the first pixel of the first target area, and obtaining a fourth weight based on the second weight of each second pixel within a preset range of the second pixel of the second target area; and fusing the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

Description

Technical Field

[0001] The present application relates to the field of image fusion technology, and more specifically, to an image processing method, an image processing device, an electronic device, and a non-volatile computer-readable storage medium. Background Art

[0002] In existing Wide Dynamic Range (WDR) solutions, multi-frame synthesis is crucial for improving the dynamic range of an image. The main existing approach matches pixel blocks by calculating the sum of absolute differences (SAD) between pixel brightness points. In practice, we've found that this SAD-based matching method is particularly susceptible to image noise and brightness variations, resulting in poor matching accuracy for images with low brightness and high noise levels. Summary of the Invention

[0003] Embodiments of the present application provide an image processing method, an image processing device, an electronic device, and a non-volatile computer-readable storage medium.

[0004] The image processing method of an embodiment of the present application includes acquiring a first image and a second image with different exposure times, and calculating a first similarity between a first pixel in the first image and a second pixel matching the first pixel in the second image; determining a first weight of the first pixel and a second weight of the second pixel based on the first similarity; acquiring a first target area containing a target object in the first image and a second target area containing the target object in the second image; updating the first weight of each first pixel in the first target area based on the first weight of the first pixel within a preset range of each first pixel in the first target area to obtain a third weight, and updating the second weight of each second pixel in the second target area based on the second weight of the second pixel within a preset range of each second pixel in the second target area to obtain a fourth weight; and fusing the first image and the second image based on the first weight, the second weight, the third weight and the fourth weight to generate a target image.

[0005] An image processing device according to an embodiment of the present application includes a first acquisition module, a first determination module, a second acquisition module, an update module, and a fusion module. The first acquisition module is configured to acquire a first image and a second image having different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel. The first determination module is configured to determine a first weight for the first pixel and a second weight for the second pixel based on the first similarity. The second acquisition module is configured to acquire a first target region containing a target object in the first image and a second target region containing the target object in the second image. The update module is configured to update the first weight of each first pixel in the first target region based on the first weight of each first pixel within a preset range of each first pixel in the first target region to obtain a third weight, and to update the second weight of each second pixel in the second target region based on the second weight of each second pixel within a preset range of each second pixel in the second target region to obtain a fourth weight. The fusion module is configured to fuse the first and second images based on the first, second, third, and fourth weights to generate a target image.

[0006] An electronic device according to an embodiment of the present application includes a processor. The processor is configured to acquire a first image and a second image having different exposure times, calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel, determine a first weight of the first pixel and a second weight of the second pixel based on the first similarity, acquire a first target region containing a target object in the first image and a second target region containing the target object in the second image, update the first weight of each first pixel in the first target region based on the first weight of each first pixel within a preset range of the first pixel in the first target region to obtain a third weight, and update the second weight of each second pixel in the second target region based on the second weight of each second pixel within a preset range of the second pixel in the second target region to obtain a fourth weight, and fuse the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0007] A non-volatile computer-readable storage medium according to an embodiment of the present application includes a computer program. When the computer program is executed by one or more processors, the processors perform the following image processing method: obtaining a first image and a second image with different exposure times, and calculating a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; determining a first weight of the first pixel and a second weight of the second pixel based on the first similarity; obtaining a first target region containing a target object in the first image and a second target region containing the target object in the second image; updating the first weight of each first pixel in the first target region based on the first weight of each first pixel within a preset range of the first pixel in the first target region to obtain a third weight, and updating the second weight of each second pixel in the second target region based on the second weight of each second pixel within a preset range of the second pixel in the second target region to obtain a fourth weight; and fusing the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0008] In the image processing method, image processing device, electronic device and non-volatile computer-readable storage medium of the embodiments of the present application, since in the process of fusing the first image and the second image, a first target area containing the target object and a second target area containing the target object will be obtained, and the first weight of the first pixel in the first target area will be updated, and the second weight of the second pixel in the second target area will be updated, and the updated first weight is the first weight of the surrounding first pixels considered, and the updated second weight is the second weight of the surrounding second pixels considered, thereby ensuring that when the first image and the second image are finally fused, the weight transition of the first pixel and the second pixel of the part located in the first target area and the second target area is relatively smooth after fusion, so as to avoid the problem of obvious signal-to-noise ratio degradation, thereby ensuring the quality of the target image.

[0009] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0011] Figure 1 is a flowchart of an image processing method according to certain embodiments of the present application;

[0012] Figure 2 Schematic diagram of an image processing device in some embodiments of the present application

[0013] Figure 3 is a schematic plan view of an electronic device according to certain embodiments of the present application;

[0014] Figure 4 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0015] Figure 5 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0016] Figure 6 is a flowchart of an image processing method according to certain embodiments of the present application;

[0017] Figure 7 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0018] Figure 8 is a flowchart of an image processing method according to certain embodiments of the present application;

[0019] Figure 9 is a flowchart of an image processing method according to certain embodiments of the present application;

[0020] Figure 10 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0021] Figure 11 is a flowchart of an image processing method according to certain embodiments of the present application;

[0022] Figure 12 is a flowchart of an image processing method according to certain embodiments of the present application;

[0023] Figure 13 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0024] Figure 14 is a flowchart of an image processing method according to certain embodiments of the present application;

[0025] Figure 15 is a schematic diagram of a scene of an image processing method in certain embodiments of the present application;

[0026] Figure 16 It is a schematic diagram of the connection status of a non-volatile computer-readable storage medium and a processor in certain embodiments of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and should not be understood as limiting the embodiments of the present application.

[0028] See also Figure 1 The present application provides an image processing method. The image processing method comprises the following steps:

[0029] 011: Acquire a first image and a second image with different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel;

[0030] 012: Determine a first weight of the first pixel and a second weight of the second pixel according to the first similarity;

[0031] 013: Acquire a first target area containing a target object in the first image and a second target area containing a target object in the second image;

[0032] 014: updating the first weight of each first pixel in the first target area according to the first weight of the first pixel within the preset range of each first pixel in the first target area to obtain a third weight, and updating the second weight of each second pixel in the second target area according to the second weight of the second pixel within the preset range of each second pixel in the second target area to obtain a fourth weight;

[0033] 015: According to the first weight, the second weight, the third weight, and the fourth weight, the first image and the second image are fused to generate a target image.

[0034] See also Figure 2, an embodiment of the present application provides an image processing device 10. The image processing device 10 includes a first acquisition module 11, a first determination module 12, a second acquisition module 13, an update module 14 and a fusion module 15. The image processing method of the embodiment of the present application can be applied to the image processing device 10. Among them, the first acquisition module 11, the first determination module 12, the second acquisition module 13, the update module 14 and the fusion module 15 are used to execute step 011, step 012, step 013, step 014 and step 015 respectively. That is, the first acquisition module 11 is used to acquire a first image and a second image with different exposure times, and calculate the first similarity of a first pixel in the first image and a second pixel matching the first pixel in the second image. The first determination module 12 is used to determine a first weight of the first pixel and a second weight of the second pixel based on the first similarity. The second acquisition module 13 is used to acquire a first target area containing a target object in the first image and a second target area containing a target object in the second image. The updating module 14 is configured to update the first weight of each first pixel in the first target area based on the first weight of the first pixel within the preset range of each first pixel in the first target area to obtain a third weight, and to update the second weight of each second pixel in the second target area based on the second weight of the second pixel within the preset range of each second pixel in the second target area to obtain a fourth weight. The fusion module 15 is configured to fuse the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0035] See also Figure 3 , the embodiment of the present application further provides an electronic device 100. The image processing method of the embodiment of the present application can be applied to the electronic device 100. The electronic device 100 includes a processor 50. The processor 50 is used for steps 011, step 012, step 013, step 014, and step 015. That is, the processor 50 is used to obtain a first image and a second image with different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; determine a first weight of the first pixel and a second weight of the second pixel based on the first similarity; obtain a first target area containing a target object in the first image and a second target area containing a target object in the second image; update the first weight of each first pixel in the first target area based on the first weight of the first pixels within a preset range of each first pixel in the first target area to obtain a third weight, and update the second weight of each second pixel in the second target area based on the second weight of the second pixels within a preset range of each second pixel in the second target area to obtain a fourth weight; and fuse the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0036] The electronic device includes a housing 30, which can be used to install functional modules such as the display device, imaging device, power supply device, and communication device of the electronic device 100, so that the housing 30 provides dustproof, drop-proof, and waterproof protection for the functional modules. The electronic device 100 can be a mobile phone, tablet computer, digital camera, laptop computer, smart watch, head-mounted display device, game console, etc. Figure 3 As shown, the embodiment of the present application is described by taking the electronic device 100 as a mobile phone as an example. It can be understood that the specific form of the electronic device 100 is not limited to a mobile phone.

[0037] Specifically, please combine Figure 3 The electronic device 100 further includes a camera 40, which can be used to capture a first image and a second image. After the camera 40 captures the first image and the second image, the processor 50 can obtain the first image and the second image. The first image and the second image have different exposure times. The first image can be a long-exposure image, and the second image can be a short-exposure image. Alternatively, the first image is a short-exposure image, and the second image is a long-exposure image. The following description takes the first image as a long-exposure image and the second image as a short-exposure image as an example.

[0038] After the processor 50 obtains the first image and the second image, the processor 50 may calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel.

[0039] The matching of the first pixel and the second pixel can be performed by traversing the first pixel in the first image in the second image to calculate the Hamming distance or Euclidean distance between each second pixel in the second image and the first pixel, and the second pixel with the smallest Hamming distance or the smallest Euclidean distance is the second pixel matching the first pixel.

[0040] like Figure 4 As shown, if a first pixel Q1 in a first image P1 matches a second pixel Q2 in a second image P2, the processor 50 can calculate a first similarity between the first pixel Q1 and the second pixel Q2 based on the Hamming distance and / or the Euclidean distance between the first pixel Q1 and the second pixel Q2. The Hamming distance is inversely proportional to the first similarity; the smaller the Hamming distance, the higher the first similarity, i.e., the higher the similarity between the first pixel Q1 and the second pixel Q2. Similarly, the Euclidean distance is inversely proportional to the first similarity; the smaller the Euclidean distance, the higher the first similarity, i.e., the higher the similarity between the first pixel Q1 and the second pixel Q2.

[0041] Next, the processor 50 may determine a first weight of the first pixel and a second weight of the second pixel according to the first similarity.

[0042] Specifically, when the exposure time of the first image is greater than the exposure time of the second image, that is, the first image is a long-exposure image and the second image is a short-exposure image, the processor 50 can determine the first weight and the second weight by comparing the first similarity and a preset similarity threshold. The preset similarity can be set by the user.

[0043] In one embodiment, when the first similarity is less than a preset similarity threshold, the first weight is proportional to the first similarity, and the second weight is inversely proportional to the second similarity.

[0044] For example, the initial first weight is 0.7 and the initial second weight is 0.3. When the processor 50 determines that the first similarity is less than the preset similarity, the first weight will decrease and the second weight will increase. For example, the first weight decreases to 0.4 and the second weight increases to 0.6. The degree of change in the first weight and the second weight can be associated with the ratio of the first similarity to the preset similarity. For example, when the first similarity is less than the preset similarity, if the ratio of the first similarity to the preset similarity is 0.9, the first weight decreases by 0.1 and the second weight increases by 0.1.

[0045] In another embodiment, when the first similarity is greater than a preset similarity threshold, the processor 50 further compares the pixel value of the first pixel with a preset maximum brightness value to determine a first weight and a second weight when the first similarity is greater than the preset similarity threshold and the pixel value of the first pixel is greater than the preset maximum brightness value. By determining the magnitude of the pixel value of the first pixel and the preset maximum brightness value, the first weight of the first pixel in the brighter area of ​​the first image can be determined, and the second weight of the corresponding second pixel can be determined.

[0046] Specifically, the difference between the pixel value of the first pixel and the maximum brightness value is proportional to the second weight of the corresponding second pixel, that is, inversely proportional to the first weight. It can be understood that the larger the difference between the pixel value of the first pixel and the maximum brightness value, the smaller the first weight and the larger the second weight.

[0047] Among them, the greater the difference between the pixel value of the first pixel and the maximum brightness value, the more it reflects that the first image is overexposed. In order to ensure the quality of the fused target image, it is necessary to reduce the weight ratio of the first image during fusion, that is, reduce the first weight, and correspondingly increase the weight ratio of the second image, that is, increase the second weight.

[0048] In another embodiment, when the first similarity is greater than a preset similarity threshold, the processor 50 further compares the pixel value of the second pixel with a preset minimum brightness value to determine the first weight and the second weight when the first similarity is greater than the preset similarity threshold and the pixel value of the second pixel is less than the preset minimum brightness value. By determining the magnitude between the pixel value of the second pixel and the preset minimum brightness value, the second weight of the second pixel in the darker area of ​​the second image can be determined, thereby determining the first weight of the corresponding first pixel.

[0049] Specifically, the difference between the pixel value of the second pixel and the minimum brightness value is proportional to the first weight of the corresponding first pixel, that is, inversely proportional to the second weight. It can be understood that the greater the difference between the pixel value of the second pixel and the minimum brightness value, the greater the first weight and the smaller the second weight.

[0050] Among them, the difference between the pixel value of the second pixel and the minimum brightness value will reflect that the second image is too dark. In order to ensure the quality of the fused target image, it is necessary to reduce the weight ratio of the second image during fusion, that is, reduce the second weight, and correspondingly increase the weight ratio of the first image, that is, increase the first weight.

[0051] It is understood that each first pixel in the first image corresponds to its own first weight, and each second pixel in the second image also corresponds to its own second weight. It is understood that the first similarity corresponds to the similarity between each first pixel in the first image and each second pixel in the second image.

[0052] In some embodiments, the processor 50 can also determine whether to fuse the corresponding first image and the second image by comparing the first similarity with a preset minimum similarity value. When the first similarity is less than the preset minimum similarity value, it means that the corresponding first pixel in the first image and the second pixel in the second image are less similar. In this case, no fusion is performed, and the data of the current frame image is directly output. That is, when fusing the target image, the first pixel and the second pixel corresponding to the first similarity are not fused, but the pixels of the current frame image are directly fused.

[0053] In this way, after the processor 50 determines the first weight of the first pixel in the first image and the second weight of the second pixel in the second image based on the first similarity, the processor 50 will also obtain the first target area containing the target object in the first image and the second target area containing the target object in the second image.

[0054] Specifically, the processor 50 detects a first target region containing a target object in the first image and a second target region containing a target object in the second image based on a preset target detection model. The target object can be an object that can be detected after targeted training of the target detection model, such as an animal, a person, or a vehicle. The first target region is the region where the target object is located in the first image, and the second target region is the region where the target object is located in the second image.

[0055] It should be noted that when the target object moves, if the first image and the second image are directly fused using the first weight of the first pixel and the second weight of the second pixel, noise will occur in the area where the target object is located in the final fused target image, thereby affecting the image quality.

[0056] Therefore, next, the processor 50 may update the first weight of each pixel in the first target area based on the first weight of the first pixel within the preset range of each first pixel in the first target area, thereby obtaining a third weight of each first pixel in the first target area. Similarly, the processor 50 may also update the second weight of each pixel in the second target area based on the second weight of the second pixel within the preset range of each first pixel in the second target area, thereby obtaining a fourth weight of each second pixel in the second target area. The preset range may be the first pixels within a certain range around the first pixel, such as the first pixels within a 3*3 range or the first pixels within a 5*5 range.

[0057] Please combine Figure 5 , the first target area in the first image P3 is S1, the second target area in the second image P4 is S2, and the size of the preset range S3 is 3*3. Then, the first weight of the first pixel Q3 in the first target area S1 is the average of the first weights of all the first pixels Q4 within the preset range S3, that is, the third weight of the first pixel Q3 is the average of the first weights of the eight first pixels Q4 surrounding the first pixel Q3. Similarly, the second weight of the second pixel Q5 in the second target area S2 is the average of the second weights of all the second pixels Q6 within the preset range S2, that is, the third weight of the second pixel Q5 is the average of the second weights of the eight second pixels Q6 surrounding the second pixel Q5.

[0058] For example, if the first weights of the eight first pixels Q4 surrounding the first pixel Q3 are 0.5, 0.8, 0.4, 0.9, 0.7, 0.3, 0.6, and 0.5, respectively, then the third weight of the first pixel Q3 is (0.5+0.8+0.4+0.9+0.7+0.3+0.6+0.5) / 8, i.e., the third weight is 0.5875. For another example, if the second weights of the eight second pixels Q6 surrounding the second pixel Q5 are 0.7, 0.6, 0.6, 0.8, 0.9, 0.3, 0.5, and 0.7, respectively, then the fourth weight of the second pixel Q5 is (0.7+0.6+0.6+0.8+0.9+0.3+0.5+0.7) / 8, i.e., the fourth weight is 0.6375.

[0059] Finally, the processor 50 can fuse the first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight. It can be understood that the first weight and the second weight are for fusing the first pixel in the first image excluding the first target area and the second pixel in the second image excluding the second target area. The third weight and the fourth weight are for fusing the first target area and the second target area, and the third weight and the fourth weight are calculated based on the weights of the surrounding pixels. This ensures that when the target area is fused, the weights are relatively smooth, that is, the weights of the first pixel and the second pixel are relatively smooth relative to the surrounding pixels, thereby improving the signal-to-noise ratio of the target area.

[0060] It should be noted that the first target area in the first image is not necessarily completely fused with the second target area in the second image. When the target object moves, the first image and the second image can be fused based on the first weight of the first pixel and the third weight of the second pixel, or the first image and the second image can be fused based on the third weight of the first pixel and the second weight of the second pixel.

[0061] In the image processing method, image processing device 10 and electronic device 100 of the embodiments of the present application, since in the process of fusing the first image and the second image, a first target area containing the target object and a second target area containing the target object will be obtained, and the first weight of the first pixel in the first target area will be updated, and the second weight of the second pixel in the second target area will be updated, and the updated first weight is the first weight of the surrounding first pixels considered, and the updated second weight is the second weight of the surrounding second pixels considered, thereby ensuring that when the first image and the second image are finally fused, the weight transition of the first pixel and the second pixel of the part located in the first target area and the second target area is relatively smooth after fusion, so as to avoid the problem of obvious signal-to-noise ratio degradation, thereby ensuring the quality of the target image.

[0062] See also Figure 2 、 Figure 3and Figure 6 The image processing method according to the embodiment of the present application further includes the steps of:

[0063] 016: Traverse the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel;

[0064] 017: Calculate the distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image; and

[0065] 018: Determine the match between the first pixel and the second pixel with the minimum distance.

[0066] In certain embodiments, the image processing device 10 further includes a traversal module 16, a calculation module 17, and a matching module 18. The traversal module 16 is configured to execute step 016. The calculation module 17 is configured to execute step 017. The matching module 18 is configured to execute step 018. Specifically, the traversal module 16 is configured to traverse the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel. The calculation module 17 is configured to calculate the distance between the first code stream for each first pixel in the first image and the second code stream for each second pixel in the second image. The matching module 18 is configured to determine whether the first pixel with the smallest distance matches the second pixel.

[0067] In some embodiments, the processor 50 is configured to execute step 016, step 017, and step 018. That is, the processor 50 is configured to traverse the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel; calculate the distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image; and determine that the first pixel with the minimum distance matches the second pixel.

[0068] Specifically, before the processor 50 determines the first weight of the first pixel in the first image and the second weight of the second pixel in the second image that matches the first pixel, the processor 50 needs to first determine the first pixel in the first image and the second pixel in the second image that matches the first pixel.

[0069] More specifically, the processor 50 may first traverse the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel. The preset window may be any size smaller than the size of the first image and the second image, such as a 3*3 preset window or a 5*5 preset window.

[0070] Furthermore, during the traversal process, the processor 50 may calculate the pixel value magnitude of the first pixel in the preset window with respect to the center thereof, with respect to each first pixel as the center, thereby performing a binarization process on the first pixel in the preset window. The obtained binarized data is the first code stream with respect to the first pixel in the center thereof. Similarly, the processor 50 may also calculate the pixel value magnitude of the second pixel in the preset window with respect to the center thereof, with respect to each second pixel as the center, thereby performing a binarization process on the second pixel in the preset window. The obtained binarized data is the second code stream with respect to the second pixel in the center thereof.

[0071] like Figure 7 As shown, the preset window in the first image X1 is M1, the preset window in the second image X2 is M2, the first pixel Y1 is located at the center of the preset window M1, and the second pixel Y2 is located at the center of the preset window M2. The processor 50 can perform a binarization process on all first pixels Y3 by comparing the pixel values ​​of the first pixel Y1 with those of other first pixels Y3 within the preset window M1. Similarly, the processor 50 can also perform a binarization process on all second pixels Y4 by comparing the pixel values ​​of the second pixel Y2 with those of other second pixels Y4 within the preset window M2.

[0072] For example, if a first pixel Y3 greater than the first pixel Y1 is recorded as 1, and a first pixel Y3 less than or equal to the first pixel Y1 is recorded as 0, if the pixel value of the first pixel Y1 is 100, the pixel values ​​of the eight first pixels Y3, from left to right and from top to bottom, are 101, 102, 88, 90, 120, 130, 93, and 102, respectively. The first code stream of the first pixel Y1 is recorded as 11001101. For another example, if a second pixel Y4 greater than the second pixel Y2 is recorded as 1, and a second pixel Y4 less than or equal to the second pixel Y2 is recorded as 0, if the pixel value of the second pixel Y2 is 101, the pixel values ​​of the eight second pixels Y4, from left to right and from top to bottom, are 101, 102, 88, 90, 120, 130, 93, and 102, respectively. The second code stream of the first pixel Y1 is recorded as 01001101.

[0073] In this way, after the processor 50 obtains the first code stream of the first pixel and the second code stream of the second pixel, the processor 50 can calculate the distance between the first code stream of each first pixel and the second code stream of each second pixel. The distance can be calculated by comparing the different number of times the first code stream and the second code stream appear at the same position. For example, if the first code stream is 11001101 and the second code stream is 01001101, then the distance between the first code stream and the second code stream is 1.

[0074] Finally, the processor 50 can obtain the second code stream of the second pixels having the smallest distance from the first code stream of the first pixel, and the processor 50 can determine that the second pixel corresponding to the second code stream matches the first pixel.

[0075] See also Figure 2 、 Figure 3 、 Figure 8 and Figure 9 In some embodiments, step 016: traversing the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel further includes the steps of:

[0076] 0161: Obtain the pixel value of a first central pixel of all first pixels within a preset window of the first image, and the pixel value of a second central pixel of all second pixels within a preset window of the second image;

[0077] 0162: When the pixel value of the first pixel in the preset window is less than the pixel value of the first central pixel, determine the pixel value of the first pixel in the preset window as the first coded value;

[0078] 0163: when the pixel value of the first pixel in the preset window is greater than or equal to the pixel value of the first central pixel, determine that the pixel value of the first pixel in the preset window is a second coding value, and the first coding value and the second coding value are different;

[0079] 0164: When the pixel value of the second pixel in the preset window is less than the pixel value of the second central pixel, determine that the pixel value of the second pixel in the preset window is the first coded value;

[0080] 0165: When the pixel value of the second pixel in the preset window is greater than or equal to the pixel value of the second central pixel, determine that the pixel value of the second pixel in the preset window is the second coding value.

[0081] Step 017: Calculating the distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image, including the steps of:

[0082] 0171: Count the number of different occurrences of the code value at the same position in the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image, and use the number as the distance.

[0083] Step 011: Calculating a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel, including:

[0084] 0111: Calculate a first similarity based on a distance between a first code stream of a first pixel in the first image and a second code stream of a matching second pixel in the second image. The smaller the distance, the greater the first similarity.

[0085] In some embodiments, the traversal module 16 is configured to execute step 0161, step 0162, step 0163, step 0164, and step 0165. The calculation module 17 is configured to execute step 0171. The first acquisition module 11 is configured to execute step 0111. That is, the traversal module 16 is configured to obtain the pixel value of the first central pixel of all first pixels within a preset window of the first image and the pixel value of the second central pixel of all second pixels within the preset window of the second image; if the pixel value of the first pixel within the preset window is less than the pixel value of the first central pixel, the pixel value of the first pixel within the preset window is determined to be a first coding value; if the pixel value of the first pixel within the preset window is greater than or equal to the pixel value of the first central pixel, the pixel value of the first pixel within the preset window is determined to be a second coding value, and the first coding value and the second coding value are different; if the pixel value of the second pixel within the preset window is less than the pixel value of the second central pixel, the pixel value of the second pixel within the preset window is determined to be the first coding value; and if the pixel value of the second pixel within the preset window is greater than or equal to the pixel value of the second central pixel, the pixel value of the second pixel within the preset window is determined to be the second coding value. The calculation module 17 is configured to count the number of different coding values ​​at the same position in the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image, to use as the distance. The first acquisition module 11 is configured to calculate a first similarity according to a distance between a first code stream of a first pixel in the first image and a second code stream of a matching second pixel in the second image. The smaller the distance, the greater the first similarity.

[0086] In some embodiments, the processor 50 is configured to execute step 0161, step 0162, step 0163, step 0614, step 0165, and step 0171. That is, the processor 50 is configured to obtain the pixel value of the first central pixel of all first pixels within the preset window of the first image, and the pixel value of the second central pixel of all second pixels within the preset window of the second image; when the pixel value of the first pixel within the preset window is less than the pixel value of the first central pixel, determine that the pixel value of the first pixel within the preset window is a first coding value; when the pixel value of the first pixel within the preset window is greater than or equal to the pixel value of the first central pixel, determine that the pixel value of the first pixel within the preset window is a second coding value, and the first coding value and the second coding value are different; when the pixel value of the second pixel within the preset window is less than the pixel value of the second central pixel, determine that the pixel value of the first pixel within the preset window is a second coding value, and the first coding value and the second coding value are different. In the case where the pixel value of the second pixel in the preset window is greater than or equal to the pixel value of the second central pixel, the pixel value of the second pixel in the preset window is determined to be the first coding value; and in the case where the pixel value of the second pixel in the preset window is greater than or equal to the pixel value of the second central pixel, the pixel value of the second pixel in the preset window is determined to be the second coding value; and the number of different occurrences of the coding value at the same position in the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image is counted as the distance; and a first similarity is calculated according to the distance between the first code stream of the first pixel in the first image and the second code stream of the matching second pixel in the second image, where the smaller the distance, the greater the first similarity.

[0087] Specifically, when the processor 50 traverses the first image and the second image based on the preset window, the processor 50 first obtains the pixel value of the first center pixel of all first pixels in the preset window of the first image, and the pixel value of the second center pixel of all second pixels in the preset window of the second image.

[0088] Please combine Figure 10 , the size of the first image Z1 is 5*5, the size of the second image Z2 is 5*5, and the size of the preset window Z3 is 3*3. Then, the first center pixel of all first pixels N1 within the preset window Z3 of the first image Z1 is N2. The second center pixel of all second pixels N3 within the preset window Z3 of the second image Z2 is N4. It can be understood that the processor 50 traverses the first image Z1 and the second image Z2 pixel by pixel to obtain the first code stream corresponding to each first pixel and the second code stream corresponding to each second pixel.

[0089] After the processor 50 obtains the pixel value of the first central pixel and the pixel value of the second central pixel, the processor 50 compares the pixel value of the first pixel and the pixel value of the first central pixel within the preset window to determine whether the pixel value of the first pixel within the preset window is the first coded value or the second coded value. The processor 50 also compares the pixel value of the second pixel within the preset window and the pixel value of the second central pixel to determine whether the pixel value of the second pixel within the preset window is the first coded value or the second coded value.

[0090] Specifically, when the pixel value of the first pixel in the preset window is less than the pixel value of the first central pixel, the pixel value of the first pixel in the preset window is the first coded value; when the pixel value of the first pixel in the preset window is greater than or equal to the pixel value of the first central pixel, the pixel value of the first pixel in the preset window is the second coded value. Similarly, when the pixel value of the second pixel in the preset window is less than the pixel value of the second central pixel, the pixel value of the second pixel in the preset window is the second coded value; when the pixel value of the second pixel in the preset window is greater than or equal to the pixel value of the second central pixel, the pixel value of the second pixel in the preset window is the second coded value.

[0091] In certain embodiments, when the pixel value of a first pixel within a preset window is less than or equal to the pixel value of a first central pixel, the pixel value of the first pixel within the preset window is a first coded value; when the pixel value of the first pixel within the preset window is greater than the pixel value of the first central pixel, the pixel value of the first pixel within the preset window is a second coded value. Similarly, when the pixel value of a second pixel within the preset window is less than or equal to the pixel value of a second central pixel, the pixel value of the second pixel within the preset window is a second coded value; when the pixel value of the second pixel within the preset window is greater than the pixel value of the second central pixel, the pixel value of the second pixel within the preset window is the second coded value.

[0092] The first code value and the second code value are different. For example, the first code value is 0 and the second code value is 1. Alternatively, the first code value is 1 and the second code value is 0.

[0093] Please combine Figure 10 Taking the pixel value of the first central pixel N2 as 100, the first coding value as 0, and the second coding value as 1 as an example, if the pixel values ​​of all the first pixels N1 except the first central pixel N2 in the preset window Z3 are 101, 98, 99, 102, 104, 80, 120, and 103 respectively, then the pixel values ​​of all the first pixels N1 except the first central pixel N2 in the preset window Z3 are 1, 0, 0, 1, 1, 0, 1, and 1 respectively, then the first code stream of the first central pixel N2 is 10011011.

[0094] Similarly, taking the pixel value of the second central pixel N3 as 98, the first coding value as 0, and the second coding value as 1 as an example, if the pixel values ​​of all second pixels N4 except the second central pixel N3 in the preset window Z3 are 101, 98, 99, 102, 104, 80, 120, and 103 respectively, then the pixel values ​​of all second pixels N4 except the second central pixel N3 in the preset window Z3 are 1, 1, 1, 1, 1, 0, 1, and 1 respectively, then the first code stream of the second central pixel N3 is 11111011.

[0095] Thus, when processor 50 calculates the distance between the first code stream for each first pixel in the first image and the second code stream for each second pixel in the second image, processor 50 may count the number of different occurrences of the code values ​​at the same position in the first and second code streams. For example, if the first code stream is 10011011 and the second code stream is 11111011, then the distance between the first and second code streams is 2.

[0096] It can be understood that the processor 50 can compare the first code stream of each first pixel in the first image with the second code streams of all second pixels in the second image, thereby obtaining the distance between the first code stream of each first pixel and the second code streams of all second pixels in the second image, that is, the distance corresponding to the first code stream of each first pixel is consistent with the number of second pixels in the second image.

[0097] Thus, when processor 50 determines the first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel, it first needs to find the smallest distance among all distances between the first and second code streams to determine the second pixel that matches the first pixel, thereby determining the first similarity. For example, there are five distances between the first code stream of the first pixel and all second pixels in the second code stream: 2, 3, 0, 1, and 5. Therefore, the second pixel with a distance of 0 matches the first pixel.

[0098] Finally, the processor 50 can determine a first similarity based on the distance. The smaller the distance, the greater the first similarity. A mapping relationship between different distances and first similarities can be pre-set. For example, when the distance is 0, the first similarity is 98%; when the distance is 1, the first similarity is 90%; when the distance is 2, the first similarity is 80%, and so on.

[0099] See also Figure 2 、 3 and Figure 11 In some embodiments, step 011: calculating a first similarity between a first pixel in a first image and a second pixel in a second image that matches the first pixel, further includes:

[0100] 0112: Obtain a first gradient value and a first pixel interpolation value of a first pixel, and a second gradient value and a second pixel interpolation value of a second pixel;

[0101] 0113: Calculate a first distance between each first gradient value and each second gradient value, and calculate a second distance between each first pixel interpolation value and each second pixel interpolation value;

[0102] 0114: Calculate a first similarity based on the distance, the first distance, and the second distance.

[0103] In some embodiments, the first acquisition module 11 is configured to execute step 0112, step 0113, and step 0114. That is, the first acquisition module 11 is configured to acquire a first gradient value and a first pixel interpolation value of a first pixel, and a second gradient value and a second pixel interpolation value of a second pixel; calculate a first distance between each first gradient value and each second gradient value, and calculate a second distance between each first pixel interpolation value and each second pixel interpolation value; and calculate a first similarity based on the distance, the first distance, and the second distance.

[0104] In some embodiments, the processor 50 is configured to execute step 0112, step 0113, and step 0114. That is, the processor 50 is configured to obtain a first gradient value and a first pixel interpolation value of a first pixel, and a second gradient value and a second pixel interpolation value of a second pixel; calculate a first distance between each first gradient value and each second gradient value, and calculate a second distance between each first pixel interpolation value and each second pixel interpolation value; and calculate a first similarity based on the distance, the first distance, and the second distance.

[0105] Specifically, when the processor 50 calculates the first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel, in addition to calculating the distance between the first code stream of the first pixel and the second code stream of the second pixel as described above, the processor 50 may also first obtain the first gradient value and the first pixel interpolation of the first pixel, and the second gradient value and the second pixel interpolation of the second pixel.

[0106] The first gradient value of each first pixel can be obtained according to the gradient information of each first pixel, and the first pixel interpolation value of the first pixel can be obtained according to the pixel values ​​of the first pixels adjacent to each first pixel.

[0107] For example, Figure 10 As shown, the first gradient value of the first pixel N2 in the first image Z1 can be obtained by the difference between the pixel values ​​of the four first pixels N1 above, below, left, and right around the first pixel N2 and the pixel value of the first pixel N2. The first pixel interpolation value of the first pixel N2 in the first image Z1 can be obtained by the average of the sum of the pixel values ​​of the two first pixels N1 to the left and right of the first pixel N2.

[0108] Next, the processor 50 may calculate the Euclidean distance between the first gradient value of each first pixel and the second gradient value of each second pixel, thereby obtaining a first distance. Similarly, the processor 50 may also calculate the Euclidean distance between the first pixel interpolation value of each first pixel and the second pixel interpolation value of each second pixel, thereby obtaining a second distance.

[0109] The smaller the first distance is, the higher the similarity between the first pixel and the second pixel is, and the smaller the second distance is, the higher the first similarity between the first pixel and the second pixel is.

[0110] In this way, the processor 50 can determine a first sub-similarity based on the distance between the first code stream of the first pixel and the second code stream of the second pixel, determine a second sub-similarity based on the second distance between the first gradient value of the first pixel and the second gradient value of the second pixel, and determine a third sub-similarity based on the first pixel interpolation of the first pixel and the second pixel interpolation of the second pixel.

[0111] Finally, the first similarity is obtained by weighted summing the first, second, and third sub-similarity. For example, if the weight of the first sub-similarity is 0.4, and the weights of the second and third sub-similarity are both 0.3, then the first similarity = 0.4 * first sub-similarity + 0.3 * second sub-similarity + 0.3 * third sub-similarity.

[0112] In this way, the sub-similarity of the first pixel and the second pixel is determined by three methods, thereby obtaining the final first similarity, which can ensure that the first similarity is relatively accurate, thereby ensuring that during the re-fusion process, the fusion ratio of the first pixel of the first image and the second pixel of the second image is relatively accurate, so as to ensure that the quality of the target image generated by the fusion is good.

[0113] See also Figure 2 、 Figure 3 and Figure 12 The image processing method according to the embodiment of the present application further includes the steps of:

[0114] 019: downsampling the second image to obtain a downsampled image;

[0115] 020: Obtain the pixel to be matched that matches the first pixel in the downsampled image;

[0116] 021: Determine a to-be-matched area in the second image that includes pixels to be matched;

[0117] 022: Obtain the second pixel that matches the first pixel in the area to be matched.

[0118] In certain embodiments, the image processing device 10 further includes a downsampling module 19, a third acquisition module 20, a second determination module 21, and a fourth acquisition module 22. The downsampling module 19 is configured to execute step 019. The third acquisition module 20 is configured to execute step 020. The second determination module 21 is configured to execute step 020. The fourth acquisition module 22 is configured to execute step 022. Specifically, the downsampling module 19 is configured to downsample the second image to obtain a downsampled image. The third acquisition module 20 is configured to obtain a pixel to be matched that matches the first pixel in the downsampled image. The second determination module 21 is configured to determine a region to be matched in the second image that contains the pixel to be matched. The fourth acquisition module 22 is configured to obtain a second pixel in the region to be matched that matches the first pixel.

[0119] In some embodiments, the processor 50 is configured to perform steps 019, 020, 021, and 022. That is, the processor 50 is configured to downsample the second image to obtain a downsampled image; obtain a pixel to be matched that matches the first pixel in the downsampled image; determine a region to be matched in the second image that contains the pixel to be matched; and obtain a second pixel in the region to be matched that matches the first pixel.

[0120] Specifically, before the processor 50 acquires the second pixel that matches the first pixel, the processor 50 may first downsample the second image to obtain a downsampled image. It will be understood that the downsampled image is an image obtained by reducing the resolution of the second image. For example, if the resolution of the second image is 2560*1440, the resolution of the first image may be 1920*1080.

[0121] Next, the processor 50 can obtain a pixel to be matched in the downsampled image that matches the first pixel. According to the above-mentioned method for matching the first pixel with the second pixel, the first pixel is traversed in the second image to find the second pixel with the smallest first bitstream distance from the first pixel, which is the second pixel that matches the first pixel.

[0122] In this way, when the processor 50 obtains the pixel to be matched that matches the first pixel by downsampling the image, it can also be obtained by obtaining the pixel to be matched that has the smallest distance from the first code stream to the first pixel.

[0123] Furthermore, the processor 50 may obtain a to-be-matched area in the second image that includes the to-be-matched pixel. The to-be-matched area may be an area of ​​a predetermined size, such as a 3*3 area centered on the to-be-matched pixel, or a 5*5 area centered on the to-be-matched pixel.

[0124] Please combine Figure 13For example, the second image W1 may be downsampled twice to obtain two downsampled images W2 and W3 of different sizes, wherein the size of the downsampled image W3 is smaller than that of the downsampled image W2.

[0125] First, the processor 50 can obtain a to-be-matched pixel U1 that matches the first pixel in the downsampled image W3. Next, the processor 50 can determine a to-be-matched region Y1 in the downsampled image W3 that includes the to-be-matched pixel U1. In this way, the processor 50 can obtain a to-be-matched pixel U2 that matches the first pixel in the to-be-matched region Y1.

[0126] Finally, the processor 50 can determine the matching area Y2 in the second image W1 that contains the matching pixel U2, and thus obtain the matching pixel U3 that matches the first pixel in the matching area Y2. It can be understood that the matching pixel U3 is the second pixel that matches the first pixel.

[0127] In summary, it can be found that the processor 50 first traverses the first pixel in the smaller downsampled image to obtain the pixel to be matched that matches the first pixel, and then obtains the area to be matched containing the pixel to be matched in the second image, and obtains the second pixel that matches the first pixel through more precise calculation. In this way, the number of traversals of the first pixel can be reduced, and the first pixel is only traversed in the smaller downsampled image and the area to be matched, thereby reducing the workload of the processor 50.

[0128] See also Figure 2 、 Figure 3 and Figure 12 The image processing method according to the embodiment of the present application further includes the steps of:

[0129] 023: perform brightness alignment on the first image and the second image;

[0130] Step 015: According to the first weight, the second weight, the third weight and the fourth weight, the first image and the second image are fused to generate a target image, including the steps of:

[0131] 0151: According to the first weight, the second weight, the third weight, and the fourth weight, the brightness-aligned first image and the second image are fused to generate a target image.

[0132] In some embodiments, the image processing device 10 further includes an alignment module 23. The alignment module 23 is configured to execute step 023. The fusion module 15 is configured to execute step 0151. Specifically, the alignment module 23 is configured to perform brightness alignment on the first image and the second image. The fusion module 15 is configured to fuse the brightness-aligned first image and the second image based on the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0133] In some embodiments, the processor 50 is configured to execute step 023 and step 0151. That is, the processor 50 is configured to perform brightness alignment on the first image and the second image; and fuse the brightness-aligned first image and the second image according to the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

[0134] Specifically, before fusing the first image and the second image, the processor 50 further performs brightness alignment on the first image and the second image.

[0135] More specifically, since the exposure parameters for capturing the first image and the second image are known or pre-set, a first preset exposure ratio can be set, which is calculated based on the exposure parameters of the first image and the exposure parameters of the second image.

[0136] Then, when aligning the brightness of the first image and the second image, the image with the longer exposure time may be divided by the first preset exposure ratio, or the image with the shorter exposure time may be multiplied by the first preset exposure ratio.

[0137] For example, if the first image is a long-exposure image and the second image is a short-exposure image, the processor 50 may divide the exposure ratio parameter in the first image by the first preset exposure ratio, or multiply the exposure ratio parameter in the second image by the first preset exposure ratio, so as to align the brightness of the first image and the second image to ensure the quality of the final fused target image.

[0138] In certain embodiments, the processor 50 may also be pre-set with a second preset exposure ratio, where the first preset exposure ratio and the second preset exposure ratio are different. When aligning the brightness of the first image and the second image, the processor 50 may divide the exposure ratio parameter of the first image by the second preset exposure ratio, and multiply the exposure ratio parameter of the second image by the second preset exposure ratio, so that the exposure ratio parameters of the first image and the second image are at the desired parameters, thereby aligning the brightness of the first image and the second image.

[0139] In some embodiments, before calculating the first distance between the first gradient value of each first pixel in the first image and the second gradient value of each second pixel in the second image, and calculating the second distance between the first pixel interpolation of each first pixel in the first image and the second pixel interpolation of each second pixel in the second image, the processor 50 may further first perform brightness alignment on the first image and the second image to ensure that the gradient information and pixel interpolation of the first pixel in the first image and the second pixel in the second image are close, so as to ensure the accuracy of the calculation of the first distance and the second distance.

[0140] See also Figure 2 、 Figure 3 and Figure 14The image processing method according to the embodiment of the present application further includes the steps of:

[0141] 024: Acquire a third image having an exposure time different from that of the first image and the second image, and calculate a second similarity between a first pixel in the first image and a third pixel in the third image that matches the first pixel;

[0142] 025: Determine a first weight of the first pixel, a second weight of the second pixel, and a fifth weight of the third pixel according to the first similarity and the second similarity;

[0143] 026: Acquire a third target area containing the target object in the third image;

[0144] 027: updating the third weight of each third pixel in the third target area according to the third weight of the third pixels within the preset range of each third pixel in the third target area to obtain a sixth weight;

[0145] 028: According to the first weight, the second weight, the third weight, the fourth weight, the fifth weight, and the sixth weight, the first image, the second image, and the third image are fused to generate a target image.

[0146] In certain embodiments, the first acquisition module 11, the first determination module 12, the second acquisition module 13, the update module 14, and the fusion module 15 are configured to execute steps 024, 025, 026, 027, and 028, respectively. Specifically, the first acquisition module 11 is configured to acquire a third image having an exposure time different from that of both the first and second images, and calculate a second similarity between a first pixel in the first image and a third pixel in the third image that matches the first pixel. The first determination module 12 is configured to determine a first weight for the first pixel, a second weight for the second pixel, and a fifth weight for the third pixel based on the first and second similarities. The second acquisition module 13 is configured to acquire a third target region in the third image that contains a target object. The update module 14 is configured to update the third weight of each third pixel in the third target region based on the third weight of each third pixel within a preset range of the third pixel in the third target region to obtain a sixth weight. The fusion module 15 is configured to fuse the first, second, and third images based on the first, second, third, fourth, fifth, and sixth weights to generate a target image.

[0147] In some embodiments, the processor 50 is configured to execute steps 024, 025, 026, 027, and 028. That is, the processor 50 is configured to obtain a third image having an exposure time different from that of the first image and the second image, and calculate a second similarity between a first pixel in the first image and a third pixel in the third image that matches the first pixel; determine a first weight of the first pixel, a second weight of the second pixel, and a fifth weight of the third pixel based on the first similarity and the second similarity; obtain a third target area containing a target object in the third image; update the third weight of each third pixel in the third target area based on the third weight of the third pixels within a preset range of each third pixel in the third target area to obtain a sixth weight; and fuse the first image, the second image, and the third image based on the first weight, the second weight, the third weight, the fourth weight, the fifth weight, and the sixth weight to generate a target image.

[0148] Specifically, the processor 50 may also acquire a third image in addition to the first and second images. The third image may have an exposure duration different from that of the first and second images. For example, the first image may be a long-exposure image, the second image may be a short-exposure image, and the third image may be a medium-exposure image. It will be appreciated that the resulting target image is not limited to being fused from two frames, but may also be composed of multiple frames, such as three, four, five, or more frames.

[0149] Next, the processor 50 may calculate a second similarity between the first pixel in the first image and the third pixel in the third image that matches the first pixel. The method for determining the match between the third pixel and the first pixel may be the same as the method for determining the match between the second pixel and the first pixel.

[0150] For example, a first code stream of each first pixel in the first image and a third code stream of each third pixel in the third image are obtained to obtain a third code stream having the smallest distance from the first code stream among all the third code streams of the third pixels. Then, the third pixel corresponding to the smallest third code stream is the third pixel that matches the first pixel.

[0151] Similarly, in the process of determining the third pixel matching the first pixel, the number of encoding values ​​at the same position in the first and third code streams can be obtained to determine the size of the distance, thereby calculating the second similarity between the first and third pixels based on the distance.

[0152] It can be understood that the calculation method of the second similarity between the first pixel and the third pixel is the same as the calculation method of the first similarity between the first pixel and the second pixel, and is not described in detail here.

[0153] After obtaining the first similarity and the second similarity, the processor 50 may determine a first weight of the first pixel, a second weight of the second pixel, and a fifth weight of the third pixel according to the first similarity and the second similarity.

[0154] As can be seen from the above, the processor 50 can determine the first weight and the second weight by presetting the similarity threshold, and the maximum brightness value and the minimum brightness value. Similarly, the processor 50 can also determine the first weight and the fifth weight by presetting the similarity threshold, and the maximum brightness value and the minimum brightness value.

[0155] For example, the processor 50 determines the first and second weights to be 0.5 and 0.5, respectively, based on the first similarity, and determines the first and fifth weights to be 0.6 and 0.4, respectively, based on the second similarity. The processor 50 can then calculate the ratios of the first and second weights, and the first and fifth weights, based on these ratios to obtain the ratios of the first, second, and fifth weights. That is, the first weight is 0.375, the second weight is 0.375, and the fifth weight is 0.25.

[0156] Furthermore, the processor 50 may obtain a third target region containing the target object in the third image. Specifically, the processor 50 detects the third target region containing the target object in the third image based on a preset target detection model, and updates the third weight of each third pixel within a preset range of the third pixel in the third target region to obtain a sixth weight.

[0157] like Figure 15 As shown, the first target area in the first image O1 is E1, the second target area in the second image O2 is E2, and the size of the preset range E3 is 3*3. Then, the first weight of the first pixel R1 in the first target area E1 is the average of the first weights of all the first pixels R2 in the preset range E3, that is, the third weight of the first pixel R1 is the average of the first weights of the eight first pixels R2 surrounding the first pixel R1. Similarly, the second weight of the second pixel R3 in the second target area E2 is the average of the second weights of all the second pixels R4 in the preset range E3, that is, the third weight of the second pixel R3 is the average of the second weights of the eight second pixels R4 surrounding the second pixel R3.

[0158] Then the fifth weight of the third pixel R5 in the third target area E4 in the third image O3 is the average of the fifth weights of all third pixels R6 in the preset range E3, that is, the sixth weight of the third pixel R5 is the average of the fifth weights of the eight third pixels R6 around the third pixel R5.

[0159] In this way, the processor 50 can fuse the first pixel in the first image, the second pixel in the second image and the third pixel in the third image according to the first weight, the second weight, the third weight, the fourth weight, the fifth weight and the sixth weight, and all three images are detected for the target object, and the corresponding weights are updated, thereby ensuring that there will be no obvious signal-to-noise ratio drop when the first image and the second image are finally fused, so as to ensure the quality of the target image.

[0160] See also Figure 16 The present application also provides a non-volatile computer-readable storage medium 300 including a computer program 301. When the computer program 301 is executed by one or more processors 50, the one or more processors 50 execute the image processing method of any of the above embodiments.

[0161] For example, when the computer program 301 is executed by one or more processors 50, the processor 50 performs the following image processing method:

[0162] 011: Acquire a first image and a second image with different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel;

[0163] 012: Determine a first weight of the first pixel and a second weight of the second pixel according to the first similarity;

[0164] 013: Acquire a first target area containing a target object in the first image and a second target area containing a target object in the second image;

[0165] 014: updating the first weight of each first pixel in the first target area according to the first weight of the first pixel within the preset range of each first pixel in the first target area to obtain a third weight, and updating the second weight of each second pixel in the second target area according to the second weight of the second pixel within the preset range of each second pixel in the second target area to obtain a fourth weight;

[0166] 015: According to the first weight, the second weight, the third weight, and the fourth weight, the first image and the second image are fused to generate a target image.

[0167] For another example, when the computer program 301 is executed by one or more processors 50, the processor 50 performs the following image processing method:

[0168] 016: Traverse the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel;

[0169] 017: Calculate the distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image; and

[0170] 018: Determine the match between the first pixel and the second pixel with the minimum distance.

[0171] In the description of this specification, the reference terms "certain embodiments", "in an example", "exemplarily", etc. mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0172] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0173] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a first image and a second image with different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; determining a first weight of the first pixel and a second weight of the second pixel according to the first similarity; Acquire a first target area containing a target object in the first image and a second target area containing the target object in the second image; updating the first weight of each of the first pixels in the first target area according to the first weight of the first pixels within a preset range of each of the first pixels in the first target area to obtain a third weight, and updating the second weight of each of the second pixels in the second target area according to the second weight of the second pixels within a preset range of each of the second pixels in the second target area to obtain a fourth weight; and The first image and the second image are fused according to the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

2. The image processing method according to claim 1, wherein: Also includes: Traversing the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel; calculating a distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image; A second pixel corresponding to a second code stream having a minimum distance from the first code stream of the first pixel is determined as a second pixel matching the first pixel.

3. The image processing method according to claim 2, wherein: The traversing the first image and the second image based on a preset window to obtain a first code stream for each first pixel and a second code stream for each second pixel includes: Obtaining a pixel value of a first central pixel of all the first pixels within the preset window of the first image, and a pixel value of a second central pixel of all the second pixels within the preset window of the second image; When the pixel value of the first pixel in the preset window is less than the pixel value of the first central pixel, determining the pixel value of the first pixel in the preset window as a first coding value; When the pixel value of the first pixel in the preset window is greater than or equal to the pixel value of the first central pixel, determining that the pixel value of the first pixel in the preset window is a second coding value, and the first coding value and the second coding value are different; When the pixel value of the second pixel in the preset window is less than the pixel value of the second central pixel, determining the pixel value of the second pixel in the preset window as the first coded value; When the pixel value of the second pixel in the preset window is greater than or equal to the pixel value of the second central pixel, determining the pixel value of the second pixel in the preset window as the second coded value; The calculating the distance between the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image includes: The number of different occurrences of the code value at the same position in the first code stream of each first pixel in the first image and the second code stream of each second pixel in the second image is counted as the distance.

4. The image processing method according to claim 3, wherein: The calculating a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel includes: The first similarity is calculated according to the distance between the first code stream of the first pixel in the first image and the second code stream of the matching second pixel in the second image, and the smaller the distance, the greater the first similarity.

5. The image processing method according to claim 3, wherein: The calculating a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel includes: Obtaining a first gradient value and a first pixel interpolation value of the first pixel, and a second gradient value and a second pixel interpolation value of the second pixel; Calculating a first distance between each of the first gradient values ​​and each of the second gradient values, and calculating a second distance between each of the first pixel interpolation values ​​and each of the second pixel interpolation values; The first similarity is calculated according to the distance, the first distance, and the second distance.

6. The image processing method according to claim 1, wherein: Also includes: downsampling the second image to obtain a downsampled image; Obtaining a pixel to be matched that matches the first pixel in the downsampled image; determining a to-be-matched area in the second image that includes the to-be-matched pixels; Acquire the second pixel in the to-be-matched area that matches the first pixel.

7. The image processing method according to claim 1, wherein: Also includes: performing brightness alignment on the first image and the second image; The step of fusing the first image and the second image according to the first weight, the second weight, the third weight, and the fourth weight to generate a target image includes: The first image and the second image after brightness alignment are fused according to the first weight, the second weight, the third weight, and the fourth weight to generate the target image.

8. The image processing method according to claim 7, wherein: The aligning the brightness of the first image and the second image includes: The image with the longer exposure time between the first image and the second image is divided by a first preset exposure ratio; or, the image with the shorter exposure time between the first image and the second image is multiplied by the first preset exposure ratio; or, the image with the longer exposure time between the first image and the second image is divided by a second preset exposure ratio, and the image with the shorter exposure time between the first image and the second image is multiplied by a second preset exposure ratio, wherein the first preset exposure ratio and the second preset exposure ratio are different in size.

9. The image processing method according to claim 8, characterized in that: The exposure time of the first image is longer than the exposure time of the second image, and when the first similarity is less than a preset similarity threshold, the first weight is proportional to the first similarity, and the second weight is inversely proportional to the first similarity; When the first similarity is greater than a preset similarity threshold and the pixel value of the first pixel is greater than a preset maximum brightness value, a difference between the pixel value of the first pixel and the maximum brightness value is proportional to the second weight of the second pixel matching the first pixel; When the first similarity is greater than the preset similarity threshold and the pixel value of the second pixel is less than a preset minimum brightness value, the difference between the minimum brightness value and the pixel value of the second pixel is proportional to the first weight of the first pixel matching the second pixel.

10. The image processing method according to claim 1, wherein: Also includes: Acquire a third image having an exposure time different from both the first image and the second image, and calculate a second similarity between the first pixel in the first image and a third pixel in the third image that matches the first pixel; determining the first weight of the first pixel, the second weight of the second pixel, and the fifth weight of the third pixel according to the first similarity and the second similarity; Acquire a third target area containing the target object in the third image; updating the third weight of each of the third pixels in the third target area according to the third weight of the third pixels within a preset range of each of the third pixels in the third target area to obtain a sixth weight; The first image, the second image, and the third image are fused according to the first weight, the second weight, the third weight, the fourth weight, the fifth weight, and the sixth weight to generate the target image.

11. An image processing device, characterized in that: include: a first acquisition module, configured to acquire a first image and a second image having different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; a first determining module, configured to determine a first weight of the first pixel and a second weight of the second pixel according to the first similarity; a second acquisition module, configured to acquire a first target region containing a target object in the first image and a second target region containing the target object in the second image; an updating module configured to update the first weight of each first pixel in the first target area according to the first weight of the first pixel within a preset range of each first pixel in the first target area to obtain a third weight, and to update the second weight of each second pixel in the second target area according to the second weight of the second pixel within the preset range of each second pixel in the second target area to obtain a fourth weight; A fusion module is used to fuse the first image and the second image according to the first weight, the second weight, the third weight and the fourth weight to generate a target image.

12. An electronic device, characterized in that: comprising a processor configured to acquire a first image and a second image having different exposure times, and calculate a first similarity between a first pixel in the first image and a second pixel in the second image that matches the first pixel; Determine a first weight of the first pixel and a second weight of the second pixel according to the first similarity; obtain a first target area containing a target object in the first image and a second target area containing the target object in the second image; updating the first weight of each of the first pixels in the first target area according to the first weight of the first pixels within a preset range of each of the first pixels in the first target area to obtain a third weight, and updating the second weight of each of the second pixels in the second target area according to the second weight of the second pixels within a preset range of each of the second pixels in the second target area to obtain a fourth weight; and fusing the first image and the second image according to the first weight, the second weight, the third weight, and the fourth weight to generate a target image.

13. A non-volatile computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by one or more processors, the processors implement the image processing method according to any one of claims 1 to 10.

Images