Low-light image detail enhancement method based on multi-exposure fusion

Abstract

The present invention discloses a method for detail enhancement of low-light images based on multi-exposure fusion. The present invention starts from two directions: preset weighted average fusion and weighted average fusion based on historical data. The weights are quickly determined based on experience, and weights can be quickly set for different exposure images for fusion according to past experience. The weighted average fusion based on historical data makes full use of past image acquisition and fusion information, can more accurately adapt to the characteristics of different scenes, improves the accuracy and flexibility of weight setting, and solves the problem that the prior art often lacks flexibility and accuracy in weight setting, resulting in unsatisfactory fusion effect and difficulty in achieving a real, natural and detail-rich image enhancement effect.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a low-light image detail enhancement method based on multi-exposure fusion. Background Art

[0002] With the continuous development of science and technology, people have higher and higher requirements for image quality, especially in some fields that require high-quality images, such as photography. Obtaining clear, natural and detail-rich images in low-light environments is crucial for applications in these fields.

[0003] Although existing image detail enhancement methods have certain effects, they still have limitations:

[0004] The existing multi-exposure fusion methods often lack flexibility and accuracy in weight setting. They either rely on fixed empirical values ​​and cannot adapt to changes in different scenes; or they do not make full use of the information of historical data, resulting in unsatisfactory fusion effects and difficulty in achieving a realistic, natural and detailed image enhancement effect.

[0005] At the same time, there is a lack of comprehensive and integrated evaluation methods to measure the quality of the fused images after multi-exposure images are fused, and the manual evaluation perspective is not incorporated in the evaluation process.

[0006] To this end, a low-light image detail enhancement method based on multi-exposure fusion is proposed. Summary of the invention

[0007] In view of this, the present invention provides a low-light image detail enhancement method based on multi-exposure fusion to solve the problems raised by the above background technology.

[0008] The objective of the present invention can be achieved by the following technical solution: A method for enhancing low-light image details based on multi-exposure fusion, comprising:

[0009] Image acquisition: Take multiple shots of the same scene at different exposure times to obtain a set of multi-exposure image sequences, including short-exposure, medium-exposure, and long-exposure images;

[0010] Image preprocessing: preprocess the collected multi-exposure image sequence;

[0011] Multi-exposure fusion: Process the pre-processed multi-exposure image sequence and obtain the final fused image after screening;

[0012] Image tone mapping: convert the final fused image into a low dynamic range image;

[0013] Image output: After the conversion is completed, the final fused image is subjected to detail enhancement processing, and the image with detail enhancement is output as the final enhancement result.

[0014] In some embodiments, the pre-processed multi-exposure image sequence is processed and screened to obtain a final fused image, specifically:

[0015] S1: using each group of preset methods to set different weights, and fusing the multi-exposure image sequences according to the different weights to obtain each group of fused images;

[0016] S2: Comprehensively evaluate each group of fused images obtained by fusion of different weights, and push them to the current user for screening, and use the fused image screened by the user as the final fused image.

[0017] In some embodiments, the multi-exposure image sequences are fused according to different weights, specifically:

[0018] S1-1: Label the short-exposure, medium-exposure, and long-exposure images as , as well as ; The preset methods include preset weighted average fusion and weighted average fusion based on historical data;

[0019] S1-2: fuse the multi-exposure image sequence using a preset weighted average fusion method;

[0020] S1-3: Fusion of multi-exposure image sequences using a weighted average fusion method based on historical data.

[0021] In some embodiments, a preset weighted average fusion method is used to fuse the multi-exposure image sequence, specifically:

[0022] S1-201: First, the pre-set weights of the short-exposure, medium-exposure, and long-exposure images are extracted. After the extraction is completed, the corresponding pixels of the multi-exposure image sequence are weighted summed according to the pre-set weights, that is, the positions of the corresponding pixels are marked as (x, y). , as well as The value at pixel (x, y) , the calculation formula is expressed as ; Wherein w1, w2 and w3 are the pre-set weights corresponding to the short exposure, medium exposure and long exposure images respectively;

[0023] S1-202: Repeat step S1-201 until all pixel points are traversed to obtain a fused image processed by a pre-weighted average fusion method.

[0024] In some embodiments, a weighted average fusion method based on historical data is used to fuse the multi-exposure image sequence, specifically:

[0025] S1-301: extracting a historical data set, extracting a scene from the historical data set that matches the current multi-exposure image sequence, discarding each set of historical data that fails to match the scene, and retaining each set of historical data that succeeds in matching the scene;

[0026] S1-302: For the historical data that has been successfully matched, the collected multi-exposure image sequence is analyzed for similarity with each group of historical data. , as well as Get the corresponding , as well as ,in , as well as Indicates the numbers corresponding to the short-exposure, medium-exposure, and long-exposure images in each set of historical data, and calculates them respectively and , and as well as and Bhattacharyya distance, Euclidean distance and cosine similarity;

[0027] Normalize the Bhattacharyya distance, Euclidean distance and cosine similarity respectively, preset the reference maximum values ​​corresponding to the Bhattacharyya distance and the Euclidean distance respectively, and calculate the ratios of the calculated Bhattacharyya distance and the Euclidean distance with the corresponding preset reference maximum values, that is, calculate the Bhattacharyya distance / corresponding reference maximum value and the Euclidean distance / corresponding reference maximum value;

[0028] Add 1 to the calculated cosine similarity, and divide the result by the integer 2 to obtain the normalized cosine similarity;

[0029] After normalization, the normalized cosine similarity, Bhattacharyya distance and Euclidean distance are multiplied by the corresponding set weight coefficients respectively, and the Bhattacharyya distance and Euclidean distance are multiplied by the corresponding weight coefficients and added to obtain the similarity value. The cosine similarity is multiplied by the corresponding weight coefficient to obtain the similarity value. The similarity value is divided by the similarity value to obtain the similarity quantization value of the two groups of images.

[0030] In some embodiments, fusing the multi-exposure image sequence using a weighted average fusion method based on historical data also includes:

[0031] S1-303: Based on step S1-302, we obtain and , and as well as and Similar quantitative values ​​of and , and as well as and The similarity quantization value of the collected multi-exposure image sequence and each group of historical data are constructed;

[0032] S1-304: and The similarity quantization value is used as the length value of the solid rectangle, and The similarity quantization value of is used as the width value of the three-dimensional rectangle, and The similarity quantization value of is used as the height value of the stereo rectangle, a stereo rectangle model is constructed, and the constructed stereo rectangle model is used as a similarity model between the collected multi-exposure image sequence and each group of historical data;

[0033] S1-305: Calculate the surface area of ​​each group of similarity models as the overall similarity index of the collected multi-exposure image sequence and each group of historical data, preset the pass index corresponding to the overall similarity index, compare the overall similarity index of each group with the preset pass index, and extract the historical data with an overall similarity index higher than the pass index as high similarity data; if the overall similarity index is lower than the corresponding pass index, execute step S1-308;

[0034] S1-306: For the highly similar data extracted in step S1-305, the weights used by the short-exposure, medium-exposure, and long-exposure images of each group of highly similar data in the historical fusion process are obtained, and corresponding pixels of the multi-exposure image sequence are weighted summed according to the weights used by the highly similar data;

[0035] S1-307: Repeat step S1-306 until all pixels are traversed to obtain a fused image processed by a weighted average fusion method based on historical data;

[0036] S1-308: Select the historical data corresponding to the highest overall similarity index as the high similarity data, and execute step S1-306.

[0037] In some embodiments, each group of fused images obtained by fusion of different weights is comprehensively evaluated and pushed to the current user for screening, specifically:

[0038] S2-1: Comprehensively evaluate each group of fused images from the perspective of image quality;

[0039] S2-101: Preset a reference ideal image of the current scene, and for each group of fused images, calculate the mean square error between each group of fused images and the reference ideal image;

[0040] S2-102: based on step S2-101, the mean square error of each group of fused images is obtained, and the peak signal-to-noise ratio of each group of fused images is further calculated;

[0041] S2-2: Comprehensively evaluate each group of fused images from the perspective of visual effects;

[0042] S2-201: Calculate the structural similarity index between each group of fused images and the reference ideal image;

[0043] S2-3: Comprehensive evaluation of each group of fused images from the perspective of manual evaluation;

[0044] S2-301: sending each group of fused images to an evaluator, who scores each group of fused images, and recording the scoring result of each group of fused images as a manual score;

[0045] S2-4: construct a detail evaluation model for each group of fused images and calculate the comprehensive evaluation value of each group of fused images, arrange the comprehensive evaluation values ​​from large to small, select the first three groups of comprehensive evaluation values ​​and the corresponding detail evaluation models and push them to the user, and after receiving them, the user selects the corresponding fused images based on the three groups of detail evaluation models and the comprehensive evaluation values;

[0046] S2-401: Convert the peak signal-to-noise ratio and structural similarity index of each group of fused images based on a preset conversion rule.

[0047] In some embodiments, the peak signal-to-noise ratio and the structural similarity index of each group of fused images are converted into scores based on a preset conversion rule, specifically:

[0048] S2-402: Preset the signal-to-noise ratio value ranges of each group corresponding to the peak signal-to-noise ratio, match the peak signal-to-noise ratio of each group of fused images with the corresponding signal-to-noise ratio value range, set each group of signal-to-noise ratio value range to correspond to a peak score, and obtain the peak score corresponding to each group of fused images;

[0049] S2-403: Similarly to step S2-402, preset the index value ranges corresponding to the structural similarity index, set each index value range to correspond to a structural score, match the structural similarity index of each group of fused images with the corresponding index value range, and obtain the structural score corresponding to each group of fused images.

[0050] In some embodiments, a detail evaluation model for each group of fused images is constructed and a comprehensive evaluation value for each group of fused images is calculated, specifically:

[0051] S2-404: extract the peak score, structure score and manual score corresponding to each group of fusion images, construct a circle, draw three line segments from the center of the circle until they are connected to the edge of the circle, and set the angle formed by every two line segments to 120°. Divide it into 5 equal parts based on the three drawn line segments, and mark the division points. Set three line segments to correspond to the peak score, structure score and manual score respectively, extract the score values ​​corresponding to the peak score, structure score and manual score, and find the corresponding division points on the corresponding line segments based on the score values, wherein the larger the score value, the farther the corresponding division point is from the center of the circle;

[0052] After finding the corresponding division points, the adjacent division points are connected to construct a triangle, and the constructed triangle is used as the detail evaluation model of each group of fused images. At the same time, the area value of the triangle is calculated as the comprehensive evaluation value of each group of fused images.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention starts from two directions: preset weighted average fusion and weighted average fusion based on historical data. The weights can be quickly determined based on experience, and weights can be quickly set for different exposure images for fusion according to past experience. The weighted average fusion based on historical data makes full use of past image acquisition and fusion information, can more accurately adapt to the characteristics of different scenes, improves the accuracy and flexibility of weight setting, and solves the problem that the prior art often lacks flexibility and accuracy in weight setting, resulting in unsatisfactory fusion effect and difficulty in achieving a real, natural and detail-rich image enhancement effect.

[0055] The present invention extracts a historical data set containing previously acquired multi-exposure image sequences and corresponding scene information, and then extracts historical data matching the current multi-exposure image sequence scene from the set, removes the unmatched ones, and retains the successfully matched ones. This ensures that the used historical data is relevant to the current scene;

[0056] The present invention uses a three-dimensional rectangular model to represent the similarity between the collected multi-exposure image sequence and each group of historical data, and uses different similarity quantization values ​​as the length, width, and height of the three-dimensional rectangle. This visual model construction method makes the expression of similarity more intuitive and comprehensive. By calculating the surface area of ​​the three-dimensional rectangular model as the overall similarity index, it can comprehensively consider the information of multiple similarity dimensions and more accurately reflect the overall similarity between image sequences;

[0057] The present invention calculates the area value of the triangle as the comprehensive evaluation value of the fused image, and arranges the comprehensive evaluation values ​​from large to small; in this way, different fused images can be quantitatively compared, and the top three groups of comprehensive evaluation values ​​with higher comprehensive evaluation values ​​and the corresponding detail evaluation models are screened out and pushed to the user. The user screens the fused image based on this information, which improves the user's participation in the image enhancement process and satisfaction with the final result, and can better meet the user's needs for image quality in different application scenarios; at the same time, this comprehensive evaluation method combines objective indicators with subjective evaluation, comprehensively and comprehensively reflects the quality and visual effect of the fused image, and provides users with a more accurate and reliable screening basis. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Further details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0059] Figure 1 is a flow chart of the present invention;

[0060] Figure 2 Schematic diagram of the detailed evaluation model in the present invention. DETAILED DESCRIPTION

[0061] Several embodiments of the present application will be described in more detail below with reference to the accompanying drawings so that those skilled in the art can implement the present application. The present application can be embodied in many different forms and purposes and should not be limited to the embodiments described herein. These embodiments are provided to make the present application comprehensive and complete, and to fully convey the scope of the present application to those skilled in the art. The embodiments do not limit the present application.

[0062] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this application belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the relevant art and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0063] See also Figure 1-Figure 2 As shown, the low-light image detail enhancement method based on multi-exposure fusion includes:

[0064] Image acquisition: Use a camera to take multiple shots of the same scene at different exposure times to obtain a set of multi-exposure image sequences, including short-exposure, medium-exposure, and long-exposure images, to capture details of different brightness in the scene.

[0065] In low-light conditions, a single image may not be able to fully present all the details, and images with different exposures can capture information about different brightness areas in the scene;

[0066] Image preprocessing: preprocess the collected multi-exposure image sequence; preprocessing includes operations such as denoising and color correction to improve the image quality;

[0067] Multi-exposure fusion: Process the pre-processed multi-exposure image sequence and obtain the final fused image after screening;

[0068] Specifically:

[0069] S1: using each group of preset methods to set different weights, and fusing the multi-exposure image sequences according to the different weights to obtain each group of fused images;

[0070] S1-1: Label the short-exposure, medium-exposure, and long-exposure images as , as well as ; The preset methods include preset weighted average fusion and weighted average fusion based on historical data;

[0071] S1-2: fuse the multi-exposure image sequence using a preset weighted average fusion method;

[0072] S1-201: First, the pre-set weights of the short-exposure, medium-exposure, and long-exposure images are extracted. After the extraction is completed, the corresponding pixels of the multi-exposure image sequence are weighted summed according to the pre-set weights, that is, the positions of the corresponding pixels are marked as (x, y). , as well as The value at pixel (x, y) , the calculation formula is expressed as ; Wherein w1, w2 and w3 are the pre-set weights corresponding to the short exposure, medium exposure and long exposure images respectively;

[0073] The choice of weights can be determined based on experience, and weights can be assigned based on the importance of different exposure images to the final effect. For example, in order to avoid the overexposed area in the long exposure image from having a greater impact on the fusion result, the weight of the long exposure image can be appropriately reduced;

[0074] S1-202: Repeat step S1-201 until all pixels are traversed to obtain a fused image processed by a pre-weighted average fusion method;

[0075] S1-3: Fusion of multi-exposure image sequences using weighted average fusion based on historical data;

[0076] S1-301: extracting a historical data set; the set includes a previously acquired multi-exposure image sequence and corresponding scene information; extracting a scene from the historical data set to match the current multi-exposure image sequence, removing each set of historical data that fails to match the scene, and retaining each set of historical data that successfully matches the scene;

[0077] S1-302: For the historical data that has been successfully matched, the collected multi-exposure image sequence is analyzed for similarity with each group of historical data. , as well as Get the corresponding , as well as ,in , as well as Indicates the numbers corresponding to the short-exposure, medium-exposure, and long-exposure images in each set of historical data, and calculates them separately and , and as well as and Bhattacharyya distance, Euclidean distance and cosine similarity;

[0078] Bhattacharyya distance is used to calculate the similarity of the color histograms of two images;

[0079] Euclidean distance is used to calculate the similarity of texture features of two sets of images;

[0080] Cosine similarity is used to calculate the similarity of semantic features of two sets of images;

[0081] Normalize the Bhattacharyya distance, Euclidean distance and cosine similarity respectively, preset the reference maximum values ​​corresponding to the Bhattacharyya distance and the Euclidean distance respectively, and calculate the ratios of the calculated Bhattacharyya distance and the Euclidean distance with the corresponding preset reference maximum values, that is, calculate the Bhattacharyya distance / corresponding reference maximum value and the Euclidean distance / corresponding reference maximum value;

[0082] Add 1 to the calculated cosine similarity, and divide the result by the integer 2 to obtain the normalized cosine similarity;

[0083] After normalization, the normalized cosine similarity, Bhattacharyya distance and Euclidean distance are multiplied by the corresponding weight coefficients respectively, and the Bhattacharyya distance and Euclidean distance are multiplied by the corresponding weight coefficients respectively and then added to obtain the similarity value 1. The cosine similarity is multiplied by the corresponding weight coefficient to obtain the similarity value 2. The similarity value 2 is divided by the similarity value 1 to finally obtain the similarity quantization value of the two sets of images.

[0084] The reason why we divide the cosine similarity by the Bhattacharyya distance and the Euclidean distance is that the smaller the Bhattacharyya distance and the Euclidean distance, the more similar they are, while the larger the cosine similarity, the more similar they are. In this way, we can get an overall similarity measure.

[0085] S1-303: Based on step S1-302, we obtain and , and as well as and Similar quantitative values ​​of and , and as well as and The similarity quantization value of the collected multi-exposure image sequence and each group of historical data are constructed;

[0086] S1-304: and The similarity quantization value is used as the length value of the solid rectangle, and The similarity quantization value of is used as the width value of the three-dimensional rectangle, and The similarity quantization value of is used as the height value of the stereo rectangle, a stereo rectangle model is constructed, and the constructed stereo rectangle model is used as a similarity model between the collected multi-exposure image sequence and each group of historical data;

[0087] S1-305: Calculate the surface area of ​​each group of similarity models as the overall similarity index of the collected multi-exposure image sequence and each group of historical data, preset the pass index corresponding to the overall similarity index, compare the overall similarity index of each group with the preset pass index, and extract the historical data with an overall similarity index higher than the pass index as high similarity data; if the overall similarity index is lower than the corresponding pass index, execute step S1-308;

[0088] S1-306: For the highly similar data extracted in step S1-305, the weights used by the short-exposure, medium-exposure, and long-exposure images of each group of highly similar data in the historical fusion process are obtained, and corresponding pixels of the multi-exposure image sequence are weighted summed according to the weights used by the highly similar data;

[0089] S1-307: Repeat step S1-306 until all pixel points are traversed to obtain a fused image processed by a weighted average fusion method based on historical data; and obtain a corresponding number of fused images according to the number of highly similar data;

[0090] S1-308: Select the historical data corresponding to the highest overall similarity index as the high similarity data, and execute step S1-306;

[0091] In summary, the two methods of preset weighted average fusion and weighted average fusion based on historical data provide a more flexible weight setting method for multi-exposure image fusion; the preset weighted average fusion can quickly set weights based on experience, while the weighted average fusion based on historical data uses the information of historical data and can better adapt to the image fusion requirements in different scenarios;

[0092] The combination of the two methods can more accurately fuse multi-exposure image sequences, thus helping users select the best fused image;

[0093] S2: Comprehensively evaluate each group of fused images obtained by fusion of different weights, and push them to the current user for screening, and use the fused image screened by the user as the final fused image;

[0094] S2-1: Comprehensively evaluate each group of fused images from the perspective of image quality;

[0095] S2-101: Preset a reference ideal image of the current scene, and for each group of fused images, calculate the mean square error between each group of fused images and the reference ideal image;

[0096] The smaller the mean square error value is, the closer the fused image is to the reference ideal image, and the better the fusion effect is;

[0097] S2-102: based on step S2-101, the mean square error of each group of fused images is obtained, and the peak signal-to-noise ratio of each group of fused images is further calculated;

[0098] The peak signal-to-noise ratio is an indicator related to the mean square error. It represents the image quality in logarithmic form. The higher the peak signal-to-noise ratio, the better the quality of the fused image, that is, the smaller the error introduced in the fusion process. The specific expression formula is: , where L is the dynamic range of the image pixel value, for example, for an 8-bit image L=255, M represents the mean square error of the fused image, and P represents the peak signal-to-noise ratio of the fused image;

[0099] S2-2: Comprehensively evaluate each group of fused images from the perspective of visual effects;

[0100] S2-201: Calculate the structural similarity index between each group of fused images and the reference ideal image;

[0101] The structural similarity index measures the naturalness by comparing the brightness, contrast, and structural information between the fused image and the reference ideal image. The range is between -1 and 1. The closer the value is to 1, the higher the naturalness.

[0102] S2-3: Comprehensive evaluation of each group of fused images from the perspective of manual evaluation;

[0103] S2-301: Send each group of fused images to an evaluator, who scores each group of fused images, and record the scoring results of each group of fused images as manual scoring; the scoring range is set to 1-5 and is a positive integer, and the higher the score, the more natural the fused image;

[0104] S2-4: construct a detail evaluation model for each group of fused images and calculate the comprehensive evaluation value of each group of fused images, arrange the comprehensive evaluation values ​​from large to small, select the first three groups of comprehensive evaluation values ​​and the corresponding detail evaluation models and push them to the user, and after receiving them, the user selects the corresponding fused images based on the three groups of detail evaluation models and the comprehensive evaluation values;

[0105] S2-401: converting the peak signal-to-noise ratio and the structural similarity index of each group of fused images into scores based on a preset conversion rule;

[0106] S2-402: preset the signal-to-noise ratio value ranges of each group corresponding to the peak signal-to-noise ratio, match the peak signal-to-noise ratio of each group of fused images with the corresponding signal-to-noise ratio value range, set each group of signal-to-noise ratio value range to correspond to a peak score; the peak score range is set to 1-5, and the higher the score, the better the quality; obtain the peak score corresponding to each group of fused images;

[0107] It should be noted that the peak signal-to-noise ratio value range [20, 50] can also be mapped to the score range [1,5]; let the peak signal-to-noise ratio value be x, and the converted score be y, the formula can be used Thus, when x=20, y=1; when x=50, y=5; if x<20, the score can be set to 1; if x>50, the score can be set to 5;

[0108] S2-403: preset the index value ranges corresponding to the structural similarity index, set each index value range to correspond to a structural score, match the structural similarity index of each group of fused images with the corresponding index value range, and obtain the structural score corresponding to each group of fused images; the structural score range is set to 1-5, and the higher the score, the better the visual effect;

[0109] It should be noted that the structural similarity index value range [0, 1] can be mapped to the score range [1, 5]. Suppose the structural similarity index value is s and the converted score is z. Then the formula z=s×4+1 can be used. In this way, when s=0, z=1, and when s=1, z=5.

[0110] S2-404: extract the peak score, structure score and manual score corresponding to each group of fusion images, construct a circle, draw three line segments from the center of the circle until they are connected to the edge of the circle, and set the angle formed by every two line segments to 120°. Divide it into 5 equal parts based on the three drawn line segments, and mark the division points. Set three line segments to correspond to the peak score, structure score and manual score respectively, extract the score values ​​corresponding to the peak score, structure score and manual score, and find the corresponding division points on the corresponding line segments based on the score values, wherein the larger the score value, the farther the corresponding division point is from the center of the circle;

[0111] After finding the corresponding division points, connect the adjacent division points to construct a triangle, and use the constructed triangle as the detail evaluation model of each group of fused images. At the same time, calculate the area value of the triangle as the comprehensive evaluation value of each group of fused images.

[0112] It should be noted that a circle is drawn, and three line segments with an angle of 120° are drawn from the center of the circle, corresponding to the peak score, structural score, and manual score respectively;

[0113] Divide each line segment into 5 equal parts according to the score range of 1-5; for the peak score, since the peak score is 3, find the third division point on the corresponding line segment; for the structure score, the structure score is 4, find the fourth division point on the corresponding line segment; for the manual score, the manual score is 3, find the third division on the corresponding line segment;

[0114] After finding these partitioning points, connect adjacent partitioning points to construct a triangle;

[0115] Image tone mapping: The final fused image is converted into a low dynamic range image through a tone mapping algorithm to adapt to the display range of ordinary display devices; the tone mapping algorithm can adopt a global tone mapping algorithm or a local tone mapping algorithm;

[0116] Image output: After the conversion is completed, the final fused image is subjected to detail enhancement processing; to further improve the detail clarity of the image, detail enhancement can be performed using a filter-based method; the image after detail enhancement is output as the final enhancement result;

[0117] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A low-light image detail enhancement method based on multi-exposure fusion, characterized in that: include: Image acquisition: Take multiple shots of the same scene at different exposure times to obtain a set of multi-exposure image sequences; Includes short exposure, medium exposure, and long exposure images; Image preprocessing: preprocess the collected multi-exposure image sequence; Multi-exposure fusion: Process the pre-processed multi-exposure image sequence and obtain the final fused image after screening; The preprocessed multi-exposure image sequence is processed and filtered to obtain the final fused image, specifically: S1: using different groups of preset methods to set different weights, and fusing the multi-exposure image sequences according to the different weights to obtain each group of fused images; the preset methods include weighted average fusion based on historical data; S2: Comprehensively evaluate each group of fused images obtained by fusion of different weights, and push them to the current user for screening, and use the fused image screened by the user as the final fused image; The multi-exposure image sequence is fused using a weighted average fusion method based on historical data, specifically: S1-301: extracting a historical data set, extracting a scene from the historical data set that matches the current multi-exposure image sequence, discarding each set of historical data that fails to match the scene, and retaining each set of historical data that succeeds in matching the scene; S1-302: For the historical data that has been successfully matched, the collected multi-exposure image sequence is analyzed for similarity with each group of historical data. , as well as Get the corresponding , as well as ,in , as well as Indicates the numbers corresponding to the short-exposure, medium-exposure, and long-exposure images in each set of historical data, and calculates them separately and , and as well as and Bhattacharyya distance, Euclidean distance and cosine similarity; Normalize the Bhattacharyya distance, Euclidean distance and cosine similarity respectively, preset the reference maximum values ​​corresponding to the Bhattacharyya distance and the Euclidean distance respectively, and calculate the ratios of the calculated Bhattacharyya distance and the Euclidean distance with the corresponding preset reference maximum values, that is, calculate the Bhattacharyya distance / corresponding reference maximum value and the Euclidean distance / corresponding reference maximum value; Add 1 to the calculated cosine similarity, and divide the result by the integer 2 to obtain the normalized cosine similarity; After normalization, the normalized cosine similarity, Bhattacharyya distance and Euclidean distance are multiplied by the corresponding weight coefficients respectively, and the Bhattacharyya distance and Euclidean distance are multiplied by the corresponding weight coefficients respectively and then added to obtain the similarity value 1. The cosine similarity is multiplied by the corresponding weight coefficient to obtain the similarity value 2. The similarity value 2 is divided by the similarity value 1 to finally obtain the similarity quantization value of the two sets of images. S1-303: Based on step S1-302, we obtain and , and as well as and Similar quantitative values ​​of and , and as well as and The similarity quantization value of the collected multi-exposure image sequence and each group of historical data are constructed; S1-304: and The similarity quantization value is used as the length value of the solid rectangle, and The similarity quantization value of is used as the width value of the three-dimensional rectangle, and The similarity quantization value of is used as the height value of the stereo rectangle, a stereo rectangle model is constructed, and the constructed stereo rectangle model is used as a similarity model between the collected multi-exposure image sequence and each group of historical data; S1-305: Calculate the surface area of ​​each group of similarity models as the overall similarity index of the collected multi-exposure image sequence and each group of historical data, preset a pass index corresponding to the overall similarity index, compare each group of overall similarity index with the preset pass index, and extract historical data with an overall similarity index higher than the pass index as high similarity data; if the overall similarity indexes are all lower than the corresponding pass index, select the historical data corresponding to the highest overall similarity index as high similarity data; S1-306: For the highly similar data extracted in step S1-305, the weights used by the short-exposure, medium-exposure, and long-exposure images of each group of highly similar data in the historical fusion process are obtained, and corresponding pixels of the multi-exposure image sequence are weighted summed according to the weights used by the highly similar data; S1-307: Repeat step S1-306 until all pixels are traversed to obtain a fused image processed by a weighted average fusion method based on historical data; Image tone mapping: convert the final fused image into a low dynamic range image; Image output: After the conversion is completed, the final fused image is subjected to detail enhancement processing, and the image with detail enhancement is output as the final enhancement result.

2. The method for low-light image detail enhancement based on multi-exposure fusion according to claim 1, characterized in that: The multi-exposure image sequences are fused according to different weights, specifically: S1-1: Label the short-exposure, medium-exposure, and long-exposure images as , as well as ; The preset methods include preset weighted average fusion and weighted average fusion based on historical data; S1-2: fuse the multi-exposure image sequence using a preset weighted average fusion method; S1-3: Fusion of multi-exposure image sequences using a weighted average fusion method based on historical data.

3. The method for low-light image detail enhancement based on multi-exposure fusion according to claim 2, characterized in that: The multi-exposure image sequence is fused using a preset weighted average fusion method, specifically: S1-201: First, the pre-set weights of the short-exposure, medium-exposure, and long-exposure images are extracted. After the extraction is completed, the corresponding pixels of the multi-exposure image sequence are weighted summed according to the pre-set weights, that is, the positions of the corresponding pixels are marked as (x, y). , as well as The value at pixel (x, y) , the calculation formula is expressed as ; Wherein w1, w2 and w3 are the pre-set weights corresponding to the short exposure, medium exposure and long exposure images respectively; S1-202: Repeat step S1-201 until all pixel points are traversed to obtain a fused image processed by a pre-weighted average fusion method.

4. The method for low-light image detail enhancement based on multi-exposure fusion according to claim 3, characterized in that: Comprehensively evaluate each group of fused images obtained by fusion with different weights and push them to the current user for screening, specifically: S2-1: Comprehensively evaluate each group of fused images from the perspective of image quality; S2-101: Preset a reference ideal image of the current scene, and for each group of fused images, calculate the mean square error between each group of fused images and the reference ideal image; S2-102: based on step S2-101, the mean square error of each group of fused images is obtained, and the peak signal-to-noise ratio of each group of fused images is further calculated; S2-2: Comprehensively evaluate each group of fused images from the perspective of visual effects; S2-201: Calculate the structural similarity index between each group of fused images and the reference ideal image; S2-3: Comprehensive evaluation of each group of fused images from the perspective of manual evaluation; S2-301: sending each group of fused images to an evaluator, who scores each group of fused images, and recording the scoring result of each group of fused images as a manual score; S2-4: construct a detail evaluation model for each group of fused images and calculate the comprehensive evaluation value of each group of fused images, arrange the comprehensive evaluation values ​​from large to small, select the first three groups of comprehensive evaluation values ​​and the corresponding detail evaluation models and push them to the user, and after receiving them, the user selects the corresponding fused images based on the three groups of detail evaluation models and the comprehensive evaluation values; S2-401: Convert the peak signal-to-noise ratio and structural similarity index of each group of fused images based on a preset conversion rule.

5. The method for low-light image detail enhancement based on multi-exposure fusion according to claim 4, characterized in that: The peak signal-to-noise ratio and structural similarity index of each group of fused images are converted into scores based on the preset conversion rules, specifically: S2-402: Preset the signal-to-noise ratio value ranges of each group corresponding to the peak signal-to-noise ratio, match the peak signal-to-noise ratio of each group of fused images with the corresponding signal-to-noise ratio value range, set each group of signal-to-noise ratio value range to correspond to a peak score, and obtain the peak score corresponding to each group of fused images; S2-403: Preset the index value ranges corresponding to the structural similarity index, set each index value range to correspond to a structural score, match the structural similarity index of each group of fused images with the corresponding index value range, and obtain the structural score corresponding to each group of fused images.

6. The method for low-light image detail enhancement based on multi-exposure fusion according to claim 5, characterized in that: Construct a detail evaluation model for each group of fused images and calculate the comprehensive evaluation value of each group of fused images, specifically: S2-404: extract the peak score, structure score and manual score corresponding to each group of fusion images, construct a circle, draw three line segments from the center of the circle until they are connected to the edge of the circle, and set the angle formed by every two line segments to 120°. Divide it into 5 equal parts based on the three drawn line segments, and mark the division points. Set three line segments to correspond to the peak score, structure score and manual score respectively, extract the score values ​​corresponding to the peak score, structure score and manual score, and find the corresponding division points on the corresponding line segments based on the score values, wherein the larger the score value, the farther the corresponding division point is from the center of the circle; After finding the corresponding division points, the adjacent division points are connected to construct a triangle, and the constructed triangle is used as the detail evaluation model of each group of fused images. At the same time, the area value of the triangle is calculated as the comprehensive evaluation value of each group of fused images.

Images