A global and local feature fusion method for image exposure correction

Abstract

The present invention relates to the field of image processing technology, and in particular to a method for image exposure correction by fusing global and local features, comprising obtaining an exposure image dataset; constructing an exposure correction model; training the exposure correction model using data in the exposure image dataset; inputting the image to be processed into the trained exposure correction model and outputting an exposure-corrected image; and performing feature extraction in parallel in the local branches of the exposure correction model by combining a small-core feature extraction module, a large-core feature extraction module, and a lightweight convolution module. The present invention can improve the shortcomings of traditional methods in detail preservation and accurately model the global semantic information of an image. The dynamic receptive field adjustment mechanism proposed in the present invention enables the network to intelligently balance the extraction accuracy and range of features at different levels, thereby significantly improving the quality of exposure correction while maintaining lightweightness, and further improving the visual effect of the image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image exposure correction method that fuses global and local features. Background Art

[0002] During image acquisition, local or global exposure anomalies, such as overexposure and underexposure, often occur due to lighting issues, limited sensor dynamic range, or improper capture parameter settings. These issues can severely degrade image visual quality and impact the accuracy of subsequent computer vision tasks, such as object detection and segmentation. Automatic exposure correction technology is particularly valuable in constrained environments, such as mobile devices and surveillance cameras.

[0003] Existing related technologies have at least the following defects: traditional exposure correction methods often rely solely on single global or local feature extraction, which makes it difficult to simultaneously take into account image detail preservation and lighting consistency, resulting in local overexposure, underexposure or color distortion in the corrected image; at the same time, existing deep learning-based models generally have too many parameters, usually exceeding 1M; the computational complexity is high, and it is difficult to meet the real-time requirements of resource-constrained scenarios such as mobile terminals; in addition, existing feature fusion strategies mostly use manual design or simple splicing or addition operations, and lack an adaptive fusion mechanism, which can easily cause brightness imbalance, artifacts or loss of details in the corrected image. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the defects existing in the above-mentioned prior art, thereby providing an image exposure correction method by fusing global and local features.

[0005] A method for image exposure correction by fusing global and local features comprises the following steps:

[0006] Get the exposure image dataset;

[0007] Build a deep learning-based exposure correction model;

[0008] The exposure correction model is trained using the data in the exposure image dataset until the training is completed;

[0009] Input the image to be processed into the trained exposure correction model and output the exposure-corrected image;

[0010] Among them, the local branch of the exposure correction model uses a small core feature extraction module combined with a lightweight convolution module, and a large core feature extraction module combined with a lightweight convolution module to perform feature extraction in parallel.

[0011] Preferably, the image to be processed is input into the trained exposure correction model to output an exposure-corrected image. Specifically, the processing flow of the image to be processed in the exposure correction model includes:

[0012] Perform 3×3 convolution on the image to be processed and output it to the local branch and global branch for processing respectively;

[0013] In the local branch, a first result is obtained based on the processing of the small core feature extraction module and the lightweight convolution module, and a second result is obtained based on the processing of the large core feature extraction module and the lightweight convolution module;

[0014] In the global branch, the data processed by the global feature extraction and enhancement module is further processed based on the color matrix and the bias adjustment to obtain a third result and a fourth result;

[0015] The original image to be processed, the first result and the second result are sequentially multiplied element by element to obtain a local branch output result;

[0016] The local branch output result is multiplied by the third result, and then element-by-element multiplied by the fourth result to obtain an exposure-corrected image.

[0017] Preferably, the global feature extraction enhancement module is composed of a global convolution and a standard convolution added in parallel;

[0018] Among them, the network structure of global convolution is: weight average-3×3 convolution-ReLU activation function;

[0019] The network structure of the standard convolution is: 3×3 convolution-Relu activation function.

[0020] Preferably, the local branch includes a first branch and a second branch arranged in parallel;

[0021] In the first branch, the first layer, the second layer, and the third layer are connected in sequence, the first layer is further summed with the third layer, and the result is subjected to 1×1 convolution to obtain the first result;

[0022] In the second branch, the fourth, fifth, and sixth layers are connected in sequence, the fourth layer is further summed with the sixth layer, and the result is subjected to 1×1 convolution to obtain the second result;

[0023] Among them, the first, second and third layers are composed of a lightweight convolution module and a small kernel feature extraction module based on 1×1 convolution;

[0024] The fourth, fifth and sixth layers are all composed of a lightweight convolution module and a large kernel feature extraction module based on 7×7 convolution.

[0025] Preferably, three channels are set in parallel in the large core feature extraction module: a fourth channel, a fifth channel, and a sixth channel; the sum of the outputs of the fourth channel, the fifth channel, and the sixth channel is the output result of the large core feature extraction module;

[0026] The fourth channel has no operation;

[0027] The network structure of the fifth channel is normalization-7×7 convolution-ReLU activation function;

[0028] The network structure of the sixth channel is upsampling-7×7 convolution-downsampling.

[0029] Preferably, the lightweight convolution module adopts a three-stage processing flow of compression-extraction-expansion, specifically including:

[0030] Use 1×1 convolution to compress the number of input channels to 1 / 4;

[0031] Spatial feature extraction through 3×3 depth-wise separable convolution;

[0032] The original number of input channels is restored using 1×1 convolution, and cross-layer residual connections are introduced.

[0033] Preferably, the global branch network structure consists of a trunk structure and two branch structures connected to the trunk structure;

[0034] Main structure: Global feature extraction enhancement module-multi-head attention mechanism:

[0035] First branch structure: color matrix;

[0036] Second branch structure: bias adjustment.

[0037] Preferably, the expression of the global feature extraction enhancement module is:

[0038] ;

[0039] The input features of the global feature extraction enhancement module are ;

[0040] Weight Range ;

[0041] The output range of the global feature extraction enhancement module is ;

[0042] Where, Indicates the batch size, Indicates the number of input channels; Indicates the number of output channels; Indicates the height of the image to be processed; Indicates the width of the image to be processed; Indicates the convolution size; Indicates the number of groups; Represents the convolution operation; represents the output features; Represents input features; Represents the height of the convolution kernel; Convolution kernel width; represents the bias term; represents the set of real numbers; The height of the output feature map; Indicates the output feature map width.

[0043] The technical solution of the present invention has the following advantages:

[0044] 1. The present invention provides a lightweight global-local feature fusion image exposure correction method. By designing a multi-scale feature extraction architecture and adopting a parallel processing mechanism of 3×3 small kernel convolution and 7×7 large kernel convolution, it achieves refined extraction and efficient fusion of image features. Among them, the small kernel convolution can effectively capture local subtle features of the image with its fine receptive field of 3×3, significantly improving the shortcomings of traditional methods in detail preservation; the large kernel convolution establishes long-distance feature dependencies by expanding the receptive field of 7×7, accurately modeling the global semantic information of the image. In particular, the dynamic receptive field adjustment mechanism proposed in the present invention adaptively optimizes the feature extraction range through up and down sampling operations, enabling the network to intelligently balance the extraction accuracy and range of features at different levels, thereby significantly improving the quality of exposure correction while maintaining lightweight, and further improving the visual effect of the image.

[0045] Second, the present invention proposes a lightweight global-local feature fusion image exposure correction method. The global branch combines standard convolution with global convolution. The standard convolution path is responsible for capturing local detail features, while the global convolution path utilizes an expanded receptive field to obtain information about the overall image structure. This design effectively addresses the feature limitations of a single convolution operation. To further enhance feature expression capabilities, the method introduces a multi-head attention mechanism, establishing long-range feature dependencies through self-attention calculations.

[0046] Third, this paper presents a lightweight global-local feature fusion image exposure correction method. This method introduces a lightweight convolution module in the local branch. This module not only significantly reduces the number of parameters but also ensures the quality of feature extraction through residual connections. Compared with traditional standard convolution, depthwise separable convolution significantly reduces computational cost while maintaining feature expression capabilities, making the model more suitable for deployment on resource-constrained devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 Schematic diagram of the overall structure of the exposure correction model of the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of the global feature extraction enhancement module of the present invention;

[0050] Figure 3 Schematic diagram of the connection structure of the lightweight convolution module and the small core feature extraction module of the present invention;

[0051] Figure 4 Schematic diagram of the connection structure of the lightweight convolution module and the large kernel feature extraction module of the present invention;

[0052] Figure 5 Schematic diagram of the exposure image correction results of the present invention on the MIT-Adobe FiveK dataset. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0055] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0056] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] Example 1

[0058] A method for image exposure correction by fusing global and local features comprises the following steps:

[0059] Get the exposure image dataset;

[0060] Build a deep learning-based exposure correction model;

[0061] The exposure correction model is trained using the data in the exposure image dataset until the training is completed;

[0062] The image to be processed is input into the trained exposure correction model, and the exposure-corrected image is output.

[0063] Specifically:

[0064] Step 1: Get the exposure image dataset:

[0065] The exposure image dataset includes images with different exposure levels and covers multiple scenes, including portraits, landscapes, indoor scenes, night scenes, and other shooting scenes. It has complex local exposure problems and samples with multiple degrees of exposure anomalies. The specific process is as follows:

[0066] Step 1.1: Download the open-source MIT-Adobe FiveK dataset. MIT-Adobe FiveK is a public benchmark dataset widely used in image enhancement and exposure correction research, built in collaboration between MIT and Adobe. MIT-Adobe FiveK contains 5,000 high-resolution RAW images, each manually graded by five professional photographers, providing realistic reference results in a variety of styles.

[0067] Step 1.2: Divide the images in the exposure image dataset obtained in step 1.1 into training, test, and validation sets in a ratio of 7:2:1; the final exposure image dataset includes 3500 training image pairs, 1000 test image pairs, and 500 validation image pairs.

[0068] Step 2: Read the exposure image dataset and perform data preprocessing; specifically:

[0069] All images in the exposure image dataset are uniformly scaled to a resolution of 512×512 pixels using the bicubic interpolation algorithm. This size strikes a balance between computational efficiency and detail preservation. Secondly, linear normalization is performed on the image pixel values, linearly mapping the original pixel values ​​from the [0, 255] interval to the [0, 1] range.

[0070] Step 3: Build a deep learning-based exposure correction model:

[0071] like Figure 1 The local branch of the exposure correction model shown uses a modular block design to achieve efficient feature extraction; it mainly combines a small-core feature extraction module with a lightweight convolution module, and a large-core feature extraction module with a lightweight convolution module to perform feature extraction in parallel.

[0072] Specifically: the local branch includes a first branch and a second branch arranged in parallel;

[0073] In the first branch, the first layer, the second layer, and the third layer are connected in sequence, the first layer is further summed with the third layer, and the result is subjected to 1×1 convolution to obtain the first result;

[0074] In the second branch, the fourth, fifth, and sixth layers are connected in sequence, the fourth layer is further summed with the sixth layer, and the result is subjected to 1×1 convolution to obtain the second result;

[0075] Among them, the first, second and third layers are composed of a lightweight convolution module and a small kernel feature extraction module based on 1×1 convolution;

[0076] The fourth, fifth and sixth layers are all composed of a lightweight convolution module and a large kernel feature extraction module based on 7×7 convolution.

[0077] Three channels are set up in parallel in the small core feature extraction module: the first channel, the second channel and the third channel; the sum of the outputs of the first channel, the second channel and the third channel is the output result of the large core feature extraction module;

[0078] No operation on the first channel;

[0079] The network structure of the second channel is normalization-3×3 convolution-ReLU activation function;

[0080] The network structure of the third channel is upsampling-3×3 convolution-downsampling.

[0081] Three channels are set in parallel in the large core feature extraction module: the fourth channel, the fifth channel, and the sixth channel; the sum of the outputs of the fourth channel, the fifth channel, and the sixth channel is the output result of the large core feature extraction module;

[0082] The fourth channel has no operation;

[0083] The network structure of the fifth channel is normalization-7×7 convolution-ReLU activation function;

[0084] The network structure of the sixth channel is upsampling-7×7 convolution-downsampling.

[0085] It should be noted that the first and fourth channels of the large-core feature extraction module and the small-core feature extraction module are used to focus on extracting dimensional features; the second and fifth channels with normalization operations extract features after their respective convolution operations; the third and sixth channels with upsampling operations extract features after sampling enhancement; finally, the features extracted by the three channels in the corresponding modules are added together to obtain local detail features that integrate multi-dimensional information. The above is a multi-path feature enhancement method based on residual connection, which gradually optimizes feature representation through position encoding, attention mechanism and MLP path. The specific process is as follows:

[0086] Position embedding enhancements:

[0087] Local branch input features ; First, triple convolution Perform position encoding and obtain enhanced features through residual connections :

[0088] ;

[0089] Attention Path:

[0090] After enhancement, the features After normalization, it goes through 1×1 convolution in sequence: , 3×3 group convolution: And 1×1 convolution: , where each layer feature is fused by element-by-element multiplication. After coefficient scaling and random depth DP processing, it is obtained through residual connection ;

[0091] ;

[0092] MLP path:

[0093] right After normalization, the multi-layer perceptron Perform feature transformation and then Coefficient scaling and random depth processing, and finally the final output is obtained through residual connection :

[0094] ;

[0095] represents element-wise multiplication; Indicates the kth normalization. In the small-core feature extraction module, A represents a 3x3 grouped convolution. In the large-core feature extraction module, A represents a 7x7 grouped convolution. are learnable scaling parameters.

[0096] like Figure 3-4 The lightweight convolution module shown uses a three-stage compression-extraction-expansion process, which includes:

[0097] Use 1×1 convolution to compress the number of input channels to 1 / 4;

[0098] Spatial feature extraction through 3×3 depth-wise separable convolution;

[0099] 1×1 convolution is used to restore the original number of input channels, and cross-layer residual connections are introduced to maintain feature integrity. Compared with traditional standard convolution, 3×3 depth-separable convolution significantly reduces the computational cost while maintaining feature expression capabilities.

[0100] like Figure 1 As shown, in this embodiment, the network structure of the global branch of the exposure correction model consists of a trunk structure and two branch structures connected to the trunk structure;

[0101] Main structure: global feature extraction enhancement module-multi-head attention mechanism;

[0102] First branch structure: color matrix;

[0103] Second branch structure: bias adjustment.

[0104] Among them, such as Figure 2 As shown in the figure, the global feature extraction and enhancement module uses a dual-path parallel architecture. The global convolution and standard convolution paths are set up in parallel to simultaneously process the input data, capturing feature information at different scales. The global convolution path is responsible for modeling the overall structure and illumination distribution characteristics of the image, while the standard convolution path focuses on extracting local detail features. The two features are added and fused and then input into the multi-head attention module. The global feature extraction and enhancement module achieves feature enhancement through the following mechanisms:

[0105] 1) Establishing long-range spatial dependencies and strengthening feature associations in semantically related regions;

[0106] 2) Dynamically adjust the fusion weights of color channels to optimize color representation;

[0107] 3) Adaptively calibrate feature bias parameters to improve the accuracy of illumination distribution.

[0108] The expression of the global feature extraction enhancement module is:

[0109] ;

[0110] The input features of the global feature extraction enhancement module are ;

[0111] Weight Range ;

[0112] The output range of the global feature extraction enhancement module is ;

[0113] Where, Indicates the batch size, Indicates the number of input channels; Indicates the number of output channels; Indicates the height of the image to be processed; Indicates the width of the image to be processed; Indicates the convolution size; Indicates the number of groups; Represents the convolution operation; represents the output features; Represents input features; Represents the height of the convolution kernel; Convolution kernel width; represents the bias term; represents the set of real numbers; The height of the output feature map; Indicates the output feature map width.

[0114] like Figure 2 The network structure of global convolution in the global feature extraction enhancement module is: weight average-3×3 convolution-ReLU activation function;

[0115] The network structure of the standard convolution is: 3×3 convolution-Relu activation function.

[0116] The image to be processed is input into the trained exposure correction model, and the exposure-corrected image is output. The specific processing flow of the image to be processed in the exposure correction model includes:

[0117] Perform 3×3 convolution on the image to be processed and output it to the local branch and global branch for processing respectively;

[0118] In the local branch, a first result is obtained based on the processing of the small core feature extraction module and the lightweight convolution module, and a second result is obtained based on the processing of the large core feature extraction module and the lightweight convolution module;

[0119] In the global branch, the data processed by the global feature extraction and enhancement module is further processed based on the color matrix and the bias adjustment to obtain a third result and a fourth result;

[0120] The original image to be processed, the first result and the second result are sequentially multiplied element by element to obtain a local branch output result;

[0121] The local branch output result is multiplied by the third result, and then element-by-element multiplied by the fourth result to obtain an exposure-corrected image.

[0122] Step 4: Train and validate the pre-built exposure correction model. The specific process is as follows:

[0123] In this embodiment, during the training process, the Adam W optimizer is used to update and optimize the network parameters; the exposure correction model based on the lightweight local-global feature fusion image principle uses two evaluation indicators, peak signal-to-noise ratio PSNR and structural similarity SSIM, to measure the similarity and quality between the corrected image and the original image.

[0124] During validation, the exposure correction model is tested using the validation set. The model's predictions are compared with the known outputs from the validation set to assess its performance. This process includes forward propagation and evaluation metrics.

[0125] Step 5: Read the test set in step 1-2 and input it into the trained exposure correction model to obtain the final exposure correction result map, as shown in Figure 5 As shown, the test set contains image comparisons under different exposure conditions:

[0126] P1 is slightly overexposed: the brightness is slightly higher than the baseline;

[0127] N1 is weak exposure: the overall brightness is insufficient.

[0128] The first column represents the original image input to the model, the second column represents the image manually restored by the expert, and the third column represents the image after being processed by the model of this embodiment. Figure 5 It is further proved that the model of this embodiment can better maintain image details and color authenticity in image enhancement tasks under different lighting conditions, and show more excellent visual enhancement effects.

[0129] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for image exposure correction by fusion of global and local features, characterized in that: The following steps are involved: Get the exposure image dataset; Build a deep learning-based exposure correction model; The exposure correction model is trained using the data in the exposure image dataset until the training is completed; Input the image to be processed into the trained exposure correction model and output the exposure-corrected image; In the local branch of the exposure correction model, a small-core feature extraction module is combined with a lightweight convolution module, and a large-core feature extraction module is combined with a lightweight convolution module to perform feature extraction in parallel. The specific process of inputting the image to be processed into the trained exposure correction model and outputting the exposure-corrected image includes: Perform 3×3 convolution on the image to be processed and output it to the local branch and global branch for processing respectively; In the local branch, a first result is obtained based on the processing of the small core feature extraction module and the lightweight convolution module, and a second result is obtained based on the processing of the large core feature extraction module and the lightweight convolution module; In the global branch, the data processed by the global feature extraction and enhancement module is further processed based on the color matrix and the bias adjustment to obtain a third result and a fourth result; The original image to be processed, the first result and the second result are sequentially multiplied element by element to obtain a local branch output result; The local branch output result is multiplied by the third result, and then element-by-element multiplied by the fourth result to obtain an exposure-corrected image.

2. The image exposure correction method based on global and local feature fusion according to claim 1, characterized in that: The global feature extraction enhancement module is composed of a global convolution and a standard convolution added in parallel; Among them, the network structure of global convolution is: weight average-3×3 convolution-ReLU activation function; The network structure of the standard convolution is: 3×3 convolution-Relu activation function.

3. The image exposure correction method based on global and local feature fusion according to claim 1, characterized in that: The local branch includes a first branch and a second branch arranged in parallel; In the first branch, the first layer, the second layer, and the third layer are connected in sequence, the first layer is further summed with the third layer, and the result is subjected to 1×1 convolution to obtain the first result; In the second branch, the fourth, fifth, and sixth layers are connected in sequence, the fourth layer is further summed with the sixth layer, and the result is subjected to 1×1 convolution to obtain the second result; Among them, the first, second and third layers are composed of a lightweight convolution module and a small kernel feature extraction module based on 1×1 convolution; The fourth, fifth and sixth layers are all composed of a lightweight convolution module and a large kernel feature extraction module based on 7×7 convolution.

4. The image exposure correction method based on global and local feature fusion according to claim 1, characterized in that: Three channels are set in parallel in the large core feature extraction module: the fourth channel, the fifth channel, and the sixth channel; the sum of the outputs of the fourth channel, the fifth channel, and the sixth channel is the output result of the large core feature extraction module; The fourth channel has no operation; The network structure of the fifth channel is normalization-7×7 convolution-ReLU activation function; The network structure of the sixth channel is upsampling-7×7 convolution-downsampling.

5. The image exposure correction method based on global and local feature fusion according to claim 1, characterized in that: The lightweight convolution module adopts a three-stage process of compression-extraction-expansion, which specifically includes: Use 1×1 convolution to compress the number of input channels to 1 / 4; Spatial feature extraction through 3×3 depth-wise separable convolution; The original number of input channels is restored using 1×1 convolution, and cross-layer residual connections are introduced.

6. The image exposure correction method based on global and local feature fusion according to claim 1, characterized in that: The network structure of the global branch consists of a trunk structure and two branch structures connected to the trunk structure; Main structure: Global feature extraction enhancement module-multi-head attention mechanism: First branch structure: color matrix; Second branch structure: bias adjustment.

7. The image exposure correction method based on global and local feature fusion according to claim 6, characterized in that: The expression of the global feature extraction enhancement module is: ； The input features of the global feature extraction enhancement module are ; Weight Range ; The output range of the global feature extraction enhancement module is ; Where, Indicates the batch size, Indicates the number of input channels; Indicates the number of output channels; Indicates the height of the image to be processed; Indicates the width of the image to be processed; Indicates the convolution size; Indicates the number of groups; Represents the convolution operation; represents the output features; Represents input features; Represents the height of the convolution kernel; Convolution kernel width; represents the bias term; represents the set of real numbers; The height of the output feature map; Indicates the output feature map width.

Images