HDR video reconstruction method based on brightness alignment

Abstract

The invention discloses an HDR video reconstruction method based on brightness alignment, and the method comprises the steps: carrying out the gamma correction of continuous frames of an input LDR video, generating an HDR image domain, and splicing the HDR image domain with an original frame to form an input tensor; constructing a brightness alignment network model, and generating brightness alignment features through a brightness attention module; generating detail features through a detail synthesis module; performing dynamic weighted fusion on the brightness alignment features and the detail features through an adaptive mixing layer, and performing up-sampling to obtain final alignment features; and finally, generating an HDR video frame by fusing the final alignment features. Through collaborative optimization of the brightness attention mechanism and the time domain alignment module, the problem of brightness inconsistency in a motion scene is effectively solved, and the HDR reconstruction quality is remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and video processing, and in particular to a method for HDR video reconstruction based on brightness alignment. Background Technology

[0002] Early high dynamic range (HDR) video generation primarily relied on dedicated hardware. HDR video capture was achieved through multi-sensor beam splitting systems (such as internal / external beam splitters) or specialized sensor designs (such as analog-to-digital sensors). For example, Tocci et al. proposed using beam splitters to distribute light to multiple sensors and simultaneously acquire images with different exposures; Zhao et al. proposed using sensor modulation techniques to generate HDR content. Furthermore, Nayar et al. extended the dynamic range by controlling exposure parameters pixel-by-pixel or line-by-line through methods such as encoded exposure, or Hajisharif et al. through dynamic ISO adjustment.

[0003] The second type of technical solution is mainly based on software algorithms for alternating exposures. For alternating exposure sequences of ordinary cameras, traditional methods generate HDR frames by optimizing optical flow alignment and weighted fusion or patch search. For example, Kang et al. used optical flow to align adjacent frames to a reference frame and then fuse them; Mangiat et al. improved this by using block motion estimation to reduce artifacts; Kalantari et al. optimized the fusion result by constraining the motion window through patch search; and Li et al. combined foreground-background separation and rank minimization techniques to improve quality.

[0004] With the rise of deep learning technology, end-to-end mapping methods based on CNNs have emerged. Wu et al. modeled HDR reconstruction as an image translation problem; Yan et al. introduced a spatial attention mechanism to suppress interference content in non-reference frames and combined it with a nonlocal network for fusion; Chen et al. performed coarse registration using optical flow and then performed finer registration by combining deformable convolution; Chung et al. registered adjacent frames with reference frames by calculating brightness-based attention scores.

[0005] Systems based on specialized hardware are expensive and have limited practicality due to their complex optical design, and their dynamic range extension capability is affected by noise.

[0006] Traditional alternating exposure algorithms rely on optical flow alignment or patch search optimization, which is computationally inefficient (processing a single frame takes several minutes to several hours) and is prone to tearing, ghosting and blurring in fast-moving or occluded scenes. At the same time, insufficient noise suppression in low-exposure frames leads to distortion in dark areas.

[0007] Existing deep learning-based technical solutions suffer from defects that lead to problems such as ghosting artifacts, loss of detail, video flickering, and poor real-time performance in video HDR reconstruction, which severely restricts their practical application in mobile devices and real-time systems. Summary of the Invention

[0008] This invention provides a brightness-aligned HDR video reconstruction method to overcome problems such as ghosting artifacts, loss of detail, video flickering, and poor real-time performance in HDR reconstruction when converting LDR video to HDR video.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows:

[0010] A brightness-aligned HDR video reconstruction method includes:

[0011] S1. Perform gamma correction on consecutive frames of the input LDR video to generate an HDR image domain; concatenate the consecutive frames of the LDR video with the HDR image domain to generate a tensor for input to the luminance alignment network model; the consecutive frames of the LDR video include a reference frame and adjacent frames.

[0012] S2. Construct a brightness alignment network model, which includes a brightness attention module, a detail synthesis module, and an adaptive hybridization layer;

[0013] The luminance attention module receives the tensor, performs downsampling on the reference frame and adjacent frames in the tensor, and performs exposure processing on the downsampled reference frame; it converts the downsampled adjacent frames and the exposure-adjusted reference frame to the YCbCr color space to extract the first Y channel; it extracts luminance features from the first Y channel, and applies an attention mechanism to the extracted luminance features and performs upsampling to obtain luminance alignment features.

[0014] The detail synthesis module converts the reference frame and adjacent frames in the tensor to the YCbCr color space to extract the second Y channel; it concatenates the second Y channel of the reference frame and adjacent frames with the reference frame and adjacent frames of the original LDR video, and applies a preset gated convolutional block to the concatenated result to obtain the intermediate features of the reference frame and adjacent frames; it uses a preset temporally deformable alignment submodule to obtain the alignment features between the intermediate features of the adjacent frames and the intermediate features of the reference frame; it then uses a preset encoder-decoder structure encoder to downsample the alignment features, and then uses a decoder to upsample them to generate detail features.

[0015] The adaptive hybridization layer fuses the brightness alignment features and detail features in a dynamic weighted manner, and then upsamples the fused features to obtain the final alignment features.

[0016] S3. Construct an HDR frame generation network, which splices and fuses the final alignment features to generate HDR frames.

[0017] Furthermore, gamma correction is performed on consecutive frames of the input LDR video to generate the expression for the HDR image domain:

[0018]

[0019] In the formula, For input LDR frames L t The frame after gamma correction; e t For L t The exposure time corresponding to the frame.

[0020] Furthermore, the specific execution process of the brightness attention module is as follows:

[0021] S211. Perform a downsampling operation on the reference frame and adjacent frames of the tensor to obtain the downsampled reference frame and adjacent frames.

[0022] S212. The exposure of the downsampled reference frame is adjusted to the exposure level of the adjacent frame using an exposure adjustment function, the expression of which is:

[0023]

[0024] In the formula, The reference frame is after exposure adjustment; γ is the gamma correction parameter.

[0025] S213. Convert the exposed reference frame and adjacent frames to the YCbCr color space and extract the first Y channel of the adjacent frames. and the first Y channel of the reference frame

[0026] S214. Use the Key-Query extractor to extract features from the first Y channel of the reference frame and adjacent frames to generate brightness features;

[0027] S215. Apply an attention mechanism to the brightness features and perform upsampling to obtain brightness-aligned features.

[0028] Furthermore, gated convolutional blocks are applied to the stitched result to obtain the expression for the intermediate features between the reference frame and adjacent frames:

[0029] F o =ELU(W f *F in )⊙Sigmoid(W g *F in (3)

[0030] In the formula, W f and W g These are the features and the gated convolution kernels, respectively; F in Refers to the result after the reference frames are stitched together. The result after splicing with adjacent frames

[0031] Furthermore, the execution process of the temporally deformable alignment submodule includes:

[0032] S221. The intermediate features of adjacent frames and the intermediate features of the reference frame are concatenated along the channel to obtain the fused intermediate features, the expression of which is:

[0033]

[0034] In the formula, [·,·] represents channel splicing;

[0035] S222. Extract the four receptive field features of the fusion intermediate features through the four dilated convolutions in the preset dilated attention fusion mechanism, and concatenate and fuse the four receptive field features to obtain the dilated features.

[0036] S223. Input the dilated features into a 3×3 convolutional layer to predict the offset field Θ required for deformable convolution. The corresponding formula is Θ=Conv(F' ti ),in, d is the number of deformable groups; K is the kernel size;

[0037] S224. Based on the predicted deformable convolution offset field, deformable convolution is used to resample the intermediate features of adjacent frames and align them to the feature space of the current reference frame to obtain aligned features.

[0038] Furthermore, the execution process of the adaptive hybrid layer is as follows:

[0039] S231. The adaptive mixing layer generates a spatial mixing feature map, which is used for operations between feature maps. Its expression is:

[0040] M∈[0,1] H×W×1 (5)

[0041] S232. The brightness alignment feature and detail feature are fused using a dynamic weighting method, and the expression is as follows:

[0042] F out =(1-M)⊙F h +M⊙(F h +F a (6)

[0043] S233, F out Perform upsampling processing to output the final aligned features.

[0044] The present invention has the following beneficial effects:

[0045] This invention constructs a brightness attention module to extract Y-channel features from a reference frame and adjacent frames, and combines an attention mechanism to focus on key brightness areas, effectively eliminating brightness jumps caused by exposure differences between multiple frames and significantly improving brightness consistency in dynamic scenes.

[0046] The detail synthesis module introduces a Y channel to enhance structure perception capabilities, and combines it with a temporal deformable alignment submodule to accurately compensate for object motion displacement. Even in complex motion scenes (such as rapid translation and rotation), it can still maintain texture clarity and avoid the common problems of detail blurring or loss in traditional methods.

[0047] The adaptive blending layer employs a dynamic weighted fusion strategy, adaptively balancing the contribution weights of brightness alignment features and detail features. This addresses the root cause of edge ghosting and afterimages caused by misalignment of moving objects, resulting in artifact-free HDR frames. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is an overall flowchart of the present invention;

[0050] Figure 2 This is a flowchart of the brightness alignment network model of the present invention;

[0051] Figure 3 This is a flowchart of the attention module of the present invention;

[0052] Figure 4 This is a flowchart of the detailed synthesis module of the present invention;

[0053] Figure 5 This is a flowchart of the time-domain deformable alignment module of the present invention;

[0054] Figure 6 This is a schematic diagram of the attention fusion module of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] This embodiment provides a brightness-aligned HDR video reconstruction method, such as... Figure 1 As shown, it includes:

[0057] S1. Perform gamma correction on consecutive frames of the input LDR video to generate an HDR image domain; concatenate the consecutive frames of the LDR video with the HDR image domain to generate a tensor for input to the luminance alignment network model; the consecutive frames of the LDR video include a reference frame and adjacent frames.

[0058] S2. Construct a brightness alignment network model, which includes a brightness attention module, a detail synthesis module, and an adaptive hybridization layer;

[0059] The luminance attention module receives the tensor, performs downsampling on the reference frame and adjacent frames in the tensor, and performs exposure processing on the downsampled reference frame; it converts the downsampled adjacent frames and the exposure-adjusted reference frame to the YCbCr color space to extract the first Y channel; it extracts luminance features from the first Y channel, and applies an attention mechanism to the extracted luminance features and performs upsampling to obtain luminance alignment features.

[0060] The detail synthesis module converts the reference frame and adjacent frames in the tensor to the YCbCr color space to extract the second Y channel; it concatenates the second Y channel of the reference frame and adjacent frames with the reference frame and adjacent frames of the original LDR video, and applies a preset gated convolutional block to the concatenated result to obtain the intermediate features of the reference frame and adjacent frames; it uses a preset temporally deformable alignment submodule to obtain the alignment features between the intermediate features of the adjacent frames and the intermediate features of the reference frame; it then uses a preset encoder-decoder structure encoder to downsample the alignment features, and then uses a decoder to upsample them to generate detail features.

[0061] The adaptive hybridization layer fuses the brightness alignment features and detail features in a dynamic weighted manner, and then upsamples the fused features to obtain the final alignment features.

[0062] S3. Construct an HDR frame generation network, which splices and fuses the final alignment features to generate HDR frames.

[0063] Specifically, gamma correction is first applied to consecutive frames of the input LDR video to generate an HDR image domain. The LDR frames and the HDR image domain are then concatenated into an input tensor. A luminance alignment network is constructed, comprising a luminance attention module, a detail synthesis module, and an adaptive mixing layer. Specifically: The luminance attention module downsamples the reference frame and adjacent frames, extracts the first Y channel through exposure adjustment and YCbCr color space conversion, fuses cross-frame luminance features using an attention mechanism, and upsamples them to generate luminance alignment features that eliminate exposure differences. The detail synthesis module extracts the second Y channel from the reference frame and adjacent frames, concatenates them, and generates intermediate features through a gated convolutional block. Motion compensation is achieved through a temporally deformable alignment submodule, and high-frequency details are preserved by combining downsampling and upsampling operations to output detail features. The adaptive mixing layer dynamically weights and fuses the luminance alignment features and detail features, upsampling to generate a final alignment feature containing global consistency and local details. Finally, the HDR frame generation network fuses the final alignment features to directly synthesize an artifact-free HDR video frame.

[0064] In a specific embodiment, the specific steps of gamma correction processing include:

[0065] S11. Extract 2N+1 consecutive frame sequences containing alternating exposure times from the LDR video, denoted as {L t-N ,…,L t ,…,L t+N}; where L t As a reference frame, adjacent frames have different exposure times; where the number of alternating exposures N is 2 (double exposure input), there are a total of 5 frames; when N is 3, there are a total of 7 frames.

[0066] S12, For each LDR reference frame L t Perform gamma correction to generate the corresponding HDR domain image X. t Its expression is:

[0067]

[0068] In the formula, e t For L t The exposure time corresponding to the frame; For input LDR frames L t The frame after gamma correction, where γ = 2.2;

[0069] S13, Transfer the original LDR reference frame L t With the corresponding HDR domain image X t The input tensor is spliced ​​along the channel dimension to form a 6-channel input tensor.

[0070] In a specific embodiment, the execution process of the brightness attention module is as follows:

[0071] S211, regarding the reference frame L of the tensor t and adjacent frames L t+i Perform downsampling processing, downsampling the resolution to 1 / 4 of the original resolution, to obtain... and

[0072] S212. The exposure of the downsampled reference frame is adjusted to the exposure level of the adjacent frame using an exposure adjustment function, the expression of which is:

[0073]

[0074] In the formula, e t for Exposure time; e t+i for Exposure time;

[0075] S213. Convert the exposed reference frame and adjacent frames to the YCbCr color space, and extract the first Y channel of the reference frame and adjacent frames. and

[0076] S214, using Key-Query Extractor F KQ Feature extraction is performed on the first Y channel of the reference frame and adjacent frames to generate luminance features, the expression of which is:

[0077]

[0078] In the formula, Q is the query for the reference frame; K is the key of the adjacent frame; k i For adjacent frame feature blocks, q i For the reference frame feature blocks, each has a dimension of 3×3×number of channels, where K and Q are expanded into multiple smaller blocks, each of which is k. i q i There are a total of n blocks, where i represents the index of each block.

[0079] The Key-Query Extractor F KQ It consists of 3×3 convolutional layers and 4 residual Fourier convolutional blocks;

[0080] S215. Calculate the cosine similarity matrix between small blocks, the expression of which is:

[0081]

[0082] In the formula, c i,j The inter-block similarity score is calculated using cosine similarity.

[0083] S216. For each query block q i Select the most relevant key block index pi and confidence level s i Its expression is:

[0084]

[0085] S217. Use the Value extractor to process adjacent downsampled frames. Extract value feature V; based on index p i The value features V are rearranged to obtain the alignment features V'; the original value features and the alignment features are concatenated, and the concatenation result is multiplied by the confidence score. The multiplication result is then upsampled to obtain the brightness alignment feature F. a .

[0086] In a specific embodiment, the execution process of the detail synthesis module is as follows:

[0087] S221. Convert the reference frame and adjacent frames of the tensor to the YCbCr color space, extract the second Y channel of the reference frame and adjacent frames of the tensor, and stitch them together with the reference frame and adjacent frames of the original LDR video along the channel.

[0088] S222. Apply a preset gated convolutional block to the stitched result to obtain intermediate features between the reference frame and adjacent frames. The calculation expression for the intermediate features is as follows:

[0089] F o =ELU(W f *F in )⊙Sigmoid(W g *F in (6)

[0090] In the formula, W f and W g These are the features and the gated convolution kernels, respectively; F in Refers to the result after the reference frames are stitched together. The result after splicing with adjacent frames When the input is The calculated output is: When the input is The calculated output is:

[0091] S223. Input the intermediate features into the preset temporal deformable alignment module to obtain the alignment features.

[0092] S224. The encoder, using a pre-defined encoder-decoder structure, aligns the features. The sample is downsampled and then upsampled by the decoder to obtain the high-resolution detail features F. h .

[0093] In a specific embodiment, the execution process of the temporal deformable alignment module is as follows:

[0094] S231. The intermediate features of adjacent frames and the intermediate features of the reference frame are concatenated along the channel to obtain the fused intermediate features, the expression of which is:

[0095]

[0096] In the formula, [·,·] represents channel splicing;

[0097] S232. The fused intermediate features are input into a preset dilation attention mechanism, and dilation is performed through four parallel 3×3 dilation convolutions with different dilation rates, which are set to 1, 2, 3, and 4 respectively, to obtain... Will and Adding them together gives Will and Adding them together gives Will and Adding them together gives Will After concatenation, the data is processed through a channel attention mechanism and a 1×1 convolution, and then combined with the intermediate feature F. o Adding them together yields the extended feature F' ti ;

[0098] S233, Expanding feature F' ti The input is fed into a 3×3 convolutional layer to predict the offset field Θ required for deformable convolution. The corresponding formula is Θ=Conv(F' ti ),in, d is the number of deformable groups; K is the kernel size;

[0099] S234. Based on the predicted deformable convolution offset field, deformable convolution is used to apply the intermediate features of adjacent frames. Resampling is performed, and the data is aligned to the feature space of the current reference frame to obtain the aligned features.

[0100] In a specific embodiment, the execution process of the adaptive hybrid layer is as follows:

[0101] S241. The adaptive mixing layer generates a spatial mixing feature map, which is used for operations between feature maps. Its expression is:

[0102] M∈[0,1] H×W×1 (8)

[0103] S242. The brightness alignment feature and detail feature are fused using a dynamic weighting method, and the expression is as follows:

[0104] F out =(1-M)⊙F h +M⊙(F h +F a (9)

[0105] S243, F out Perform upsampling processing to output the final aligned features.

[0106] In a specific embodiment, an HDR frame generation network is constructed using one 3×3 convolutional layer and five residual Fourier convolutional blocks. Features output from multiple brightness-aligned network models are concatenated along the channel dimension, and global feature fusion is performed through the HDR frame generation network. Finally, sigmoid activation is used to generate HDR frames. t .

[0107] In this embodiment, random flipping, 90° rotation, and color enhancement are preferably used as data augmentation methods;

[0108] Preferably, since HDR images typically require tone mapping before display, the loss function uses a difference μ-law function to calculate the predicted HDR frame after color mapping. Compared to true HDR frames T(H) t The loss between ) is expressed as:

[0109]

[0110] In the formula, μ is set to 5000.

[0111] Preferably, a joint loss function is constructed using L1 loss, perceptual loss, frequency loss, and temporal loss, where:

[0112] The L1 loss formula is:

[0113]

[0114] The formula for perceived loss is:

[0115]

[0116] In the formula, φ(·) represents the feature extraction network obtained by the pre-trained ResNet-50;

[0117] The formula for frequency loss is:

[0118]

[0119] In the formula, Represents Fast Fourier Transform;

[0120] The formula for time loss is:

[0121]

[0122] In the formula, ΔH t =T(H t )-Τ(H t -1); ε is set to 10 -3 ;

[0123] The constructed joint loss function is:

[0124] L total =λ1L1+λ per L per +λ freq L freq +λ temp L temp (15)

[0125] In the formula, λ1 is set to 1; λ per , λ freq , λ temp Set it to 0.1.

[0126] Preferably, for triple exposure input, there is a total of 7 frame sequences; each adjacent frame is aligned with a reference frame through an independent luminance alignment network, and the HDR frame generation network fuses all features to generate an HDR frame; the network structure is consistent with the double exposure scene, only the number of input frames and the number of branches of the luminance alignment network are increased accordingly.

[0127] Preferably, the datasets used in this invention are the Cinematic Video and DeepHDRVideo datasets. The Cinematic Video dataset contains high dynamic range video sequences covering different lighting scenes, providing multi-exposure LDR frames and their HDR ground truth, and supporting temporal consistency evaluation; the DeepHDRVideo dataset contains three-exposure LDR video pairs, accompanied by accurate optical flow data and dynamic region masks, and specifically handles motion blur and strong exposure differences, providing benchmark tests for motion compensation and detail recovery algorithms.

[0128] The present invention has the following beneficial effects:

[0129] This invention constructs a brightness attention module to extract Y-channel features from a reference frame and adjacent frames, and combines an attention mechanism to focus on key brightness areas, effectively eliminating brightness jumps caused by exposure differences between multiple frames and significantly improving brightness consistency in dynamic scenes.

[0130] The detail synthesis module introduces a Y channel to enhance structure perception capabilities, and combines it with a temporal deformable alignment submodule to accurately compensate for object motion displacement. Even in complex motion scenes (such as rapid translation and rotation), it can still maintain texture clarity and avoid the common problems of detail blurring or loss in traditional methods.

[0131] The adaptive blending layer employs a dynamic weighted fusion strategy, adaptively balancing the contribution weights of brightness alignment features and detail features. This addresses the root cause of edge ghosting and afterimages caused by misalignment of moving objects, resulting in artifact-free HDR frames.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for HDR video reconstruction based on brightness alignment, characterized in that, include: S1. Perform gamma correction on consecutive frames of the input LDR video to generate an HDR image domain; concatenate the consecutive frames of the LDR video with the HDR image domain to generate a tensor for input to the luminance alignment network model; the consecutive frames of the LDR video include a reference frame and adjacent frames. S2. Construct a brightness alignment network model, which includes a brightness attention module, a detail synthesis module, and an adaptive hybridization layer; The luminance attention module receives the tensor, performs downsampling on the reference frame and adjacent frames in the tensor, and performs exposure processing on the downsampled reference frame; it converts the downsampled adjacent frames and the exposure-adjusted reference frame to the YCbCr color space to extract the first Y channel; it extracts luminance features from the first Y channel, and applies an attention mechanism to the extracted luminance features and performs upsampling to obtain luminance alignment features. The detail synthesis module converts the reference frame and adjacent frames in the tensor to the YCbCr color space to extract the second Y channel; it concatenates the second Y channel of the reference frame and adjacent frames with the reference frame and adjacent frames of the original LDR video, and applies a preset gated convolutional block to the concatenated result to obtain the intermediate features of the reference frame and adjacent frames; it uses a preset temporally deformable alignment submodule to obtain the alignment features between the intermediate features of the adjacent frames and the intermediate features of the reference frame; it then uses a preset encoder-decoder structure encoder to downsample the alignment features, and then uses a decoder to upsample them to generate detail features. The adaptive hybridization layer fuses the brightness alignment features and detail features in a dynamic weighted manner, and then upsamples the fused features to obtain the final alignment features. S3. Construct an HDR frame generation network, which splices and fuses the final alignment features to generate HDR frames.

2. The HDR video reconstruction method based on brightness alignment according to claim 1, characterized in that, The expression for generating the HDR image domain by performing gamma correction on consecutive frames of the input LDR video is: In the formula, For input LDR frames L t The frame after gamma correction; e t For L t The exposure time corresponding to the frame.

3. The HDR video reconstruction method based on brightness alignment according to claim 1, characterized in that, The specific execution process of the brightness attention module is as follows: S211. Perform a downsampling operation on the reference frame and adjacent frames of the tensor to obtain the downsampled reference frame and adjacent frames. S212. The exposure of the downsampled reference frame is adjusted to the exposure level of the adjacent frame using an exposure adjustment function, the expression of which is: In the formula, The reference frame is after exposure adjustment; γ is the gamma correction parameter. S213. Convert the exposed reference frame and adjacent frames to the YCbCr color space and extract the first Y channel of the adjacent frames. and the first Y channel of the reference frame S214. Use the Key-Query extractor to extract features from the first Y channel of the reference frame and adjacent frames to generate brightness features; S215. Apply an attention mechanism to the brightness features and perform upsampling to obtain brightness-aligned features.

4. The HDR video reconstruction method based on brightness alignment according to claim 1, characterized in that, Applying a pre-defined gated convolutional block to the stitched result to obtain the expression for the intermediate features between the reference frame and adjacent frames is as follows: F o =ELU(W f *F in )⊙Sigmoid(W g *F in ) (3) In the formula, W f and W g These are the features and the gated convolution kernels, respectively; F in Refers to the result after the reference frames are stitched together. The result after splicing with adjacent frames 5. The HDR video reconstruction method based on brightness alignment according to claim 1, characterized in that, The execution process of the temporally deformable alignment submodule includes: S221. The intermediate features of adjacent frames and the intermediate features of the reference frame are concatenated along the channel to obtain the fused intermediate features, the expression of which is: In the formula, [·,·] represents channel splicing; S222. Extract the four receptive field features of the fusion intermediate features through the four dilated convolutions in the preset dilated attention fusion mechanism, and concatenate and fuse the four receptive field features to obtain the dilated features. S223. Input the dilated features into a 3×3 convolutional layer to predict the offset field Θ required for deformable convolution. The corresponding formula is Θ=Conv(F′ ti ),in, d is the number of deformable groups; K is the kernel size; S224. Based on the predicted deformable convolution offset field, deformable convolution is used to resample the intermediate features of adjacent frames and align them to the feature space of the current reference frame to obtain aligned features.

6. The HDR video reconstruction method based on brightness alignment according to claim 1, characterized in that, The execution process of the adaptive hybrid layer is as follows: S231. The adaptive mixing layer generates a spatial mixing feature map, which is used for operations between feature maps. Its expression is: M∈[0,1] H×W×1 (5) S232. The brightness alignment feature and detail feature are fused using a dynamic weighting method, and the expression is as follows: F out =(1-M)⊙F h +M⊙(F h +F a ) (6) S233, F out Perform upsampling processing to output the final aligned features.

Images