A high dynamic range image synthesis method and system

Abstract

The application relates to the technical field of image processing, and discloses a high-dynamic-range image synthesis method and system, wherein a first image synthesis model for first-stage processing is constructed based on a U-shaped network improved by adopting an affine transformation network and a channel attention mechanism; the improved U-shaped network is used to obtain pseudo-exposure image data according to prior information and middle-exposure image data; the prior information comprises information extraction on low-exposure image data or high-exposure image data and the middle-exposure image data, and first low-exposure region prior information and first high-exposure region prior information are obtained correspondingly; a diffusion network is integrated between a ground truth encoder and the U-shaped network to regenerate the first low-exposure region prior information and the first high-exposure region prior information, and a second image synthesis model for second-stage processing is obtained; and the second image synthesis model is trained to obtain a target image synthesis model. The method improves the quality of the generated high-dynamic-range image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a high dynamic range image synthesis method and system. Background Technology

[0002] With the rapid development of computer vision, intelligent photography, and image enhancement technologies, high dynamic range (HDR) images have become an important means to enhance visual experience and can effectively solve the problem of detail loss in traditional low dynamic range images under extreme lighting conditions.

[0003] Existing high dynamic range (HVR) image synthesis methods include multi-image HVR image reconstruction methods and single-image HVR image reconstruction methods. Among them, multi-image HVR image reconstruction methods are based on multi-exposure image alignment and weighted fusion, which requires strict alignment of input images. However, due to slight shaking of the shooting equipment, dynamic changes in the scene, and / or limitations of shooting conditions, it is difficult to obtain perfectly aligned multi-exposure image sequences. Furthermore, when dealing with complex lighting distributions, multi-image HVR image reconstruction methods are prone to unreasonable weight allocation, resulting in artifacts such as halos, ghosting, or blurred details in the fusion results, which affect image quality.

[0004] Therefore, improving the quality of high dynamic range image synthesis has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides a high dynamic range image synthesis method and system to solve the technical problem of how to improve the quality of high dynamic range image synthesis, and to achieve the effect of effectively improving the quality of high dynamic range image synthesis.

[0006] In a first aspect, the present invention provides a high dynamic range image synthesis method, the method comprising: The acquired image data set to be synthesized is input into a pre-constructed target image synthesis model for sequential first-stage processing and second-stage processing to obtain high dynamic range image data. The construction of the target image synthesis model includes: Based on the improved U-shaped network using affine transformation network and channel attention mechanism, a first image synthesis model for the first stage of processing is constructed. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium exposure image data. The prior information includes first low exposure region prior information obtained by extracting information from low exposure image data and medium exposure image data, and first high exposure region prior information obtained by extracting information from high exposure image data and medium exposure image data. The pseudo-exposure image data is used to fuse with the medium exposure image data to obtain the high dynamic range image data. The diffusion network is integrated between the trained ground truth encoder and the improved U-shaped network to obtain the second image synthesis model of the second stage processing. The diffusion network is used to regenerate the first low exposure region prior information and the first high exposure region prior information based on the condition vector extracted from the middle exposure image data, respectively, to obtain the second low exposure region prior information and the second high exposure region prior information. The second image synthesis model is trained to obtain the target image synthesis model.

[0007] Preferably, obtaining the false exposure image data based on prior information and medium exposure image data includes: The exposure time is sequentially expanded and broadcast to obtain time features, and the time features and the middle exposure image data are concatenated by channel dimension to obtain input image data. The input image data is convolved through the coding layer of the improved U-shaped network to obtain the first feature image data; The improved U-shaped network is used to fuse the first feature image data and the prior information to obtain the second feature image data. The second feature image data is enhanced by the channel attention mechanism of the improved U-shaped network to obtain the third feature image data. After the third feature image data is decoded by the decoding layer of the improved U-shaped network, it is skipped and connected with the first feature image data to obtain exposure time feature and image feature data. The exposure time feature and the image feature data are multiplied pixel by pixel to obtain the pseudo exposure image data. The pseudo exposure image data includes pseudo low exposure image data corresponding to the low exposure image data and pseudo high exposure image data corresponding to the high exposure image data.

[0008] Preferably, the step of fusing the first feature image data and the prior information using the affine transformation network of the improved U-shaped network to obtain the second feature image data includes: The first feature image data is subjected to tensor reconstruction processing to obtain the fourth feature image data; The prior information is adjusted in terms of the number of channels to obtain adjusted prior information with the same number of channels as the fourth feature image data; After multiplying the fourth feature image data and the adjusted prior information pixel by pixel, the adjusted prior information is added pixel by pixel to obtain the second feature map data.

[0009] Preferably, the first low-exposure region prior information obtained by extracting information from the low-exposure image data and the medium-exposure image data, and the first high-exposure region prior information obtained by extracting information from the high-exposure image data and the medium-exposure image data, include: After concatenating the low-exposure image data and the medium-exposure image data along the channel dimension, residual processing, convolution processing and average pooling processing are performed sequentially to obtain the first fused image feature. The low-exposure image data is then subjected to the residual processing, convolution processing and average pooling processing sequentially to obtain the low-exposure light region feature. After stitching the first fused image features and the low-exposure light region features in the dimensional direction, multi-layer perception processing is performed to obtain the prior information of the first low-exposure region. After concatenating the high-exposure image data and the medium-exposure image data along the channel dimension, the residual processing, convolution processing, and average pooling processing are performed sequentially to obtain the second fused image feature. The high-exposure image data is then subjected to the residual processing, convolution processing, and average pooling processing sequentially to obtain the high-exposure light region feature. After stitching the second fused image features and the high-exposure light region features along the dimensional direction, the multilayer perception processing is performed to obtain the prior information of the first high-exposure region.

[0010] Preferably, the step of regenerating the prior information of the first low-exposure region and the prior information of the first high-exposure region based on the conditional vector extracted from the exposed image data, respectively, to obtain the prior information of the second low-exposure region and the prior information of the second high-exposure region, includes: The intermediate exposure image data is sequentially subjected to the residual processing, the convolution processing, and the average pooling processing to obtain intermediate features, and the intermediate features are subjected to the multilayer perceptron processing to obtain a conditional vector; The prior information of the first low-exposure region is subjected to forward diffusion processing to obtain the noise vector of the low-exposure region. After the condition vector and the noise vector of the low-exposure region are concatenated in the dimensional direction, multi-layer perceptual denoising processing is performed to obtain the prior information of the second low-exposure region. The prior information of the first high-exposure region is subjected to the forward diffusion process to obtain the noise vector of the high-exposure region. After the condition vector and the noise vector of the high-exposure region are concatenated in the dimensional direction, multilayer perceptual denoising processing is performed to obtain the prior information of the second high-exposure region.

[0011] Preferably, the high dynamic range image data obtained by fusing the high dynamic range image data with the intermediate exposure image data includes: Obtain the brightness data of the medium exposure image data, and based on the brightness data and a pre-set dynamic weighting trigonometric function, obtain the pseudo-low exposure image weight, pseudo-high exposure image weight, and medium exposure image weight. The pseudo-low exposure image data, the pseudo-high exposure image data, and the medium exposure image data are respectively subjected to gamma correction to obtain the corresponding corrected pseudo-low exposure image data, pseudo-high exposure image data, and medium exposure image data; The high dynamic range image data is obtained by weighting the corrected pseudo-low exposure image data, pseudo-high exposure image data, and medium exposure image data according to the pseudo-low exposure image weight, the pseudo-high exposure image weight, and the medium exposure image weight.

[0012] Preferably, the method further includes: The first image synthesis model is configured with a first loss function, which includes a first average absolute value error loss term, a second average absolute value error loss term, and a third average absolute value error loss term. The first average absolute value error loss term represents the loss between the predicted value and the true value of the high dynamic range image data, the second average absolute value error loss term represents the loss between the pseudo low exposure image data and the low exposure image data, and the third average absolute value error loss term represents the loss between the pseudo high exposure image data and the high exposure image data. A second loss function is set for the second image synthesis model. The second loss function includes: a first average absolute value error loss term, a second average absolute value error loss term, a third average absolute value error loss term, a fourth average absolute value error loss term, and a fifth average absolute value error loss term. The fourth average absolute value error loss term represents the loss between the prior information of the first low-exposure area and the prior information of the second low-exposure area. The fifth average absolute value error loss term represents the loss between the prior information of the first high-exposure area and the prior information of the second high-exposure area.

[0013] In a second aspect, the present invention also provides a high dynamic range image synthesis system to implement the high dynamic range image synthesis method described above, the system comprising: a high dynamic range image synthesis module; The high dynamic range image synthesis module is used to input the acquired image data set to be synthesized into the pre-constructed target image synthesis model for first-stage processing and second-stage processing in sequence to obtain high dynamic range image data. The high dynamic range image synthesis module includes: a first image synthesis model building unit, a second image synthesis model building unit, and a model training unit; The first image synthesis model construction unit is used to construct a first image synthesis model for the first stage of processing based on an improved U-shaped network and a truth encoder. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes a first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and a first high-exposure region prior information obtained by extracting information from high-exposure image data and medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain the high dynamic range image data. The second image synthesis model building unit is used to integrate the diffusion network between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model that performs the second stage of processing. The diffusion network is used to regenerate the first low-exposure region prior information and the first high-exposure region prior information based on the condition vector extracted from the middle exposure image data, respectively, to obtain the second low-exposure region prior information and the second high-exposure region prior information. The model training unit is used to freeze the weight parameters of the truth encoder and train the second image synthesis model to obtain the target image synthesis model.

[0014] Preferably, the first image synthesis model building unit includes: a channel dimension splicing subunit, a convolution processing subunit, an affine transformation subunit, a feature enhancement processing subunit, a decoding connection subunit, and an image output subunit; The channel dimension stitching subunit is used to perform dimensional expansion and broadcasting operations on the exposure time sequentially to obtain time features, and then stitch the time features and the middle exposure image data together in the channel dimension to obtain input image data. The convolution processing subunit is used to perform convolution processing on the input image data through the coding layer of the improved U-shaped network to obtain the first feature image data; The affine transformation subunit is used to perform feature fusion on the first feature image data and the prior information through the affine transformation network of the improved U-shaped network to obtain the second feature image data. The feature enhancement processing subunit is used to perform key feature enhancement processing on the second feature image data through the channel attention mechanism of the improved U-shaped network to obtain the third feature image data. The decoding connection subunit is used to decode the third feature image data through the decoding layer of the improved U-shaped network, and then make a skip connection with the first feature image data to obtain exposure time feature and image feature data. The image output subunit is used to perform a pixel-by-pixel multiplication operation on the exposure time feature and the image feature data to obtain the pseudo exposure image data. The pseudo exposure image data includes pseudo low exposure image data corresponding to the low exposure image data and pseudo high exposure image data corresponding to the high exposure image data.

[0015] Preferably, the second image synthesis model construction unit includes: a conditional vector acquisition subunit, a low-exposure region prior information acquisition subunit, and a high-exposure region prior information acquisition subunit; The conditional vector acquisition subunit is used to sequentially perform the residual processing, the convolution processing, and the average pooling processing on the intermediate exposure image data to obtain intermediate features, and then perform the multilayer perceptron processing on the intermediate features to obtain a conditional vector. The low-exposure region prior information acquisition subunit is used to perform the forward diffusion processing on the first low-exposure region prior information to obtain a low-exposure region noise vector, and after performing the dimensional direction concatenation on the condition vector and the low-exposure region noise vector, perform multi-layer perception denoising processing to obtain the second low-exposure region prior information. The high-exposure region prior information acquisition subunit performs forward diffusion processing on the first high-exposure region prior information to obtain a high-exposure region noise vector. After concatenating the condition vector and the high-exposure region noise vector in the dimensional direction, it performs multi-layer perceptual denoising processing to obtain the second high-exposure region prior information.

[0016] This application provides a high dynamic range image synthesis method and system. Compared with the prior art, the beneficial effects of the embodiments of this application are as follows: This application discloses a high dynamic range (HDL) image synthesis method, proposing a two-stage HDL image synthesis network. It employs an affine transformation network and a channel attention mechanism to improve the U-shaped network, enabling the prediction of structure and brightness information of multi-exposure images from medium-exposure image data. A diffusion model is used to regenerate prior information, better utilizing the prior information from multi-exposure images and improving the quality of the generated HDL images. By incorporating exposure time into the target image synthesis model, strict alignment of multi-exposure images is not required to achieve time-based image enhancement, resulting in more accurate generated HDL images. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the steps of a method for constructing a target image synthesis model according to a preferred embodiment of the present invention; Figure 2 This is a schematic diagram of the target image synthesis model provided in a preferred embodiment of the present invention; Figure 3This is a schematic diagram of the structure of a truth encoder provided in a preferred embodiment of the present invention; Figure 4 This is a schematic diagram of the coding layer structure of the improved U-shaped network provided in a preferred embodiment of the present invention; Figure 5 This is a schematic diagram of the decoding layer of the improved U-shaped network provided in a preferred embodiment of the present invention; Figure 6 This is a schematic diagram of the overall structure of the improved U-shaped network provided in a preferred embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of an affine transformation network provided in a preferred embodiment of the present invention; Figure 8 This is a schematic diagram of the channel attention mechanism provided in a preferred embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of an input encoder provided in a preferred embodiment of the present invention; Figure 10 This is a schematic diagram of the diffusion network provided in a preferred embodiment of the present invention; Figure 11 This is a schematic diagram of the structure of a high dynamic range image synthesis system provided in a preferred embodiment of the present invention; Figure label: 1-High dynamic range image synthesis module, 11-First image synthesis model construction unit, 12-Second image synthesis model construction unit, 13-Model training unit. Detailed Implementation

[0018] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are provided for illustrative purposes only and should not be construed as limiting the invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of this invention. In the description of this invention, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0019] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Those skilled in the art will understand the specific meaning of these terms in this invention based on the specific circumstances.

[0020] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0021] In an embodiment of the present invention, a high dynamic range (HDR) image synthesis method is provided. The method includes: inputting a group of acquired image data to be synthesized into a pre-constructed target image synthesis model and sequentially performing a first-stage processing and a second-stage processing to obtain HDR image data; the group of image data to be synthesized includes low-exposure image data, medium-exposure image data, and high-exposure image data. HDR images are images capable of recording and displaying a wider range of brightness, capturing light changes from extremely dark to extremely bright, covering a brightness range far exceeding that of the human eye and traditional image formats. Traditional LDR images (low dynamic range images) typically only represent a brightness ratio of approximately 100:1, while HDR images can reach a brightness ratio of 100,000:1 or even higher. HDR images retain complete detail. In traditional LDR images, bright areas are prone to "pure white overexposure," while dark areas may appear as "pure black with no detail," whereas HDR images can simultaneously retain the texture of bright areas and the detail of dark areas. The high dynamic range image synthesis method provided in this application is based on low-exposure image data, medium-exposure image data, and high-exposure image data. A target image synthesis model constructed using convolutional networks and diffusion networks is used to perform first-stage processing and second-stage processing sequentially to obtain a high dynamic range image, achieving image detail enhancement and dynamic range expansion. The construction of the target image synthesis model is shown below: Please see Figure 1 The diagram illustrates the steps of a method for constructing a target image synthesis model, wherein the construction of the target image synthesis model includes: S1. Based on an improved U-shaped network and a truth encoder, a first image synthesis model is constructed to perform the first stage of processing. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes first low-exposure region prior information obtained by extracting information from low-exposure image data and the medium-exposure image data, and first high-exposure region prior information obtained by extracting information from high-exposure image data and the medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain the high dynamic range image data. In a preferred embodiment of this application, as shown... Figure 2The diagram shows the structure of the target image synthesis model. The first type of dashed line (dotted line) in the diagram illustrates the first-stage processing flow. For the first image synthesis model undergoing the first-stage processing, a truth encoder, a U-shaped network, an affine transformation network, a channel attention mechanism, and an image fusion network are employed. In a preferred embodiment of this application, as... Figure 2 The diagram shows the structure of the target image synthesis model, which includes three image data processing branches. The first branch processes low-exposure image data I1 and medium-exposure image data I2 to obtain pseudo-low-exposure image data, and includes a first ground truth encoder and a first U-shaped network. The second branch processes high-exposure image data I3 and medium-exposure image data I2 to obtain pseudo-high-exposure image data, and includes a second ground truth encoder and a second U-shaped network. The structures of the first and second branches are identical, differing only in the objects they process. The third branch fuses the pseudo-low-exposure image data output from the first branch, the pseudo-high-exposure image data output from the second branch, and the medium-exposure image data to obtain high dynamic range image data, and includes an image fusion network. An affine transformation network and a channel attention mechanism are integrated into the U-shaped network to improve it, resulting in an improved U-shaped network.

[0022] like Figure 3The diagram shows the structure of a truth encoder, including branch 1 and branch 2. Branch 1 receives and processes a stitched image of medium-exposure image data and low-exposure image data or high-exposure image data along the channel dimension. Branch 2 receives and processes low-exposure image data or high-exposure image data. Both branch 1 and branch 2 sequentially include 9 residual blocks, 2 branch convolutional layers, and 1 average pooling layer (Avg Pool shown in the diagram). The residual blocks sequentially include Conv (convolutional layer), ReLU activation function (rectified linear activation function), and Conv again. The input of each residual block passes through 2 convolutional layers with a kernel size of 3×3 and C channels, and is then added pixel by pixel to obtain the output of the residual block. The ReLU activation function is used between the convolutional layers. The branch convolutional layers sequentially include Conv and Leaky ReLU (Leaky Rectified Linear Activation Function). The kernel size of the branch convolutional layers is 3×3, and the number of channels is C. The outputs of the two branches are concatenated along the dimensional direction and then input into the multilayer perceptron to obtain the prior information of the first low-exposure region. The multilayer perceptron consists of four linear layers to adjust the number of feature channels to 256. Each linear layer is followed by a Leaky ReLU activation function. The input to branch 1 of the ground truth encoder is a concatenated image of medium-exposure image data and low-exposure image data or high-exposure image data along the channel dimension, which has rich and stable feature representation. Therefore, a relatively simple structure of convolutional layers and Leaky ReLU function is used to improve efficiency. The input to branch 2 of the truth encoder is either low-exposure or high-exposure image data. Both low-exposure and high-exposure image data lack the normally exposed image features from the high-exposure image data, resulting in extreme brightness and detail loss, leading to unstable feature distribution. To enhance the stability and robustness of learning under these conditions, a normalization layer (BatchNorm, illustrated as BN) is introduced, combined with a convolutional layer and the Leaky ReLU activation function to enhance adaptability to unstable feature distributions. The outputs of the two branches of the truth encoder are compressed to a 1×1 storage size by an average pooling layer and then input into a multilayer perceptron, corresponding to the prior information of the first low-exposure region, represented as a feature vector. The outputs of the two branches of the truth encoder are compressed to a 1×1 storage size by an average pooling layer and then input into a multilayer perceptron to obtain the prior information of the first high-exposure region, which is represented as a feature vector. .

[0023] In a preferred embodiment of this application, an affine transformation network and a channel attention mechanism are added to each coding layer of the encoder to improve the U-shaped network encoder. The input of the improved U-shaped network is the exposure time, the mid-exposure image data, and prior information. The exposure time is sequentially expanded and broadcast to obtain temporal features. The temporal features and the mid-exposure image data are then concatenated along the channel dimension to obtain the input image data. The input image data is processed by the encoder of the improved U-shaped network. The processing flow of each coding layer of the encoder is the same: the input image data is first convolved to obtain the first feature image data. The second feature image data is obtained by fusing features from the first feature image data and prior information using an affine transformation network. The second feature image data is enhanced using a channel attention mechanism to obtain the third feature image data. .like Figure 6 The diagram shows the overall structure of the improved U-shaped network. The third feature image data includes the third feature data output from the first coding layer. The third feature data output by the second coding layer The third feature data output by the third coding layer The third feature data output by the fourth coding layer The third feature data output from the fifth coding layer is then decoded sequentially through the decoding layer of the U-shaped network. This decoded data is then connected in a skip connection with the first feature image data from the corresponding layer to obtain the exposure time feature. and image feature data The exposure time features and image feature data are multiplied pixel by pixel to obtain pseudo-exposure image data.

[0024] like Figure 4 The diagram shows the structure of the encoding layer in the improved U-shaped network. The encoder of the U-shaped network has five encoding layers. These layers sequentially use 3×3 Conv (convolutional layers) to adjust the number of input image data channels to 64, 128, 256, 512, and 512 respectively. The number of convolutional layers within each encoding layer is 2, 2, 3, 3, and 3 in sequence. Each convolutional layer is followed by a normalization layer (BN) and a ReLU activation function (rectified linear activation function). The first feature image data after ReLU activation is shown below. The second feature image data is obtained by fusing it with the prior information of either the first low-exposure region or the first high-exposure region through an affine transformation network. Then, the second feature image data is processed through a channel attention mechanism. Key feature enhancement processing is performed, and the third feature image data is obtained after residual connection. , used as the input decoder for the improved U-shaped network.

[0025] like Figure 5 The diagram shows the structure of the decoding layer of the improved U-shaped network. The decoder of the improved U-shaped network has 4 decoding layers. The number of channels is adjusted to 512, 256, 128 and 64 in turn through convolutional layers with a kernel size of 3×3. Each decoding layer contains two convolutional layers, and each convolutional layer is followed by a BatchNorm (normalization layer) and a ReLU activation function (linear rectified activation function). Finally, the exposure time feature and image feature data are output.

[0026] like Figure 6 The diagram shows the overall structure of the improved U-shaped network. An exposure time fusion network is connected to the input of the encoder to control the exposure time. and exposure image data Before channel-dimensional concatenation, the exposure time needs to be sequentially expanded in both dimensions and broadcast. Specifically, the one-dimensional exposure time dimension is expanded to the same dimension as the medium-exposure image data using the unsqueeze function (dimensionality increase function), and then the exposure time is broadcast to the same size as the medium-exposure image using a broadcast operation to obtain the temporal feature. The temporal feature is then concatenated with the input medium-exposure image data along the channel dimension to obtain the input image data, which is used as input to the first encoding layer of the improved U-shaped network encoder. Multi-scale features are obtained through max pooling. In each encoding layer, the input image data undergoes convolution processing, affine transformation network feature fusion, and key feature enhancement processing using the channel attention mechanism to obtain the third feature image data. The output features of the fourth decoding layer and the fifth encoding layer of the U-shaped network are gradually upsampled to adjust the resolution of the third feature image. The upsampled feature map is then connected to the output of the corresponding encoding layer via skip connections: the fourth encoding layer is connected to the fourth decoding layer, the third encoding layer is connected to the third decoding layer, the second encoding layer is connected to the second decoding layer, and the first encoding layer is connected to the first decoding layer, achieving channel-level concatenation. After passing through all decoding layers, the number of channels is compressed to 4 by an output convolutional layer with a kernel size of 3×3. From the 4-channel feature map, one channel of exposure time feature and three channel of image feature are separated. The one-channel exposure time feature and the three-channel image feature are multiplied element-wise to obtain the enhanced feature guided by high exposure time. After passing through the Sigmoid activation function, pseudo-exposure image data is generated. The improved first U-shaped network and the improved second U-shaped network have the same structure, but the processing object is different. For the first U-shaped network with the first image data processing branch, the prior information is specifically the prior information of the first low-exposure region, and the obtained pseudo-exposure image data is pseudo-low-exposure image data. For the second U-shaped network of the second image data processing branch, the prior information is specifically the prior information of the first high-exposure region, and the resulting pseudo-exposure image data is pseudo-high-exposure image data. .

[0027] like Figure 7 The diagram shows the structure of an affine transformation network. The input to the affine transformation network is the first feature map data. and prior information This method integrates first feature image data and prior information to supplement missing information in low- or high-exposure areas of the mid-exposure image data. First, the first feature image data is normalized (Norm) to obtain fourth feature image data. Then, the prior information is processed through a linear layer to adjust the number of channels to match the fourth feature image data. Finally, the fourth feature image data and the adjusted prior information are multiplied pixel-by-pixel, and then added pixel-by-pixel to obtain the second feature map data. By using an affine transformation network, features extracted from medium-exposure image data are fused with prior information extracted from low-exposure or high-exposure image data to compensate for the lack of information in extreme exposure areas in medium-exposure image data.

[0028] like Figure 8 The diagram shows the structure of the channel attention mechanism, used for the second feature image data. Key feature enhancement processing is performed to obtain the third feature image data. Second feature image data The weight map is obtained by sequentially passing the data through an adaptive average pooling layer (AAvgPool), a linear layer (Linear), a ReLU activation function, another linear layer (Linear), and a Sigmoid activation function. Then, the weight map is compared with the second feature image data. Element-wise multiplication is performed to enhance key features, resulting in third feature image data. .

[0029] For the third image data processing branch, an image fusion network is used to fuse medium-exposure image data, pseudo-low-exposure image data, and pseudo-high-exposure image data to obtain the first high dynamic range image data. Specifically, the brightness data of the medium-exposure image data is obtained, and the pseudo-low-exposure image weight, pseudo-high-exposure image weight, and medium-exposure image weight are obtained based on the brightness data and a pre-set dynamic weight allocation trigonometric function. In a preferred embodiment of this application, the dynamic weight allocation trigonometric function is: in, For the first dynamically assigned weight trigonometric function, For the second dynamically assigned weight trigonometric function, For the third dynamically assigned weight trigonometric function, This represents the brightness value of the medium-exposure image data.

[0030] Furthermore, based on the first, second, and third dynamic weight allocation trigonometric functions, the pseudo-low exposure image weight, pseudo-high exposure image weight, and medium exposure image weight are obtained, as follows: in, Weighting for pseudo-low exposure images For the weight of the exposure image, Weights for pseudo-high exposure images.

[0031] Furthermore, gamma correction is applied to the pseudo-underexposed image data, pseudo-overexposed image data, and medium-exposed image data to obtain the corrected pseudo-underexposed image data, pseudo-overexposed image data, and medium-exposed image data. The calculation formula is as follows: in, This represents the corrected pseudo-underexposure image data. This indicates pseudo-low exposure image data. The hyperparameter is set to 2.2. This represents the corrected medium exposure image data. The image data is shown in the figure. This represents the corrected pseudo-overexposure image data. This represents pseudo-high-exposure image data.

[0032] Based on the weights of the pseudo-underexposure image, pseudo-overexposure image, and medium exposure image, the corrected pseudo-underexposure image data, medium exposure image data, and pseudo-overexposure image data are weighted and calculated to obtain the high dynamic range image data. The calculation formula is as follows: in, Represents high dynamic range image data. .

[0033] S2. Integrate the diffusion network between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model for the second stage of processing. The diffusion network is used to regenerate the prior information of the first low-exposure region and the first high-exposure region based on the conditional vector extracted from the exposure image data, respectively, to obtain the prior information of the second low-exposure region and the prior information of the second high-exposure region. In a preferred embodiment of this application, 466 training sets and 92 test sets are downloaded from a publicly available HDR dataset. Each set contains three static low dynamic range images with different exposure times, a file storing the image exposure information, and the corresponding HDR image. The training set and test set are preprocessed, including data augmentation and normalization. The image resolution is cropped to 128×128 before data augmentation. The data augmentation strategy includes rotating the image counterclockwise by 90°, 180°, horizontally flipping, vertically flipping, and combinations of the above operations. Finally, the augmented image data is normalized. The first image synthesis model is trained based on the preprocessed training set. The loss function of the first image synthesis model consists of three parts: the first mean absolute error loss term between the predicted high dynamic range image data and the high dynamic range image data in the training set, the second mean absolute error loss term between the pseudo low exposure image data and the low exposure image data, and the third mean absolute error loss term between the pseudo high exposure image data and the high exposure image data.

[0034] like Figure 2 The diagram shown illustrates the structure of the target image synthesis model, where, Figure 2 The second type of dashed line (short dash) illustrates the second-stage processing flow, integrating the diffusion network between the trained ground truth encoder and the improved U-shaped network. This aims to leverage the generative capabilities of the diffusion network to regenerate prior information for either the second low-exposure region or the second high-exposure region, containing information about the structure and brightness of the multi-exposure image. For the first data processing branch, this is the first diffusion network; for the second data processing branch, it is the second diffusion network. The first diffusion network is integrated between the trained first ground truth encoder and the first U-shaped network, and the second diffusion network is integrated between the trained second ground truth encoder and the second U-shaped network. The weight parameters of the first and second ground truth encoders are frozen, and feature information from the mid-exposure image data is extracted using either the first or second input encoder, resulting in either the first or second conditional vector. Based on the first conditional vector, the first diffusion network regenerates the prior information for the first low-exposure region to obtain the prior information for the second low-exposure region. The second diffusion network, based on the second conditional vector, regenerates the prior information of the first high-exposure region to obtain the prior information of the second high-exposure region. Subsequently, the prior information of the second low-exposure region is input into the first U-shaped network to generate new pseudo-low-exposure image data, and the prior information of the second high-exposure region is input into the second U-shaped network to generate new pseudo-high-exposure image data.

[0035] like Figure 9 The diagram shows the structure of the input encoder. The first input encoder includes 9 residual blocks, 2 branch convolutional layers, and an average pooling layer (Avg Pool). The structure of the residual blocks and branch convolutional layers is the same as that of the ground truth encoder, and will not be described again here. After the average pooling layer processes the exposure image data, it obtains intermediate features. Then, it performs multi-layer perceptual processing on the intermediate features through two linear layers with 256 channels each, obtaining the first conditional vector. The second input encoder has the same structure as the first input encoder, and correspondingly yields the second condition vector. .

[0036] like Figure 10 The diagram shows the structure of a diffusion network. For the first diffusion network, the first condition vector is received. The prior information of the first low-exposure region output after freezing the weight parameters of the first ground truth encoder trained by the first image synthesis model, and the added time step vector. Channel stitching is performed, and forward diffusion is carried out using a linear scheduler. Noise is gradually added to the prior information of the first low-exposure region. After 4 time steps, the prior information of the first low-exposure region is transformed into pure noise, resulting in the noise vector of the low-exposure region. and for the first condition vector and noise vector in low-exposure areas After dimensional stitching, multilayer perceptron denoising is performed to obtain prior information for the second low-exposure region. This denoising is implemented using a multilayer perceptron, which consists of six linear layers (illustrated as Linear) and a Leaky ReLU activation function, achieving lightweight denoising without affecting model performance. The second diffusion network has the same structure as the first diffusion network and correspondingly obtains prior information for the second high-exposure region.

[0037] For the second loss function of the second image synthesis model, compared with the first image synthesis model, a fourth average absolute value error loss term between the prior information of the first low-exposure area and the prior information of the second low-exposure area and a fifth average absolute value error loss term between the prior information of the first high-exposure area and the prior information of the second high-exposure area are added.

[0038] Based on the second loss function and the training set, the second image synthesis model is trained to obtain the target image synthesis model. In practical applications, the collected image data set containing low-exposure, medium-exposure, and high-exposure image data is input into the target image synthesis model. The low-exposure, medium-exposure, and high-exposure image data undergo a first-stage processing through a ground truth encoder, an improved U-shaped network, and an image fusion network. Then, a frozen ground truth encoder, an input encoder, a diffusion network, an improved U-shaped network, and an image fusion network are used to process the medium-exposure image data to obtain high dynamic range image data.

[0039] The model was tested using a test set. The evaluation metrics included standard peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) calculated directly in high dynamic range image data, as well as peak signal-to-noise ratio (PSNR-μ) and structural similarity (SSIM-μ) calculated in μ-law tone-mapped images.

[0040] The standard peak signal-to-noise ratio (PSNR) is a metric for measuring the pixel-level reconstruction quality of two images; a higher value indicates a smaller reconstruction error.

[0041] Structural similarity comprehensively evaluates the similarity between two images from three aspects: brightness, contrast, and structure, which is more in line with human subjective perception.

[0042] To better approximate the human eye's sensitivity to details in dark areas, μ-law tone mapping is applied to the image, and then peak signal-to-noise ratio and structural similarity are calculated.

[0043] Table 1 shows the results of testing using the test set. As shown in Table 1, the target image synthesis model of this application has significant improvements in PSNR, SSIM, PSNR-μ and SSIM-μ, indicating that prior information and temporal features provide effective guidance in the high dynamic range image generation task of this application, making the generated high dynamic range images clearer and improving the quality of high dynamic range images generated by the image synthesis model.

[0044] In a preferred embodiment of the present invention, the acquired image data set to be synthesized is input into a pre-constructed target image synthesis model for sequential first-stage processing and second-stage processing to obtain high dynamic range image data. The construction of the target image synthesis model includes: constructing a first image synthesis model for the first-stage processing based on an improved U-shaped network using an affine transformation network and a channel attention mechanism, and a truth encoder. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and prior information from high-exposure image data and medium-exposure image data. The high dynamic range (HMR) image synthesis method disclosed in this application proposes a two-stage HMR image synthesis network. It employs an affine transformation network and a channel attention mechanism to improve the U-shaped network, enabling the prediction of structure and brightness information of multi-exposure images from medium-exposure image data. The pseudo-exposure image data is then fused with medium-exposure image data to obtain high dynamic range image data. A diffusion network is integrated between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model for the second stage of processing. The diffusion network is used to regenerate the prior information of the first low-exposure region and the first high-exposure region based on the conditional vector extracted from the medium-exposure image data, correspondingly obtaining the prior information of the second low-exposure region and the second high-exposure region. The weight parameters of the ground truth encoder are frozen, and the second image synthesis model is trained to obtain the target image synthesis model. By incorporating exposure time into the target image synthesis model, time-based image enhancement can be achieved without strict alignment of multiple exposure images, and the generated high dynamic range images are more accurate.

[0045] like Figure 11 The diagram shows a high dynamic range image synthesis system. Correspondingly, based on a high dynamic range image synthesis method, this embodiment of the invention also provides a high dynamic range image synthesis system to implement the high dynamic range image synthesis method disclosed in this embodiment of the invention. The system includes: a high dynamic range image synthesis module 1. The high dynamic range image synthesis module 1 is used to input the acquired image data set to be synthesized into the pre-constructed target image synthesis model for first-stage processing and second-stage processing in sequence to obtain high dynamic range image data.

[0046] The high dynamic range image synthesis module 1 includes: a first image synthesis model construction unit 11, a second image synthesis model construction unit 12, and a model training unit 13; The first image synthesis model construction unit 11 is used to construct a first image synthesis model for the first stage of processing based on an improved U-shaped network and a truth encoder. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes a first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and a first high-exposure region prior information obtained by extracting information from high-exposure image data and medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain the high dynamic range image data. The second image synthesis model building unit 12 is used to integrate the diffusion network between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model that performs the second stage processing. The diffusion network is used to regenerate the first low exposure region prior information and the first high exposure region prior information based on the condition vector extracted from the middle exposure image data, respectively, to obtain the second low exposure region prior information and the second high exposure region prior information. The model training unit 13 is used to freeze the weight parameters of the truth encoder and train the second image synthesis model to obtain the target image synthesis model.

[0047] Furthermore, the first image synthesis model construction unit includes: a channel dimension splicing subunit, a convolution processing subunit, an affine transformation subunit, a feature enhancement processing subunit, a decoding connection subunit, and an image output subunit; The channel dimension stitching subunit is used to perform dimensional expansion and broadcasting operations on the exposure time in sequence to obtain time features, and then stitch the time features and the mid-exposure image data together in the channel dimension to obtain input image data. The convolution processing subunit is used to perform convolution processing on the input image data through the coding layer of the improved U-shaped network to obtain the first feature image data; The affine transformation subunit is used to perform feature fusion on the first feature image data and prior information through the affine transformation network of the improved U-shaped network to obtain the second feature image data. The feature enhancement processing subunit is used to perform key feature enhancement processing on the second feature image data through the channel attention mechanism of the improved U-shaped network to obtain the third feature image data. The decoding connection subunit is used to decode the third feature image data through the decoding layer of the improved U-shaped network, and then make a skip connection with the first feature image data to obtain exposure time feature and image feature data. The image output subunit is used to perform pixel-by-pixel multiplication on the exposure time feature and the image feature data to obtain pseudo-exposure image data. The pseudo-exposure image data includes pseudo-low exposure image data corresponding to the low exposure image data and pseudo-high exposure image data corresponding to the high exposure image data.

[0048] Furthermore, the second image synthesis model construction unit includes: a conditional vector acquisition subunit, a low-exposure region prior information acquisition subunit, and a high-exposure region prior information acquisition subunit; The conditional vector acquisition subunit is used to sequentially perform residual processing, convolution processing, and average pooling processing on the intermediate exposure image data to obtain intermediate features, and then perform multi-layer perceptual processing on the intermediate features to obtain a conditional vector. The low-exposure region prior information acquisition subunit is used to perform forward diffusion processing on the first low-exposure region prior information to obtain a low-exposure region noise vector, and after concatenating the condition vector and the low-exposure region noise vector in the dimensional direction, perform multi-layer perception denoising processing to obtain the second low-exposure region prior information. The high-exposure region prior information acquisition subunit performs forward diffusion processing on the first high-exposure region prior information to obtain a high-exposure region noise vector. After concatenating the condition vector and the high-exposure region noise vector in the dimensional direction, it performs multi-layer perception denoising processing to obtain the second high-exposure region prior information.

[0049] For specific limitations regarding a high dynamic range image synthesis system, please refer to the above-described limitations regarding a high dynamic range image synthesis method, which will not be repeated here. Those skilled in the art will recognize that the various modules and steps described in conjunction with the embodiments disclosed in this invention can be implemented in hardware, software, or a combination of both. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0050] In summary, the high dynamic range image synthesis method and system provided in this application address the technical problem of improving the quality of high dynamic range image synthesis. The method includes: inputting the acquired image data set to be synthesized into a pre-constructed target image synthesis model for sequential first-stage processing and second-stage processing to obtain high dynamic range image data. The construction of the target image synthesis model includes: constructing a first image synthesis model for the first stage processing based on a U-shaped network improved using an affine transformation network and a channel attention mechanism; the improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data; the prior information includes a first pseudo-exposure image data obtained by extracting information from low-exposure image data and medium-exposure image data. The high dynamic range (HMR) image synthesis method disclosed in this application proposes a two-stage HMR image synthesis network. It employs an affine transformation network and a channel attention mechanism to improve the U-shaped network. This network can predict the structure and brightness information of multi-exposure images from medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain high dynamic range (HMR) image data. A diffusion network is integrated between a trained ground truth encoder and an improved U-shaped network to obtain a second-stage image synthesis model. The diffusion network is used to regenerate the first low-exposure and first high-exposure prior information based on the conditional vector extracted from the medium-exposure image data, resulting in the second low-exposure and second high-exposure prior information. The second image synthesis model is then trained to obtain the target image synthesis model. By incorporating exposure time into the target image synthesis model, time-based image enhancement can be achieved without strict alignment of multiple exposure images, and the generated high dynamic range images are more accurate.

[0051] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0052] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this application, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A high dynamic range image synthesis method, characterized in that, The method includes: The acquired image data set to be synthesized is input into a pre-constructed target image synthesis model for sequential first-stage processing and second-stage processing to obtain high dynamic range image data. The construction of the target image synthesis model includes: Based on an improved U-shaped network and a truth encoder, a first image synthesis model is constructed to perform the first stage of processing. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes a first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and a first high-exposure region prior information obtained by extracting information from high-exposure image data and medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain the high dynamic range image data. The diffusion network is integrated between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model for the second stage of processing. The diffusion network is used to regenerate the first low-exposure region prior information and the first high-exposure region prior information based on the condition vector extracted from the middle exposure image data, respectively, to obtain the second low-exposure region prior information and the second high-exposure region prior information. The weight parameters of the truth encoder are frozen, and the second image synthesis model is trained to obtain the target image synthesis model.

2. The high dynamic range image synthesis method as described in claim 1, characterized in that, The step of obtaining false exposure image data based on prior information and medium exposure image data includes: The exposure time is sequentially expanded and broadcast to obtain time features, and the time features and the middle exposure image data are concatenated by channel dimension to obtain input image data. The input image data is convolved through the coding layer of the improved U-shaped network to obtain the first feature image data; The improved U-shaped network is used to fuse the first feature image data and the prior information to obtain the second feature image data. The second feature image data is enhanced by the channel attention mechanism of the improved U-shaped network to obtain the third feature image data. After the third feature image data is decoded by the decoding layer of the improved U-shaped network, it is skipped and connected with the first feature image data to obtain exposure time feature and image feature data. The exposure time feature and the image feature data are multiplied pixel by pixel to obtain the pseudo exposure image data. The pseudo exposure image data includes pseudo low exposure image data corresponding to the low exposure image data and pseudo high exposure image data corresponding to the high exposure image data.

3. The high dynamic range image synthesis method as described in claim 2, characterized in that, The step of fusing the first feature image data and the prior information using the affine transformation network of the improved U-shaped network to obtain the second feature image data includes: The first feature image data is subjected to tensor reconstruction processing to obtain the fourth feature image data; The prior information is adjusted in terms of the number of channels to obtain adjusted prior information with the same number of channels as the fourth feature image data; After multiplying the fourth feature image data and the adjusted prior information pixel by pixel, the adjusted prior information is added pixel by pixel to obtain the second feature map data.

4. The high dynamic range image synthesis method as described in claim 1, characterized in that, The first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and the first high-exposure region prior information obtained by extracting information from high-exposure image data and medium-exposure image data, include: After concatenating the low-exposure image data and the medium-exposure image data along the channel dimension, residual processing, convolution processing and average pooling processing are performed sequentially to obtain the first fused image feature. The low-exposure image data is then subjected to the residual processing, convolution processing and average pooling processing sequentially to obtain the low-exposure light region feature. After stitching the first fused image features and the low-exposure light region features in the dimensional direction, multi-layer perception processing is performed to obtain the prior information of the first low-exposure region. After concatenating the high-exposure image data and the medium-exposure image data along the channel dimension, the residual processing, convolution processing, and average pooling processing are performed sequentially to obtain the second fused image feature. The high-exposure image data is then subjected to the residual processing, convolution processing, and average pooling processing sequentially to obtain the high-exposure light region feature. After stitching the second fused image features and the high-exposure light region features along the dimensional direction, the multilayer perception processing is performed to obtain the prior information of the first high-exposure region.

5. The high dynamic range image synthesis method as described in claim 4, characterized in that, The conditional vector extracted from the exposed image data is used to regenerate the prior information of the first low-exposure region and the prior information of the first high-exposure region, respectively, to obtain the corresponding prior information of the second low-exposure region and the second high-exposure region, including: The intermediate exposure image data is sequentially subjected to the residual processing, the convolution processing, and the average pooling processing to obtain intermediate features, and the intermediate features are subjected to the multilayer perceptron processing to obtain a conditional vector; The prior information of the first low-exposure region is subjected to forward diffusion processing to obtain the noise vector of the low-exposure region. After the condition vector and the noise vector of the low-exposure region are concatenated in the dimensional direction, multi-layer perceptual denoising processing is performed to obtain the prior information of the second low-exposure region. The prior information of the first high-exposure region is subjected to the forward diffusion process to obtain the noise vector of the high-exposure region. After the condition vector and the noise vector of the high-exposure region are concatenated in the dimensional direction, multilayer perceptual denoising processing is performed to obtain the prior information of the second high-exposure region.

6. The high dynamic range image synthesis method as described in claim 2, characterized in that, The high dynamic range image data obtained by fusing the intermediate exposure image data includes: Obtain the brightness data of the medium exposure image data, and based on the brightness data and a pre-set dynamic weighting trigonometric function, obtain the pseudo-low exposure image weight, pseudo-high exposure image weight, and medium exposure image weight. The pseudo-low exposure image data, the pseudo-high exposure image data, and the medium exposure image data are respectively subjected to gamma correction to obtain the corresponding corrected pseudo-low exposure image data, pseudo-high exposure image data, and medium exposure image data; The high dynamic range image data is obtained by weighting the corrected pseudo-low exposure image data, pseudo-high exposure image data, and medium exposure image data according to the pseudo-low exposure image weight, the pseudo-high exposure image weight, and the medium exposure image weight.

7. The high dynamic range image synthesis method as described in claim 2, characterized in that, The method further includes: The first image synthesis model is configured with a first loss function, which includes a first average absolute value error loss term, a second average absolute value error loss term, and a third average absolute value error loss term. The first average absolute value error loss term represents the loss between the predicted value and the true value of the high dynamic range image data, the second average absolute value error loss term represents the loss between the pseudo low exposure image data and the low exposure image data, and the third average absolute value error loss term represents the loss between the pseudo high exposure image data and the high exposure image data. A second loss function is set for the second image synthesis model. The second loss function includes: a first average absolute value error loss term, a second average absolute value error loss term, a third average absolute value error loss term, a fourth average absolute value error loss term, and a fifth average absolute value error loss term. The fourth average absolute value error loss term represents the loss between the prior information of the first low-exposure area and the prior information of the second low-exposure area. The fifth average absolute value error loss term represents the loss between the prior information of the first high-exposure area and the prior information of the second high-exposure area.

8. A high dynamic range image synthesis system, used to implement the high dynamic range image synthesis method according to any one of claims 1-7, characterized in that, The system includes: a high dynamic range image synthesis module; The high dynamic range image synthesis module is used to input the acquired image data set to be synthesized into the pre-constructed target image synthesis model for first-stage processing and second-stage processing in sequence to obtain high dynamic range image data. The high dynamic range image synthesis module includes: a first image synthesis model building unit, a second image synthesis model building unit, and a model training unit; The first image synthesis model construction unit is used to construct a first image synthesis model for the first stage of processing based on an improved U-shaped network and a truth encoder. The improved U-shaped network is used to obtain pseudo-exposure image data based on prior information and medium-exposure image data. The prior information includes a first low-exposure region prior information obtained by extracting information from low-exposure image data and medium-exposure image data, and a first high-exposure region prior information obtained by extracting information from high-exposure image data and medium-exposure image data. The pseudo-exposure image data is used to fuse with the medium-exposure image data to obtain the high dynamic range image data. The second image synthesis model building unit is used to integrate the diffusion network between the trained ground truth encoder and the improved U-shaped network to obtain a second image synthesis model that performs the second stage of processing. The diffusion network is used to regenerate the first low-exposure region prior information and the first high-exposure region prior information based on the condition vector extracted from the middle exposure image data, respectively, to obtain the second low-exposure region prior information and the second high-exposure region prior information. The model training unit is used to freeze the weight parameters of the truth encoder and train the second image synthesis model to obtain the target image synthesis model.

9. The high dynamic range image synthesis system as described in claim 8, characterized in that, The first image synthesis model construction unit includes: a channel dimension splicing subunit, a convolution processing subunit, an affine transformation subunit, a feature enhancement processing subunit, a decoding connection subunit, and an image output subunit; The channel dimension stitching subunit is used to perform dimensional expansion and broadcasting operations on the exposure time sequentially to obtain time features, and then stitch the time features and the middle exposure image data together in the channel dimension to obtain input image data. The convolution processing subunit is used to perform convolution processing on the input image data through the coding layer of the improved U-shaped network to obtain the first feature image data; The affine transformation subunit is used to perform feature fusion on the first feature image data and the prior information through the affine transformation network of the improved U-shaped network to obtain the second feature image data. The feature enhancement processing subunit is used to perform key feature enhancement processing on the second feature image data through the channel attention mechanism of the improved U-shaped network to obtain the third feature image data. The decoding connection subunit is used to decode the third feature image data through the decoding layer of the improved U-shaped network, and then make a skip connection with the first feature image data to obtain exposure time feature and image feature data. The image output subunit is used to perform a pixel-by-pixel multiplication operation on the exposure time feature and the image feature data to obtain the pseudo exposure image data. The pseudo exposure image data includes pseudo low exposure image data corresponding to the low exposure image data and pseudo high exposure image data corresponding to the high exposure image data.

10. The high dynamic range image synthesis system as described in claim 8, characterized in that, The second image synthesis model construction unit includes: a conditional vector acquisition subunit, a low-exposure region prior information acquisition subunit, and a high-exposure region prior information acquisition subunit; The conditional vector acquisition subunit is used to sequentially perform the residual processing, the convolution processing, and the average pooling processing on the intermediate exposure image data to obtain intermediate features, and then perform the multilayer perceptron processing on the intermediate features to obtain a conditional vector. The low-exposure region prior information acquisition subunit is used to perform the forward diffusion processing on the first low-exposure region prior information to obtain a low-exposure region noise vector, and after performing the dimensional direction concatenation on the condition vector and the low-exposure region noise vector, perform multi-layer perception denoising processing to obtain the second low-exposure region prior information. The high-exposure region prior information acquisition subunit performs forward diffusion processing on the first high-exposure region prior information to obtain a high-exposure region noise vector. After concatenating the condition vector and the high-exposure region noise vector in the dimensional direction, it performs multi-layer perceptual denoising processing to obtain the second high-exposure region prior information.

Images