Image reconstruction method and apparatus

Abstract

The present application provides an image reconstruction method and apparatus. The method includes: obtaining an image reconstruction instruction, obtaining an edge image of an LDR image according to the LDR image, inputting the LDR image into a pre-trained global neural network model to obtain an intermediate image of the LDR image, inputting the edge image of the LDR image into a pre-trained edge neural network model to obtain an intermediate edge image of the LDR image, and inputting the intermediate image of the LDR image and the intermediate edge image of the LDR image into a pre-trained fusion neural network model to obtain an HDR image corresponding to the LDR image. Therefore, it is possible to reconstruct an HDR image with vivid colors, rich brightness information, and clear texture details from a single-frame LDR image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular, to an image reconstruction method and apparatus. Background Art

[0002] With the development of the times, people have higher and higher requirements for high-quality images in multimedia life. High-Dynamic Range (HDR) images can meet this need of people. HDR images are images rendered with high dynamic range lighting. Compared with ordinary Low-Dynamic Range (LDR) images, they can provide more dynamic range and image details. Using LDR images corresponding to the best details at each exposure time to synthesize the final HDR image can better reflect the visual effect in the real environment.

[0003] Currently, the commonly used way to obtain HDR images is to collect LDR images by using multiple exposures of an image, and then synthesize multiple LRD images to process them into an image with HDR effect. However, the above method has poor enhancement effect on the texture details in the low-brightness area of the image. Summary of the Invention

[0004] The present application provides an image reconstruction method and apparatus to solve the problem of poor enhancement effect on the texture details in the low-brightness area of the HDR image reconstructed from LDR images.

[0005] In a first aspect, the present application provides an image reconstruction method, including:

[0006] Obtain an image reconstruction instruction, where the image reconstruction instruction is used to indicate to reconstruct an HDR image from an LDR image by using a preset image reconstruction neural network model. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model;

[0007] Obtain an edge image of the LDR image according to the LDR image.

[0008] Input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image.

[0009] Input the edge image of the LDR image into the pre-trained edge neural network model to obtain an intermediate edge image of the LDR image.

[0010] Input the intermediate image of the LDR image and the intermediate edge image of the LDR image into the pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0011] Optionally, the pre-set image reconstruction neural network model is obtained by the following steps:

[0012] Obtain multiple HDR sample images;

[0013] According to the multiple HDR sample images, obtain corresponding multiple LDR sample images;

[0014] According to the multiple LDR sample images, obtain edge images of the multiple LDR sample images;

[0015] Input the multiple LDR sample images into the global neural network model for training to obtain a trained global neural network model; and input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images;

[0016] Input the edge images of the multiple LDR sample images into the edge neural network model for training to obtain a trained edge neural network model, and input the edge images of the multiple LDR sample images into the trained edge neural network model respectively to obtain intermediate edge images of the multiple LDR sample images;

[0017] Input the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model;

[0018] According to the trained global neural network model, trained edge neural network model, and trained fusion neural network model, obtain the image reconstruction neural network model.

[0019] Optionally, the global neural network model includes a first compression sub-network, a first Gated Recurrent Unit (GRU) network, and a first decompression sub-network;

[0020] Among them, the first compression sub-network includes 3 convolutional layers, and the first decompression sub-network includes 3 deconvolutional layers.

[0021] Optionally, the edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network;

[0022] Among them, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

[0023] Optionally, the fusion neural network model includes a third compression sub-network and a third decompression sub-network;

[0024] Among them, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

[0025] Optionally, input multiple LDR sample images into the global neural network model for training to obtain a trained global neural network model, including:

[0026] Input multiple LDR sample images into the global neural network model for training, and use the loss function shown in Formula 1 below during the training process to obtain a trained global neural network model;

[0027] L global = α1L1 + α2L hist Formula 1

[0028] where L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the LDR sample image by the global neural network model at the i-th pixel, I represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the HDR sample image histogram grading statistical loss, where M represents the number of statistical histogram types of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents the number of times the predicted pixel value of the LDR sample image by the global neural network model is count m (I) represents the number of pixels with the value of I in the HDR sample image, and α1 and α2 represent weight parameters.

[0029] Optionally, input the edge images of multiple LDR sample images into the edge neural network model for training to obtain a trained edge neural network model, including:

[0030] Input the edge images of multiple LDR sample images into the edge neural network model for training, and use the loss function shown in Formula 2 below during the training process to obtain a trained edge neural network model;

[0031]

[0032] where L edge represents the edge loss function value, N represents the number of pixels of the edge image of the LDR sample image, represents the predicted pixel value of the edge image of the LDR sample image by the edge neural network model at the i-th pixel, edge(I i ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

[0033] Optionally, input the intermediate image and the intermediate edge image of multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model, including:

[0034] Input the intermediate image and the intermediate edge image of multiple LDR sample images into the fusion neural network model for training. During the training process, use the loss function shown in Formula 3 below to obtain a trained fusion neural network model;

[0035] L recon = α3L global + α4L edge + α5SSMI(I recon ,I HDR ) Formula 3

[0036] Wherein, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, I recon represents the HDR image output by the fusion neural network model, I HDR represents the HDR sample image, and SSMI(I recon ,I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image, Wherein, μ recon represents the average of the pixel values of I recon , μ HDR represents the average of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR , c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

[0037] In a second aspect, the present application provides an image reconstruction device, including:

[0038] An acquisition module, configured to acquire an image reconstruction instruction, where the image reconstruction instruction is used to indicate reconstructing an HDR image from an LDR image using a preset image reconstruction neural network model, and the preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model;

[0039] The first processing module is used to obtain the edge image of the LDR image according to the LDR image;

[0040] The second processing module is used to input the LDR image into a pre-trained global neural network model to obtain an intermediate image of the LDR image;

[0041] The third processing module is used to input the edge image of the LDR image into a pre-trained edge neural network model to obtain an intermediate edge image of the LDR image;

[0042] The fourth processing module is used to input the intermediate image of the LDR image and the intermediate edge image of the LDR image into a pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0043] Optionally, the image reconstruction device further includes:

[0044] The training module is used to:

[0045] Obtain multiple HDR sample images;

[0046] Obtain corresponding multiple LDR sample images according to the multiple HDR sample images;

[0047] Obtain the edge images of the multiple LDR sample images according to the multiple LDR sample images;

[0048] Input the multiple LDR sample images into the global neural network model for training to obtain a trained global neural network model; and input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images;

[0049] Input the edge images of the multiple LDR sample images into the edge neural network model for training to obtain a trained edge neural network model, and input the edge images of the multiple LDR sample images into the trained edge neural network model respectively to obtain intermediate edge images of the multiple LDR sample images;

[0050] Input the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model;

[0051] Obtain an image reconstruction neural network model according to the trained global neural network model, the trained edge neural network model, and the trained fusion neural network model.

[0052] Optionally, the global neural network model includes a first compression sub-network, a first GRU network, and a first decompression sub-network;

[0053] Among them, the first compression sub-network includes 3 convolutional layers, and the first decompression sub-network includes 3 deconvolutional layers.

[0054] Optionally, the edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network;

[0055] Among them, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

[0056] Optionally, the fusion neural network model includes a third compression sub-network and a third decompression sub-network;

[0057] Among them, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

[0058] Optionally, the training module is specifically used for:

[0059] Inputting multiple LDR sample images into the global neural network model for training, and using the loss function shown in the following formula (1) during the training process to obtain a trained global neural network model;

[0060] L global = α1L1 + α2L hist Formula (1)

[0061] Among them, L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the global neural network model for the LDR sample image at the i-th pixel, I represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the HDR sample image histogram classification statistical loss, Among them, M represents the number of types of the statistical histogram of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents that the predicted pixel value of the global neural network model for the LDR sample image is the number of, count m (I) represents the number of pixels with the value I in the HDR sample image, and α1 and α2 represent weight parameters.

[0062] Optionally, the training module is specifically used for:

[0063] Inputting the edge images of multiple LDR sample images into the edge neural network model for training, and using the loss function shown in the following formula (2) during the training process to obtain a trained edge neural network model;

[0064]

[0065] Among them, L edge represents the edge loss function value, N represents the number of pixels of the edge image of the LDR sample image, represents the predicted pixel value of the edge neural network model for the edge image of the LDR sample image at the i-th pixel, edge(I i ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

[0066] Optionally, the training module is specifically used for:

[0067] Input the intermediate image and the intermediate edge image of multiple LDR sample images into the fusion neural network model for training, and use the loss function shown in the following formula three during the training process to obtain a trained fusion neural network model;

[0068] L recon = α3L global + α4L edge + α5SSMI(I recon , I HDR ) Formula three

[0069] Among them, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, I recon represents the HDR image output by the fusion neural network model, I HDR represents the HDR sample image, SSMI(I recon , I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image, Among them, μ recon represents the average value of the pixel values of I recon , μ HDR represents the average value of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR , c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent the stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

[0070] In a third aspect, the present application provides an image reconstruction device, including: a memory and a processor;

[0071] The memory is used for storing program instructions;

[0072] The processor is used for calling the program instructions in the memory to execute the image reconstruction method according to the first aspect of the present application.

[0073] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer program instructions are stored. When the computer program instructions are executed, the image reconstruction method according to any item of the first aspect of the present application is implemented.

[0074] In a fifth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the image reconstruction method according to the first aspect of the present application is implemented.

[0075] For the image reconstruction method and device provided by the present application, since the global neural network model of the present application balances the overall high brightness and enhances the low brightness of a single-frame LDR image, and the edge neural network model extracts and sharpens the texture features of the low-brightness area of the single-frame LDR image, and then fuses the processed results above, an HDR image with vivid colors, rich brightness information, and clear texture details can be obtained. It avoids the way in the prior art that multiple LDR images collected by exposure are synthesized into an HDR image, which requires relying on expensive professional HDR devices. Therefore, the present application can also reduce the post-processing of the captured images and the amount of collected data, and improve the real-time performance of HDR image generation. Description of the Drawings

[0076] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0077] Figure 1 It is a flowchart of the image reconstruction method provided by an embodiment of the present application;

[0078] Figure 2 It is a schematic diagram of the principle of reconstructing an LDR image into an HDR image provided by an embodiment of the present application;

[0079] Figure 3 It is a flowchart of the image reconstruction method provided by another embodiment of the present application;

[0080] Figure 4Schematic structural diagram of an image reconstruction device provided by an embodiment of the present application;

[0081] Figure 5 Schematic structural diagram of an image reconstruction device provided by another embodiment of the present application;

[0082] Figure 6 Schematic structural diagram of an image reconstruction device provided by another embodiment of the present application. Detailed implementation manners

[0083] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0084] An HDR image is an image with high dynamic range lighting rendering. Compared with ordinary LDR images, it can provide more dynamic range and image details. By using LDR images corresponding to the best details for each exposure time to synthesize the final HDR image, the visual effect in the real environment can be better reflected. Currently, the common way to obtain HDR images is to use multiple exposures to capture LDR images of the image, and then synthesize multiple LRD images to process them into an image with HDR effect. The HDR images obtained in this way have problems such as ghosting and jaggedness. Reconstructing HDR images from single-frame LDR images has made good progress in enhancing the low-brightness areas of the image, but not much consideration has been given to the texture details in the low-brightness areas, and the enhancement effect in some areas is not good.

[0085] Therefore, the present application provides an image reconstruction method and device, which pre-train a global neural network model for balancing overall high brightness and enhancing low brightness, an edge neural network model for extracting texture features and sharpening enhancement in low-brightness areas, and a fusion neural network model. Then, use the global neural network model to balance the overall high brightness and enhance the low brightness of the single-frame LDR image, use the edge neural network model to extract the texture features and sharpen the enhancement of the low-brightness areas of the single-frame LDR image, and use the fusion neural network model to fuse the processed images to obtain an HDR image with vivid colors, rich brightness information, and clear texture details.

[0086] Figure 1 Flowchart of an image reconstruction method provided by an embodiment of the present application. The method of this embodiment can be applied to an electronic device, which can be a terminal device, a server, etc. The terminal device can be, for example, a mobile phone, a tablet computer, a laptop computer, a desktop computer, etc. As Figure 1As shown, the method of this embodiment includes:

[0087] S101. Obtain an image reconstruction instruction, which is used to indicate reconstructing an HDR image from an LDR image by using a preset image reconstruction neural network model. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model.

[0088] In this embodiment, a preset image reconstruction neural network model has been deployed on an electronic device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, or a server. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model. Input the LDR image captured by an LDR image acquisition device such as a camera or a webcam into the preset image reconstruction neural network model, and start reconstructing the HDR image from the LDR image by using the preset image reconstruction neural network model according to the image reconstruction instruction.

[0089] Among them, the above image reconstruction instruction can be input by a user to the electronic device executing this method embodiment, or sent by another device to the electronic device executing this method embodiment.

[0090] S102. Obtain the edge image of the LDR image according to the LDR image.

[0091] In this embodiment, the LDR image of the HDR image to be reconstructed has been obtained, that is, the LDR image. Figure 2 FIG. is a schematic diagram of the principle of reconstructing an LDR image into an HDR image provided by an embodiment of the present application. As Figure 2 shown, obtain the edge image of the LDR image according to the LDR image.

[0092] S103. Input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image.

[0093] In this embodiment, the LDR image of the HDR image to be reconstructed and the pre-trained global neural network model have been obtained. Therefore, as Figure 2 shown, input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image.

[0094] S104. Input the edge image of the LDR image into the pre-trained edge neural network model to obtain an intermediate edge image of the LDR image.

[0095] In this embodiment, the LDR image of the HDR image to be reconstructed and the pre-trained edge neural network model have been obtained. Therefore, as Figure 2As shown, the edge image of the LDR image is input into a pre-trained edge neural network model to obtain the intermediate edge image of the LDR image.

[0096] S105. Input the intermediate image of the LDR image and the intermediate edge image of the LDR image into a pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0097] In this embodiment, the intermediate image and the intermediate edge image of the LDR image have been obtained, as well as the pre-trained fusion neural network model. Therefore, as Figure 2 shown, input the intermediate image and the intermediate edge image of the LDR image into a pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0098] The image reconstruction method provided by this application obtains an image reconstruction instruction, which is used to indicate that a preset image reconstruction neural network model is used to reconstruct an HDR image from an LDR image. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model. According to the LDR image, the edge image of the LDR image is obtained, the LDR image is input into the pre-trained global neural network model to obtain the intermediate image of the LDR image, the edge image of the LDR image is input into the pre-trained edge neural network model to obtain the intermediate edge image of the LDR image, and the intermediate image of the LDR image and the intermediate edge image of the LDR image are input into the pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image. Since the global neural network model of this application balances the overall high brightness and enhances the low brightness of a single-frame LDR image, and the edge neural network model extracts and sharpens the texture features of the low-brightness area of a single-frame LDR image, and then fuses the processed results above, an HDR image with distinct colors, rich brightness information, and clear texture details can be obtained. It avoids the method in the prior art of synthesizing multiple LDR images collected by exposure into an HDR image, which requires relying on expensive professional HDR equipment. Therefore, this application can also reduce the post-processing of the captured images and the amount of collected data, and improve the real-time performance of HDR image generation.

[0099] Next, a description will be given on how to obtain the above image reconstruction neural network model. Figure 3 It is a flowchart of the image reconstruction method provided by another embodiment of this application. As Figure 3 shown, the method of this embodiment may include:

[0100] S301. Obtain multiple HDR sample images.

[0101] In this embodiment, multiple HDR images are collected. For example, multiple HDR images can be obtained by downloading the publicly available HDR image dataset HDR-Eye from the Internet, and an HDR image database is established. The obtained multiple HDR sample images are used to train the image reconstruction neural network model.

[0102] S302. Obtain corresponding multiple LDR sample images according to the multiple HDR sample images.

[0103] In this embodiment, since multiple HDR sample images have been obtained, image transformation methods such as random cropping and affine transformation of the images can be used to obtain corresponding multiple LDR sample images. Reset the size of each LDR sample image in the multiple LDR sample images. For example, interpolation methods can be used to reset the size of the LDR sample images so that the sizes of the multiple LDR sample images are the same as those of the corresponding HDR sample images and have the same number of pixels, and then an LDR sample image database is constructed.

[0104] S303. Obtain edge images of the multiple LDR sample images according to the multiple LDR sample images.

[0105] In this embodiment, since multiple LDR sample images have been obtained, the Canny edge detection algorithm can be used, for example, to extract the edges of the LDR sample images to obtain edge images of the multiple LDR sample images, and an edge image library of the LDR sample images is constructed.

[0106] S304. Input the multiple LDR sample images into the global neural network model for training to obtain a trained global neural network model; and input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images.

[0107] In this embodiment, since multiple LDR sample images have been obtained and a global neural network model before training has been constructed, therefore, the multiple LDR sample images are input into the global neural network model for training to obtain a trained global neural network model, which is used to balance the overall high brightness and enhance the low brightness of the LDR sample images. According to the trained global neural network model, the multiple LDR sample images are input into the trained global neural network model respectively, and thus, intermediate images of the multiple LDR sample images are obtained.

[0108] Optionally, the global neural network model includes a first compression sub-network, a first GRU network, and a first decompression sub-network, where the first compression sub-network includes 3 convolutional layers and the first decompression sub-network includes 3 deconvolutional layers.

[0109] In this embodiment, as Figure 2The global neural network model in

[0110] Optionally, multiple LDR sample images are input into the global neural network model for training. During the training process, the loss function shown in Formula 1 below is used to obtain a trained global neural network model.

[0111] L global = α1L1 + α2L hist Formula 1

[0112] Where L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the LDR sample image by the global neural network model at the i-th pixel, I represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the histogram grading statistical loss of the HDR sample image, Where M represents the number of types of statistical histograms of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents the number of times that the predicted pixel value of the LDR sample image by the global neural network model is count m (I) represents the number of pixels with the value I in the HDR sample image, and α1 and α2 represent weight parameters.

[0113] In this embodiment, in order to achieve the balance of the overall high brightness and the enhancement of the low brightness of the LDR sample image, therefore, the total pixel statistical loss L of the LDR sample image is calculated by Formula 1 global . Specifically, L1 represents the pixel statistical loss of the HDR sample image, which is used to characterize the overall brightness loss of the reconstructed image. During the training process of the global neural network model, I is the true pixel value of the HDR sample image at the i-th pixel. For each I, the global neural network model will output a which is the predicted pixel value of the LDR sample image by the global neural network model at the i-th pixel, The smaller the difference between and I is, the better the training result of the global neural network model is. Convert the RGB of the LDR sample image to grayscale intensity. Optionally, the conversion method is, for example: I = 0.299r + 0.587g + 0.114b, where r, g, and b are the values of the three RGB channels of the image respectively. L hist Represents the histogram classification statistical loss of the HDR sample image, which is used to balance the difference between the high-brightness area and the low-brightness area of the reconstructed image. Specifically, L hist For Perform classification data statistics. For example, the grayscale intensity of the LDR sample image is 0 - 255, with 50 as one level, divided into 5 levels: 0 - 50, 50 - 100, 100 - 150, 150 - 200, 200 - 255. Then M is 5, that is, the number of types of the statistical histogram of the HDR sample image is 5. The statistical histogram is used to judge the brightness of the image. ω m Represents the weight of the statistical histogram of the HDR sample image. For example: when m < 5, ω m = 1.2, when m >= 5, ω m = 0.9, which is used to balance the contrast difference between the high-brightness area and the low-brightness area of the reconstructed image. Represents that the predicted pixel value of the LDR sample image by the global neural network model is The quantity, count m (I) represents the quantity of pixels with the value I in the HDR sample image. And count m (I) are both rounded according to the rounding principle. α1 and α2 represent weight parameters. α1 is, for example, 1, and α2 is, for example, 0.1. Through Formula 1, train the global neural network model. When the training converges, stop the training. When the value of the loss function L global keeps getting smaller and oscillates slightly near the preset threshold, it can be judged that the training process converges, and then a trained global neural network model is obtained.

[0114] S305. Input the edge images of multiple LDR sample images into the edge neural network model for training to obtain a trained edge neural network model, and input the edge images of multiple LDR sample images into the trained edge neural network model respectively to obtain the intermediate edge images of multiple LDR sample images.

[0115] In this embodiment, the edge images of multiple LDR sample images have been obtained, and the edge neural network model before training has been constructed. Therefore, the edge images of multiple LDR sample images are input into the edge neural network model for training to obtain a trained edge neural network model, which is used to extract and sharpen the texture features of the low-brightness regions of the LDR sample images. According to the trained edge neural network model, the edge images of multiple LDR sample images are respectively input into the trained edge neural network model. Therefore, the intermediate edge images of multiple LDR sample images are obtained.

[0116] Optionally, the edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network. Among them, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

[0117] In this embodiment, as Figure 2 the edge neural network model in includes a second compression sub-network, a second GRU network, and a second decompression sub-network. The second compression sub-network is used to compress and encode the LDR sample images. Considering that the texture of the edge images of the LDR sample images is relatively clear, the number of layers of the second compression sub-network is reduced and it consists of 2 convolutional layers; the GRU network is used for training and inference. Compared with the traditional LSTM network, the GRU network has a higher training efficiency, can reduce the training and inference time, and improve the real-time performance of training and inference; the second decompression sub-network is used to decompress and restore the LDR sample images and consists of 2 deconvolutional layers.

[0118] Optionally, the edge images of multiple LDR sample images are input into the edge neural network model for training, and the loss function shown in Formula 2 below is used during the training process to obtain a trained edge neural network model.

[0119]

[0120] Among them, L edge represents the edge loss function value, N represents the number of pixels of the edge image of the LDR sample image, represents the predicted pixel value of the edge neural network model for the edge image of the LDR sample image at the i-th pixel, and edge(I i ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

[0121] In this embodiment, in order to realize the extraction and sharpening enhancement of the texture features of the low-brightness regions of the LDR sample images, therefore, the total pixel statistical loss L edge of the edge image of the LDR sample image is calculated by Formula 2. Specifically, Denote the predicted pixel value of the edge image of the LDR sample image by the edge neural network model at the \(i\)-th pixel, edge(\(I\) i ) denote the true pixel value of the edge image of the HDR sample image at the \(i\)-th pixel, and edge(\(I\) i ) are both rounded according to the rounding principle. Through Formula 2, the edge neural network model is trained. When the training converges, the training stops. When the loss function value \(L\) edge keeps getting smaller and oscillates slightly near a preset threshold during the training process, it can be judged that the training process converges, and then a trained edge neural network model is obtained.

[0122] S306. Input the intermediate image and the intermediate edge image of multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model.

[0123] In this embodiment, the intermediate images and intermediate edge images of multiple LDR sample images have been obtained, and the fusion neural network model before training has been constructed. Therefore, input the intermediate images and intermediate edge images of multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model. According to the trained fusion neural network model, input the intermediate images and intermediate edge images of multiple LDR sample images into the fusion neural network model for training. Therefore, a trained fusion neural network model is obtained.

[0124] Optionally, the fusion neural network model includes a third compression sub-network and a third decompression sub-network. Among them, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

[0125] In this embodiment, as Figure 2 the fusion neural network model in includes a third compression sub-network and a third decompression sub-network. The third compression sub-network is used to compress and encode the LDR sample image and consists of 4 convolutional layers; the third decompression sub-network is used to decompress and restore the LDR sample image and consists of 3 deconvolutional layers. In order to improve the fusion efficiency, the GRU network is no longer used here.

[0126] Optionally, input the intermediate images and intermediate edge images of multiple LDR sample images into the fusion neural network model for training. During the training process, use the loss function shown in Formula 3 below to obtain a trained fusion neural network model.

[0127] \(L\) recon =\(\alpha_3L\) global +\(\alpha_4L\) edge +\(\alpha_5\)SSMI(\(I\) recon ,\(I\) HDR ) Formula 3

[0128] Among them, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, and I recon represents the HDR image output by the fusion neural network model, and I HDR represents the HDR sample image. SSMI(I recon , I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image. Among them, μ recon represents the average of the pixel values of I recon , μ HDR represents the average of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR . c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

[0129] In this embodiment, the total pixel statistical loss L recon of the HDR image is reconstructed through Formula 3. Specifically, L global has been obtained through Formula 1, and L edge has been obtained through Formula 2. α3 is, for example, 1, α4 is, for example, 0.1, and α5 is, for example, 0.2. According to the pixels of the HDR image output by the fusion neural network model and the pixels of the HDR sample image, SSMI(I recon , I HDR ) is obtained, where k1 is, for example, 0.01 and k2 is, for example, 0.03. Through Formula 3, the fusion neural network model is trained. When the training converges, the training is stopped. When the loss function value L recon continually becomes smaller and oscillates slightly near a preset threshold, it can be determined that the training process converges, and then the trained fusion neural network model is obtained.

[0130] S307. Obtain an image reconstruction neural network model according to the trained global neural network model, the trained edge neural network model, and the trained fusion neural network model.

[0131] In this embodiment, a trained global neural network model, a trained edge neural network model, and a trained fusion neural network model have been obtained. Therefore, an image reconstruction neural network model is obtained. Specifically, the data output by the global neural network model and the data output by the edge neural network model are input into the fusion neural network model together to form an image reconstruction neural network model for reconstructing the LDR sample image.

[0132] The image reconstruction method provided by this application obtains corresponding multiple LDR sample images and the edge images of multiple LDR sample images according to multiple HDR sample images. The multiple LDR sample images are input into the global neural network model for training to obtain a trained global neural network model and the intermediate images of multiple LDR sample images. The edge images of multiple LDR sample images are input into the edge neural network model for training to obtain a trained edge neural network model and the intermediate edge images of multiple LDR sample images. The intermediate images and intermediate edge images of multiple LDR sample images are input into the fusion neural network model for training to obtain a trained fusion neural network model, and then an image reconstruction neural network model is obtained. By the above method, this application can obtain a global neural network model for balancing overall high brightness and enhancing low brightness, an edge neural network model for extracting texture features of low-brightness regions and sharpening enhancement, and a fusion neural network model for fusing the images obtained by the above processing into an HDR image. Therefore, this application can reconstruct an HDR image with distinct colors, rich brightness information, and clear texture details according to a single-frame LDR image.

[0133] Figure 4 It is a schematic structural diagram of an image reconstruction device provided in an embodiment of this application. As Figure 4 shown, the image reconstruction device 400 in this embodiment includes: an acquisition module 401, a first processing module 402, a second processing module 403, a third processing module 404, and a fourth processing module 405.

[0134] The acquisition module 401 is used to acquire an image reconstruction instruction, which is used to indicate reconstructing an HDR image from an LDR image using a preset image reconstruction neural network model. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model.

[0135] The first processing module 402 is used to acquire the edge image of the LDR image according to the LDR image.

[0136] The second processing module 403 is used to input the LDR image into the pre-trained global neural network model to obtain the intermediate image of the LDR image.

[0137] The third processing module 404 is configured to input the edge image of the LDR image into a pre-trained edge neural network model to obtain an intermediate edge image of the LDR image.

[0138] The fourth processing module 405 is configured to input the intermediate image of the LDR image and the intermediate edge image of the LDR image into a pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0139] Based on any of the above-described embodiments, the image reconstruction apparatus 400 of this embodiment further includes: a training module 406.

[0140] The training module 406 is configured to:

[0141] Obtain multiple HDR sample images; obtain corresponding multiple LDR sample images according to the multiple HDR sample images; obtain edge images of the multiple LDR sample images according to the multiple LDR sample images; input the multiple LDR sample images into a global neural network model for training to obtain a trained global neural network model; and input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images; input the edge images of the multiple LDR sample images into an edge neural network model for training to obtain a trained edge neural network model, and input the edge images of the multiple LDR sample images into the trained edge neural network model respectively to obtain intermediate edge images of the multiple LDR sample images; input the intermediate images and intermediate edge images of the multiple LDR sample images into a fusion neural network model for training to obtain a trained fusion neural network model; and obtain an image reconstruction neural network model according to the trained global neural network model, the trained edge neural network model, and the trained fusion neural network model.

[0142] Based on any of the above-described embodiments, the global neural network model includes a first compression sub-network, a first GRU network, and a first decompression sub-network; wherein, the first compression sub-network includes 3 convolutional layers, and the first decompression sub-network includes 3 deconvolutional layers.

[0143] Based on any of the above-described embodiments, the edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network; wherein, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

[0144] Based on any of the above-described embodiments, the fusion neural network model includes a third compression sub-network and a third decompression sub-network; wherein, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

[0145] Based on any of the above-described embodiments, the training module 406 is specifically configured to:

[0146] Input multiple LDR sample images into the global neural network model for training, and use the loss function shown in Formula 1 below during the training process to obtain a trained global neural network model;

[0147] L global = α1L1 + α2L hist Formula 1

[0148] where L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the global neural network model for the LDR sample image at the i-th pixel, I represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the HDR sample image histogram classification statistical loss, where M represents the number of types of statistical histograms of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents the number of predicted pixel values of the global neural network model for the LDR sample image as , count m (I) represents the number of pixels with the value I in the HDR sample image, and α1 and α2 represent weight parameters.

[0149] Based on any of the above-described embodiments, the training module 406 is specifically configured to:

[0150] Input the edge images of multiple LDR sample images into the edge neural network model for training, and use the loss function shown in Formula 2 below during the training process to obtain a trained edge neural network model;

[0151]

[0152] where L edge represents the edge loss function value, N represents the number of pixels of the edge image of the LDR sample image, represents the predicted pixel value of the edge neural network model for the edge image of the LDR sample image at the i-th pixel, edge(I i ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

[0153] Based on any of the above-described embodiments, the training module 406 is specifically configured to:

[0154] Input the intermediate image and the intermediate edge image of multiple LDR sample images into the fusion neural network model for training. During the training process, use the loss function shown in Formula 3 below to obtain the trained fusion neural network model;

[0155] L recon = α3L global + α4L edge + α5SSMI(I recon , I HDR ) Formula 3

[0156] Among them, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, I recon represents the HDR image output by the fusion neural network model, I HDR represents the HDR sample image, and SSMI(I recon , I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image. Among them, μ recon represents the average of the pixel values of I rec on, μ HDR represents the average of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR , c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent the stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

[0157] The device of this embodiment can be used to execute the technical solutions of any of the above - shown method embodiments. Its implementation principle and technical effects are similar and will not be elaborated here.

[0158] Figure 5 is the structural schematic diagram of an image reconstruction device provided in another embodiment of this application. As Figure 5 shown, the image reconstruction device 500 of this embodiment includes: a memory 501 and a processor 502. Among them, the memory 501 and the processor 502 are connected through a bus.

[0159] The memory 501 is used to store program instructions.

[0160] The processor 502 is used to call program instructions in the memory to execute:

[0161] Obtain an image reconstruction instruction, which is used to indicate reconstructing an HDR image from an LDR image by using a preset image reconstruction neural network model. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model. According to the LDR image, obtain the edge image of the LDR image. Input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image. Input the edge image of the LDR image into the pre-trained edge neural network model to obtain an intermediate edge image of the LDR image. Input the intermediate image of the LDR image and the intermediate edge image of the LDR image into the pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image.

[0162] Based on any of the above-described embodiments, the processor 502 is further used for:

[0163] Obtain multiple HDR sample images; according to the multiple HDR sample images, obtain corresponding multiple LDR sample images; according to the multiple LDR sample images, obtain the edge images of the multiple LDR sample images; input the multiple LDR sample images into the global neural network model for training to obtain a trained global neural network model; and input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images; input the edge images of the multiple LDR sample images into the edge neural network model for training to obtain a trained edge neural network model, and input the edge images of the multiple LDR sample images into the trained edge neural network model respectively to obtain intermediate edge images of the multiple LDR sample images; input the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training to obtain a trained fusion neural network model; according to the trained global neural network model, the trained edge neural network model, and the trained fusion neural network model, obtain the image reconstruction neural network model.

[0164] Based on any of the above-described embodiments, the global neural network model includes a first compression sub-network, a first GRU network, and a first decompression sub-network; wherein, the first compression sub-network includes 3 convolutional layers, and the first decompression sub-network includes 3 deconvolutional layers.

[0165] Based on any of the above-described embodiments, the edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network; wherein, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

[0166] Based on any of the above-described embodiments, the fusion neural network model includes a third compression sub-network and a third decompression sub-network; wherein, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

[0167] Based on any of the above-described embodiments, the processor 502 is specifically configured to:

[0168] Input multiple LDR sample images into the global neural network model for training, and adopt the loss function shown in Formula 1 below during the training process to obtain a trained global neural network model;

[0169] L global = α1L1 + α2L hist Formula 1

[0170] Wherein, L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the global neural network model for the LDR sample image at the i-th pixel, I represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the HDR sample image histogram classification statistical loss, Wherein, M represents the number of types of statistical histograms of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents that the predicted pixel value of the global neural network model for the LDR sample image is the number of, count m (I) represents the number of pixels with the pixel value I in the HDR sample image, and α1 and α2 represent weight parameters.

[0171] Based on any of the above-described embodiments, the processor 502 is specifically configured to:

[0172] Input the edge images of multiple LDR sample images into the edge neural network model for training, and adopt the loss function shown in Formula 2 below during the training process to obtain a trained edge neural network model;

[0173]

[0174] Wherein, L edge represents the edge loss function value, N represents the number of pixels of the edge image of the LDR sample image, represents the predicted pixel value of the edge neural network model for the edge image of the LDR sample image at the i-th pixel, edge(Ii ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

[0175] Based on any of the above-described embodiments, the processor 502 is specifically configured to:

[0176] Input the intermediate image and the intermediate edge image of multiple LDR sample images into a fusion neural network model for training, and use the loss function shown in Formula 3 below during the training process to obtain a trained fusion neural network model;

[0177] L recon = α3L global + α4L edge + α5SSMI(I recon , I HDR ) Formula 3

[0178] Wherein, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, I recon represents the HDR image output by the fusion neural network model, I HDR represents the HDR sample image, SSMI(I recon , I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image, Wherein, μ recon represents the average value of the pixel values of I recon , μ HDR represents the average value of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR , c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

[0179] The device of this embodiment can be used to execute the technical solutions of any of the above-described method embodiments. The implementation principle and technical effects are similar and will not be elaborated here.

[0180] Figure 6 is a schematic structural diagram of an image reconstruction device provided in another embodiment of the present application, as Figure 6As shown, for example, the image reconstruction device 600 may be provided as a server or a computer. Referring to Figure 6 , the image reconstruction device 600 includes a processing component 601, which further includes one or more processors, and memory resources represented by a memory 602 for storing instructions executable by the processing component 601, such as application programs. The application programs stored in the memory 602 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 601 is configured to execute instructions to perform any of the above method embodiments.

[0181] The device 600 may further include a power supply component 603 configured to perform power management of the device 600, a wired or wireless network interface 604 configured to connect the device 600 to a network, and an input / output (I / O) interface 605. The device 600 may operate based on an operating system stored in the memory 602, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0182] This application also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the above image reconstruction method.

[0183] For the above computer-readable storage medium, the above-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk or an optical disk. The readable storage medium may be any available medium accessible by a general-purpose or special-purpose computer.

[0184] An exemplary readable storage medium is coupled to the processor such that the processor can read information from and write information to the readable storage medium. Of course, the readable storage medium may also be a component of the processor. The processor and the readable storage medium may be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium may also exist as discrete components in the image reconstruction device.

[0185] Those of ordinary skill in the art will understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image reconstruction method, characterized in that, Including: Obtain an image reconstruction instruction for instructing to reconstruct a high-dynamic range (HDR) image from a current single-frame low-dynamic range (LDR) image using a preset image reconstruction neural network model. The preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model. Obtain an edge image of the LDR image according to the LDR image. Input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image. The global neural network model is used to balance the overall high brightness and enhance the low brightness of the LDR image. Input the edge image of the LDR image into the pre-trained edge neural network model to obtain an intermediate edge image of the LDR image. The edge neural network model is used to extract and sharpen the texture features of the low-brightness region of the LDR image. Input the intermediate image of the LDR image and the intermediate edge image of the LDR image into the pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image. The global neural network model is trained based on the following method: Input multiple LDR sample images into the global neural network model for training. During the training process, use the loss function shown in the following formula (1) to obtain the trained global neural network model. L global = α1L1 + α2L hist Formula 1 Among them, L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the global neural network model for the LDR sample image at the i-th pixel, I i represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the histogram grading statistical loss of the HDR sample image, Among them, M represents the number of types of statistical histograms of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents that the predicted pixel value of the global neural network model for the LDR sample image is the number of, count m (I) represents the number of pixels with the value of I in the HDR sample image, and α1 and α2 represent weight parameters; The LDR sample images are obtained based on the HDR sample images.

2. The method according to claim 1, wherein The preset image reconstruction neural network model is obtained by the following steps: Obtain multiple HDR sample images. Obtain corresponding multiple LDR sample images according to the multiple HDR sample images. Obtain edge images of the multiple LDR sample images according to the multiple LDR sample images. Input the multiple LDR sample images into the global neural network model for training to obtain the trained global neural network model. And input the multiple LDR sample images into the trained global neural network model respectively to obtain intermediate images of the multiple LDR sample images. Input the edge images of the multiple LDR sample images into the edge neural network model for training to obtain the trained edge neural network model. And input the edge images of the multiple LDR sample images into the trained edge neural network model respectively to obtain intermediate edge images of the multiple LDR sample images. Input the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training to obtain the trained fusion neural network model. Obtain the image reconstruction neural network model according to the trained global neural network model, the trained edge neural network model, and the trained fusion neural network model.

3. The method according to claim 2, characterized in that, The global neural network model includes a first compression sub-network, a first gated recurrent unit (GRU) network, and a first decompression sub-network. Among them, the first compression sub-network includes 3 convolutional layers, and the first decompression sub-network includes 3 deconvolutional layers. The edge neural network model includes a second compression sub-network, a second GRU network, and a second decompression sub-network; Among them, the second compression sub-network includes 2 convolutional layers, and the second decompression sub-network includes 2 deconvolutional layers.

4. The method according to claim 2, wherein The fusion neural network model includes a third compression sub-network and a third decompression sub-network; Among them, the third compression sub-network includes 4 convolutional layers, and the third decompression sub-network includes 3 deconvolutional layers.

5. The method according to claim 2, characterized in that, The step of inputting the edge images of the multiple LDR sample images into the edge neural network model for training to obtain the trained edge neural network model includes: Inputting the edge images of the multiple LDR sample images into the edge neural network model for training, and using the loss function shown in Formula 2 below during the training process to obtain the trained edge neural network model; Among them, L edge represents the edge loss function value, N represents the number of pixels in the edge image of the LDR sample image, represents the predicted pixel value of the edge neural network model for the edge image of the LDR sample image at the i-th pixel, edge(I i ) represents the true pixel value of the edge image of the HDR sample image at the i-th pixel.

6. The method according to any one of claims 2-5, characterized in that The step of inputting the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training to obtain the trained fusion neural network model includes: Inputting the intermediate images and intermediate edge images of the multiple LDR sample images into the fusion neural network model for training, and using the loss function shown in Formula 3 below during the training process to obtain the trained fusion neural network model; L recon = α3L global + α4L edge + α5SSMI(I recon ,I HDR ) Equation 3 Among them, L recon represents the loss function value of the reconstructed image, α3, α4, and α5 represent weight parameters, and I recon represents the HDR image output by the fusion neural network model, and I HDR represents the HDR sample image. SSMI(I recon , I HDR ) represents the structural similarity between the HDR image output by the fusion neural network model and the HDR sample image. Among them, μ recon represents the average value of the pixel values of I recon , μ HDR represents the average value of the pixel values of I HDR , σ recon represents the variance of I recon , σ HDR represents the variance of I HDR , σ recon,HDR represents the covariance of I recon and I HDR . c1 = (k1I range ) 2 and c2 = (k2I range ) 2 represent stability coefficients, I range represents the dynamic range of the HDR sample image, and k1 and k2 represent empirical coefficients.

7. An image reconstruction device, characterized in that, It includes: An acquisition module, configured to acquire an image reconstruction instruction, where the image reconstruction instruction is used to indicate to reconstruct a high-dynamic range (HDR) image from a current single-frame low-dynamic range (LDR) image using a preset image reconstruction neural network model, and the preset image reconstruction neural network model includes a pre-trained global neural network model, an edge neural network model, and a fusion neural network model; A first processing module, configured to obtain an edge image of the LDR image according to the LDR image; A second processing module, configured to input the LDR image into the pre-trained global neural network model to obtain an intermediate image of the LDR image, and the global neural network model is used to balance the overall high brightness and enhance the low brightness of the LDR image; A third processing module, configured to input the edge image of the LDR image into the pre-trained edge neural network model to obtain an intermediate edge image of the LDR image, and the edge neural network model is used to extract and sharpen the texture features of the low-brightness area of the LDR image; A fourth processing module, configured to input the intermediate image of the LDR image and the intermediate edge image of the LDR image into the pre-trained fusion neural network model to obtain the HDR image corresponding to the LDR image, and the fusion neural network model includes a third compression sub-network and a third decompression sub-network; A training module, when training the global neural network model, is configured to: Input multiple LDR sample images into the global neural network model for training, and use the loss function shown in Formula 1 below during the training process to obtain the trained global neural network model; L global = α1L1 + α2L hist Formula 1 Among them, L global represents the global loss function value, L1 represents the pixel statistical loss of the HDR sample image, N represents the number of pixels of the HDR sample image, represents the predicted pixel value of the global neural network model for the LDR sample image at the i-th pixel, I i represents the true pixel value of the HDR sample image at the i-th pixel, L hist represents the histogram grading statistical loss of the HDR sample image, Among them, M represents the number of types of statistical histograms of the HDR sample image, m represents the type of the statistical histogram of the m-th HDR sample image, ω m represents the weight of the statistical histogram of the HDR sample image, represents that the predicted pixel value of the global neural network model for the LDR sample image is the number of, count m (I) represents the number of pixels with the pixel value of I in the HDR sample image, and α1 and α2 represent the weight parameters.

8. An image reconstruction device, characterized in that, It includes: A memory and a processor; The memory is used to store program instructions; The processor is configured to call program instructions in the memory to execute the image reconstruction method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the image reconstruction method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the image reconstruction method according to any one of claims 1-7 is implemented.

Images