HDR image saliency detection method, device, image processing equipment and storage medium

Abstract

The present invention is applicable to the field of image processing technology, and provides a HDR image saliency detection method, device, image processing equipment and storage medium. The method comprises: decomposing an input HDR image to be detected into multiple exposure images and a tone mapping image; performing a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain multi-exposure inspired local salient features; performing global context-aware feature extraction on the tone mapping image to obtain global salient features driven by heuristic semantic priors; fusing the local salient features with the global salient features to generate a saliency map of the HDR image, thereby effectively utilizing the intrinsic relationship between the multiple exposure images and the tone mapping image, and combining the global context-aware features to improve the accuracy of HDR image saliency detection.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a HDR image saliency detection method, device, image processing equipment and storage medium. Background Art

[0002] Compared with traditional low-dynamic range (LDR) imaging, high-dynamic range (HDR) imaging can obtain more realistic scenes and provide better aesthetic appreciation. HDR images can provide a wider range of brightness, a wider color gamut, and richer details. Identifying salient areas in high dynamic range images is a very important issue in HDR imaging and plays a vital role in fully understanding the scene. In order to adapt to various HDR devices, there is an urgent need for automatic detection of visual saliency in HDR images, which aims to identify the most attractive areas or objects in the scene. By perceiving and retaining important content, visual saliency has been widely concerned and applied to many HDR image applications, such as HDR image tone mapping and HDR image compression.

[0003] In recent years, due to the successful development of deep neural networks (DNN), great progress has been made in saliency detection of LDR images. Due to its powerful feature representation ability, the DNN model is able to extract high-level semantic priors from the scene that are helpful for visual saliency reasoning. However, most DNN-based LDR image saliency detection methods are only suitable for traditional 8-bit LDR images. It is obviously not feasible to infer the saliency of HDR images by directly applying deep learning-based LDR image saliency methods. According to psychophysical research, compared with LDR images, the visual attention of HDR images is different from that of LDR images in areas with higher contrast and brightness. For example, Figure 1 As shown in the figure, significant differences in the content and eye fixation positions of LDR and HDR images can be observed. For example, in HDR, subjects can focus on more objects compared to LDR images, and their attention can be shifted from the brighter areas of the LDR image to the darker detailed areas in the HDR image, as shown in the third row of the figure. Therefore, HDR imaging can indeed have an impact on the pattern and intensity of visual attention.

[0004] Currently, only a few studies focus on saliency detection of HDR images. Different from 2D saliency detection models, existing HDR saliency detection models have improved the performance of HDR image saliency detection by considering some characteristics of HDR imaging. However, there are still some challenging problems in the characterization of HDR image content. Traditional methods are based on manually extracted underlying features, ignoring the high-level semantic information that represents the image content, resulting in limited performance. Existing deep learning-based methods do not fully consider the characteristics of HDR images and ignore the correlation between images with different exposure levels. Therefore, existing HDR image saliency detection models lack the feature representation of diverse image content and do not fully consider the imaging characteristics of HDR images. Summary of the invention

[0005] The purpose of the present invention is to provide a HDR image saliency detection method, apparatus, image processing device and storage medium, aiming to solve the problem that the accuracy of the generated saliency map is not high because the prior art cannot provide an effective HDR image saliency map generation method.

[0006] In one aspect, the present invention provides a method for detecting saliency of an HDR image, the method comprising:

[0007] Decomposing the input HDR image to be detected into multiple exposure images and a tone mapping image;

[0008] performing a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain multi-exposure inspired local salient features;

[0009] performing global context-aware feature extraction on the tone-mapped image to obtain global saliency features driven by heuristic semantic priors;

[0010] The local salient features and the global salient features are fused to generate a saliency map of the HDR image.

[0011] In another aspect, the present invention provides a HDR image saliency detection device, the device comprising:

[0012] An image decomposition unit, used for decomposing an input HDR image to be detected into multiple exposure images and a tone mapping image;

[0013] a first feature acquisition unit, configured to perform a multi-exposure inspired salient feature aggregation operation on the multi-exposure image and the tone-mapped image to obtain a multi-exposure inspired local salient feature;

[0014] a second feature acquisition unit, configured to perform global context-aware feature extraction on the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior; and

[0015] The saliency map generating unit is used to fuse the local saliency features with the global saliency features to generate a saliency map of the HDR image.

[0016] On the other hand, the present invention further provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0017] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0018] The present invention decomposes an input HDR image to be detected into multiple exposure images and a tone mapping image, performs a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain multi-exposure inspired local salient features, performs global context-aware feature extraction on the tone mapping image to obtain global salient features driven by heuristic semantic priors, and finally fuses the local salient features with the global salient features to generate a saliency map of the HDR image, thereby effectively utilizing the intrinsic relationship between the multiple exposure images and the tone mapping image, and combining the global context-aware features to improve the accuracy of HDR image saliency detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a specific example diagram of existing HDR images and LDR images;

[0020] Figure 2 is a flowchart of an implementation method of an HDR image saliency detection method provided in Embodiment 1 of the present invention;

[0021] Figure 3 is an example diagram of an HDR image and a corresponding multiple exposure image provided by the first embodiment of the present invention;

[0022] Figure 4 It is a flowchart for implementing a multi-exposure inspired salient feature aggregation operation on a multi-exposure image and a tone-mapped image in the HDR image saliency detection method provided in the second embodiment of the present invention;

[0023] Figure 5 Schematic diagram of the structure of the HDR image saliency detection architecture provided by the fourth embodiment of the present invention;

[0024] Figure 6 is a schematic diagram of applying the HDR image saliency detection method provided in the fourth embodiment of the present invention to the HDR image saliency detection architecture to perform saliency detection;

[0025] Figure 7 is a structural diagram of an HDR image saliency detection device provided in Embodiment 5 of the present invention; and

[0026] Figure 8 It is a structural schematic diagram of an image processing device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0029] Embodiment 1:

[0030] Figure 2 The implementation process of the HDR image saliency detection method provided by the first embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0031] In step S201, the input HDR image to be detected is decomposed into multiple exposure images and a tone mapping image;

[0032] The embodiment of the present invention is applicable to an image processing device with image processing function, such as a personal computer, a server or a camera device, etc., to perform saliency detection on an input or captured HDR image. The multiple exposure images are a series of LDR images with different exposure levels (exposure time), and the tone mapping image is an LDR image. When the input HDR image to be detected is decomposed into multiple exposure images and a tone mapping image, a group of LDR images with different exposure levels are synthesized as an HDR image by camera simulation. In a preferred embodiment, when a group of LDR images with different exposure levels are synthesized as an HDR image by camera simulation, the formula L is used. d =f(H*Δt d ) is used to synthesize a set of LDR images with different exposure levels for the HDR image, where L d represents the LDR image, f represents the camera response function, H and Δt dDenote the HDR image and exposure duration respectively, so as to accurately obtain the LDR image with different exposure levels. Specifically, first, the average pixel value of the HDR image is normalized to 0.5, and then the power function is used to define Δt d , Where T is an even number, d = 1, 2, ..., T + 1, and finally the above formula is used to synthesize LDR images with different exposure levels. Preferably, τ and T are set to and 8, thereby further accurately acquiring LDR images with different exposure levels.

[0033] As an example, Figure 3 As shown, it can be observed that some objects can be highlighted in underexposed images, such as the green lawn shown in the first row of the figure, the blue sky and white clouds shown in the second row, while these objects cannot be seen in overexposed images, such as the three images shown after the first and second rows in the figure. Similarly, some objects will be displayed in overexposed images, such as the football goal frame shown in the first row of the figure and the building shown in the second row. It can be seen that multiple exposures help identify the most salient objects or areas in HDR images, thereby improving the performance of HDR image saliency detection.

[0034] In step S202, a multi-exposure inspired salient feature aggregation operation is performed on the multi-exposure image and the tone-mapped image to obtain multi-exposure inspired local salient features;

[0035] In an embodiment of the present invention, a multi-exposure inspired salient feature aggregation module based on a graph neural network is used to perform a multi-exposure inspired salient feature aggregation operation on multiple exposure images and tone-mapped images to model the intrinsic relationship between the multiple exposure images and the tone-mapped images (including the spatial correlation between the multiple exposure images and the tone-mapped images, and the brightness correlation between multiple exposure images of different exposures), thereby generating local salient features driven by their intrinsic relationship, namely, multi-exposure inspired local salient features.

[0036] In step S203, global context-aware feature extraction is performed on the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior;

[0037] In an embodiment of the present invention, considering that high-level semantic information plays a crucial role in image saliency detection, a global context-aware HDR image saliency reasoning module is used to perform global context-aware feature extraction on tone-mapped images to obtain global saliency features driven by heuristic semantic priors.

[0038] In step S204, the local salient features and the global salient features are fused to generate a saliency map of the HDR image.

[0039] In the embodiment of the present invention, when the local salient features and the global salient features are fused, specifically, the local salient features and the global salient features can be fused using a convolution operation to generate a saliency map of the final HDR image.

[0040] The embodiment of the present invention decomposes an input HDR image to be detected into multiple exposure images and a tone mapping image, performs a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain multi-exposure inspired local salient features, performs global context-aware feature extraction on the tone mapping image to obtain global salient features driven by heuristic semantic priors, and finally fuses the local salient features with the global salient features to generate a saliency map of the HDR image, thereby effectively utilizing the intrinsic relationship between the multiple exposure images and the tone mapping image, and combining the global context-aware features to improve the accuracy of HDR image saliency detection.

[0041] Embodiment 2:

[0042] Figure 4 The implementation process of performing a multi-exposure inspired salient feature aggregation operation on a multi-exposure image and a tone mapping image in the HDR image saliency detection method provided in the second embodiment of the present invention is shown. For the sake of convenience, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0043] In step S401, the multiple exposure images are divided into a preset number of image groups according to the exposure levels, a feature representation of each image group is obtained by a feature extraction module, and a feature representation of a tone mapping image is obtained;

[0044] In an embodiment of the present invention, the exposure degree can be set to multiple levels, for example, three levels of underexposure, medium exposure and overexposure, or other number of levels. The number of image groups is the same as the number of exposure level levels. After the multiple exposure images are divided into a preset number of image groups, a feature representation of each image group is obtained through a feature extraction module. The feature extraction module may include a residual network module, a feature conversion module and a cascade module to perform feature extraction on each exposure image. The residual network module is used to extract features from each exposure image. The feature conversion module is used to reduce the feature dimension of the residual network module input so that the input features can maintain a uniform dimension. Then, the cascade is performed through the cascade module to obtain the feature representation of each image group. The feature representation of the tone mapping image can also be obtained through its corresponding residual network module.

[0045] As an example, in a specific embodiment, the exposure degree can be divided into three levels: underexposure, medium exposure and overexposure. When n images with different exposure degrees are obtained by decomposing the HDR image, and a tone-mapped image When , n exposure images can be divided into three groups: and Here n can be set to 9. The feature representation of the first image group can be obtained through the feature extraction module. Among them, w 2d and b 2d represents the weight and bias of the two-dimensional convolution kernel; [·] represents the cascade operation, F1,…,F n / 3 The feature representation of each exposure image in the first group is represented, and the feature representations of other image groups can be obtained in the same way.

[0046] In step S402, the feature representation of each image group and the feature representation of the tone mapping image are set as input nodes of a preset graph neural network, the state representation of each node is learned and updated through the state representation of adjacent nodes in the graph neural network, and the feature representation of all nodes after the final iteration is output;

[0047] In an embodiment of the present invention, the number of nodes of the graph neural network is the same as the number and the number of feature representations of each image group and tone mapping image. In addition, in order to clearly mine the intrinsic relationship between the multiple exposure images and the tone mapping images, the graph neural network is a fully connected graph network. After obtaining the feature representation of each image group and the feature representation of the tone mapping image, the feature representation of each image group and the feature representation of the tone mapping image are set as input nodes of the preset graph neural network, and the state representation of each node is learned and updated through the state representation of the adjacent nodes in the graph neural network, and the feature representation of all nodes after the final iteration is output, so as to use the graph neural network to deeply mine the spatial and brightness complementary correlations between the multiple exposure images and the mapping images, and improve the accuracy of saliency detection of subsequent HDR images.

[0048] In step S403, the feature representations of all nodes are cascaded, and a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image is obtained according to the cascaded features;

[0049] In an embodiment of the present invention, after obtaining the feature representations of all nodes after the final iteration in the graph neural network, the feature representations of all nodes are cascaded, and then a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image is obtained based on the cascaded features.

[0050] When obtaining a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image based on the cascaded features, a stacked convolutional layer is used to learn the correlation between the cascaded features and reduce the dimension of the cascaded features to generate a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image.

[0051] In step S404, local saliency features inspired by multiple exposures are obtained according to the feature vector.

[0052] In an embodiment of the present invention, when obtaining multi-exposure inspired local salient features based on a feature vector, the feature vector obtained in step S403 is multiplied by the feature representations of all previously acquired multiple exposure images, all the multiplied features are cascaded, and the cascaded features are input into a preset convolutional layer to finally generate multi-exposure inspired local salient features, thereby utilizing a graph neural network to learn the relationship between multi-exposure images and mapping images, and finally obtaining multi-exposure inspired local salient features.

[0053] Embodiment three:

[0054] In an embodiment of the present invention, when performing global context-aware feature extraction on a tone-mapped image, a global context-aware HDR image saliency reasoning module is used to process the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior, thereby improving the accuracy of a saliency map of a subsequent HDR image.

[0055] In a specific embodiment, the HDR image saliency reasoning module includes a feature extraction module, three global perception units, a cascade module and a feature conversion module. The feature extraction module uses a residual network to obtain three features of the tone mapping image (for example, denoted by ), each global perception unit Pe consists of five parallel convolutional layers, four of which are dilated convolutional layers with dilation rates of 1, 6, 12, and 18, respectively. Benefiting from the dilated convolution, the HDR image saliency reasoning module can capture rich contextual information at multiple scales without sacrificing image resolution. The outputs of the three global perception units can be expressed as in, as well as Afterwards, the cascade module combines the three contextual semantic features The cascade is then input into the feature transformation module to finally obtain the global saliency feature driven by the heuristic semantic prior.

[0056] Embodiment 4:

[0057] Figure 5The structure of the HDR image saliency detection architecture provided by the fourth embodiment of the present invention is shown. For the sake of convenience, only the part related to the embodiment of the present invention is shown.

[0058] In the embodiment of the present invention, the HDR image saliency detection architecture includes an image decomposition module 51, a multi-exposure inspired salient feature aggregation module 52, a global context-aware HDR image saliency reasoning module 53 and a feature fusion module 54, wherein:

[0059] An image decomposition module 51 is used to decompose the input HDR image to be detected into multiple exposure images and a tone mapping image;

[0060] A global context-aware salient feature aggregation module 52, configured to perform a multi-exposure inspired salient feature aggregation operation on the multi-exposure image and the tone-mapped image to obtain multi-exposure inspired local salient features;

[0061] In the embodiment of the present invention, the multi-exposure inspired salient feature aggregation module 52 is a salient feature aggregation module based on a graph neural network, and the multi-exposure inspired salient feature aggregation module 52 includes a first feature extraction module, a graph neural network, a first cascade module, a first feature conversion module, a multiplication module, and a second cascade module. The first feature extraction module is used to obtain the feature representation of each image group, and input the feature representation of each image group into the graph neural network. The graph neural network deeply mines the spatial and brightness complementary correlations between the multiple exposure images and the mapped images based on the feature representation of each image group and the feature representation of the tone mapping image, and outputs the feature representations after the final iteration of all nodes. The first cascade module cascades the feature representations of all nodes output by the graph neural network. The first feature conversion module learns the correlation between the cascaded features and reduces the dimension of the cascaded features, and generates a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping images. The multiplication module is used to multiply the feature vector obtained by the first feature conversion module with the feature representations of all the multiple exposure images obtained by the first feature extraction module. The second cascade module cascades all the multiplied features, and inputs the cascaded features into a preset convolutional layer, and finally generates the multi-exposure inspired local salient features.

[0062] The first feature extraction module includes a residual network module, a second feature conversion module and a third cascade module, wherein the residual network module is used to extract features from each exposure image, and the second feature conversion module is used to reduce the feature dimension of the input of the residual network module so that the input features can maintain a uniform dimension, and then cascade through the third cascade module to obtain the feature representation of each image group. The first feature conversion module uses stacked convolutional layers to learn the correlation between cascaded features and reduce the dimension of the cascaded features to generate a feature vector representing the spatial and brightness relationship between multiple exposure images and tone mapping images.

[0063] A global context-aware HDR image saliency inference module 53, for performing global context-aware feature extraction on the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior;

[0064] In an embodiment of the present invention, the global context-aware HDR image saliency reasoning module 53 includes a second feature extraction module, three global perception units, a fourth cascade module and a third feature conversion module. The second feature extraction module uses a residual network to obtain three features of the tone mapping image. The three global perception units learn the global context-aware salient features in the tone mapping image based on the three features obtained by the second feature extraction module and alleviate information dilution at the same time. Afterwards, the fourth cascade module cascades the three contextual semantic features output by the three global perception units, and then inputs them into the third feature conversion module, and finally obtains the global saliency features driven by heuristic semantic priors.

[0065] The feature fusion module 54 is used to fuse the local salient features obtained by the global context-aware salient feature aggregation module 52 and the global salient features obtained by the global context-aware HDR image salient reasoning module 53 to generate a saliency map of the HDR image.

[0066] The HDR image saliency detection method described in Examples 1 to 3 can be applied to the embodiment of the present invention, so that through the HDR image saliency detection architecture of the embodiment of the present invention, the intrinsic relationship between the multi-exposure image and the tone mapping image is effectively utilized, and the global context perception features are combined to improve the accuracy of the saliency map. Figure 6 The schematic diagram of applying the HDR image saliency detection method to the HDR image saliency detection architecture for saliency detection is shown. Table 1 shows the comparison of the result parameters of saliency detection on two benchmark datasets (HDR-Eye and ETHyma datasets) of the existing representative HDR image saliency detection models (Dong et al.'s method, LBVS-HDR, DF-HSal) and the embodiments of the present invention. It can be seen that the embodiments of the present invention have better saliency detection performance.

[0067] Table 1

[0068]

[0069]

[0070] Embodiment five:

[0071] Figure 7 The structure of the HDR image saliency detection device provided by the fifth embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, including:

[0072] An image decomposition unit 71 is used to decompose the input HDR image to be detected into multiple exposure images and a tone mapping image;

[0073] A first feature acquisition unit 72, configured to perform a multi-exposure inspired salient feature aggregation operation on the multi-exposure image and the tone-mapped image to obtain a multi-exposure inspired local salient feature;

[0074] A second feature acquisition unit 73, configured to perform global context-aware feature extraction on the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior; and

[0075] The saliency map generating unit 74 is used to fuse the local saliency features with the global saliency features to generate a saliency map of the HDR image.

[0076] In the embodiment of the present invention, each unit of the HDR image saliency detection device can be implemented by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which is not intended to limit the present invention. The specific implementation of each unit can refer to the description of embodiments 1 to 3, which will not be repeated here.

[0077] Embodiment six:

[0078] Figure 8 The structure of the image processing device provided by the sixth embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown.

[0079] The image processing device 8 of the embodiment of the present invention includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, the steps in the above-mentioned HDR image saliency detection method embodiment are implemented, for example: Figure 2Alternatively, when the processor 80 executes the computer program 82, the functions of each unit in the above-mentioned device embodiment are realized, for example Figure 7 The functions of the units 71 to 74 are shown.

[0080] The image processing device of the embodiment of the present invention may be a personal computer, a server or a camera device. The steps implemented when the processor 80 in the image processing device 8 executes the computer program 82 to implement the HDR image saliency detection method can refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0081] Embodiment seven:

[0082] In an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned HDR image saliency detection method embodiment are implemented, for example, Figure 2 Alternatively, when the computer program is executed by a processor, the functions of each unit in the above-mentioned HDR image saliency detection device embodiment are realized, for example Figure 7 The functions of the units 71 to 74 are shown.

[0083] The embodiment of the present invention decomposes an input HDR image to be detected into multiple exposure images and a tone mapping image, performs a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain multi-exposure inspired local salient features, performs global context-aware feature extraction on the tone mapping image to obtain global salient features driven by heuristic semantic priors, and finally fuses the local salient features with the global salient features to generate a saliency map of the HDR image, thereby effectively utilizing the intrinsic relationship between the multiple exposure images and the tone mapping image, and combining the global context-aware features to improve the accuracy of HDR image saliency detection.

[0084] The computer-readable storage medium of the embodiment of the present invention may include any entity or device or recording medium capable of carrying computer program code, for example, ROM / RAM, magnetic disk, optical disk, flash memory and other memories.

[0085] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A HDR image saliency detection method, characterized in that: The method comprises: Decomposing the input HDR image to be detected into multiple exposure images and a tone mapping image; A multi-exposure inspired salient feature aggregation operation is performed on the multiple exposure images and the tone mapping image to obtain a multi-exposure inspired local salient feature. The step of performing a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image comprises: dividing the multiple exposure images into a preset number of image groups according to the exposure degree, obtaining a feature representation of each image group through a feature extraction module, and obtaining a feature representation of the tone mapping image; setting the feature representation of each image group and the feature representation of the tone mapping image as input nodes of a preset graph neural network, learning and updating the state representation of each node through the state representation of adjacent nodes in the graph neural network, and outputting the feature representation of all nodes after the final iteration; cascading the feature representations of all nodes, and obtaining a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image according to the cascaded features; and obtaining the multi-exposure inspired local salient feature according to the feature vector; performing global context-aware feature extraction on the tone-mapped image to obtain global saliency features driven by heuristic semantic priors; The local salient features and the global salient features are fused to generate a saliency map of the HDR image.

2. The method according to claim 1, characterized in that The step of decomposing the input HDR image to be detected into multiple exposure images and a tone mapping image includes: A set of LDR images with different exposure levels are synthesized for the HDR image through camera simulation.

3. The method according to claim 2, characterized in that The steps of synthesizing a set of LDR images with different exposure levels for the HDR image through camera simulation are specifically: By formula L d =f(H*Δt d ) synthesizes a set of LDR images with different exposure levels for the HDR image, where L d represents the LDR image, f represents the camera response function, H and Δt d represent the HDR image and exposure duration respectively.

4. The method according to claim 1, characterized in that The step of obtaining a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image according to the cascaded features is specifically: Stacked convolutional layers are used to learn the correlation between the cascaded features and reduce the dimension of the cascaded features, so as to generate a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image.

5. The method according to claim 1, characterized in that The step of obtaining the local saliency feature inspired by multiple exposures according to the feature vector is specifically as follows: The feature vector is multiplied by the feature representation of the multiple exposure image, all the multiplied features are cascaded, and the cascaded features are input into a preset convolutional layer to generate the local saliency features inspired by the multiple exposures.

6. The method according to claim 1, characterized in that The step of performing global context-aware feature extraction on the tone-mapped image comprises: The tone mapping image is processed using a global context-aware HDR image saliency reasoning module to obtain a global saliency feature driven by a heuristic semantic prior, wherein the HDR image saliency reasoning module includes three global perception units.

7. A HDR image saliency detection device, characterized in that: The device comprises: An image decomposition unit, used for decomposing an input HDR image to be detected into multiple exposure images and a tone mapping image; A first feature acquisition unit is used to perform a multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image to obtain a multi-exposure inspired local salient feature. When performing the multi-exposure inspired salient feature aggregation operation on the multiple exposure images and the tone mapping image, the unit includes: dividing the multiple exposure images into a preset number of image groups according to the exposure degree, obtaining a feature representation of each image group through a feature extraction module, and obtaining a feature representation of the tone mapping image; setting the feature representation of each image group and the feature representation of the tone mapping image as input nodes of a preset graph neural network, learning and updating the state representation of each node through the state representation of adjacent nodes in the graph neural network, and outputting the feature representation of all nodes after the final iteration; cascading the feature representations of all nodes, and obtaining a feature vector representing the spatial and brightness relationship between the multiple exposure images and the tone mapping image according to the cascaded features; and obtaining the multi-exposure inspired local salient feature according to the feature vector; a second feature acquisition unit, configured to perform global context-aware feature extraction on the tone-mapped image to obtain a global saliency feature driven by a heuristic semantic prior; and The saliency map generating unit is used to fuse the local saliency features with the global saliency features to generate a saliency map of the HDR image.

8. An image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images