Image visual saliency extraction method

Abstract

The invention provides a method for extracting visual saliency of an image, which comprises the following steps of: acquiring an image, and carrying out superpixel segmentation on the acquired image to obtain a superpixel set; expressing the constructed hypergraph by using an incidence matrix; performing feature extraction on the image input through a deep learning network; calculating the features of the superpixels; calculating saliency values of the superpixels; calculating a saliency value of the hyperedge; updating the saliency value of the super-pixel associated with the hyperedge; according to the position coordinates of the pixel points in all the superpixels, restoring the saliency value of each pixel point of the whole image; and mapping the feature space importance in an image physical space to obtain a final saliency map of the image. According to the method, super-pixels are re-clustered, the super-pixels and clustering results are subjected to association representation through a hypergraph, two branches are arranged behind two full-connection layers of a network structure, one branch can obtain the importance of a feature channel, and the features are weighted; and the other branch can obtain the importance of the feature space.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for extracting image visual saliency. Background Art

[0002] When facing a scene, humans automatically process the areas of interest and selectively ignore the areas of no interest. These areas of interest are called salient areas. The human visual system has the ability to quickly search and locate objects of interest when facing natural scenes. This visual attention mechanism is an important mechanism for processing visual information in people's daily lives. With the spread of large amounts of big data brought by the Internet, how to quickly obtain important information from massive amounts of image and video data has become a key issue in the field of computer vision. The visual saliency detection model is a process of predicting which information in an image or video is more visually noticed through computer vision algorithms. By introducing this visual attention mechanism, namely visual saliency, into computer vision tasks, a series of significant help and improvements can be brought to visual information processing tasks. The advantages of introducing visual saliency are mainly reflected in two aspects. First, it can allocate limited computing resources to more important information in images and videos. Second, the results of introducing visual saliency are more in line with people's visual cognitive needs. Visual saliency detection has important application value in target recognition, image and video compression, image retrieval, image redirection, etc.

[0003] Visual saliency includes two mechanisms: bottom-up and top-down. Bottom-up can also be considered as data-driven, that is, the image itself attracts people, while top-down is to pay attention to the image under the control of human consciousness. Traditional saliency detection models, whether bottom-up or top-down salient object detection algorithms, are mostly based on texture clues inside the image, using hand-crafted low-level features and heuristic methods for detection. Traditional salient object detection methods are difficult to extend to other complex application scenarios due to their lack of ability to capture image semantic information. In recent years, with the development of deep learning technology, more and more researchers have begun to use deep learning technology for saliency detection. Deep learning methods learn more accurate and effective salient features by training neural network models, thereby achieving more accurate and robust saliency detection. Common deep learning methods include: methods based on fully convolutional networks, methods based on recurrent neural networks, and methods based on attention mechanisms. The attention mechanism simulates the human visual system, focuses on important areas in the image, and generates corresponding saliency maps by calculating the importance of each pixel in the image. Although the saliency detection method based on deep learning has achieved good results, there is still room for improvement.

[0004] In 2017, Shandong University published Hao Jing's master's thesis "Research on Saliency Detection Method Based on Multi-scale Hypergraph of Adaptively Scaling Images". The article mentioned: Saliency detection is an important research topic in the field of image processing and computer vision. This paper explores the method of using multi-scale hypergraphs of adaptively scaling images with independent channels for saliency detection. The work done is of great significance to the development of image processing and computer vision. There are many classic saliency detection methods at present, but almost none of the existing methods consider the impact of the difference in the R, G, and B channel values ​​of the image on the target saliency detection results. This paper introduces the difference in the R, G, and B channel values ​​of the image into the saliency detection method based on hypergraph modeling, and proposes a method for saliency detection using independent channel adaptive scaling. The specific research contents include: (1) Taking into account the difference in sensitivity of the human eye to the three primary colors R, G, and B, the image pixel values ​​are adaptively scaled independently to obtain an independent channel adaptively scaled image of the initial image; (2) The independent channel adaptive scaling method of pixel values ​​is introduced into the construction of hypervertices and hyperedges, and the adaptive scaling factor is combined with the fixed scale empirical value to obtain adaptive multi-scale hyperedges and hypervertices, thereby designing an adaptive multi-scale hypergraph based on independent channel adaptive scaling images; (3) The designed adaptive multi-scale hypergraph based on independent channel adaptive scaling images is introduced into the saliency detection method, and a saliency detection method based on the independent channel adaptive scaling image hypergraph is designed, and the effectiveness of the designed saliency detection method is demonstrated using an image segmentation example. Firstly, the gradient map is used to calculate the saliency of the hypervertices and hyperedges based on the independent channel adaptive scaling image, and the saliency map of the multi-scale hypergraph is obtained. Then, the saliency maps of the multi-scale hypergraph are fused to obtain the final saliency map based on the independent channel adaptive scaling image hypergraph. Finally, the designed saliency detection method is applied to the image segmentation example to illustrate its effectiveness. (4) The designed saliency detection method based on the independent channel adaptive scaling image hypergraph is tested on the publicly available image sets MSRA-1000, SOD, SED and ImgSal-5, and compared with the previous six classic saliency detection methods. A large number of experiments show that the proposed method improves the saliency detection effect to a certain extent for images with narrow differences in the range of R, G, and B channel values. The saliency detection method based on independent channel adaptive scaling images proposed in this paper can be applied to many image processing and computer vision applications such as image segmentation, object recognition, image adaptive compression, content-based image retrieval, etc., which helps to improve its performance and efficiency. Although this article uses a hypergraph method to model images, the saliency detection method still uses traditional computing methods, does not use deep learning methods, and does not achieve the mapping of feature saliency and image physical space. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method for extracting image visual saliency, re-clustering superpixels, and associatively representing superpixels and clustering results through a hypergraph. In the network structure, there are two branches behind two fully connected layers, one branch can obtain the importance of feature channels and weight the features; the other branch can obtain the importance of feature space.

[0006] The present invention provides a method for extracting visual saliency of an image, comprising acquiring an image, and further comprising the following steps:

[0007] Step 1: Perform superpixel segmentation on the image to obtain a superpixel set {a n}, 1≤n≤N, N is the number of superpixels;

[0008] Step 2: Take the incidence matrix K∈R N×M Represent the constructed hypergraph, where N represents the number of superpixels and M represents the number of hyperedges;

[0009] Step 3: Input the image into the deep learning network for feature extraction;

[0010] Step 4: Calculate the features of superpixels;

[0011] Step 5: Calculate the saliency value of the superpixel;

[0012] Step 6: Calculate the saliency value of the hyperedge;

[0013] Step 7: Update the saliency value of the superpixel associated with the hyperedge;

[0014] Step 8: According to the position coordinates of the pixels in all superpixels, the saliency value of each pixel in the entire image is restored, and the corresponding saliency map is represented as X;

[0015] Step 9: Map the importance of feature space in the physical space of the image to obtain the final saliency map of the image.

[0016] Preferably, the step 1 further comprises: n}Clustering is performed, and superpixels of the same class are connected through a hyperedge to build a hypergraph.

[0017] In any of the above schemes, preferably, the structure of the hypergraph is defined as Among them, v n represents superpixel a n The corresponding vertex, v n ∈V, E represents the superpixel set {a n The set of all hyperedges after clustering. The mth element in this set is denoted as e m∈E, 1≤m≤M, W represents the saliency matrix of the hyperedge in the hypergraph, and its elements are denoted by ω(e i ), which can be initialized to a 1×M zero matrix.

[0018] In any of the above solutions, preferably, the element k(v n ,e m ) is expressed as

[0019]

[0020] In any of the above schemes, preferably, the deep learning network has a dual-branch structure, F represents the original feature map, and its shape is (Q, C, H, W), where Q is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0021] In any of the above solutions, preferably, the first branch of the deep learning network performs global average pooling on each channel of the feature map to obtain a feature vector whose feature dimension is the number of channels, and the calculation formula is:

[0022]

[0023] Among them, ⊙ is the dot product symbol, g2 and g1 are the weights of the two fully connected layers, σ is the sigmoid function, G() is the global average pooling function, and F1 is the weighted feature map.

[0024] In any of the above solutions, preferably, the second branch of the deep learning network performs global average pooling on all feature channels of the second fully connected layer, and the calculation method is:

[0025]

[0026] F1 and F2 are fused to obtain the final feature map F3;

[0027] F3=h(F1,F2)

[0028] Among them, h() is the feature fusion function.

[0029] In any of the above solutions, preferably, the super pixel a n Features The calculation formula is

[0030]

[0031] Among them, |a n | represents superpixel a n where p(j) is the number of pixels in the superpixel, p(j) indicates that the jth element in the superpixel corresponds to the p(j)th pixel in the image, and F3(p(j)) represents the feature of pixel p(j).

[0032] In any of the above solutions, preferably, superpixel a n The calculation method of the significant value is as follows:

[0033]

[0034] Among them, dist(a n ,a j ) represents superpixel a n and a j The distance

[0035] Get the saliency matrix S of all superpixels,

[0036]

[0037] Preferably, in any of the above solutions, the method for calculating the saliency value of the hyperedge is as follows:

[0038]

[0039] in, It is a matrix multiplication operation, and count_ones(K) means calculating the number of 1s in each column of matrix K.

[0040] In any of the above schemes, preferably, S2 represents the saliency value of all superpixels,

[0041]

[0042] Among them, W T Represents the transpose of W.

[0043] In any of the above solutions, preferably, the formula of the final saliency map is:

[0044]

[0045] Where σ is the sigmoid function, f 1×1 It is a convolution operation with a convolution kernel of 1x1.

[0046] The present invention proposes a method for extracting image visual saliency. Through superpixel segmentation and superpixel re-clustering, the problem of too much workload of extracting features from image pixels is avoided on the one hand, and the consistency of object region saliency is improved on the other hand. In terms of network structure, two branches are provided behind two fully connected layers, one branch can obtain the importance of feature channels, and the other branch can obtain the importance of feature space. The mapping of the importance of feature space to physical space is realized, and the extraction effect of image saliency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The flowchart is a preferred embodiment of the method for extracting image visual saliency according to the present invention.

[0048] Figure 2 The figure is a schematic diagram of segmentation results according to an embodiment of a superpixel segmentation method of a method for extracting visual saliency of an image according to the present invention.

[0049] Figure 3 The figure is an example of an embodiment of the original image simplification according to the method for extracting image visual saliency of the present invention.

[0050] Figure 4 The figure is an example of an embodiment of superpixel segmentation of the method for extracting image visual saliency according to the present invention.

[0051] Figure 5 The figure is a schematic diagram of an embodiment of a method for extracting visual saliency of an image based on superpixel clustering according to the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0053] Embodiment 1

[0054] like Figure 1 As shown, step 100 is executed to obtain an image from which visual saliency needs to be extracted.

[0055] Execute step 110 to perform superpixel segmentation on the image to obtain a superpixel set {a n}, 1≤n≤N, N is the number of superpixels;

[0056] For a superpixel set {a n} clustering, connecting superpixels of the same class through a hyperedge to build a hypergraph. The structure of the hypergraph is defined as Among them, v n represents the vertex corresponding to the superpixel, v n ∈V, E represents the superpixel set {a n The set of all hyperedges after clustering. The mth element in this set is denoted as e m ∈E, 1≤m≤M, W represents the saliency matrix of the hyperedge in the hypergraph, and its elements are denoted by ω(e i ), which can be initialized to a 1×M zero matrix.

[0057] Execute step 120, with the incidence matrix K∈R N×M The constructed hypergraph is represented by N, where N represents the number of superpixels and M represents the number of hyperedges. n ,e m ) is expressed as

[0058]

[0059] Execute step 130, input the image and perform feature extraction through a deep learning network, wherein the deep learning network has a dual-branch structure, F represents the original feature map, and its shape is (Q, C, H, W), where Q is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0060] The first branch of the deep learning network performs global average pooling on each channel of the feature map to obtain a feature vector whose feature dimension is the number of channels. The calculation formula is:

[0061]

[0062] Among them, ⊙ is the dot product symbol, g2 and g1 are the weights of the two fully connected layers, σ is the sigmoid function, G() is the global average pooling function, and F1 is the weighted feature map.

[0063] The second branch of the deep learning network performs global average pooling on all feature channels of the second fully connected layer, and the calculation method is:

[0064]

[0065] F1 and F2 are fused to obtain the final feature map F3;

[0066] F3=h(F1,F2)

[0067] Among them, h() is the feature fusion function.

[0068] Execute step 140 to calculate the features of superpixels. The calculation formula is:

[0069]

[0070] Among them, |a n | represents superpixel a n where p(j) is the number of pixels in the superpixel, p(j) indicates that the jth element in the superpixel corresponds to the p(j)th pixel in the image, and F3(p(j)) represents the feature of pixel p(j).

[0071] Execute step 150 to calculate the saliency value of the superpixel. The calculation method is as follows:

[0072]

[0073] Among them, dist(a n ,a j ) represents superpixel a n and a jThe distance

[0074] Get the saliency matrix S of all superpixels,

[0075]

[0076] Execute step 160 to calculate the saliency value of the hyperedge. The calculation method is as follows:

[0077]

[0078] in, is a matrix multiplication operation, count_ones(K) represents the number of 1s in each column of the matrix K. Execute step 170 to update the saliency value of the superpixel associated with the hyperedge, S2 represents the saliency value of all superpixels,

[0079]

[0080] Among them, W T Represents the transpose of W.

[0081] Step 180 is executed to restore the saliency value of each pixel in the entire image according to the position coordinates of the pixels in all superpixels, and the corresponding saliency map is represented as X.

[0082] Execute step 190 to map the feature space importance in the image physical space to obtain the final saliency map of the image. The formula is:

[0083]

[0084] Where σ is the sigmoid function, f 1×1 It is a convolution operation with a convolution kernel of 1x1.

[0085] Embodiment 2

[0086] In order to reduce the problem of too much workload in extracting features from image pixels, the usual practice is to perform superpixel segmentation on the image. However, this will cause the same object area in the image to be often divided into multiple different superpixel areas, and the subsequent saliency map generated by this object area will have the problem of uneven saliency. Therefore, the present invention re-clustered the superpixels and associated the superpixels and clustering results through a supergraph. In addition, the saliency values ​​of superpixels of the same class are balanced through the association matrix to improve the consistency of the saliency of the object area. In terms of the network structure, there are two branches behind the two fully connected layers, one branch can obtain the importance of the feature channel and weight the features; the other branch can obtain the importance of the feature space, which can not only weight the features by the importance of the feature space, but also realize the mapping of the importance of the feature space to the importance of the image physical space. Through the key processing described above, the present invention can obtain a better image saliency extraction effect.

[0087] A method for extracting visual saliency of an image comprises the following steps:

[0088] (1) Perform superpixel segmentation on the input image / to obtain a superpixel set {a n}, 1≤n≤N, N is the number of superpixels. n} clustering, connecting superpixels of the same class through a hyperedge to build a hypergraph. The hypergraph structure is defined as Among them, v n represents superpixel a n The corresponding vertex, v n ∈V, E represents the set of all hyperedges, and the mth element in the set is denoted by e m ∈E, 1≤m≤M, W represents the saliency matrix of the hyperedge in the hypergraph, and its elements are denoted by ω(e i ), which can be initialized to a 1×M zero matrix.

[0089] (2) With the incidence matrix K∈R N×M The constructed hypergraph is represented as follows, where N represents the number of superpixels, M represents the number of hyperedges, and the element k(v n ,e m ) is expressed as

[0090]

[0091] (3) The image is input through a deep learning network for feature extraction.

[0092] The deep learning network has a dual-branch structure. F represents the original feature map, whose shape is (Q, C, H, W), where Q is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0093] One branch performs global average pooling on each channel of the feature map to obtain a feature vector with a feature dimension equal to the number of channels. The calculation formula is:

[0094]

[0095] Among them, ⊙ is the dot product symbol, g2 and g1 are the weights of the two fully connected layers, σ is the sigmoid function, G() is the global average pooling function, and F1 is the weighted feature map.

[0096] The other branch performs global average pooling on all feature channels of the second fully connected layer, and the calculation method is:

[0097]

[0098] F1 and F2 are fused to obtain the final feature map F3.

[0099] F3=h(F1,F2)

[0100] h() is the feature fusion function.

[0101] (4) Calculate the features of superpixels. The calculation formula is as follows:

[0102]

[0103] |a n | represents superpixel a n where p(j) is the number of pixels in the superpixel, p(j) indicates that the jth element in the superpixel corresponds to the p(j)th pixel in the image, and F3(p(j)) represents the feature of pixel p(j).

[0104] (5) Calculate the saliency value of the superpixel. n The calculation method of the significant value is as follows:

[0105]

[0106] Among them, dist(a n ,a j ) represents superpixel a n and a j The distance. Get the saliency value matrix S of all superpixels,

[0107]

[0108] (6) Calculate the saliency value of the hyperedge. The saliency value calculation method of the hyperedge is as follows:

[0109]

[0110] It is a matrix multiplication operation, and count_ones(K) means calculating the number of 1s in each column of matrix K.

[0111] (7) Update the saliency value of the superpixel associated with the hyperedge. S2 represents the saliency value of all superpixels,

[0112]

[0113] W T Represents the transpose of W.

[0114] (8) According to the position coordinates of the pixels in all superpixels, the saliency value of each pixel in the entire image is restored, and the corresponding saliency map is represented as X.

[0115] (9) Map the importance of feature space in the physical space of the image to obtain the final saliency map of the image.

[0116]

[0117] σ is the sigmoid function, f 1×1 (X) is the feature convolution operation with a convolution kernel size of 1x1.

[0118] The present invention achieves:

[0119] (1) Through superpixel segmentation and superpixel re-clustering, on the one hand, the problem of excessive workload in extracting features from image pixels is avoided, and on the other hand, the consistency of the saliency of object regions is improved.

[0120] (2) In terms of network structure, there are two branches behind the two fully connected layers, one branch can obtain the importance of the feature channel, and the other branch can obtain the importance of the feature space.

[0121] (3) Realize the mapping of the importance of feature space to physical space and improve the extraction effect of image saliency.

[0122] Embodiment 3

[0123] This example demonstrates the generation of superpixels.

[0124] like Figure 2 As shown, the SLIC superpixel segmentation method is adopted, and the segmentation results with different numbers of blocks are used during segmentation. Figure 2 The number of blocks in the upper left image is 20. Figure 2 The number of blocks in the upper right image is 50. Figure 2 The number of blocks in the lower left image is 100. Figure 2 The number of blocks in the lower right image is 300.

[0125] Embodiment 4

[0126] This example demonstrates the clustering of superpixels.

[0127] To illustrate the operation process, we simplify the original image to Figure 3 The simplified original image is segmented into superpixels to obtain 8 superpixels, as shown in the figure. Figure 4 As shown. Clustering the superpixels yields 4 classes. The results are as follows Figure 5 shown.

[0128] Embodiment 5

[0129] This embodiment shows a method for constructing a hypergraph. In a constructed hypergraph, v is a vertex (a vertex is a superpixel), and e is a hyperedge (a hyperedge is a class obtained by clustering superpixels). Hyperedge e1 includes two vertices v1, v3, v5, and v7, hyperedge e2 includes two vertices v2, v4, v6, and v8, and hyperedge e3 includes two vertices v9, v10, v11, v12, v13, v14, v15, and v16. 11 、v 13 、v 15 Four vertices, hyperedge e4 includes v 10 、v 12 、v 14 、v 16 Two vertices. Figure 4 is the incidence matrix. The hypergraph is represented by the incidence matrix. If the vertex v is in the hyperedge e, the element in the matrix is ​​1, otherwise it is 0.

[0130] Embodiment 6

[0131] This example demonstrates a deep learning network structure.

[0132] Input layer: The network receives the input image.

[0133] Convolutional layer 1: The first convolutional layer, followed by an activation function, which can be ReLU.

[0134] Convolutional layer 2: The second convolutional layer, followed by an activation function.

[0135] Convolutional layer 3: The third convolutional layer, continues with the activation function.

[0136] Pooling layer (optional): After the convolutional layer, a pooling layer can be used to reduce the spatial dimension of the feature map.

[0137] Fully connected layer 1: The feature map output by the convolutional layer will be flattened and passed through the first fully connected layer.

[0138] Fully connected layer 2: Then it passes through the second fully connected layer.

[0139] Branch 1: Use global average pooling to aggregate spatial information and learn channel importance weights through one or more fully connected layers. The channel importance weights are normalized using the sigmoid function and multiplied by the corresponding channel features to obtain weighted channel features.

[0140] Branch 2: Starting from the output of fully connected layer 2, it is possible to pass additional layers to learn feature space importance, for example, using the SE module in SENet to calculate.

[0141] Feature fusion: The output features of branch 1 and branch 2 are fused, possibly through stacking, weighted averaging, or more complex fusion strategies.

[0142] Output layer: The fused features pass through the final output layer for classification or prediction of other tasks.

[0143] Embodiment 7

[0144] This example shows the calculation result of the saliency value.

[0145] The saliency value of each superpixel is calculated and the results are shown in Table 1.

[0146] Table 1 Calculation results of the saliency value of each superpixel

[0147]

[0148]

[0149] The saliency value of each hyperedge is calculated and the results are shown in Table 2.

[0150] Table 2 Calculation results of the saliency value of each hyperedge

[0151] Super Edge e1 e2 e3 e4 Significant value 0.28 0.5 0.55 0.18

[0152] According to the saliency value of the hyperedge, the saliency value of the superpixel is updated, as shown in Table 3.

[0153] Table 3 Calculation results of the saliency values ​​of the updated superpixels

[0154] Super Pixel v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 Significant value 0.28 0.5 028 0.5 0.28 0.5 0.28 0.5 0.55 0.18 0.55 0.18 0.55 0.18 0.55 0.18

[0155] In order to better understand the present invention, the above is described in detail in conjunction with the specific embodiments of the present invention, but it is not intended to limit the present invention. Any simple modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention. Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referenced to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for extracting visual saliency of an image, comprising acquiring an image, characterized in that: The following steps are also included: Step 1: Perform superpixel segmentation on the image to obtain a superpixel set {a n }, 1≤n≤N, N is the number of superpixels; Step 2: Take the incidence matrix K∈R N×M Represent the constructed hypergraph, where N represents the number of superpixels and M represents the number of hyperedges; Step 3: Input the image into the deep learning network for feature extraction; Step 4: Calculate the features of superpixels; Step 5: Calculate the saliency value of the superpixel; Step 6: Calculate the saliency value of the hyperedge; Step 7: Update the saliency value of the superpixel associated with the hyperedge; Step 8: According to the position coordinates of the pixels in all superpixels, the saliency value of each pixel in the entire image is restored, and the corresponding saliency map is represented as X; Step 9: Map the importance of feature space in the physical space of the image to obtain the final saliency map of the image.

2. The method for extracting visual saliency of an image according to claim 1, wherein: The step 1 also includes the following steps: n }Clustering is performed, and superpixels of the same class are connected through a hyperedge to build a hypergraph.

3. The method for extracting image visual saliency according to claim 2, characterized in that: The structure of the hypergraph is defined as Among them, v n represents superpixel a n The corresponding vertex, v n ∈V, E represents the superpixel set {a n The set of all hyperedges after clustering. The mth element in this set is denoted as e m ∈E, 1≤m≤M, W represents the saliency matrix of the hyperedge in the hypergraph, and its elements are denoted by ω(e i ), which can be initialized to a 1×M zero matrix.

4. The method for extracting visual saliency of an image according to claim 3, wherein: The element k(v n , e m ) is expressed as 5. The method for extracting image visual saliency according to claim 4, characterized in that: The deep learning network has a dual-branch structure, F represents the original feature map, and its shape is (Q, C, H, W), where Q is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

6. The method for extracting visual saliency of an image according to claim 5, characterized in that: The first branch of the deep learning network performs global average pooling on each channel of the feature map to obtain a feature vector whose feature dimension is the number of channels. The calculation formula is: Among them, ⊙ is the dot product symbol, g2 and g1 are the weights of the two fully connected layers, σ is the sigmoid function, G() is the global average pooling function, and F1 is the weighted feature map.

7. The method for extracting image visual saliency according to claim 6, characterized in that: The second branch of the deep learning network performs global average pooling on all feature channels of the second fully connected layer, and the calculation method is: F1 and F2 are fused to obtain the final feature map F3; F3=h(F1,F2) Among them, h() is the feature fusion function.

8. The method for extracting image visual saliency according to claim 7, characterized in that: The superpixel a n Features The calculation formula is Among them, |a n | represents superpixel a n where p(j) is the number of pixels in the superpixel, p(j) indicates that the jth element in the superpixel corresponds to the p(j)th pixel in the image, and F3(p(j)) represents the feature of pixel p(j).

9. The method for extracting image visual saliency according to claim 8, characterized in that: Super Pixel a n The calculation method of the significant value is as follows: Among them, dist(a n , a j ) represents superpixel a n and a j The distance Get the saliency matrix S of all superpixels, 10. The method for extracting image visual saliency according to claim 9, characterized in that: The calculation method of the saliency value of the hyperedge is as follows: in, It is a matrix multiplication operation. counr_ones(K) means calculating the number of 1s in each column of the matrix K, that is, how many superpixels are associated with each hyperedge.

Images