A water body salient target detection method based on multi-graph learning

Abstract

The application discloses a water body saliency target detection method based on multi-graph learning, relates to the technical field of saliency target detection, and comprises the following steps: step one: data acquisition: water body information is acquired through unmanned aerial vehicle images or remote sensing images; step two: data preprocessing: an adjacency matrix is generated based on SLIC superpixel segmentation of the acquired images, and a complete graph is constructed; step three: multi-feature extraction: (1) color feature map extraction; (2) contrast feature map extraction; step four: saliency detection by a manifold ordering method; step five: combination of multiple saliency maps; step six: saliency optimization: the images are processed through a saliency optimization framework, and a new saliency effect drawing is obtained; the color feature map, the contrast feature map and the saliency detection map obtained by the manifold ordering method are weighted and fused with different weights, a new saliency map is obtained, and finally saliency optimization is performed, so that a new saliency target detection method based on a multi-graph learning model is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of salient object detection, and particularly relates to a water salient object detection method based on multi-graph learning. BACKGROUND

[0002] In many geographic scene applications, we need to extract the information of ground objects from images, such as color, shape and texture features, etc. In this paper, salient object detection technology is used in geographic scene recognition, which can effectively reduce labor costs and improve work efficiency by simulating human eye to extract ground object information with the help of computer.

[0003] After searching, the patent for invention with the Chinese patent number CN112085020A discloses a visual salient object detection method and device. The method maps the RGB color images obtained on different devices from the GRB color space to the LAB color space; in the converted LAB color space, the region segmentation of the input image is realized based on the SLIC method of superpixel image segmentation, and the superpixel image segmentation region is obtained. The visual salient object detection method based on multi-scale local feature contrast proposed by the invention can be widely applied in image segmentation, object recognition, adaptive image compression and other related fields.

[0004] The above-mentioned method is based on the background prior local region similarity measurement mechanism, uses the multi-scale idea, calculates and obtains the visual salient region of the color input image, and realizes the visual salient object detection. However, in the actual use process, only through image recognition and using the multi-scale idea, the detection effect of water body is not ideal, and it is not sensitive to color and contrast, and some regions may not be detected, therefore, a water salient object detection method based on multi-graph learning is proposed. SUMMARY

[0005] The purpose of the present application is to provide a water salient object detection method based on multi-graph learning, which solves the problem of lack of protection for the display screen in the prior art.

[0006] In order to achieve the above-mentioned purpose, the embodiments of the present application provide the following technical scheme:

[0007] A water salient object detection method based on multi-graph learning, comprising the following steps:

[0008] Step one: obtaining water information by using unmanned aerial vehicle and remote sensing information;

[0009] Step two: data preprocessing: SLIC superpixel segmentation is performed on the obtained data, an adjacency matrix is generated, and a complete graph is constructed;

[0010] Step three: extract multi-features: (1) color feature map extraction; (2) contrast feature map extraction;

[0011] Step four: manifold ranking method saliency detection;

[0012] Step five: multi-saliency map combination;

[0013] Step six: saliency optimization: process the image through the saliency optimization framework to obtain a new saliency effect map;

[0014] Step seven: saliency effect map verification: perform experiments through ASD, MSRA, SOD, and SED2, and compare optMR and optSO by drawing PR curves and calculating MAE.

[0015] The steps of color feature map extraction are as follows:

[0016] First, convert the RGB color space to the LAB color space, and calculate the average L, a, and b values of all superpixels, as shown in formula (2.1):

[0017]

[0018] Where N is the number of superpixels, p L (i), p A (i), p B (i) represent the L, a, and b values of the i-th superpixel

[0019] Second, calculate the Euclidean distance d col (i) of each superpixel to the Lab mean, as shown in formula (2.2):

[0020]

[0021] Third, normalize the Euclidean distance d col (i) to obtain as shown in formula (2.3):

[0022]

[0023] Thus, the saliency map is obtained from the color features of the image.

[0024] First, calculate the Euclidean distance of each superpixel in the LAB color space, as shown in formula (2.4):

[0025]

[0026] Second, calculate the spatial position Euclidean distance between each superpixel, as shown in formula (2.5):

[0027]

[0028] where p x (i) and p y (i) represent the center row and column value of the ith superpixel respectively, the formula is shown as (2.6):

[0029]

[0030] where pix x (i) and pix y (i) represent the row and column value of each pixel in the superpixel respectively, n is the number of pixels in the superpixel. After calculating the position distance, we calculate the weight of the spatial position, the formula is shown as (2.7):

[0031]

[0032] where σ spa is a constant, here we take 0.4. After calculating the spatial position weight w spa (p, q), we fuse the LAB color space distance d col (p, q) to obtain the contrast weight w con (p, q) between superpixels, the formula is shown as (2.8):

[0033] w con (p, q) = d spa (p, q) · w spa (p, q) (2.8)

[0034] The third step is to add the contrast weight of each superpixel to other superpixels to obtain the saliency value extracted from the contrast feature of each superpixel, the formula is shown as (2.9):

[0035]

[0036] where N is the number of superpixels, and finally the saliency value is normalized, the formula is shown as (2.10):

[0037]

[0038] Thus the saliency map is obtained from the image contrast color feature.

[0039] The invention manifold sorting method includes two stages, the first stage: respectively through the four superpixel boundaries of the image as seed nodes to obtain four different saliency maps, and then integrate to obtain a saliency map, the second stage: on the premise of the first stage, taking the foreground region of the first stage as a seed point to perform foreground priori.

[0040] The present application combines the weighted fusion method;

[0041]

[0042] wherein σ col , σ con and σ MR are constants, according to the fused saliency detection effect map and multiple test data, we take σ col as 0.2, σ con as 0.3, and σ MR as 0.5.

[0043] The present application has the following advantages:

[0044] Compared with the prior art, the present application has the following beneficial effects:

[0045] A new saliency object detection method based on a multi-map learning model is realized by obtaining a color feature map, a contrast feature map and a manifold ranking method detection map, respectively, weighted fusion of different weights to obtain a new saliency map, and finally saliency optimization.

[0046] The present application is compared with two famous saliency object detection methods, i.e., a manifold ranking method and a saliency optimization method, through PR curves and mean absolute errors by using ASD, MSRA, SOD and SED2, famous data sets, to evaluate the detection effect, solve the problems of color and contrast insensitivity, and possible partial area detection failure. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0048] The structures, proportions, sizes, etc. shown in the present specification are only used to cooperate with the content disclosed in the specification, to be understood and read by those skilled in the art, and do not define the limiting conditions for the implementation of the present application, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0049] Figure 1 A flow chart of a water saliency object detection method based on multi-map learning is provided.

[0050] Figure 2 Figure 1 is a first comparative diagram of the present application;

[0051] Figure 3 Figure 2 is a second comparative diagram of the present application;

[0052] Figure 4 Figure 3 is a third comparative diagram of the present application;

[0053] Figure 5 Figure 4 is a fourth comparative diagram of the present application. DETAILED DESCRIPTION

[0054] The present application will be described in greater detail by way of specific embodiments, which should not be construed as limiting the present application to those specific embodiments. Without intending to limit the scope of the application, the following examples are provided for the purpose of illustration. The embodiments described herein are exemplary and are not intended to be limiting, except in the specific embodiments set forth herein. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of the present application.

[0055] The terms such as "upper", "lower", "left", "right", "middle" and the like used in the present specification are merely for the convenience of description and are not intended to limit the scope of the present application. Changes or adjustments of the relative relationship without substantial changes in the technical content are also considered as the scope of the present application.

[0056] Embodiment 1

[0057] As shown in Figures 1-5 The present application proposes a water saliency target detection method based on multi-graph learning, which includes the following steps:

[0058] 1. Obtain water information by unmanned aerial vehicle and remote sensing information, search remote sensing image data by computer, and download the searched water remote sensing information data;

[0059] Meanwhile, the unmanned aerial vehicle takes aerial photographs, splices the photographed images, and orthorectifies the images, and finally organizes the data set;

[0060] 2. Segment the obtained SLIC superpixels, generate an adjacency matrix, and construct a complete graph;

[0061] 3. Extract multiple features: (1) color feature map extraction:

[0062] Firstly, convert the RGB color space to the LAB color space, and calculate the average L, a, and b values of all superpixels, as shown in formula (2.1):

[0063]

[0064] where N is the number of superpixels, p L (i), p A (i), p B (i) represent the L, a, b value of the i-th superpixel

[0065] Second step, calculate the Euclidean distance d col (i) of each superpixel to the Lab mean value, the formula is shown in (2.2):

[0066]

[0067] Third step, normalize the Euclidean distance d col (i) to get The formula is shown in (2.3):

[0068]

[0069] Thus, the saliency map is obtained from the color feature of the image;

[0070] (2) Contrast feature map extraction;

[0071] First step, calculate the Euclidean distance of each superpixel in the LAB color space, the formula is shown in (2.4):

[0072]

[0073] Second step, calculate the spatial position Euclidean distance between each superpixel, the formula is shown in (2.5):

[0074]

[0075] where p x (i) and p y (i) represent the center row and column value of the i-th superpixel, the formula is shown in (2.6):

[0076]

[0077] where pix x (i) and pix y (i) represent the row and column value of each pixel in the superpixel, n is the number of pixels in the superpixel. After calculating the position distance, we calculate the spatial position weight, the formula is shown in (2.7):

[0078]

[0079] where σ spa is a constant, here we take 0.4. After calculating the spatial position weight wspa After we get the LAB color space distance d col (p, q) between superpixels, we fuse them to get the contrast weight w con (p, q) between superpixels, formula as (2.8) shows:

[0080] w con (p, q) = d spa (p, q)·w spa (p, q) (2.8)

[0081] Third, we add the contrast weight of each superpixel to other superpixels to get the saliency value of each superpixel from the contrast feature, formula as (2.9) shows:

[0082]

[0083] Where N is the number of superpixels, and finally normalize the saliency value, formula as (2.10) shows:

[0084]

[0085] Thus we get the saliency map from the image contrast color feature;

[0086] 4. Manifold ranking method saliency detection:

[0087] Manifold ranking method includes two stages, the first stage: through the four superpixel boundaries of the image as seed nodes to get four different saliency maps, and then integrate to get a saliency map, the second stage: on the premise of the first stage, the foreground region of the first stage as seed point to do foreground prior

[0088] 5. Multi-saliency map combination:

[0089] Multi-saliency map combination adopts weighted fusion method;

[0090]

[0091] Where σ col , σ con and σ MR are constants, according to the saliency detection effect map after fusion and multiple test data, we take σ col as 0.2, σ con as 0.3, σ MR as 0.5;

[0092] 6. Saliency optimization: through the saliency optimization framework to process the image, get the new saliency effect map.

[0093] Example two

[0094] The saliency detection of images by Precision-Recall curve: the detection effects of 1000 images in the ASD data set are compared and PR curves are drawn, as shown in the curve Figure 2 The PR curves are almost above the other two saliency detection methods, and the effects of the optSO method and the optMR method are close.

[0095] The detection effects of 100 images in the SED2 data set are compared and PR curves are drawn, as shown in the curve Figure 3 The best method is the saliency optimization method (optSO), the manifold ranking method (optMR) and the effect of the present patent is close.

[0096] The detection effects of 300 images in the SOD data set are compared and PR curves are drawn, as shown in the curve Figure 4 When the recall rate is greater than 0.88, the (optMG) is slightly lower than the other two, but most of them are above the other two methods, and the optMR method is close to the effect of the optSO method, but the performance is better in the area with a recall rate less than 0.52.

[0097] The detection effects of 10000 images in the MSRA data set are compared and drawn, as shown in the curve Figure 5 Although the optMG in the present application is slightly consistent with the precision of the other two saliency detection methods optMR and optSO when the recall rate is greater than 0.95, but overall, the optMG is slightly better than the other two saliency detection methods.

[0098] Although the present application has been described in detail above with general description and specific embodiments, some modifications or improvements can be made on the basis of the present application, which is obvious to those skilled in the art. Therefore, these modifications or improvements made on the basis of not deviating from the spirit of the present application, all belong to the scope of the present application.

Claims

1. A water body saliency target detection method based on multi-graph learning, comprising the following steps: Step one: obtaining water body information through unmanned aerial vehicle and remote sensing information; Step two: data preprocessing: SLIC superpixel segmentation is performed on the obtained data, an adjacency matrix is generated, and a complete graph is constructed; Step three: extracting multiple features: (1) color feature map extraction; (2) contrast feature map extraction; Step four: saliency detection by manifold ranking method; Step five: combination of multiple saliency maps; Step six: saliency optimization: the image is processed through a saliency optimization framework to obtain a new saliency effect diagram; Step seven: saliency effect diagram verification: experiments are performed through ASD, MSRA, SOD and SED2, and PR curves are drawn and MAE is calculated to compare optMR and optSO; The color feature map extraction step is as follows: Firstly, the RGB color space is converted into the LAB color space, and the average L, a and b values of all superpixels are calculated, as shown in formula (2.1): （2.1） where N is the number of superpixels, L, a, b values of the i-th superpixel Second, calculate the Euclidean distance of each superpixel to the Lab mean The formula is shown in (2.2): （2.2） Third step, the Euclidean distance is normalized to get , as shown in equation (2.3): （2.3） Thus, the saliency map is obtained from the color feature of the image; The image contrast feature extraction step is as follows: Firstly, the Euclidean distance of each superpixel in the LAB color space is calculated, as shown in formula (2.4): （2.4） Secondly, the spatial position Euclidean distance between each superpixel is calculated, as shown in formula (2.5): （2.5） where and denote the center row and column values of the ith superpixel, respectively, as given by equation (2.6): （2.6） wherein and respectively represent the row and column values of each pixel within the superpixel, n is the number of pixels within the superpixel, and after calculating the position distance, we calculate the weight of the spatial position, as shown in equation (2.7): （2.7） where is a constant, here we take 0.4 to calculate the spatial position weight After that, we calculate the LAB color space distance between superpixels, and then fuse them to get the superpixel contrast weight , as shown in equation (2.8): （2.8） Thirdly, the contrast weight of each superpixel with other superpixels is added to obtain the saliency value extracted from the contrast feature of each superpixel, as shown in formula (2.9): （2.9） Wherein N is the number of superpixels, and finally the saliency value is normalized, as shown in formula (2.10): （2.10） Thus, the saliency map is obtained from the contrast color feature of the image; The manifold ranking method includes two stages, the first stage: the four superpixel boundaries of the image are taken as seed nodes to obtain four different saliency maps, and then a saliency map is obtained by integration; The multiple saliency map combination adopts a weighted fusion method; （2.11） wherein , and are constants, according to the fused saliency detection effect diagram and multiple test data, we take 0.2, 0.3, 0.

5.

2. The method of claim 1, wherein the method is based on multi-graph learning. The second stage: on the premise of the first stage, the foreground region of the first stage is taken as a seed point to perform foreground priori.

Images