Seagrass bed area acquisition method, system and device based on multi-scale graph convolution

Abstract

The invention relates to the field of image processing technology, and in particular to a method, system and device for acquiring seagrass bed area based on multi-scale graph convolution. In order to solve the problem of low accuracy and stability of the current results of acquiring the seagrass bed area, the invention firstly uses pixel points whose neighborhood channel difference is less than a neighborhood channel difference threshold as seed pixels for superpixel growth processing to obtain a superpixel seagrass bed map; then, after residual processing and segmentation of the superpixel seagrass bed map, a multi-scale adjacency matrix is ​​obtained, and the multi-scale adjacency matrix is ​​respectively subjected to graph convolution processing with a superpixel seagrass bed segmentation map and then fused to obtain a seagrass bed fusion feature map; finally, the seagrass bed fusion feature map is subjected to classification processing to obtain the number of seagrass pixels, and then the number of seagrass pixels is combined with the seagrass bed image area to calculate the seagrass bed area; the acquisition method is applied in the field of seagrass bed area calculation, which not only saves time and effort, but also improves the accuracy and stability of seagrass bed area acquisition.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, system and device for acquiring seagrass bed area based on multi-scale graph convolution. Background Art

[0002] The identification and distribution of seagrass beds can provide important information for seagrass bed protection, carbon estimation and coastal ecological protection. In recent years, field surveys have been often used to obtain the distribution of seagrass beds. However, field surveys are time-consuming and labor-intensive, limited to a small scale, and unable to predict the existence of seagrass beds in advance, making it difficult to accurately calculate the area of ​​seagrass beds.

[0003] Manual field measurement of seagrass area is time-consuming and costly. Therefore, at present, the area of ​​seagrass beds is extracted by analyzing the remote sensing images of the seagrass bed area in the past. However, sometimes due to factors such as light, the pixel values ​​of the same seagrass bed area in the obtained seagrass bed image may be different, affecting the accuracy of the seagrass bed area identification results; if the seagrass bed area in the image is manually circled, it not only takes a lot of time and effort, but more importantly, it is easily affected by subjective influences, reducing the accuracy and stability of the seagrass bed area results. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system and device for acquiring seagrass bed area based on multi-scale graph convolution.

[0005] The technical solution of the present invention is as follows:

[0006] A method for obtaining seagrass bed area based on multi-scale graph convolution includes the following operations:

[0007] S1. Obtain the mean of the color channel value differences between each pixel and other pixels in the respective neighborhood range in the seagrass bed image to obtain the neighborhood pixel difference mean of each pixel; based on the neighborhood pixel difference mean of each pixel, obtain the neighborhood channel difference degree of each pixel;

[0008] The pixel points whose neighborhood channel difference is less than the neighborhood channel difference threshold are taken as seed pixel points; all seed pixel points form a seed pixel point set;

[0009] Take the seed pixel set, each seed pixel as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area;

[0010] During the superpixel growth process, if the regional channel difference within the superpixel region corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, the superpixel growth of the current seed pixel stops; after all seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed image;

[0011] S2, the super-pixel seagrass bed image is subjected to residual processing to obtain a super-pixel seagrass bed residual image;

[0012] The super-pixel seagrass bed residual image is segmented to obtain a super-pixel seagrass bed segmentation image containing multiple block areas;

[0013] The adjacency matrices corresponding to the block areas directly connected and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are obtained respectively, and a direct adjacency matrix and an indirect adjacency matrix are obtained; the direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix;

[0014] The multi-scale adjacency matrix is ​​processed with the segmentation pixel matrix of the superpixel seagrass bed segmentation map through graph convolution to obtain a multi-scale graph convolution seagrass bed map;

[0015] The multi-scale graph convolution seagrass bed map is fused to obtain the seagrass bed fusion feature map;

[0016] S3, the fusion feature map of the seagrass bed is processed by classification to obtain a classification map of the seagrass bed containing classification labels and classification probabilities;

[0017] The pixel points whose classification label is seaweed and whose corresponding classification probability is greater than the probability threshold are regarded as seaweed pixel points; the number of seaweed pixel points is counted to obtain the number of seaweed pixel points;

[0018] Based on the number of seagrass pixels and the seagrass bed image area, the seagrass bed area is obtained.

[0019] The specific operation of superpixel growth in S1 is as follows: obtain the mean and variance of all pixels in the superpixel area corresponding to the current seed pixel at the previous moment, and obtain the mean value of the currently generated pixel growth at the previous moment and the variance of the current pixel growth at the previous moment; divide the difference between the pixel value of each pixel in the superpixel area corresponding to the current seed pixel at the previous moment and the mean value of the currently generated pixel growth at the previous moment by the variance of the current pixel growth at the previous moment, and then multiply them by their respective pixel values ​​to obtain the pixel balance of each pixel at the current moment; add the pixel balance of each pixel at the current moment to their respective pixel values ​​at the previous moment to obtain the superpixel area corresponding to the current seed pixel at the current moment.

[0020] In S1, if the regional growth channel difference within the superpixel area corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, and at the current moment, the ratio of the regional growth channel difference within the superpixel area corresponding to the current seed pixel to the regional growth channel difference within the superpixel area corresponding to the current seed pixel at the previous moment is greater than the difference ratio threshold, the superpixel growth of the current seed pixel stops.

[0021] The specific residual processing operations in S2 are as follows: the superpixel seagrass bed map is subjected to convolution, batch normalization and nonlinear processing to obtain a superpixel seagrass bed nonlinear map; the superpixel seagrass bed nonlinear map and the superpixel seagrass bed map are fused to obtain a superpixel seagrass bed residual map.

[0022] During the S2 graph convolution process, the adjacency matrix corresponding to the output of the previous layer is multiplied element by element with the initial adjacency matrix to obtain the updated adjacency matrix; the updated adjacency matrix is ​​used as the adjacency matrix of the current layer input.

[0023] The method for obtaining the growth radius in S2 is specifically as follows: obtain the pixel point whose pixel difference with the current seed pixel point is equal to the standard pixel difference to obtain the candidate pixel point; all the candidate pixel points form a candidate pixel point set; in the candidate pixel point set, the candidate pixel point with the shortest distance to the current seed pixel point is the target distance pixel point; the straight-line distance between the target distance pixel point and the current seed pixel point is the growth radius of the current seed pixel point.

[0024] In S3, the operation of obtaining the seagrass bed area is as follows:

[0025] The ratio of the number of seagrass pixels to the total number of pixels in the seagrass bed fusion feature map or the seagrass bed image is obtained to obtain the seagrass bed proportion; the seagrass bed proportion is multiplied by the seagrass bed image area and the corresponding image scale to obtain the seagrass bed area.

[0026] A seagrass bed area acquisition system based on multi-scale graph convolution, comprising:

[0027] The superpixel seagrass bed image generation module is used to obtain the mean of the color channel value difference between each pixel point and other pixels within their respective neighborhoods in the seagrass bed image, and obtain the neighborhood pixel difference mean of each pixel point; based on the neighborhood pixel difference mean of each pixel point, obtain the neighborhood channel difference of each pixel point; take the pixel point whose neighborhood channel difference is less than the neighborhood channel difference threshold as the seed pixel point; all the seed pixels form a seed pixel point set; take the seed pixel point set, each seed pixel point as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area; during the superpixel growth process, if the regional channel difference of the current seed pixel point corresponding to the superpixel area reaches the corresponding growth channel difference threshold, the current seed pixel point stops superpixel growth; after all the seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed image;

[0028] The seagrass bed fusion feature map generation module is used for residual processing of the superpixel seagrass bed map to obtain a superpixel seagrass bed residual map; the superpixel seagrass bed residual map is segmented to obtain a superpixel seagrass bed segmentation map containing multiple block areas; the adjacency matrices corresponding to the block areas directly and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are respectively obtained to obtain a direct adjacency matrix and an indirect adjacency matrix; the direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; the multi-scale adjacency matrix is ​​respectively processed with the segmentation pixel matrix of the superpixel seagrass bed segmentation map by graph convolution to obtain a multi-scale graph convolution seagrass bed map; the multi-scale graph convolution seagrass bed map is fused to obtain a seagrass bed fusion feature map;

[0029] The seagrass bed area generation module is used to classify the fused feature map of the seagrass bed to obtain a seagrass bed classification map containing classification labels and classification probabilities; the pixels whose classification labels are seagrass and whose corresponding classification probabilities are greater than the probability threshold are regarded as seagrass pixels; the number of seagrass pixels is counted to obtain the number of seagrass pixels; and the seagrass bed area is obtained based on the number of seagrass pixels and the area of ​​the seagrass bed image.

[0030] A device for acquiring seagrass bed area based on multi-scale graph convolution comprises a processor and a memory, wherein the processor implements the above-mentioned method for acquiring seagrass bed area based on multi-scale graph convolution when executing a computer program stored in the memory.

[0031] A computer-readable storage medium is used to store a computer program, wherein when the computer program is executed by a processor, the method for obtaining the seagrass bed area based on multi-scale graph convolution is implemented.

[0032] The beneficial effects of the present invention are:

[0033] The present invention provides a method for acquiring the area of ​​a seagrass bed based on multi-scale graph convolution. First, the pixel points whose neighborhood channel difference is less than the neighborhood channel difference threshold are used as seed pixels, and the seed pixels are subjected to superpixel growth processing of expanding the area, so that pixels with similar pixels form a larger area, and a superpixel seagrass bed image that is convenient for image analysis and processing is obtained; then, the superpixel seagrass bed image is subjected to residual processing and segmentation to obtain a multi-scale adjacency matrix that can express different spatial distance information in the seagrass bed image, and the multi-scale adjacency matrix and the superpixel seagrass bed segmentation image are subjected to graph convolution processing and then fused to fully extract the rich local features of the seagrass bed and effectively explore context information at different levels, so as to obtain a richer fusion feature map of the seagrass bed; finally, the fusion feature map of the seagrass bed is subjected to classification processing to obtain the number of seagrass pixels, and the number of seagrass pixels is combined with the area of ​​the seagrass bed image to calculate the area of ​​the seagrass bed; the acquisition method not only saves time and labor, but also improves the accuracy and stability of seagrass bed area acquisition. DETAILED DESCRIPTION

[0034] This embodiment provides a method for obtaining seagrass bed area based on multi-scale graph convolution, including the following operations:

[0035] S1. Obtain the mean of the color channel value difference between each pixel and other pixels in their respective neighborhoods in the seagrass bed image, and obtain the mean neighborhood pixel difference of each pixel; obtain the neighborhood channel difference of each pixel based on the mean neighborhood pixel difference of each pixel; use the pixel whose neighborhood channel difference is less than the neighborhood channel difference threshold as the seed pixel; all the seed pixels form a seed pixel set; use the seed pixel set, each seed pixel as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area; during the superpixel growth process, if the regional channel difference of the current seed pixel corresponding to the superpixel area reaches the corresponding growth channel difference threshold, the current seed pixel stops superpixel growth; after all the seed pixels stop superpixel growth, output the seagrass bed image at the current moment to obtain a superpixel seagrass bed map;

[0036] S2. The superpixel seagrass bed map is subjected to residual processing to obtain a superpixel seagrass bed residual map; the superpixel seagrass bed residual map is subjected to segmentation processing to obtain a superpixel seagrass bed segmentation map containing multiple block areas; the adjacency matrices corresponding to the block areas directly and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are respectively obtained to obtain a direct adjacency matrix and an indirect adjacency matrix; the direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; the multi-scale adjacency matrix is ​​subjected to graph convolution processing with the segmentation pixel matrix of the superpixel seagrass bed segmentation map to obtain a multi-scale graph convolution seagrass bed map; the multi-scale graph convolution seagrass bed map is subjected to fusion processing to obtain a seagrass bed fusion feature map;

[0037] S3. The fusion feature map of the seagrass bed is classified and processed to obtain a seagrass bed classification map containing classification labels and classification probabilities; pixels whose classification labels are seagrass and whose corresponding classification probabilities are greater than the probability threshold are regarded as seagrass pixels; the number of seagrass pixels is counted to obtain the number of seagrass pixels; the area of ​​the seagrass bed is obtained based on the number of seagrass pixels and the area of ​​the seagrass bed image.

[0038] S1. Obtain the mean of the color channel value difference between each pixel and other pixels in the respective neighborhood range in the seagrass bed image to obtain the neighborhood pixel difference mean of each pixel; obtain the neighborhood channel difference of each pixel based on the neighborhood pixel difference mean of each pixel; take the pixel whose neighborhood channel difference is less than the first channel difference threshold as the seed pixel; all the seed pixels form a seed pixel set; take the seed pixel set, each seed pixel as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area; the growth radius of the seed pixel is from the seed pixel to the shortest distance between the pixel points whose pixel difference with the seed pixel is equal to the pixel difference marked; during the superpixel growth process, if the regional channel difference of the current seed pixel corresponding to the superpixel area reaches the corresponding growth channel difference threshold, the current seed pixel stops superpixel growth; after all the seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed map.

[0039] In order to reduce the image noise and redundant information of the seagrass bed image caused by light problems and the like while retaining the structural information of the seagrass bed image, this embodiment takes into account the differences between pixels in the seagrass bed image, and uses the pixels in the seagrass bed image whose neighborhood channel difference is less than the neighborhood channel difference threshold as the seed pixels for superpixel growth; then, based on the corresponding growth radius and growth rate, the seed pixels are subjected to superpixel growth processing to expand the area, so that pixels with similar pixel values ​​form a larger area, which is convenient for subsequent rapid analysis; when the regional channel difference within the superpixel area corresponding to each seed pixel reaches the respective growth similarity threshold, the growth is stopped respectively; when all seed pixels stop superpixel growth, the seagrass bed image after superpixel growth is output to obtain a superpixel seagrass bed image that is convenient for image analysis and processing.

[0040] First, obtain the seagrass bed image of the target area through remote sensing data websites or drone aerial photography. Extract the pixel value of each location point in the seagrass bed image, and calculate the mean of the color channel difference value of each pixel point with other pixels in its own neighborhood, and obtain the mean of the neighborhood pixel difference of each pixel point.

[0041] Then, based on the mean difference of the neighborhood pixels of each pixel, the neighborhood channel difference of each pixel is obtained.

[0042] The operation of obtaining the neighborhood channel difference of each pixel point may be to use the reciprocal of the natural exponential function value of the neighborhood pixel difference mean of each pixel point as the neighborhood channel difference of the corresponding pixel point.

[0043] The calculation formula is:

[0044] ,

[0045] I a Pixel a The neighborhood channel difference, a c Pixel a exist c Color values ​​on color channels, b c Pixel b exist c Color values ​​on color channels, C is the total number of color channels, which is the red channel, or the green channel, and / or the blue channel, B Pixel a The total number of pixels in the neighborhood.

[0046] The operation of obtaining the neighborhood channel difference of each pixel point may also be to use the inverse of the neighborhood pixel difference mean of each pixel point as the neighborhood channel difference of the corresponding pixel point.

[0047] The calculation formula is:

[0048] ,

[0049] I a Pixel a The neighborhood channel difference, a c Pixel a exist c Color values ​​on color channels, b c Pixel b exist c Color values ​​on color channels, C is the total number of color channels, is the red channel, and / or the green channel, and / or the blue channel, B Pixel a The total number of pixels in the neighborhood.

[0050] Next, the pixels whose neighborhood channel difference is less than the neighborhood channel difference threshold are used as seed pixels; all seed pixels form a seed pixel set. The smaller the neighborhood channel difference, the closer the color of the pixels in the neighborhood is, and vice versa, the greater the difference; by selecting pixels with smaller neighborhood channel difference as seed pixels for subsequent superpixel growth, it is easy to generate a large area containing more similar pixels.

[0051] Subsequently, the seed pixel set is used, with each seed pixel as the growth center, and the superpixel growth of the region expansion is performed with the corresponding growth radius and growth speed to form a superpixel region.

[0052] Among them, the growth radius of all seed pixels can be set to the same fixed value according to actual needs. Or in order to make the generation of superpixels more consistent and reasonable, the growth radius of the seed pixel can be the shortest distance from the seed pixel to the pixel whose pixel difference with the seed pixel is equal to the standard pixel difference. Specifically, the pixel whose pixel difference with the current seed pixel is equal to the standard pixel difference is obtained to obtain the pixel to be selected; all the pixel to be selected form a set of pixel to be selected; in the set of pixel to be selected, the pixel to be selected with the shortest distance to the current seed pixel is the target distance pixel; the straight-line distance between the target distance pixel and the current seed pixel is the growth radius of the current seed pixel.

[0053] The operation of superpixel growth can be: obtain the mean and variance of all pixels in the superpixel area corresponding to the current seed pixel at the previous moment, obtain the mean value of the currently generated pixel growth at the previous moment and the variance of the current pixel growth at the previous moment, divide the difference between the pixel value of each pixel in the superpixel area corresponding to the current seed pixel at the previous moment and the mean value of the currently generated pixel growth at the previous moment by the variance of the current pixel growth at the previous moment, and multiply them by their respective pixel values ​​to obtain the pixel balance of each pixel at the current moment; add the pixel balance of each pixel at the current moment to its respective pixel value at the previous moment to obtain the superpixel area corresponding to the current seed pixel at the current moment.

[0054] The calculation formula is:

[0055] ,

[0056] x i(t) for t At this moment, the current seed pixel corresponds to the pixel in the superpixel area i The pixel value of x i(t-1) for t-1 At this moment, the current seed pixel corresponds to the pixel in the superpixel area i The pixel value of Xi(t-1) for t-1 At this moment, the pixel mean of all pixels in the superpixel area corresponding to the current seed pixel is s (t-1) for t-1 At this moment, the pixel variance of all pixels in the superpixel area corresponding to the current seed pixel is λ, and λ is the first parameter.

[0057] The operation of superpixel growth can also be as follows: obtain the mean value of all pixels in the superpixel area corresponding to the current seed pixel at the previous moment, and obtain the mean value of the currently generated pixel growth at the previous moment; multiply the pixel value of each pixel in the superpixel area corresponding to the current seed pixel at the previous moment by the mean value of the currently generated pixel growth at the previous moment and the corresponding growth coefficient to obtain the superpixel area corresponding to the current seed pixel at the current moment.

[0058] Finally, during the superpixel growth process, if the regional channel difference within the superpixel area corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, the superpixel growth of the current seed pixel stops; when all seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed map.

[0059] Different growth channel difference thresholds are set for different seed pixels to ensure that different seed pixels can adaptively generate pixel similarities in different areas according to the local characteristics of the image, realize the adaptive growth of superpixels, and retain the difference information in the initial seagrass bed image on the basis of expanding the area of ​​similar pixel values, thereby improving the accuracy of subsequent analysis.

[0060] Among them, the growth channel difference threshold of the seed pixel point with a relatively high neighborhood channel difference is relatively large, that is, if the neighborhood channel difference of the first seed pixel point is smaller than the neighborhood channel difference of the second seed pixel point, then the growth channel difference threshold of the first seed pixel point is smaller than the growth channel difference threshold of the second seed pixel point.

[0061] When the difference between the neighborhood channels is small, it means that the regional pixel values ​​of the corresponding seed pixels tend to be the same. The threshold can be appropriately increased to stop the growth, forming a relatively large superpixel area. On the contrary, when the difference between the neighborhood channels is large, it means that the pixel difference in the region is large. The threshold is lowered for growth to form a relatively large superpixel area. This can not only make the superpixel area of ​​the seagrass bed area with simple content larger, speeding up the subsequent analysis process, but also make the superpixel area of ​​the seagrass bed area with complex content smaller than that of the seagrass bed area with simple content. During the superpixel growth process, the original structural information in the initial seagrass bed image can be retained, improving the accuracy of subsequent analysis.

[0062] Furthermore, in order to improve the superpixel growth effect, if the regional growth channel difference within the superpixel area corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, and at the current moment, the ratio of the regional growth channel difference within the superpixel area corresponding to the current seed pixel and the regional growth channel difference within the superpixel area corresponding to the current seed pixel at the previous moment is greater than the difference ratio threshold, the superpixel growth of the current seed pixel stops.

[0063] S2. The superpixel seagrass bed map is subjected to residual processing to obtain a superpixel seagrass bed residual map; the superpixel seagrass bed residual map is subjected to segmentation processing to obtain a superpixel seagrass bed segmentation map containing multiple block areas; the adjacency matrices corresponding to the block areas directly and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are obtained respectively to obtain a direct adjacency matrix and an indirect adjacency matrix; the direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; the multi-scale adjacency matrix is ​​subjected to graph convolution processing with the segmentation pixel matrix of the superpixel seagrass bed segmentation map to obtain a multi-scale graph convolution seagrass bed map; the multi-scale graph convolution seagrass bed map is subjected to fusion processing to obtain a seagrass bed fusion feature map.

[0064] In order to improve the information expression ability of superpixel seagrass bed images, the superpixel seagrass bed images are subjected to residual processing to obtain superpixel seagrass bed residual images containing higher-level semantic information, which helps to understand the complex structure and texture of seagrass bed images more comprehensively and deeply; then the superpixel seagrass bed residual images are segmented to obtain a multi-scale adjacency matrix that can express different spatial distance information in the seagrass bed images; then, the multi-scale adjacency matrix is ​​convolved with the segmented pixel matrix of the superpixel seagrass bed segmentation image to fully extract the rich local features of the seagrass bed and effectively explore contextual information at different levels to obtain a multi-scale graph convolution seagrass bed image; finally, the multi-scale graph convolution seagrass bed image is fused to obtain richer seagrass bed feature information, which helps to improve the accuracy of subsequent seagrass bed classification and recognition.

[0065] Firstly, the superpixel seagrass bed map is subjected to residual processing. The residual processing operations can be: the superpixel seagrass bed map is subjected to convolution, batch normalization and nonlinear processing to obtain a superpixel seagrass bed nonlinear map; the superpixel seagrass bed nonlinear map and the superpixel seagrass bed map are fused to obtain a superpixel seagrass bed residual map, which provides high-quality feature input for subsequent graph convolution.

[0066] The residual processing operation can also be: the superpixel seagrass bed map is subjected to convolution, batch normalization and nonlinear processing to obtain a superpixel seagrass bed nonlinear map; the superpixel seagrass bed nonlinear map is subjected to attention processing and pooling processing to obtain a superpixel seagrass bed pooled map; the superpixel seagrass bed pooled map and the superpixel seagrass bed map are fused to obtain a superpixel seagrass bed residual map.

[0067] Then, the superpixel seagrass bed residual map is segmented to obtain a superpixel seagrass bed segmentation map containing multiple uniform block areas; in the superpixel seagrass bed segmentation map, a block area is randomly selected as the target block area, and with the target block area as the center, the direct adjacency matrix formed by other block areas directly connected to the target block area and the indirect adjacency matrix formed by other block areas indirectly connected to the target block area are obtained to obtain a multi-scale adjacency matrix.

[0068] The operation of obtaining the direct adjacency matrix is ​​as follows: obtain the block area that is connected to the target block area only by one edge or one corner to obtain the directly connected block area; obtain the connection value between the directly connected block area and the target block area to obtain the direct connection value; the direct connection values ​​of all directly connected block areas and their positions in the superpixel seagrass bed segmentation map form the direct adjacency matrix of the target block area, which is also the first-order adjacency matrix of the target block area. The position of the direct connection value in the superpixel seagrass bed segmentation map corresponds to the position of the direct connection value in the direct adjacency matrix. In the direct adjacency matrix, the value at the corresponding position of the non-directly connected block area is 0.

[0069] The operation of obtaining the indirect adjacency matrix is ​​as follows: obtain the block area that is connected to the target block area through several block areas to obtain the indirectly connected block area; obtain the connection value between the indirectly connected block area and the target block area to obtain the indirect connection value; the indirect connection values ​​of all indirectly connected block areas and their positions in the superpixel seagrass bed segmentation map form the indirect adjacency matrix of the target block area. The position of the indirect connection value in the superpixel seagrass bed segmentation map corresponds to the position of the indirect connection value in the direct adjacency matrix. In the indirect adjacency matrix, the value at the corresponding position of the directly connected block area is 0.

[0070] For example, all block regions that are indirectly connected to the target block region through a block region are second-order connected block regions; the second-order connection values ​​of all second-order connected block regions and their positions in the superpixel seagrass bed segmentation map form the second-order adjacency matrix of the target block region.

[0071] The operation of obtaining the connection value is: obtain the square of the integer of the difference between the average pixel value of the current block area and the average pixel value of the target block area to obtain the first connection number; multiply the first connection number by the connection parameter to obtain the second connection number; obtain the inverse of the natural exponential function of the second connection number as the connection value between the current block area and the target block area.

[0072] The connection value is calculated as:

[0073] ,

[0074] A u,v The target block areau and block area v The connection value of p u The target block area u The average pixel value, p v The target block area v The average pixel value, β It is a connection parameter, which is variable and is related to the number of block areas through which the target block area passes to achieve the connection.

[0075] Then, the multi-scale adjacency matrix is ​​respectively subjected to graph convolution processing with the segmentation pixel matrix of the superpixel seagrass bed segmentation map to obtain a multi-scale graph convolution seagrass bed map.

[0076] The formula for graph convolution is:

[0077] H l =sigmoid(A l ·H l-1 ·W l )，

[0078] H l For the l The output of the convolutional layer, H l-1 For the l-1 The output of the convolutional layer, H 0 is the segmentation pixel matrix of the superpixel seagrass bed segmentation map, is the pixel matrix formed by the average pixel value of each block area in the superpixel seagrass bed segmentation map, W l For the l The weights of the convolutional layers, A l For the l The adjacency matrix of the convolutional layer input.

[0079] Furthermore, in order to improve the effect of graph convolution processing, during the graph convolution processing, the adjacency matrix corresponding to the output of the previous layer is added to the initial adjacency matrix to obtain an updated adjacency matrix as the adjacency matrix input to the current layer. Taking the direct adjacency matrix as an example, the direct adjacency matrix corresponding to the output of the previous layer is multiplied element by element with the initial direct adjacency matrix to obtain an updated adjacency matrix as the adjacency matrix input to the current layer.

[0080] Finally, the multi-scale graph convolution seagrass bed map is fused by superposition to obtain the seagrass bed fusion feature map.

[0081] S3. The fusion feature map of the seagrass bed is classified and processed to obtain a seagrass bed classification map containing classification labels and classification probabilities; pixels whose classification labels are seagrass and whose corresponding classification probabilities are greater than the probability threshold are regarded as seagrass pixels; the number of seagrass pixels is counted to obtain the number of seagrass pixels; the area of ​​the seagrass bed is obtained based on the number of seagrass pixels and the area of ​​the seagrass bed image.

[0082] First, the seagrass bed fusion feature map is classified. The classification operation can be: obtain the neighborhood pixel value distribution map of the current pixel point in the seagrass bed fusion feature map, and the similarity with the standard feature neighborhood distribution map in the standard seagrass feature distribution database. The type of the standard feature neighborhood distribution map corresponding to the maximum similarity value is used as the classification label of the current pixel point, and the maximum similarity value is used as the classification probability of the current pixel point.

[0083] The classification operation can also be: obtaining a standard seagrass bed data set, training a neural network using the standard seagrass bed data set to obtain a trained neural network; processing the seagrass bed fusion feature map through the trained neural network to obtain a seagrass bed classification map containing classification labels and classification probabilities.

[0084] Then, the pixels whose classification label is seaweed and whose corresponding classification probability is greater than the probability threshold are regarded as seaweed pixels; the number of seaweed pixels is counted to obtain the number of seaweed pixels.

[0085] Finally, the seagrass bed area is obtained based on the number of seagrass pixels and the seagrass bed image area.

[0086] The operation of obtaining the seagrass bed area can be: obtaining the ratio of the number of seagrass pixels to the total number of pixels in the seagrass bed fusion feature map or the seagrass bed image to obtain the seagrass bed proportion; multiplying the seagrass bed proportion by the seagrass bed image area and the corresponding image scale to obtain the seagrass bed area.

[0087] The operation to obtain the seagrass bed area can also be: based on the number of seagrass pixels, a connected domain analysis is performed on the seagrass bed classification map to obtain the area on the seagrass bed map; the ratio of the area on the seagrass bed map to the area of ​​the seagrass bed image is multiplied by the area of ​​the actual area corresponding to the seagrass bed image to obtain the seagrass bed area.

[0088] This embodiment also provides a seagrass bed area acquisition system based on multi-scale graph convolution, including:

[0089] The superpixel seagrass bed image generation module is used to obtain the mean of the color channel value difference between each pixel point and other pixels within their respective neighborhoods in the seagrass bed image, and obtain the neighborhood pixel difference mean of each pixel point; based on the neighborhood pixel difference mean of each pixel point, obtain the neighborhood channel difference of each pixel point; take the pixel point whose neighborhood channel difference is less than the neighborhood channel difference threshold as the seed pixel point; all the seed pixels form a seed pixel point set; take the seed pixel point set, each seed pixel point as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area; during the superpixel growth process, if the regional channel difference of the current seed pixel point corresponding to the superpixel area reaches the corresponding growth channel difference threshold, the current seed pixel point stops superpixel growth; after all the seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed image;

[0090] The seagrass bed fusion feature map generation module is used for residual processing of the superpixel seagrass bed map to obtain a superpixel seagrass bed residual map; the superpixel seagrass bed residual map is segmented to obtain a superpixel seagrass bed segmentation map containing multiple block areas; the adjacency matrices corresponding to the block areas directly and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are respectively obtained to obtain a direct adjacency matrix and an indirect adjacency matrix; the direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; the multi-scale adjacency matrix is ​​respectively processed with the segmentation pixel matrix of the superpixel seagrass bed segmentation map by graph convolution to obtain a multi-scale graph convolution seagrass bed map; the multi-scale graph convolution seagrass bed map is fused to obtain a seagrass bed fusion feature map;

[0091] The seagrass bed area generation module is used to classify the fused feature map of the seagrass bed to obtain a seagrass bed classification map containing classification labels and classification probabilities; the pixels whose classification labels are seagrass and whose corresponding classification probabilities are greater than the probability threshold are regarded as seagrass pixels; the number of seagrass pixels is counted to obtain the number of seagrass pixels; and the seagrass bed area is obtained based on the number of seagrass pixels and the area of ​​the seagrass bed image.

[0092] This embodiment also provides a seagrass bed area acquisition device based on multi-scale graph convolution, including a processor and a memory, wherein the processor implements the above-mentioned seagrass bed area acquisition method based on multi-scale graph convolution when executing a computer program stored in the memory.

[0093] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-mentioned method for obtaining the seagrass bed area based on multi-scale graph convolution.

[0094] The present embodiment provides a method for acquiring the area of ​​a seagrass bed based on multi-scale graph convolution. First, the pixel points whose neighborhood channel difference is less than the neighborhood channel difference threshold are used as seed pixels, and the seed pixels are subjected to superpixel growth processing to expand the area, so that pixels with similar pixels form a larger area, and a superpixel seagrass bed map that is convenient for image analysis and processing is obtained; then, the superpixel seagrass bed map is subjected to residual processing and segmentation to obtain a multi-scale adjacency matrix that can express different spatial distance information in the seagrass bed image, and the multi-scale adjacency matrix and the superpixel seagrass bed segmentation map are subjected to graph convolution processing and then fused to fully extract the rich local features of the seagrass bed and effectively explore context information at different levels, so as to obtain a richer fusion feature map of the seagrass bed; finally, the fusion feature map of the seagrass bed is subjected to classification processing to obtain the number of seagrass pixels, and the number of seagrass pixels is combined with the area of ​​the seagrass bed image to calculate the area of ​​the seagrass bed; the acquisition method not only saves time and effort, but also improves the accuracy and stability of seagrass bed area acquisition.

Claims

1. A method for obtaining seagrass bed area based on multi-scale graph convolution, characterized in that: The following operations are included: S1. Obtain the mean of the color channel value differences between each pixel and other pixels in the respective neighborhood range in the seagrass bed image to obtain the neighborhood pixel difference mean of each pixel; based on the neighborhood pixel difference mean of each pixel, obtain the neighborhood channel difference degree of each pixel; The pixel points whose neighborhood channel difference is less than the neighborhood channel difference threshold are taken as seed pixel points; all seed pixel points form a seed pixel point set; Taking the seed pixel set, each seed pixel is used as a growth center, and superpixel growth is performed with a corresponding growth radius and growth speed to form a superpixel region; During the superpixel growth process, if the regional channel difference within the superpixel region corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, the superpixel growth of the current seed pixel stops; after all seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed image; S2, the super-pixel seagrass bed image is subjected to residual processing to obtain a super-pixel seagrass bed residual image; The superpixel seagrass bed residual image is segmented to obtain a superpixel seagrass bed segmentation image containing a plurality of block areas; Respectively obtain adjacency matrices corresponding to block areas that are directly connected and indirectly connected to the target block area in the superpixel seagrass bed segmentation map to obtain a direct adjacency matrix and an indirect adjacency matrix; The direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; The multi-scale adjacency matrix is ​​respectively processed with the segmentation pixel matrix of the superpixel seagrass bed segmentation map through graph convolution to obtain a multi-scale graph convolution seagrass bed map; The multi-scale graph convolution seagrass bed image is fused to obtain a seagrass bed fusion feature image; S3, the seagrass bed fusion feature map is subjected to classification processing to obtain a seagrass bed classification map containing classification labels and classification probabilities; The pixel points whose classification label is seaweed and whose corresponding classification probability is greater than the probability threshold are regarded as seaweed pixel points; the number of seaweed pixel points is counted to obtain the number of seaweed pixel points; The seagrass bed area is obtained based on the number of seagrass pixels and the seagrass bed image area.

2. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: The operation of superpixel growth in S1 is specifically as follows: Get the mean and variance of all pixels in the superpixel area corresponding to the current seed pixel at the previous moment, and get the mean value of the current pixel growth at the previous moment and the variance of the current pixel growth at the previous moment; The difference between the pixel value of each pixel point in the superpixel area corresponding to the current seed pixel point at the previous moment and the mean value of the pixel growth currently generated at the previous moment is divided by the variance of the current pixel growth at the previous moment, and then multiplied by the respective pixel values ​​to obtain the pixel balance degree of each pixel point at the current moment; The pixel balance of each pixel at the current moment is added to the pixel value of each pixel at the previous moment to obtain the superpixel area corresponding to the current seed pixel at the current moment.

3. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: In S1, if the regional growth channel difference within the superpixel region corresponding to the current seed pixel reaches the corresponding growth channel difference threshold, When the ratio of the difference of the regional growth channel in the superpixel area corresponding to the current seed pixel at the current moment to the difference of the regional growth channel in the superpixel area corresponding to the current seed pixel at the previous moment is greater than the difference ratio threshold, the superpixel growth of the current seed pixel stops.

4. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: The specific operation of residual processing in S2 is: The super-pixel seagrass bed image is subjected to convolution, batch normalization and non-linear processing to obtain a super-pixel seagrass bed non-linear image; The superpixel seagrass bed nonlinear map and the superpixel seagrass bed map are fused to obtain the superpixel seagrass bed residual map.

5. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: During the S2 graph convolution process, the adjacency matrix corresponding to the output of the previous layer is multiplied element by element with the initial adjacency matrix to obtain an updated adjacency matrix; the updated adjacency matrix is ​​used as the adjacency matrix input to the current layer.

6. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: The method for obtaining the growth radius in S2 is specifically as follows: Obtain a pixel point whose pixel difference with the current seed pixel point is equal to the standard pixel difference to obtain a candidate pixel point; all the candidate pixel points form a candidate pixel point set; in the candidate pixel point set, the candidate pixel point with the shortest distance to the current seed pixel point is the target distance pixel point; the straight-line distance between the target distance pixel point and the current seed pixel point is the growth radius of the current seed pixel point.

7. The method for obtaining seagrass bed area based on multi-scale graph convolution according to claim 1, characterized in that: In S3, the operation of obtaining the seagrass bed area is specifically as follows: The number of seagrass pixels is obtained, and the ratio of the number of pixels in the seagrass bed fusion feature map or the seagrass bed image is obtained to obtain the seagrass bed proportion; the seagrass bed proportion is multiplied by the seagrass bed image area and the corresponding image scale to obtain the seagrass bed area.

8. A seagrass bed area acquisition system based on multi-scale graph convolution, characterized in that: include: The superpixel seagrass bed image generation module is used to obtain the mean of the color channel value difference between each pixel point and other pixels within their respective neighborhoods in the seagrass bed image, and obtain the neighborhood pixel difference mean of each pixel point; based on the neighborhood pixel difference mean of each pixel point, obtain the neighborhood channel difference of each pixel point; take the pixel point whose neighborhood channel difference is less than the neighborhood channel difference threshold as the seed pixel point; all the seed pixel points form a seed pixel point set; take the seed pixel point set, each seed pixel point as the growth center, and perform superpixel growth with the corresponding growth radius and growth speed to form a superpixel area; if the regional channel difference within the superpixel area corresponding to the current seed pixel point reaches the corresponding growth channel difference threshold, the current seed pixel point stops superpixel growth; after all the seed pixels stop superpixel growth, the seagrass bed image at the current moment is output to obtain a superpixel seagrass bed image; A seagrass bed fusion feature map generation module is used to perform residual processing on the superpixel seagrass bed map to obtain a superpixel seagrass bed residual map; the superpixel seagrass bed residual map is segmented to obtain a superpixel seagrass bed segmentation map containing multiple block areas; and the adjacency matrices corresponding to the block areas directly connected and indirectly connected to the target block area in the superpixel seagrass bed segmentation map are respectively obtained to obtain a direct adjacency matrix and an indirect adjacency matrix; The direct adjacency matrix and the indirect adjacency matrix form a multi-scale adjacency matrix; The multi-scale adjacency matrix is ​​respectively processed with the segmentation pixel matrix of the superpixel seagrass bed segmentation map through graph convolution to obtain a multi-scale graph convolution seagrass bed map; the multi-scale graph convolution seagrass bed map is fused to obtain a seagrass bed fusion feature map; The seagrass bed area generation module is used to classify the seagrass bed fusion feature map to obtain a seagrass bed classification map containing classification labels and classification probabilities; the pixel points whose classification label is seagrass and whose corresponding classification probability is greater than the probability threshold are regarded as seagrass pixel points; the number of seagrass pixel points is counted to obtain the number of seagrass pixel points; and the seagrass bed area is obtained based on the number of seagrass pixel points and the seagrass bed image area.

9. A seagrass bed area acquisition device based on multi-scale graph convolution, characterized in that: It comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, the method for obtaining the seagrass bed area based on multi-scale graph convolution as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for obtaining the seagrass bed area based on multi-scale graph convolution as described in any one of claims 1 to 7 is implemented.