Building waste identification method and device in ecological protection red line based on remote sensing image

Abstract

The application discloses a kind of ecological protection red line in building waste identification method and device based on remote sensing image, belong to satellite remote sensing technical field.The method includes: obtaining the remote sensing image covering the region to be studied ecological protection red line and pre-processing;According to the remote sensing image after pre-processing, vegetation area and water area are extracted and eliminated;The remote sensing image after eliminating vegetation area and water area is converted into gray scale image, and texture feature and gray scale feature are extracted based on gray scale image;The texture feature and gray scale feature extracted are fused, to obtain fusion feature;The fusion feature is input into the deep learning model based on convolutional neural network trained, the deep layer feature extraction is carried out to fusion feature by deep learning model, and classification identification is carried out according to the deep layer feature extracted, to obtain the category of building waste and the distribution area of each category building waste.The application realizes fully automated, accurate, fast building waste extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite remote sensing, in particular to a method and device for identifying construction waste within an ecological protection red line based on remote sensing images. BACKGROUND

[0002] The ecological protection red line refers to a region within an ecological space that has special important ecological functions and must be strictly protected, usually including important ecological function areas with important water conservation, biodiversity maintenance, soil and water conservation, wind and sand prevention, and coastal ecological stability, as well as ecologically sensitive and fragile areas with problems such as soil erosion, land desertification, and rocky desertification. The ecological protection red line is the bottom line and lifeline for protecting and maintaining ecological security.

[0003] Construction waste is all kinds of waste generated during the construction of new, reconstructed, expanded and demolished buildings, pipelines, transportation facilities and renovated houses, including waste concrete, slag, waste bricks, excess mud, discarded materials and other construction waste. The illegal occupation of the ecological protection red line by construction waste has caused pollution and damage to land, water resources, air and ecosystems, posing a serious environmental risk.

[0004] Traditional methods for identifying construction waste based on remote sensing images mainly include manual visual interpretation, human-computer interaction interpretation, and multispectral index method. Manual visual interpretation and human-computer interaction interpretation rely on expert experience and interpretation tools, and are inefficient and subjective. The adaptability of the multispectral index method under different environmental and lighting conditions is limited, and its ability to identify non-biological features such as construction waste needs to be improved.

[0005] The application of deep learning technology is a major breakthrough in recent years. By training deep learning models such as convolutional neural networks (CNN), construction waste can be automatically identified and classified, significantly improving the efficiency and accuracy of identification. However, deep learning models usually require a large amount of labeled data for training, which is difficult to obtain and costly to train, and requires high generalization ability of the model.

[0006] The ecological protection red line is large in area and wide in range, and construction waste is scattered and of varying sizes. How to provide a high-precision and more convenient method for identifying construction waste within the ecological protection red line is a problem that needs to be solved in the field. SUMMARY

[0007] To solve the above technical problems, the present application provides a method and device for identifying construction waste within an ecological protection red line based on remote sensing images, which realizes full automation, accuracy and rapid extraction of construction waste, and provides strong technical support for the management of ecological protection red lines.

[0008] The application provides the technical scheme as follows:

[0009] An ecological protection red line building waste identification method based on remote sensing images, the method comprises the following steps:

[0010] S1: obtaining remote sensing images covering the region to be studied within the ecological protection red line and performing preprocessing;

[0011] S2: extracting and removing the vegetation area and the water area according to the preprocessed remote sensing images;

[0012] S3: converting the remote sensing images after removing the vegetation area and the water area into a gray-scale image, and extracting texture features and gray-scale features based on the gray-scale image;

[0013] S4: fusing the extracted texture features and gray-scale features to obtain fused features;

[0014] S5: inputting the fused features into a trained deep learning model based on a convolutional neural network, performing deep feature extraction on the fused features through the deep learning model, and performing classification and identification according to the extracted deep features to obtain the category of building waste and the distribution area of building waste of each category.

[0015] Further, the S1 comprises the following steps:

[0016] S11: obtaining remote sensing images covering the region to be studied within the ecological protection red line, wherein the remote sensing images are multi-source remote sensing images;

[0017] S12: performing radiation correction, geometric correction, orthographic correction, and image cropping and splicing on the multi-source remote sensing images;

[0018] S13: fusing low-resolution multispectral data and high-resolution panchromatic data contained in the multi-source remote sensing images to obtain high-resolution color remote sensing images.

[0019] Further, the S2 comprises the following steps:

[0020] S21: calculating the normalized vegetation index and the normalized water index of each pixel according to the preprocessed remote sensing images;

[0021] S22: performing geographic registration on the calculated normalized vegetation index and normalized water index data and the preprocessed remote sensing images;

[0022] S23: comparing the normalized vegetation index and the normalized water index of each pixel with the set vegetation index threshold and water index threshold respectively, and classifying the pixels with the normalized vegetation index and the normalized water index greater than the vegetation index threshold and the water index threshold respectively as the vegetation area and the water area;

[0023] S24: respectively remove the vegetation area and the water area from the remote sensing image after pretreatment.

[0024] Further, the S3 comprises:

[0025] S301: according to the weight of each band of the remote sensing image set, the remote sensing image after removing the vegetation area and the water area is converted into a gray image by using a weighted calculation method;

[0026] S302: for each pixel point of the gray image, a neighborhood range of a certain scale is set with the pixel point as the center pixel point;

[0027] S303: for each neighborhood range, the gray value of the center pixel point in the neighborhood range is compared with the gray values of each surrounding pixel point around the center pixel point, if the gray value of the surrounding pixel point is greater than the gray value of the center pixel point, the value of the surrounding pixel point is set to 1, otherwise the value of the surrounding pixel point is set to 0;

[0028] S304: for each neighborhood range of the center pixel point, the values of each surrounding pixel point in the neighborhood range are arranged in a certain order to obtain the binary feature code corresponding to each center pixel point of the gray image;

[0029] S305: for each binary feature code corresponding to each center pixel point, the binary feature code is cyclically right shifted bit by bit, and the minimum value of each binary feature code obtained by cyclically right shifting bit by bit is taken as the rotation invariant binary feature code;

[0030] S306: the binary feature code and the rotation invariant binary feature code are converted into decimal data to obtain the decimal code value and the rotation invariant decimal code value corresponding to each center pixel point;

[0031] S307: the probability of the occurrence of the rotation invariant binary feature code B is calculated by using the total probability formula ;

[0032]

[0033] is the rotation invariant binary feature code B corresponding to the binary feature code with equal probability, is the probability of the occurrence of , is the probability of the occurrence of , is the probability of the occurrence of under the condition that B occurs, and n is the total number of ;

[0034] S308: calculate the product of the rotation-invariant decimal encoding value corresponding to each center pixel point and the probability, to obtain the texture feature value of each center pixel point, and the texture feature values of all center pixel points form the texture feature.

[0035] Further, the remote sensing image after removing the vegetation area and the water area is converted into a gray image by the following formula:

[0036]

[0037] Wherein, G is a gray image, R, G, B are red, green and blue bands of a remote sensing image respectively, W1, W2, W3 are weights of the red, green and blue bands respectively.

[0038] Further, the S3 further comprises:

[0039] S309: set a neighborhood range of different scales, return to S303 and repeat execution to obtain texture features under multiple scales;

[0040] S310: perform gray level histogram statistics on the texture features under multiple scales to obtain gray features under multiple scales.

[0041] Further, the S4 comprises:

[0042] The texture features and the gray features under multiple scales are concatenated and fused to obtain the fusion features.

[0043] Further, the S5 comprises:

[0044] S51: constructing a classification system sample set including construction waste samples;

[0045] S52: constructing a BP neural network model, the BP neural network model comprising an input layer, a hidden layer and an output layer;

[0046] S53: training the BP neural network model using the classification system sample set, using gradient descent method and back propagation algorithm to constantly adjust the weights and biases of the BP neural network model to minimize the output error;

[0047] S54: inputting the fusion features into the BP neural network model, performing deep feature extraction on the fusion features by the BP neural network model, and classifying according to the extracted deep features to obtain the category of construction waste and the distribution area of each category of construction waste.

[0048] Further, the method further comprises:

[0049] ​S6: Screen the distribution area of construction waste according to the set area threshold.

[0050] An ecological protection red line construction waste identification device based on remote sensing images, the device comprises:

[0051] A data acquisition and processing module is configured to acquire remote sensing images covering an area within an ecological protection red line to be studied and perform preprocessing.

[0052] A removal module is configured to extract and remove vegetation areas and water areas from the preprocessed remote sensing images.

[0053] A feature extraction module is configured to convert the remote sensing images after removing the vegetation areas and water areas into grayscale images, and extract texture features and grayscale features based on the grayscale images.

[0054] A feature fusion module is configured to fuse the extracted texture features and grayscale features to obtain fused features.

[0055] A classification module is configured to input the fused features into a trained deep learning model based on a convolutional neural network, extract deep features from the fused features through the deep learning model, and perform classification and identification based on the extracted deep features to obtain the category of construction waste and the distribution area of construction waste of each category.

[0056] Further, the data acquisition and processing module comprises:

[0057] A data acquisition unit is configured to acquire remote sensing images covering an area within an ecological protection red line to be studied, wherein the remote sensing images are multi-source remote sensing images.

[0058] A first processing unit is configured to perform radiation correction, geometric correction, orthographic correction, and image cropping and stitching on the multi-source remote sensing images.

[0059] A second processing unit is configured to fuse low-resolution multispectral data and high-resolution panchromatic data contained in the multi-source remote sensing images to obtain high-resolution color remote sensing images.

[0060] Further, the removal module comprises:

[0061] An index calculation unit is configured to calculate normalized vegetation indices and normalized water indices for each pixel based on the preprocessed remote sensing images.

[0062] A geographic registration unit is configured to geographically register the calculated normalized vegetation indices and normalized water indices with the preprocessed remote sensing images.

[0063] The extraction unit is configured to compare the normalized vegetation index and the normalized water index of each pixel with a set vegetation index threshold and a set water index threshold respectively, and classify the pixels with the normalized vegetation index and the normalized water index greater than the vegetation index threshold and the water index threshold respectively as a vegetation region and a water region respectively.

[0064] The elimination unit is configured to eliminate the vegetation region and the water region from the preprocessed remote sensing image respectively.

[0065] Further, the feature extraction module comprises:

[0066] The gray-scale conversion unit is configured to convert the remote sensing image after eliminating the vegetation region and the water region into a gray-scale image by using a weighted calculation method according to the weights of the respective bands of the remote sensing image.

[0067] The neighborhood setting unit is configured to set a neighborhood range of a set scale for each pixel of the gray-scale image with the pixel as a center pixel.

[0068] The comparison unit is configured to compare the gray-scale value of the center pixel with the gray-scale values of the surrounding pixels around the center pixel in each neighborhood range, and set the value of the surrounding pixel as 1 if the gray-scale value of the surrounding pixel is greater than the gray-scale value of the center pixel, or set the value of the surrounding pixel as 0 otherwise.

[0069] The encoding unit is configured to arrange the values of the surrounding pixels in the neighborhood range of each center pixel in a specific order to obtain a binary feature code corresponding to each center pixel of the gray-scale image.

[0070] The rotation invariant encoding unit is configured to cyclically right-shift the binary feature code corresponding to each center pixel, find the minimum value of each binary feature code obtained by the cyclic right shift, and take the minimum value as a rotation invariant binary feature code.

[0071] The base conversion unit is configured to convert the binary feature code and the rotation invariant binary feature code into decimal data to obtain a decimal encoding value and a rotation invariant decimal encoding value corresponding to each center pixel.

[0072] The probability calculation unit is configured to calculate the probability of occurrence of the rotation invariant binary feature code B by using a total probability formula.

[0073]

[0074] ​A rotation invariant binary feature code B corresponds to a binary feature code with equal probability of occurrence, A rotation invariant binary feature code B corresponds to a binary feature code with equal probability of occurrence, A rotation invariant binary feature code B corresponds to a binary feature code with equal probability of occurrence, A rotation invariant binary feature code B corresponds to a binary feature code with equal probability of occurrence, A rotation invariant binary feature code B corresponds to a binary feature code with equal probability of occurrence, n is The total number of B.

[0075] A texture feature calculation unit is configured to calculate a product of a rotation invariant decimal code value corresponding to each center pixel and the probability A texture feature calculation unit is configured to calculate a product of a rotation invariant decimal code value corresponding to each center pixel and the probability

[0076] Further, the remote sensing image after removing the vegetation area and the water area is converted into a gray image through the following formula;

[0077]

[0078] Wherein, G is a gray image, R, G and B are red, green and blue bands of a remote sensing image respectively, and W1, W2 and W3 are weights of the red, green and blue bands respectively.

[0079] Further, the feature extraction module further comprises:

[0080] A multi-scale unit is configured to set a neighborhood range of different scales, and return the comparison unit to repeatedly execute to obtain texture features under multiple scales;

[0081] A gray feature extraction unit is configured to perform gray histogram statistics on the texture features under multiple scales to obtain gray features under multiple scales.

[0082] Further, the feature fusion module comprises:

[0083] A serial unit is configured to serially fuse the texture features and the gray features under multiple scales to obtain the fusion features.

[0084] Further, the classification module comprises:

[0085] A sample set construction unit is configured to construct a classification system sample set comprising construction waste samples;

[0086] A model construction unit is configured to construct a BP neural network model, and the BP neural network model comprises an input layer, a hidden layer and an output layer.

[0087] a training unit configured to train the BP neural network model using the sample set of the classification system, and to constantly adjust the weights and biases of the BP neural network model by using a gradient descent method and a back propagation algorithm, so as to minimize the output error;

[0088] a classification unit configured to input the fusion features into the BP neural network model, to perform deep feature extraction on the fusion features by using the BP neural network model, and to perform classification and recognition according to the extracted deep features, so as to obtain the categories of construction waste and the distribution areas of construction waste of each category.

[0089] Further, the device further comprises:

[0090] a post-processing module configured to screen the distribution areas of construction waste according to a set area threshold.

[0091] The present application has the following beneficial effects:

[0092] The present application reduces the influence of environmental changes on the recognition result by using the multi-source satellite remote sensing image fusion technology, and effectively alleviates the limitations of single feature recognition by combining multiple feature fusion, so as to realize automatic, accurate and rapid extraction of construction waste, and provide strong technical support for the management of ecological protection red lines. BRIEF DESCRIPTION OF DRAWINGS

[0093] Figure 1 a flowchart of the method for recognizing construction waste in an ecological protection red line based on remote sensing images according to the present application;

[0094] Figure 2 a schematic diagram of the calculation and encoding of texture features according to the present application;

[0095] Figure 3 a schematic diagram of the device for recognizing construction waste in an ecological protection red line based on remote sensing images according to the present application. DETAILED DESCRIPTION

[0096] In order to make the technical problems, technical solutions and advantages of the present application clearer, the following will be described in detail in combination with the drawings and specific embodiments.

[0097] Embodiment 1:

[0098] The embodiment of the present application provides a method for recognizing construction waste in an ecological protection red line based on remote sensing images, as shown in the figure, the method comprises: Figure 1

[0099] S1: obtaining remote sensing images covering a region in an ecological protection red line to be studied and performing preprocessing.

[0100] In one example, the method of preprocessing comprises:​

[0101] S11: Obtain remote sensing images covering the area within the ecological protection red line to be studied, wherein the remote sensing images are multi-source remote sensing images.

[0102] For example, the multi-source remote sensing images can include Sentinel-2 (Sentinel-2 satellite) data and Jilin-1 remote sensing data.

[0103] S12: Perform radiation correction, geometric correction, orthorectification, and image cropping and stitching on the multi-source remote sensing images.

[0104] Radiation correction: unify the radiation values of images obtained by different sensors to the same standard for subsequent accurate analysis.

[0105] Geometric correction: eliminate geometric distortion of the image, so that the image coordinates match the actual geographic coordinates.

[0106] Image cropping and stitching: crop the image according to the range of the study area and stitch multiple images into a complete large-area image.

[0107] S13: Fuse low-resolution multispectral data and high-resolution panchromatic data contained in the multi-source remote sensing images to obtain high-resolution color remote sensing images.

[0108] For example, the multispectral data of Jilin-1 satellite and the panchromatic data can be fused into high-resolution remote sensing images using the NNDiffuse Pan Sharpening tool. The NNDiffuse Pan Sharpening tool uses the Pan Sharping algorithm to fuse low-resolution multispectral images with high-resolution grayscale images to produce high-resolution color images. This tool not only preserves the color information of the multispectral image, but also enhances the contrast and details of the image, improving the spatial resolution of the image and laying the foundation for automatic identification of construction waste.

[0109] S2: Extract and remove vegetation and water areas from the preprocessed remote sensing images.

[0110] Since the ecological protection red line is mostly covered with vegetation, the vegetation coverage is high, and there is a considerable amount of water, masking the grassland and water areas will greatly improve the model efficiency and facilitate visual interpretation of the subsequent model classification results.

[0111] Specifically, S2 includes:

[0112] S21: Calculate the normalized vegetation index (NDVI) and normalized water index (NDWI) of each pixel based on the preprocessed remote sensing images.

[0113]

[0114]

[0115] wherein, , and are the reflectance or brightness values of green band, near-infrared band and red band pixels of the Sentinel-2 image respectively.

[0116] S22: Geographically registering the calculated normalized vegetation index and normalized water body index data with the pre-processed remote sensing image.

[0117] S23: Comparing the normalized vegetation index and normalized water body index of each pixel with the set vegetation index threshold and water body index threshold respectively, and classifying the pixels with normalized vegetation index and normalized water body index greater than the vegetation index threshold and water body index threshold respectively as vegetation area and water body area.

[0118] S24: Removing the vegetation area and water body area from the pre-processed remote sensing image respectively.

[0119] The present application calculates the normalized vegetation index (NDVI) and normalized water body index (NDWI) based on Sentinel-2 image data. After extracting the areas with low NDVI and NDWI values through reclassification to establish a mask, the data is geographically matched to remove the vegetation coverage area and water body distribution area within the ecological protection red line of the original image, thereby refining the study area to improve the working efficiency of model automatic recognition.

[0120] S3: Converting the remote sensing image after removing the vegetation area and water body area into a gray-scale image, and extracting texture features and gray-scale features based on the gray-scale image.

[0121] This step is used to extract classification features, including texture features and gray-scale features, and the present application does not limit the specific implementation manner of extracting texture features and gray-scale features, one of which is as follows:

[0122] S301: According to the weights of each band of the remote sensing image set, the remote sensing image after removing the vegetation area and water body area is converted into a gray-scale image by using a weighted calculation manner.

[0123] When extracting texture features and gray-scale features, the image usually used is a gray-scale image, because the texture feature and gray-scale feature extraction method of the present application is based on gray-scale value calculation, and the gray-scale image can simplify the calculation process while retaining enough information for feature extraction.

[0124] In order to highlight the construction waste area, the present application uses a weighted average method to convert the remote sensing image into a gray-scale image, and the specific formula is as follows:

[0125]

[0126] Where G is the grayscale image, R, G, and B are the red, green, and blue bands of the remote sensing image, respectively, and W1, W2, and W3 are the weights of the red, green, and blue bands, respectively.

[0127] Adjusting the weights can make the grayscale values ​​of the construction waste area more distinct from other areas, thus facilitating extraction and identification. The specific weights need to be adjusted based on actual data and experimental results; commonly used weights are 0.3, 0.6, and 0.1.

[0128] S302: For each pixel of the grayscale image, set a neighborhood range of a specified scale with the pixel as the center pixel.

[0129] The scale size is determined by the size of the defined neighborhood (such as 3×3, 5×5, and 7×7). If the scale is too large, the amount of data is small, but too much detail information is lost; if the scale is too small, the amount of data is too large, but the detail information is preserved in greater detail. This invention can select the scale size according to the obtained spatial resolution and spectral resolution.

[0130] S303: For each neighborhood range, compare the gray value of the center pixel within the neighborhood range with the gray values ​​of all surrounding pixels around the center pixel. If the gray value of the surrounding pixels is greater than the gray value of the center pixel, set the value of the surrounding pixels to 1; otherwise, set the value of the surrounding pixels to 0.

[0131] S304: For the neighborhood range of each center pixel, arrange the values ​​of each surrounding pixel in a specific order to obtain the binary feature code corresponding to each center pixel of the grayscale image.

[0132] A d×d neighborhood contains (d×d-1) surrounding pixels, which, after comparison, can generate a (d×d-1)-bit binary number. Taking a 3×3 scale as an example, comparing the 8 surrounding pixels in a 3×3 neighborhood can generate an 8-bit binary number. Arranging these pixels in a specific order, such as clockwise from the top left corner, yields a binary feature code. The N pixels in the neighborhood set corresponding to a grayscale image can form... A different binary mode.

[0133] by Figure 2 Taking the 3×3 neighborhood shown as an example, the obtained binary feature code is 11010011.

[0134] S305: For each center pixel point corresponding to the binary feature code, the binary feature code is cyclically right shifted bit by bit, and the minimum value of each binary feature code obtained by cyclically right shifting bit by bit is taken as the rotation invariant binary feature code.

[0135] The aforementioned binary feature code has gray scale invariance, but still does not have rotation invariance, and the rotation of the image will obtain different binary feature codes. Therefore, the application obtains a series of binary feature codes in the manner of continuously rotating the neighborhood (cyclically right shifting bit by bit is the rotation neighborhood), takes the minimum value as the rotation invariant binary feature code. The rotation invariant binary feature code has rotation invariance, and will not change no matter how the image is rotated.

[0136] The rotation invariant binary feature code can eliminate some redundant coding information and reduce the data dimension, but may cause the loss of effective image information. Through experiments, when the neighborhood radius R = 1, N = 8 (that is, a 3*3 neighborhood), the feature scale is reduced from 256 dimensions to 36 dimensions, which can eliminate some redundant coding information, while considering the rotation invariance of the image, greatly retaining the image information while reducing the feature scale.

[0137] S306: Convert the binary feature code and the rotation invariant binary feature code into decimal data to obtain the decimal encoding value and the rotation invariant decimal encoding value corresponding to each center pixel point.

[0138] The converted decimal number is the texture feature code of the dxd neighborhood of the center pixel point, and the encoding value can be used to reflect the texture structure information of the center pixel and the neighborhood region.

[0139] S307: Calculate the probability of the occurrence of the rotation invariant binary feature code B by using the total probability formula .

[0140]

[0141] For the rotation invariant binary feature code B corresponding to the binary feature code that appears with equal probability, is the probability of the occurrence of , and is the probability of the occurrence of under the condition that the probability of the occurrence of is n.

[0142] Since the rotation invariant binary feature code corresponds to multiple encoding values of the original binary feature code, the application uses the total probability formula to solve the problem that the influence of the prior probability causes the correlation between the histogram features to be unable to be processed.

[0143] S308: Calculate the product of the rotation-invariant decimal encoding value corresponding to each center pixel point and the probability, to obtain the texture feature value of each center pixel point, and the texture feature values of all center pixel points form a texture feature.

[0144] S309: Set different scale neighborhood ranges, return to S303 for repeated execution, to obtain texture features under multiple scales.

[0145] The texture of an image reflects the structural features of objects in the image, and has characteristics such as scale, anisotropy, and rhythm. The present application characterizes the arrangement and combination mode of image internal texture primitives from multiple scales and their changes in multiple scales, so as to better capture the overall structural features and detailed information of the image and exhibit the unique characteristics of the image under different scale receptive fields. By performing multi-scale texture feature extraction, the obtained multiple scale texture features are concatenated and fused to obtain the comprehensive texture features of the image.

[0146] S310: Perform gray level histogram statistics on the texture features under multiple scales to obtain gray level features under multiple scales.

[0147] The overall gray level distribution statistical feature of an image is the overall distribution of the gray level values obtained from the gray level image. The gray level in the gray level image is 0-255, which describes the significant gray level attribute features of the image. Different types of images have their own characteristics in gray level distribution, and thus the gray level features of each type of image can be obtained by separately performing statistics on the gray level features of each type of image, which plays a certain role in the classification and recognition of images.

[0148] When calculating the gray level distribution features of an image, the probability of occurrence of each gray level in each type of image is calculated to obtain the gray level probability distribution histogram corresponding to each type of image. Common image gray level feature indicators include mean, variance, skewness, kurtosis, energy, and entropy. The original gray level image can obtain texture features under multiple scales through the above transformation, and the gray level probability distribution histogram is performed on the texture features to obtain the texture structure and gray level distribution corresponding to each code in the statistical histogram under each scale, as well as the gray level feature indicators of the structure corresponding to each code under each scale, thereby describing the gray level distribution performance of the local region of the image with certain structural features. Finally, the obtained image local features under multiple scales are concatenated and fused to obtain the multi-scale local gray level features of the image.

[0149] ​The application extracts multi-scale texture features of remote sensing images and obtains corresponding multi-scale gray attribute features, can select appropriate multiple scales according to application requirements and actual effects, and respectively processes the selected multi-scale texture features and multi-scale gray attribute features by using feature fusion technology, and respectively obtains multi-scale comprehensive texture features and multi-scale comprehensive gray features.

[0150] S4: Fuse the extracted texture features and gray features to obtain fused features.

[0151] Specifically, the texture features and gray features under multiple scales can be serially fused to obtain fused features as overall features for image classification.

[0152] The serial fusion is to sequentially connect the multi-scale gray features and multi-scale texture features to form a longer feature vector. This method can fully utilize the feature information under different scales and improve the expression ability of the features.

[0153] S5: Input the fused features into the trained deep learning model based on the convolutional neural network, perform deep feature extraction on the fused features by the deep learning model, and perform classification and recognition according to the extracted deep features to obtain the category of construction waste and the distribution area of each category of construction waste.

[0154] As an implementation mode, S5 includes:

[0155] S51: Construct a classification system sample set including construction waste samples.

[0156] The deep learning model needs to be trained before use, and a sample set needs to be established for each classification system before training. Specifically, the spatial distribution of construction waste and non-construction waste such as housing construction, road, and bare land in the remote sensing image can be found out by visual interpretation of remote sensing images and field investigation, and these categories of image patches can be drawn out, pixel by pixel, to extract the fused features of the image patches, construct a construction waste sample library, and facilitate subsequent research.

[0157] S52: Construct a BP neural network model, which includes an input layer, a hidden layer, and an output layer.

[0158] The number of neurons in the input layer corresponds to the number of extracted features, the number of neurons in the hidden layer is adjusted according to the complexity of the problem and the experimental results, and the number of neurons in the output layer corresponds to the number of categories of the classification task.

[0159] S53: Train the BP neural network model using the classification system sample set, use the gradient descent method and the back propagation algorithm, and constantly adjust the weights and biases of the BP neural network model to minimize the output error.

[0160] Specifically, the fusion features extracted from the sample set of the classification system are taken as inputs in the training process, and the model is trained to classify and identify construction waste and building, road, bare land and other regions. A certain amount of labeled remote sensing image samples can be used to train the model, and the weight parameters of the model can be continuously optimized to improve the recognition accuracy. However, compared with the commonly used deep learning, the model for extracting suspected construction waste established by the present application requires fewer samples and has higher classification accuracy. Therefore, the present model is suitable for research with low prior sample size and high classification accuracy requirements.

[0161] In the training process, momentum method, adaptive learning rate and other strategies can be used to improve the training efficiency and convergence speed.

[0162] S54: input the fusion features into the BP neural network model, perform deep feature extraction on the fusion features through the BP neural network model, and perform classification and identification according to the extracted deep features to obtain the category of construction waste and the distribution area of each category of construction waste.

[0163] The present application can classify remote sensing images into construction waste and non-construction waste categories, and then extract the spatial position attributes of construction waste, such as obtaining the distribution area and calculating the area.

[0164] S6: according to the set area threshold, the distribution area of construction waste is screened.

[0165] According to experience, the area of construction waste has a certain range, and too large or too small does not meet the characteristics of construction waste, so the present application sets an area threshold, discards the areas that do not meet the area threshold requirements, and selects the construction waste map patches that meet the research requirements. The specific setting value of the area threshold can be adjusted according to actual needs, and the present application does not limit it.

[0166] Finally, the image map patches of the construction waste category and their attribute information such as area can be exported.

[0167] The present application is mainly applied in the ecological red line protection zone, and provides a method for identifying construction waste in the ecological protection red line based on remote sensing images according to the characteristics of the research area. The technical advantages of multi-source satellite remote sensing monitoring are used to realize efficient and accurate monitoring and management of construction waste in the ecological protection red line area, to protect and improve the ecological environment, promote the rational recycling of resources, and provide technical support for the supervision of the ecological protection red line.

[0168] In addition, the present application not only has the characteristics of high classification result accuracy, but also can make up for the shortcoming that deep learning requires a large number of samples to ensure classification accuracy.

[0169] In the prior art, the conventional practice is to directly extract features and classify remote sensing images (after certain preprocessing) through a deep learning model, but the data volume of remote sensing images is large and the redundant information is more, which not only has low processing efficiency, but also usually requires a large amount of labeled data for training, has great difficulty in data acquisition, high model training cost, and high requirement for the generalization ability of the model, which makes it unsuitable for the identification of construction waste within the ecological protection red line which has large area, wide range, scattered distribution and different scales.

[0170] Compared with the prior art, the present application does not directly use a deep learning model to directly process remote sensing images, but first extracts texture features and gray level features from remote sensing images, and then uses a deep learning model to process the extracted texture features and depth features. Compared with directly using a deep model, extracting texture and gray level features can reduce redundant information to highlight key features and enhance class boundaries to improve feature separability. Lower data dimension means that the neural network can use a simpler structure to learn features and perform classification, thereby simplifying the model structure to reduce data dimension and complexity. Moreover, texture and gray level features have certain robustness to some common interference factors in remote sensing images, such as changes in light, atmospheric effects, etc. These features can eliminate the influence of these interference factors on classification and identification to some extent, so that the model can maintain stable performance under different environmental conditions. Due to the reduction of data dimension and the more representative features, the neural network can focus more on learning the essential rules in the data during training, rather than overfitting the noise and details in the training samples. This makes the model have stronger generalization ability when facing new remote sensing image samples, and can achieve better recognition effect with less sample amount. In summary, the present application greatly reduces the required sample amount, which not only reduces the workload of data collection and labeling, but also speeds up the model training and deployment, so that the technology can adapt to new data sets faster.

[0171] In addition, the present application integrates texture features and gray level attribute features, comprehensively considers multiple classification features, and uses a neural network model for classification to ensure that the model can accurately and efficiently extract the distribution of construction waste. This multi-feature fusion method improves the accuracy of identification and reduces the missed detection problem that may be caused by a single feature.

[0172] Moreover, the present application integrates multi-source remote sensing data to finely process the research area, effectively reduces the data volume, and thus improves the working efficiency of the model. This method reduces the consumption of computing resources, so that the model can quickly respond and process a large amount of data.

[0173] To sum up, the application reduces the influence of environmental changes on the recognition result through the multi-source satellite remote sensing image fusion technology, effectively alleviates the limitations of single feature recognition by combining multiple feature fusion, realizes the full automation, accuracy and rapid building waste extraction, and provides strong technical support for the management of ecological protection red lines.

[0174] Embodiment 2:

[0175] The embodiment of the application provides a kind of building waste identification device in ecological protection red line based on remote sensing image, as shown in Figure 3 The device comprises:

[0176] Data acquisition and processing module 1 is used to obtain remote sensing image covering the area to be studied in ecological protection red line and pre-process.

[0177] The elimination module 2 is used to extract and eliminate the vegetation area and water area according to the pre-processed remote sensing image.

[0178] The feature extraction module 3 is used to convert the remote sensing image after eliminating the vegetation area and water area into a gray scale image, and extract texture features and gray scale features based on the gray scale image.

[0179] The feature fusion module 4 is used to fuse the extracted texture features and gray scale features to obtain the fusion features.

[0180] The classification module 5 is used to input the fusion features into the trained deep learning model based on convolutional neural network, extract deep features from the fusion features through the deep learning model, and classify and identify according to the extracted deep features to obtain the category of building waste and the distribution area of each category of building waste.

[0181] As an improvement of the embodiment, the aforementioned data acquisition and processing module comprises:

[0182] The data acquisition unit is used to obtain remote sensing image covering the area to be studied in ecological protection red line, wherein the remote sensing image is multi-source remote sensing image.

[0183] The first processing unit is used to perform radiation correction, geometric correction, orthographic correction and image cropping and splicing on the multi-source remote sensing image.

[0184] The second processing unit is used to fuse the low-resolution multispectral data and high-resolution panchromatic data contained in the multi-source remote sensing image to obtain high-resolution color remote sensing image.

[0185] In one example, the aforementioned elimination module comprises:

[0186] An index calculation unit is configured to calculate normalized vegetation index and normalized water index of each pixel according to the preprocessed remote sensing image.

[0187] A geographical registration unit is configured to register the data of the calculated normalized vegetation index and normalized water index with the preprocessed remote sensing image.

[0188] An extraction unit is configured to compare the normalized vegetation index and normalized water index of each pixel with the set vegetation index threshold and water index threshold respectively, and classify the pixels with normalized vegetation index and normalized water index greater than the vegetation index threshold and water index threshold respectively as vegetation area and water area.

[0189] A removal unit is configured to remove the vegetation area and water area from the preprocessed remote sensing image respectively.

[0190] In the present application, the aforementioned feature extraction module specifically comprises:

[0191] A gray-scale unit is configured to convert the remote sensing image after removing the vegetation area and water area into a gray-scale image by using a weighted calculation method according to the weights of each band of the remote sensing image.

[0192] The calculation formula is:

[0193] Wherein, G is the gray-scale image, R, G and B are the red band, green band and blue band of the remote sensing image respectively, and W1, W2 and W3 are the weights of the red band, green band and blue band respectively.

[0194] A neighborhood setting unit is configured to set a neighborhood range of a certain scale for each pixel point of the gray-scale image with the pixel point as the center pixel point.

[0195] A comparison unit is configured to compare the gray-scale value of the center pixel point with the gray-scale values of each surrounding pixel point around the center pixel point in each neighborhood range, and if the gray-scale value of the surrounding pixel point is greater than the gray-scale value of the center pixel point, the value of the surrounding pixel point is set to 1, otherwise the value of the surrounding pixel point is set to 0.

[0196] An encoding unit is configured to arrange the values of each surrounding pixel point in the neighborhood range of each center pixel point in a specific order to obtain the binary feature code corresponding to each center pixel point of the gray-scale image.

[0197] A rotation invariant encoding unit is configured to cyclically right shift each binary feature code corresponding to each center pixel point, find the minimum value of each binary feature code obtained by the cyclic right shift, and take the minimum value as the rotation invariant binary feature code.

[0198] a decimal conversion unit, configured to convert the binary feature code and the rotation-invariant binary feature code into decimal data to obtain a decimal code value corresponding to each center pixel and a rotation-invariant decimal code value.

[0199] a probability calculation unit, configured to calculate a probability of occurrence of the rotation-invariant binary feature code B by using a total probability formula .

[0200]

[0201] a binary feature code corresponding to the rotation-invariant binary feature code B, a probability of occurrence of the binary feature code a probability of occurrence of the binary feature code under a condition that the binary feature code occurs n times.

[0202] a texture feature calculation unit, configured to calculate a product of the rotation-invariant decimal code value and the probability to obtain a texture feature value of each center pixel, and the texture feature values of all the center pixels form a texture feature.

[0203] a multi-scale unit, configured to set a neighborhood range of different scales, and return the comparison unit to repeatedly execute to obtain the texture features under multiple scales.

[0204] a gray feature extraction unit, configured to perform gray histogram statistics on the texture features under the multiple scales to obtain gray features under the multiple scales.

[0205] Correspondingly, the feature fusion module described above comprises:

[0206] a concatenation unit, configured to concatenate and fuse the texture features and the gray features under the multiple scales to obtain a fusion feature.

[0207] Illustratively, the classification module described above comprises:

[0208] a sample set construction unit, configured to construct a classification system sample set comprising construction waste samples.

[0209] a model construction unit, configured to construct a BP neural network model, the BP neural network model comprising an input layer, a hidden layer and an output layer.

[0210] a training unit, configured to train the BP neural network model using the classification system sample set, and adopt a gradient descent method and a back propagation algorithm to constantly adjust weights and biases of the BP neural network model to minimize output errors. ​

[0211] The classification unit is configured to input the fusion features into a BP neural network model, extract deep features of the fusion features by the BP neural network model, and perform classification recognition according to the extracted deep features to obtain the category of the construction waste and the distribution area of the construction waste of each category.

[0212] As another improvement of the embodiment of the present application, the device of the present application can further include:

[0213] The post-processing module is configured to screen the distribution area of the construction waste according to the set area threshold.

[0214] The device provided by the embodiment of the present application has the same implementation principle and technical effects as the foregoing method embodiment. For brevity, the part not mentioned in the device embodiment can refer to the corresponding content in the foregoing method embodiment 1. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the foregoing described device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be described here again.

[0215] Finally, it should be noted that: the above-described embodiments are merely specific implementations of the present application, which are used to illustrate the technical solutions of the present application, rather than limit the same. The protection scope of the present application is not limited thereto. Although the foregoing embodiments of the present application have been described in detail, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or replace some technical features with equivalent ones within the technical range disclosed by the present application. Such modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application.

Claims

1. A method for identifying construction waste within ecological protection red lines based on remote sensing imagery, characterized in that, The method includes: S1: Acquire remote sensing images covering the area within the ecological protection red line to be studied and perform preprocessing; S2: Extract and remove vegetation areas and water areas from the preprocessed remote sensing images; S3: Convert the remote sensing image after removing vegetation and water areas into a grayscale image, and extract texture features and grayscale features based on the grayscale image; S4: The extracted texture features and grayscale features are fused to obtain the fused features; S5: Input the fused features into a trained deep learning model based on a convolutional neural network, extract deep features from the fused features through the deep learning model, and classify and identify the extracted deep features to obtain the categories of construction waste and the distribution areas of each category of construction waste. S3 includes: S301: Based on the weights of each band of the remote sensing image as set, a weighted calculation method is used to convert the remote sensing image after removing vegetation areas and water areas into a grayscale image. S302: For each pixel of the grayscale image, a neighborhood range of a set scale is set with the pixel as the center pixel. S303: For each neighborhood range, compare the gray value of the center pixel within the neighborhood range with the gray values ​​of each surrounding pixel around the center pixel. If the gray value of the surrounding pixel is greater than the gray value of the center pixel, set the value of the surrounding pixel to 1; otherwise, set the value of the surrounding pixel to 0. S304: For the neighborhood range of each center pixel, arrange the values ​​of each surrounding pixel within the neighborhood range in a specific order to obtain the binary feature code corresponding to each center pixel of the grayscale image. S305: For the binary feature code corresponding to each center pixel, the binary feature code is cyclically shifted right bit by bit, and the minimum value of each binary feature code obtained by cyclically shifting right bit by bit is found as the rotation-invariant binary feature code. S306: Convert the binary feature code and rotation-invariant binary feature code into decimal data to obtain the decimal code value and rotation-invariant decimal code value corresponding to each center pixel. S307: Calculate the probability P(B) of the occurrence of the rotation-invariant binary feature code B using the law of total probability; A i Let B be the binary feature code with equal probability of occurrence corresponding to the rotation-invariant binary feature code B, and P(A) be the binary feature code that appears with equal probability. i ) is A i The probability of occurrence, P(B|A) i ) is A i Given that B occurs, the probability of B occurring is given, where n is the probability of A occurring. i The total number; S308: Calculate the product of the rotation-invariant decimal code value corresponding to each center pixel and the probability P(B) to obtain the texture feature value of each center pixel. The texture feature values ​​of all center pixels constitute the texture feature.

2. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 1, characterized in that, S1 includes: S11: Acquire remote sensing images covering the area within the ecological protection red line to be studied, wherein the remote sensing images are multi-source remote sensing images; S12: Perform radiometric correction, geometric correction, orthorectification, and image cropping and stitching on the multi-source remote sensing images; S13: The low-resolution multispectral data contained in the multi-source remote sensing image are fused with the high-resolution panchromatic data to obtain a high-resolution color remote sensing image.

3. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 1, characterized in that, S2 includes: S21: Calculate the normalized vegetation index and normalized water index for each pixel based on the preprocessed remote sensing image. S22: Georegister the calculated normalized vegetation index and normalized water index data with the preprocessed remote sensing image. S23: Compare the normalized vegetation index and normalized water index of each pixel with the set vegetation index threshold and water index threshold respectively, and classify the pixels with normalized vegetation index and normalized water index greater than the vegetation index threshold and water index threshold respectively into vegetation area and water area respectively. S24: Remove the vegetation area and water area from the preprocessed remote sensing image respectively.

4. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 1, characterized in that, The remote sensing image after removing vegetation and water areas is converted into a grayscale image using the following formula; G = W1·R + W2·G + W3·B Where G is the grayscale image, R, G, and B are the red, green, and blue bands of the remote sensing image, respectively, and W1, W2, and W3 are the weights of the red, green, and blue bands, respectively.

5. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 1, characterized in that, S3 further includes: S309: Set the neighborhood range at different scales, return to S303 and repeat the execution to obtain texture features at multiple scales; S310: Perform grayscale histogram statistics on the texture features at the multiple scales to obtain grayscale features at the multiple scales.

6. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 5, characterized in that, S4 includes: The texture features and grayscale features at multiple scales are concatenated and fused to obtain the fused feature.

7. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to any one of claims 1-6, characterized in that, S5 includes: S51: Construct a classification system sample set including construction waste samples; S52: Construct a BP neural network model, which includes an input layer, a hidden layer, and an output layer; S53: The BP neural network model is trained using the sample set of the classification system. Gradient descent and backpropagation algorithms are used to continuously adjust the weights and biases of the BP neural network model to minimize the output error. S54: Input the fused features into the BP neural network model, perform deep feature extraction on the fused features through the BP neural network model, and classify and identify the extracted deep features to obtain the category of construction waste and the distribution area of ​​each category of construction waste.

8. The method for identifying construction waste within ecological protection red lines based on remote sensing imagery according to claim 7, characterized in that, The method further includes: S6: Filter the distribution areas of construction waste based on the set area threshold.

9. A device for identifying construction waste within ecological protection red lines based on remote sensing imagery, characterized in that, The device includes: The data acquisition and processing module is used to acquire remote sensing images covering the area within the ecological protection red line to be studied and to perform preprocessing. The removal module is used to extract and remove vegetation areas and water areas from the preprocessed remote sensing images. The feature extraction module is used to convert the remote sensing image after removing vegetation and water areas into a grayscale image, and extract texture features and grayscale features based on the grayscale image; The feature fusion module is used to fuse the extracted texture features and grayscale features to obtain fused features; The classification module is used to input the fused features into a trained deep learning model based on a convolutional neural network, extract deep features from the fused features through the deep learning model, and classify and identify the construction waste based on the extracted deep features to obtain the categories of construction waste and the distribution areas of each category of construction waste. The feature extraction module includes: The grayscale unit is used to convert the remote sensing image after removing vegetation and water areas into a grayscale image by weighted calculation according to the weight of each band of the remote sensing image. The neighborhood setting unit is used to set a neighborhood range of a set scale for each pixel of the grayscale image, with the pixel as the center pixel. The comparison unit is used to compare the gray value of the center pixel in each neighborhood range with the gray values ​​of each surrounding pixel around the center pixel. If the gray value of the surrounding pixel is greater than the gray value of the center pixel, the value of the surrounding pixel is set to 1; otherwise, the value of the surrounding pixel is set to 0. The encoding unit is used to arrange the values ​​of each surrounding pixel in the neighborhood of each center pixel in a specific order to obtain the binary feature code corresponding to each center pixel of the grayscale image. The rotation-invariant coding unit is used to encode the binary feature corresponding to each center pixel. The binary feature code is cyclically shifted right bit by bit, and the minimum value of each binary feature code obtained by cyclically shifting right bit by bit is found as the rotation-invariant binary feature code. The binary conversion unit is used to convert the binary feature code and rotation-invariant binary feature code into decimal data to obtain the decimal code value and rotation-invariant decimal code value corresponding to each center pixel. The probability calculation unit is used to calculate the probability P(B) of the occurrence of the rotation-invariant binary feature code B using the law of total probability. A i Let B be the binary feature code with equal probability of occurrence corresponding to the rotation-invariant binary feature code B, and P(A) be the binary feature code that appears with equal probability. i ) is A i The probability of occurrence, P(B|A) i ) is A i Given that B occurs, the probability of B occurring is given, where n is the probability of A occurring. i The total number; The texture feature calculation unit is used to calculate the product of the rotation-invariant decimal code value corresponding to each center pixel and the probability P(B) to obtain the texture feature value of each center pixel. The texture feature values ​​of all center pixels constitute the texture feature.

Images