A method, system, device and storage medium for identifying land underlying surface type
By defog removal and feature extraction of optical remote sensing maps, and feature-level fusion combined with radar remote sensing maps, using border regression and probability mapping processing, the low accuracy of land underwater surface type recognition and high error rate in the prior art is solved. The problem of high accuracy and high accuracy recognition results are achieved.
Patent Information
- Application Number
- CN202410078146.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-19
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-01-19
AI Technical Summary
The existing underground surface type identification technology has the problem of low classification recognition accuracy and high misjudgment rate for complex scenarios. Especially when the quality of remote sensing data declines, especially due to the decrease in the quality of remote sensing data caused by fog coverage, the classification is further reduced. The accuracy of the identification results.
By obtaining the optical remote sensing map of the area to be tested for defog treatment, its low-level and advanced feature maps are extracted, and multi-scale feature extraction, channel attention processing, spatial attention processing and convolution processing are carried out to generate high-quality optical remote sensing. Feature map; At the same time, the radar remote sensing map is integrated with the optical remote sensing feature map, and the border regression and probability mapping process are used to filter out the optimal identification box based on the intersection ratio and confidence, and obtain the identification results of the lower surface type .
It improves the accuracy and accuracy of the identification of underground surface surface types, reduces the misjudgment rate in complex scenarios, and ensures the accuracy of the identification results.
Smart Images
Figure CN117789036B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method, system, device and storage medium for identifying the type of land underlying surface. Background Art
[0002] Land surface refers to the combination of geology, soil and other natural materials under the surface, forming the foundation of the earth's surface. Land surface identification is an important research direction in the fields of geoscience, remote sensing and geographic information systems. This research focuses on how to use remote sensing data, geographic information technology and machine learning methods to accurately identify and classify land cover, geology, soil and other characteristics under the surface. The identification of land surface types provides important information and data support for environmental monitoring, natural resource management, urban planning, weather forecasting, agricultural decision-making and other fields, helping to make scientific decisions and achieve sustainable social, economic and environmental development.
[0003] Existing land surface type recognition technologies include: classification methods based on traditional machine learning algorithms and semantic segmentation algorithms based on deep learning. However, these two methods are prone to the disadvantages of low classification and recognition accuracy and high misjudgment rate in complex scenes. In addition, high-quality remote sensing data is crucial for land surface type recognition. However, in some areas, the land surface is covered by fog due to weather reasons, resulting in a decrease in the quality of remote sensing data, which will further reduce the accuracy of land surface classification and recognition results. Summary of the invention
[0004] The purpose of the present invention is to provide a method, system, device and storage medium for identifying the type of land underlying surface.
[0005] The technical solution of the present invention is as follows:
[0006] A method for identifying the type of land underlying surface includes the following operations:
[0007] S1, obtaining an optical remote sensing image of the area to be measured, and obtaining an optical remote sensing enhanced image by defogging the optical remote sensing image;
[0008] S2, obtaining a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; the low-level feature map is sequentially subjected to channel attention processing and convolution processing to obtain a second optical feature map; the low-level feature map and the high-level feature map are respectively subjected to spatial attention processing and convolution processing, and then fused to obtain a third optical feature map; the first optical feature map and the second optical feature map are fused to obtain a fourth optical feature map; the fourth optical feature map is subjected to upsampling processing and fused with the third optical feature map to obtain a fifth optical feature map; the fifth optical feature map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing feature map;
[0009] S3, obtaining a radar remote sensing image of the area to be measured, and performing feature-level fusion processing on the radar remote sensing image and the optical remote sensing feature image to obtain a fused remote sensing feature image;
[0010] S4, the fused remote sensing feature map is processed by frame regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; the intersection-over-union ratio of all recognition frames in the same recognition area in the initial classification map is obtained; based on the intersection-over-union ratio, the recognition frames are screened to obtain an optimized classification map; the confidence of each recognition frame in the optimized classification map is obtained, and the recognition frame with the highest confidence in the same recognition area is retained to obtain the optimal classification map;
[0011] S5. Obtain the classification result corresponding to the maximum classification probability in each identification box in the optimal classification map to obtain the underlying surface type recognition result.
[0012] The defogging operation in S1 is specifically as follows: based on the minimum value of the dark channel of the optical remote sensing image, the diffraction rate of each position point in the optical remote sensing image is obtained; based on the diffraction rate and pixel value of each position point, the fog concentration of each position point is obtained; after subtracting the pixel value of each position point from the fog concentration in the optical remote sensing image, the pixel value is divided by the diffraction rate of the corresponding position point to obtain the optical remote sensing defogging image as the optical remote sensing enhanced image.
[0013] The operation of the multi-scale feature extraction processing in S2 is: the high-level feature map is subjected to convolution processing of different scales to obtain different high-level convolution feature maps; the high-level feature map is subjected to pooling processing to obtain a high-level pooling feature map; the high-level pooling feature map and different high-level convolution feature maps are spliced and then convolution processing is performed to obtain the first feature map.
[0014] The operations of the feature-level fusion processing in S3 are specifically as follows: performing wavelet decomposition processing of different frequency bands on the radar remote sensing image and the optical remote sensing image respectively to obtain a multi-band radar remote sensing image and a multi-band optical remote sensing image; based on the obtained wavelet basis function, respectively obtaining the wavelet coefficients of different frequency band images in the multi-band radar remote sensing image and the multi-band optical remote sensing image to form a radar remote sensing image wavelet coefficient set and an optical remote sensing image wavelet coefficient set; each wavelet coefficient in the radar remote sensing image wavelet coefficient set is multiplied by the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed radar remote sensing image; each wavelet coefficient in the optical remote sensing image wavelet coefficient set is multiplied by the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed optical remote sensing image; the wavelet transformed radar remote sensing image and the wavelet transformed optical remote sensing image are superimposed with the frequency band images of the same frequency band to obtain the fused remote sensing feature image.
[0015] Before the wavelet decomposition processing of different frequency bands, the method also includes performing histogram equalization processing on the radar remote sensing image and the optical remote sensing image respectively to obtain a balanced radar remote sensing image and a balanced optical remote sensing image; the balanced radar remote sensing image and the balanced optical remote sensing image are used to perform the wavelet decomposition processing of different frequency bands.
[0016] After the defogging operation in S1, color correction processing is also included; specifically, the color temperature in the optical remote sensing defogging image obtained after the defogging process is obtained, and the color temperature difference between the color temperature and the standard color temperature is calculated; based on the color temperature difference, the channel gain of different color channels of each position point in the optical remote sensing defogging image is obtained to obtain the red gain, green gain and blue gain of each position point; based on the red gain, green gain and blue gain of each position point, the white balance coefficient of each position point is obtained; the pixel value of each position point in the optical remote sensing defogging image is divided by the corresponding white balance coefficient to obtain the updated pixel value of each position point, and the optical remote sensing color correction image is obtained as the optical remote sensing enhanced image.
[0017] After the color correction operation, the optical remote sensing color correction image is also subjected to detail enhancement processing; specifically, the optical remote sensing color correction image is subjected to Gaussian blur processing to obtain an optical remote sensing smooth image; the optical remote sensing color correction image and the optical remote sensing smooth image are subjected to pixel subtraction processing at corresponding positions to obtain an optical remote sensing detail image; the optical remote sensing detail image and the optical remote sensing color correction image are superimposed to obtain an optical remote sensing detail enhancement image as the optical remote sensing enhanced image.
[0018] A land underlying surface type identification system, comprising:
[0019] An optical remote sensing enhanced image generation module is used to obtain an optical remote sensing image of the area to be measured, and the optical remote sensing image is defogged to obtain an optical remote sensing enhanced image;
[0020] The optical remote sensing feature map generation module is used to obtain a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; the low-level feature map is sequentially subjected to channel attention processing and convolution processing to obtain a second optical feature map; the low-level feature map and the high-level feature map are respectively subjected to spatial attention processing and convolution processing, and then fused to obtain a third optical feature map; the first optical feature map and the second optical feature map are fused to obtain a fourth optical feature map; the fourth optical feature map is subjected to upsampling processing and fused with the third optical feature map to obtain a fifth optical feature map; the fifth optical feature map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing feature map;
[0021] A fused remote sensing feature map generation module is used to obtain a radar remote sensing map of the area to be measured, and the radar remote sensing map and the optical remote sensing feature map are subjected to feature-level fusion processing to obtain a fused remote sensing feature map;
[0022] The optimal classification map generation module is used for the fused remote sensing feature map to be processed by frame regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; obtain the intersection-and-union ratio of all recognition frames in the same recognition area in the initial classification map; based on the intersection-and-union ratio, the recognition frames are screened to obtain an optimized classification map; obtain the confidence of each recognition frame in the optimized classification map, retain the recognition frame with the highest confidence in the same recognition area, and obtain the optimal classification map;
[0023] The underlying surface type recognition result generation module is used to obtain the classification result corresponding to the maximum classification probability in each recognition frame in the optimal classification map to obtain the underlying surface type recognition result.
[0024] A device for identifying the type of underlying land surface comprises a processor and a memory, wherein the processor implements the above-mentioned method for identifying the type of underlying land surface when executing a computer program stored in the memory.
[0025] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned method for identifying the type of land underlying surface when executed by a processor.
[0026] The beneficial effects of the present invention are:
[0027] The invention provides a method for identifying the type of underlying land surface. First, an optical remote sensing image that can reflect the pixel features of ground objects is defogged, and then features in channel dimensions and spatial dimensions are extracted to obtain an optical remote sensing feature image with rich features and high resolution. Then, a radar remote sensing image with elevation information and an optical remote sensing feature image with pixel features are subjected to feature-level fusion processing to obtain a fused remote sensing feature image that is easy to classify and identify. Finally, the fused remote sensing feature image is processed by using frame regression processing and probability mapping, and based on the intersection-and-union ratio and confidence, an identification frame with the most accurate classification result is screened out, and the classification result corresponding to the maximum classification probability of the identification frame is used as the underlying surface type recognition result. The recognition method has high accuracy and precision and low misjudgment rate for complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] By reading the detailed description of the preferred embodiment below, the scheme and advantages of the present application will become clear to those skilled in the art. The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.
[0029] In the attached picture:
[0030] Figure 1 In the embodiment, a schematic diagram of the flow of the identification method of this embodiment;
[0031] Figure 2 This is a result diagram of scene 1 recognized by the method of this embodiment and the existing method DeepLabv3+ model in the embodiment;
[0032] Figure 3 This is a result diagram of scene 2 recognized by the method of this embodiment and the existing method DeepLabv3+ model in the embodiment;
[0033] Figure 4 This is a result diagram of scene 3 recognized by the method of this embodiment and the existing method DeepLabv3+ model in the embodiment;
[0034] Figure 5 This is a result diagram of scene 4 recognized by the method of this embodiment and the existing method DeepLabv3+ model in the embodiment. DETAILED DESCRIPTION
[0035] This embodiment provides a method for identifying the type of land underlying surface. Figure 1 , including the following operations:
[0036] S1, obtaining an optical remote sensing image of the area to be measured, and obtaining an optical remote sensing enhanced image by defogging the optical remote sensing image;
[0037] S2, obtaining a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; the low-level feature map is sequentially subjected to channel attention processing and convolution processing to obtain a second optical feature map; the low-level feature map and the high-level feature map are respectively subjected to spatial attention processing and convolution processing, and then fused to obtain a third optical feature map; the first optical feature map and the second optical feature map are fused to obtain a fourth optical feature map; the fourth optical feature map is subjected to upsampling processing and fused with the third optical feature map to obtain a fifth optical feature map; the fifth optical feature map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing feature map;
[0038] S3, obtaining a radar remote sensing image of the area to be measured, and performing feature-level fusion processing on the radar remote sensing image and the optical remote sensing feature image to obtain a fused remote sensing feature image;
[0039] S4, the fused remote sensing feature map is processed by frame regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; the intersection-and-union ratio of all recognition frames in the same recognition area in the initial classification map is obtained; based on the intersection-and-union ratio, the recognition frames are screened to obtain an optimized classification map;
[0040] S5. Obtain the confidence of each recognition box in the optimized classification map, retain the recognition box with the highest confidence in the same recognition area, and obtain the optimal classification map.
[0041] S1. Obtain an optical remote sensing image of the area to be measured, and obtain an optical remote sensing enhanced image by defogging the optical remote sensing image.
[0042] Obtain an optical remote sensing image of the area to be measured that can reflect the pixel characteristics of the ground objects. Affected by weather and other factors, the fog on the optical remote sensing image will block the underlying surface information below it and affect the identification of the underlying surface type. In order to solve the above technical problems, the optical remote sensing image is defogged to enhance the details of the optical remote sensing image.
[0043] The defogging operation is as follows: based on the minimum value of the dark channel of the optical remote sensing image, the diffraction rate of each position point in the optical remote sensing image is obtained; based on the diffraction rate and pixel value of each position point, the fog concentration of each position point is obtained; after subtracting the pixel value of each position point in the optical remote sensing image from the fog concentration, the pixel value is divided by the diffraction rate of the corresponding position point to obtain the optical remote sensing defogging image as the optical remote sensing enhanced image.
[0044] First, the optical remote sensing image is grayed to obtain an optical remote sensing grayscale image; the minimum value of the grayscale value in the optical remote sensing grayscale image is obtained as the minimum value of the dark channel, and the diffraction coefficient of each position point is obtained based on the minimum value of the dark channel. The diffraction coefficient of each position point can be obtained by the following formula: D i =1-w i ·m , D i For location point i The diffraction rate, w i For location point i The weight of m is the minimum value of the dark channel.
[0045] Next, the pixel value corresponding to the position point corresponding to the minimum value of the dark channel in the optical remote sensing grayscale image is used as the atmospheric light value. Based on the atmospheric light value, the diffraction index and pixel value of each position point, the fog concentration corresponding to each position point is obtained. The fog concentration corresponding to each position point can be obtained by the following formula: F i =p 0i / D i +b , F i For location point i The fog concentration, p 0i For location point i The initial pixel value (pixel value in the optical remote sensing grayscale image), D i For location point i The diffraction rate, b is the atmospheric light value.
[0046] Finally, the pixel value of each position point in the optical remote sensing grayscale image is subtracted from the corresponding fog concentration and divided by the diffraction rate of the position point to obtain the optical remote sensing defogging image, which can be achieved by the following formula: p 2i =(p 0i -F i ) / D i , p 2i For location point i The pixel value after dehazing concentration.
[0047] To ensure the effect of dehazing processing, if the structural similarity between the optical remote sensing dehazing image (optical remote sensing enhanced image) and the optical remote sensing image exceeds the preset threshold, it proves that the optical remote sensing dehazing image (optical remote sensing enhanced image) at this time has a better effect and can be used to perform the operation in S2. If the structural similarity between the optical remote sensing dehazing image (optical remote sensing enhanced image) and the optical remote sensing image does not exceed the preset threshold, the optical remote sensing dehazing image (optical remote sensing enhanced image) has a poor effect and the dehazing processing can continue until the structural similarity requirement is met and then the operation in S2 is performed.
[0048] That is, after the defogging operation in S1, the structural similarity between the optical remote sensing defogging image (optical remote sensing enhanced image) and the optical remote sensing image is obtained, and it is determined whether the structural similarity exceeds a first threshold; if it exceeds, S2 is executed; if it does not exceed, the optical remote sensing defogging image (optical remote sensing enhanced image) is defogged.
[0049] The operation for obtaining the structural similarity between the optical remote sensing dehazed image (optical remote sensing enhanced image) and the optical remote sensing image is as follows: the optical remote sensing dehazed image (optical remote sensing enhanced image) is divided into multiple block areas, and the pixel standard deviation of each block area is compared with the pixel standard deviation of the initial block corresponding to each block area in the optical remote sensing image, and the pixel covariance of each block area in the optical remote sensing image is compared with the initial block, so as to obtain the regional structural similarity between each block area and the corresponding initial block; all regional structural similarities are multiplied by the weight values of the corresponding block areas respectively, and then summed to obtain the structural similarity.
[0050] In order to make the optical remote sensing dehazed image (optical remote sensing enhanced image) more consistent with the color under real lighting conditions, the dehazing operation also includes color correction processing, specifically: obtaining the color temperature in the optical remote sensing dehazed image obtained after the dehazing process, and calculating the color temperature difference between the color temperature and the standard color temperature; based on the color temperature difference, obtaining the channel gain of different color channels of each position point in the optical remote sensing dehazed image, and obtaining the red gain, green gain and blue gain of each position point; based on the red gain, green gain and blue gain of each position point, obtaining the white balance coefficient of each position point; dividing the pixel value of each position point in the optical remote sensing dehazed image by the corresponding white balance coefficient to obtain the updated pixel value of each position point, realizing color correction processing, and obtaining the optical remote sensing color correction image as the optical remote sensing enhanced image.
[0051] In order to further highlight the details of the optical remote sensing dehazed image or the optical remote sensing color correction image, after the dehazed image or the color correction image is processed, the optical remote sensing dehazed image or the optical remote sensing color correction image is also subjected to detail enhancement processing; specifically, the optical remote sensing dehazed image or the optical remote sensing color correction image is subjected to Gaussian blur processing to obtain an optical remote sensing smooth image; the optical remote sensing dehazed image or the optical remote sensing color correction image is subjected to pixel subtraction processing at corresponding positions from the optical remote sensing smooth image to obtain an optical remote sensing detail image; the optical remote sensing detail image is superimposed with the optical remote sensing dehazed image or the optical remote sensing color correction image to achieve detail enhancement, and an optical remote sensing detail enhanced image is obtained as the optical remote sensing enhanced image.
[0052] Since the resolution of optical remote sensing images is relatively high, in order to improve the computational efficiency, the optical remote sensing images can be segmented into standard image blocks, and then defogging, color correction, and detail enhancement can be performed. The segmentation operation can be to divide the optical remote sensing images into image blocks of 512 pixels by 512 pixels in units of pixels, and fill the parts less than 512 pixels with pixels whose RGB value is (0,0,0).
[0053] S2. Obtain a low-level feature map and a high-level feature map of the optical remote sensing enhanced image; the high-level feature map is processed by multi-scale feature extraction to obtain a first optical feature map; the low-level feature map is processed by channel attention and convolution in sequence to obtain a second optical feature map; the low-level feature map and the high-level feature map are processed by spatial attention and convolution in sequence, respectively, and then fused to obtain a third optical feature map; the first optical feature map and the second optical feature map are fused to obtain a fourth optical feature map; the fourth optical feature map is up-sampled and fused with the third optical feature map to obtain a fifth optical feature map; the fifth optical feature map is sequentially processed by convolution and up-sampling to obtain an optical remote sensing feature map.
[0054] First, obtain a low-level feature map with spatial pixel information and a high-level feature map with more semantic information of the optical remote sensing enhancement map. The operation of obtaining the low-level feature map and the high-level feature map of the optical remote sensing enhancement map is a prior art, and is not described here in detail to save space. For example, the low-level feature map can be obtained by processing the optical remote sensing enhancement map using a shallow network of a convolutional layer in a deep convolutional neural network (DCNN). The optical remote sensing enhancement map can be processed using a deep convolutional neural network with the output layer removed to obtain the feature representation of the penultimate layer or the middle layer as the high-level features of the optical remote sensing enhancement map, thereby obtaining a high-level feature map.
[0055] Next, the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map with rich high-level features. The multi-scale feature extraction processing operation is as follows: the high-level feature map is subjected to convolution processing of different scales to obtain different high-level convolution feature maps; the high-level feature map is subjected to pooling processing to obtain a high-level pooling feature map; the high-level pooling feature map and different high-level convolution feature maps are spliced and then subjected to convolution processing to obtain a first feature map.
[0056] At the same time, the low-level feature map is subjected to channel attention processing and convolution processing to obtain a second optical feature map focusing on the channel features of the image. The channel attention processing operation is as follows: the low-level feature map is subjected to global maximum pooling processing and global average pooling processing respectively to obtain a first maximum pooling map and a first average pooling map; the first maximum pooling map and the first average pooling map are processed by a multi-layer perceptron to obtain a first channel weight and a second channel weight; based on the first channel weight and the second channel weight, the first maximum pooling map and the first average pooling map are weighted, and the obtained pooling fusion map is multiplied element-by-element with the low-level feature map to obtain a channel attention map, which is used to perform the convolution processing operation.
[0057] At the same time, the low-level feature map and the high-level feature map are processed with spatial attention and convolution respectively, and then connected in the channel dimension to achieve fusion processing, so as to obtain a third optical feature map with rich spatial features. The operation of spatial attention processing is as follows: the low-level feature map is processed with global maximum pooling, global average pooling and convolution in turn to obtain a first spatial convolution map; the first spatial convolution map is multiplied element by element with the low-level feature map to obtain a first spatial attention feature map; the high-level feature map is processed with global maximum pooling, global average pooling and convolution in turn to obtain a second spatial convolution map; the second spatial convolution map is multiplied element by element with the high-level feature map to obtain a second spatial attention feature map; the first spatial attention feature map and the second spatial attention feature map are respectively used to perform the convolution processing operation.
[0058] Then, the first optical feature map rich in high-level features and the second optical feature map focusing on image channel features are connected in the channel dimension to achieve fusion, so as to obtain a fourth optical feature map rich in feature information and easy to identify.
[0059] Subsequently, after upsampling the fourth optical characteristic map to increase the resolution, it is connected with the third optical characteristic map in the channel dimension to achieve fusion processing, so as to obtain a fifth optical characteristic map with rich features and good resolution.
[0060] Finally, the fifth optical feature map is convolved and upsampled to further improve the resolution and feature richness, thereby obtaining an optical remote sensing feature map with strong feature expression and easy identification and classification.
[0061] S3. Obtain a radar remote sensing image of the area to be measured, and perform feature-level fusion processing on the radar remote sensing image and the optical remote sensing feature image to obtain a fused remote sensing feature image.
[0062] Obtain a radar remote sensing image of the area to be measured that can reflect the elevation feature information of the objects, perform feature-level fusion processing on the radar remote sensing image with elevation information and the optical remote sensing feature image with pixel features of the objects, and obtain a fused remote sensing feature image that is easy to classify and identify.
[0063] The operation of feature-level fusion processing can be: respectively perform wavelet decomposition processing of different frequency bands on the radar remote sensing image and the optical remote sensing image to obtain a multi-band radar remote sensing image and a multi-band optical remote sensing image; based on the obtained wavelet basis function, respectively obtain the wavelet coefficients of different frequency band images in the multi-band radar remote sensing image and the multi-band optical remote sensing image to form a radar remote sensing image wavelet coefficient set and an optical remote sensing image wavelet coefficient set; each wavelet coefficient in the radar remote sensing image wavelet coefficient set is respectively multiplied by the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed radar remote sensing image; each wavelet coefficient in the optical remote sensing image wavelet coefficient set is respectively multiplied by the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed optical remote sensing image; the wavelet transformed radar remote sensing image and the wavelet transformed optical remote sensing image are superimposed on the frequency band images of the same frequency band to obtain a fused remote sensing feature map. The operation of wavelet decomposition processing is an existing technology, and the wavelet basis function can be selected or set according to actual needs. In order to save space, it is not described in detail here.
[0064] In order to improve the quality of the fused remote sensing feature map, before the wavelet decomposition processing of different frequency bands, the radar remote sensing image and the optical remote sensing image are respectively subjected to histogram equalization processing to obtain a balanced radar remote sensing image and a balanced optical remote sensing image; the balanced radar remote sensing image and the balanced optical remote sensing image are used to perform the wavelet decomposition processing of different frequency bands.
[0065] The operation of feature-level fusion processing can also be achieved by performing principal component analysis on the radar remote sensing images and the optical remote sensing images.
[0066] The operation of feature-level fusion processing can also be: processing the radar remote sensing image and the optical remote sensing image through a graph convolutional network to obtain the radar remote sensing image topological structure and the optical remote sensing image topological structure, combining the radar remote sensing image topological structure and the optical remote sensing image topological structure at corresponding positions to obtain a fused remote sensing feature map.
[0067] S4. The fused remote sensing feature map is processed by bounding box regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; the intersection-over-union ratio of different recognition frames in the same recognition area in the initial classification map is obtained, and the recognition frames are screened based on the intersection-over-union ratio to obtain an optimized classification map; the confidence of each recognition frame in the optimized classification map is obtained, and the recognition frame with the highest confidence in the same recognition area is retained to obtain the optimal classification map.
[0068] First, the fused remote sensing feature map is processed by frame regression (existing technology, in order to save space, it will not be described here in detail), and a feature map of multiple recognition frames and a true frame (the true frame can be formed by manual annotation) is obtained. Then, the feature map is processed by the softmax function to realize probability mapping processing, and the similarity between each recognition frame and all standard feature maps in the standard feature database is obtained. The classification label of the standard feature map with a similarity greater than the second threshold is used as the classification result corresponding to the recognition frame, and the corresponding similarity is used as the classification probability corresponding to the classification result. After all recognition frames are operated to obtain the classification results and classification probabilities, an initial classification map containing multiple recognition frames in a recognition area and multiple classification results and corresponding classification probabilities can be obtained on each recognition frame.
[0069] Among them, the classification labels of the standard feature maps in the standard feature database can be manually marked. For example, the features in the hardened ground (road surface) whose neighborhood area elevation change gradient is greater than the third threshold and whose neighborhood area elevation difference is greater than the fourth threshold can be identified as hardened roofs; the features in the green ground whose neighborhood area elevation change gradient is greater than the fifth threshold and whose neighborhood area elevation difference is greater than the sixth threshold can also be identified as green roofs. In the water area, the flow direction is determined according to the height difference distribution characteristics, the confluence point of the water flow is calculated, and the water area at the confluence point is identified as a lake.
[0070] Next, in order to obtain a more accurate classification result, the intersection-and-union ratios of all recognition boxes and the true box in the same recognition area are obtained, and the recognition boxes with an intersection-and-union ratio lower than the seventh threshold are deleted to implement the screening process of the recognition boxes, and obtain an optimized classification map with a more accurate classification result.
[0071] Finally, the pixel distribution similarity between each recognition box in the optimized classification map and the standard feature map corresponding to the classification result corresponding to the maximum classification probability of the recognition box is obtained to obtain the confidence; in the same recognition area, only the recognition box with the highest confidence is retained to obtain the optimal classification map with more accurate classification results.
[0072] S5. Obtain the classification result corresponding to the maximum classification probability in each recognition box in the optimal classification map to obtain the underlying surface type recognition result.
[0073] The classification result corresponding to the maximum classification probability in each identification box in the optimal classification map is used as the type recognition result of the object in the identification box, thereby obtaining the type recognition result of each ground object in the underlying surface. If there are multiple identification boxes with the same confidence in the same identification area in the optimal classification map, the classification results corresponding to the maximum classification probability in multiple identification boxes are used as the type recognition result of the ground object in the area.
[0074] In order to verify the effect of the identification method provided by this embodiment (hereinafter referred to as the method of this embodiment), the following experiment was conducted.
[0075] In the experiment, the method of this embodiment and the existing method DeepLabv3+ model were used to identify the underlying surface types of four different complex scenes. The original image of the area to be tested can be found in Figure 2 , Figure 3 , Figure 4 , Figure 5 The leftmost figure shows the recognition results of the existing method DeepLabv3+ model. Figure 2 , Figure 3 , Figure 4 , Figure 5 The recognition results of the method of this embodiment can be found in the middle figure of Figure 2 , Figure 3 , Figure 4 , Figure 5 The rightmost image in .
[0076] By comparing the original image of the area to be tested with the recognition results of the method of this embodiment, it is found that the method of this embodiment can be used for the recognition of the underlying surface type of complex scenes, and has a high accuracy in the recognition of the underlying surface type of complex scenes. In addition, by comparing the regional contours in the marked box of the recognition results of the method of this embodiment with the regional contours of the corresponding positions in the recognition results of the DeepLabv3+ model, it can be found that the recognition accuracy of this embodiment is higher and the recognition results are better.
[0077] This embodiment also provides a system for identifying the type of land underlying surface, including:
[0078] The optical remote sensing enhanced image generation module is used to obtain the optical remote sensing image of the area to be measured. The optical remote sensing image is defogged to obtain the optical remote sensing enhanced image;
[0079] The optical remote sensing feature map generation module is used to obtain a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; the low-level feature map is sequentially subjected to channel attention processing and convolution processing to obtain a second optical feature map; the low-level feature map and the high-level feature map are respectively subjected to spatial attention processing and convolution processing, and then fused to obtain a third optical feature map; the first optical feature map and the second optical feature map are fused to obtain a fourth optical feature map; the fourth optical feature map is subjected to upsampling processing and then fused with the third optical feature map to obtain a fifth optical feature map; the fifth optical feature map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing feature map;
[0080] The fused remote sensing feature map generation module is used to obtain the radar remote sensing map of the area to be measured. The radar remote sensing map and the optical remote sensing feature map are fused at the feature level to obtain the fused remote sensing feature map;
[0081] The optimal classification map generation module is used to fuse the remote sensing feature map through border regression processing and probability mapping processing to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; obtain the intersection-and-union ratio of all recognition frames in the same recognition area in the initial classification map; based on the intersection-and-union ratio, the recognition frames are screened to obtain an optimized classification map; obtain the confidence of each recognition frame in the optimized classification map, retain the recognition frame with the highest confidence in the same recognition area, and obtain the optimal classification map;
[0082] The underlying surface type recognition result generation module is used to obtain the classification result corresponding to the maximum classification probability in each recognition box in the optimal classification map, and obtain the underlying surface type recognition result.
[0083] This embodiment also provides a device for identifying the type of underlying land surface, including a processor and a memory, wherein the processor implements the above-mentioned method for identifying the type of underlying land surface when executing a computer program stored in the memory.
[0084] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned method for identifying the type of underlying land surface when executed by a processor.
[0085] The present embodiment provides a method for identifying the type of underlying land surface. First, after defogging an optical remote sensing image that can reflect the pixel features of ground objects, feature extraction is performed in channel dimensions and spatial dimensions to obtain an optical remote sensing feature map with rich features and high resolution. Then, a radar remote sensing image with elevation information and an optical remote sensing feature map with pixel features are subjected to feature-level fusion processing to obtain a fused remote sensing feature map that is easy to classify and identify. Finally, the fused remote sensing feature map is processed by using frame regression processing and probability mapping, and based on the intersection-over-union ratio and confidence level, an identification frame with the most accurate classification result is screened out, and the classification result corresponding to the maximum classification probability of the identification frame is used as the underlying surface type recognition result. The recognition method has high accuracy and precision and low misjudgment rate for complex scenes.
Claims
1. A method for identifying the type of land underlying surface, characterized in that: The following operations are included: S1, obtaining an optical remote sensing image of the area to be measured, and obtaining an optical remote sensing enhanced image by defogging the optical remote sensing image; specifically, obtaining the diffraction rate of each position point in the optical remote sensing image based on the minimum value of the dark channel of the obtained optical remote sensing image; and obtaining the fog concentration of each position point based on the diffraction rate and pixel value of each position point; After subtracting the pixel value of each position point from the fog concentration in the optical remote sensing image, the result is divided by the diffraction rate of the corresponding position point to obtain an optical remote sensing defogging image as an optical remote sensing enhancement image; S2, obtaining a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; specifically: the high-level feature map is subjected to convolution processing of different scales to obtain different high-level convolution feature maps; the high-level feature map is subjected to pooling processing to obtain a high-level pooling feature map; the high-level pooling feature map and different high-level convolution feature maps are spliced and then subjected to convolution processing to obtain the first optical feature map; The low-level feature map is sequentially processed by channel attention and convolution to obtain a second optical feature map; The low-level feature map and the high-level feature map are respectively subjected to spatial attention processing and convolution processing, and then fused to obtain a third optical feature map; The first optical characteristic graph and the second optical characteristic graph are fused to obtain a fourth optical characteristic graph; The fourth optical characteristic graph is upsampled and then fused with the third optical characteristic graph to obtain a fifth optical characteristic graph; The fifth optical characteristic map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing characteristic map; S3, obtaining a radar remote sensing image of the area to be measured, and performing feature-level fusion processing on the radar remote sensing image and the optical remote sensing feature image to obtain a fused remote sensing feature image; Specifically, the radar remote sensing image and the optical remote sensing image are subjected to wavelet decomposition processing of different frequency bands respectively to obtain a multi-band radar remote sensing image and a multi-band optical remote sensing image; based on the obtained wavelet basis function, the wavelet coefficients of different frequency band images in the multi-band radar remote sensing image and the multi-band optical remote sensing image are obtained respectively to form a radar remote sensing image wavelet coefficient set and an optical remote sensing image wavelet coefficient set; each wavelet coefficient in the radar remote sensing image wavelet coefficient set is multiplied with the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed radar remote sensing image; each wavelet coefficient in the optical remote sensing image wavelet coefficient set is multiplied with the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed optical remote sensing image; the wavelet transformed radar remote sensing image and the wavelet transformed optical remote sensing image are subjected to frequency band image superposition of the same frequency band to obtain the fused remote sensing feature image; S4, the fused remote sensing feature map is processed by frame regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; Obtaining the intersection-over-union ratio of all recognition boxes in the same recognition area in the initial classification map; Based on the intersection-union ratio, the recognition frame is screened to obtain an optimized classification map; Obtaining the confidence of each recognition frame in the optimized classification map, retaining the recognition frame with the highest confidence in the same recognition area, and obtaining the optimal classification map; S5. Obtain the classification result corresponding to the maximum classification probability in each identification box in the optimal classification map to obtain the underlying surface type recognition result.
2. The method for identifying the type of land underlying surface according to claim 1, characterized in that: Before the wavelet decomposition processing of different frequency bands, the process further includes performing histogram equalization processing on the radar remote sensing image and the optical remote sensing image respectively to obtain an equalized radar remote sensing image and an equalized optical remote sensing image; The balanced radar remote sensing image and the balanced optical remote sensing image are used to perform the wavelet decomposition processing operation of the different frequency bands.
3. The method for identifying the type of land underlying surface according to claim 1, characterized in that: After the defogging process in S1, a color correction process is also included; specifically: Acquiring the color temperature of the optical remote sensing defogging image obtained after the defogging process, and calculating the color temperature difference between the color temperature and the standard color temperature; Based on the color temperature difference, the channel gain of different color channels at each position point in the optical remote sensing defogging image is obtained to obtain the red gain, green gain and blue gain of each position point; Based on the red gain, green gain and blue gain of each position point, a white balance coefficient of each position point is obtained; The pixel value of each position point in the optical remote sensing defogging image is divided by the corresponding white balance coefficient to obtain an updated pixel value of each position point, and an optical remote sensing color correction image is obtained as the optical remote sensing enhancement image.
4. The method for identifying the type of land underlying surface according to claim 3, characterized in that: After the color correction process, the optical remote sensing color correction image is further subjected to detail enhancement process; specifically: The optical remote sensing color correction image is subjected to Gaussian blur processing to obtain an optical remote sensing smooth image; the optical remote sensing color correction image and the optical remote sensing smooth image are subjected to pixel subtraction processing at corresponding positions to obtain an optical remote sensing detail image; the optical remote sensing detail image and the optical remote sensing color correction image are superimposed to obtain an optical remote sensing detail enhancement image as the optical remote sensing enhanced image.
5. A system for identifying the type of land underlying surface, characterized in that: include: An optical remote sensing enhanced image generation module is used to obtain an optical remote sensing image of the area to be measured, and the optical remote sensing image is defogged to obtain an optical remote sensing enhanced image; Specifically, based on the minimum value of the dark channel of the optical remote sensing image, the diffraction rate of each position point in the optical remote sensing image is obtained; based on the diffraction rate and pixel value of each position point, the fog concentration of each position point is obtained; After subtracting the pixel value of each position point from the fog concentration in the optical remote sensing image, the result is divided by the diffraction rate of the corresponding position point to obtain an optical remote sensing defogging image as an optical remote sensing enhancement image; The optical remote sensing feature map generation module is used to obtain a low-level feature map and a high-level feature map of the optical remote sensing enhancement map; the high-level feature map is subjected to multi-scale feature extraction processing to obtain a first optical feature map; specifically: the high-level feature map is subjected to convolution processing of different scales to obtain different high-level convolution feature maps; the high-level feature map is subjected to pooling processing to obtain a high-level pooling feature map; the high-level pooling feature map is spliced with different high-level convolution feature maps and then subjected to convolution processing to obtain the first optical feature map; the low-level feature map is subjected to channel attention processing and convolution processing in turn to obtain a second optical feature map; the low-level feature map and the high-level feature map are subjected to spatial attention processing and convolution processing in turn, and then subjected to fusion processing to obtain a third optical feature map; the first optical feature map and the second optical feature map are subjected to fusion processing to obtain a fourth optical feature map; the fourth optical feature map is subjected to upsampling processing and fusion processing with the third optical feature map to obtain a fifth optical feature map; The fifth optical characteristic map is sequentially subjected to convolution and upsampling processing to obtain an optical remote sensing characteristic map; A fused remote sensing feature map generation module is used to obtain a radar remote sensing map of the area to be measured, and the radar remote sensing map and the optical remote sensing feature map are subjected to feature-level fusion processing to obtain a fused remote sensing feature map; Specifically, the radar remote sensing image and the optical remote sensing image are subjected to wavelet decomposition processing of different frequency bands respectively to obtain a multi-band radar remote sensing image and a multi-band optical remote sensing image; based on the obtained wavelet basis function, the wavelet coefficients of different frequency band images in the multi-band radar remote sensing image and the multi-band optical remote sensing image are obtained respectively to form a radar remote sensing image wavelet coefficient set and an optical remote sensing image wavelet coefficient set; each wavelet coefficient in the radar remote sensing image wavelet coefficient set is multiplied with the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed radar remote sensing image; each wavelet coefficient in the optical remote sensing image wavelet coefficient set is multiplied with the pixel of the corresponding frequency band image in the multi-band optical remote sensing image to obtain a wavelet transformed optical remote sensing image; the wavelet transformed radar remote sensing image and the wavelet transformed optical remote sensing image are subjected to frequency band image superposition of the same frequency band to obtain the fused remote sensing feature image; The optimal classification map generation module is used for the fused remote sensing feature map to be processed by frame regression and probability mapping to obtain an initial classification map in which a recognition area contains multiple recognition frames, and each recognition frame contains multiple classification results; Obtaining the intersection-over-union ratio of all recognition boxes in the same recognition area in the initial classification map; Based on the intersection-union ratio, the recognition frame is screened to obtain an optimized classification map; Obtaining the confidence of each recognition frame in the optimized classification map, retaining the recognition frame with the highest confidence in the same recognition area, and obtaining the optimal classification map; The underlying surface type recognition result generation module is used to obtain the classification result corresponding to the maximum classification probability in each recognition frame in the optimal classification map to obtain the underlying surface type recognition result.
6. A device for identifying the type of land underlying surface, characterized in that: The method comprises a processor and a memory, wherein the processor implements the method for identifying the land underlying surface type as described in any one of claims 1 to 4 when executing the computer program stored in the memory.
7. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for identifying the land underlying surface type according to any one of claims 1 to 4 is implemented.