A deep learning-based land use out-of-limit identification method and system for remote sensing images

Abstract

The application discloses a kind of based on deep learning's remote sensing image land use over-limit identification method and system, the method includes: obtaining land use remote sensing image, and land use remote sensing image is preprocessed, to obtain the picture pair needing identification;Utilize pre-trained Siamese-Resnet50 model and grey level co-occurrence matrix to the feature extraction and matching of the picture pair needing identification, to obtain depth similarity and texture similarity;Based on the depth similarity and texture similarity of picture pair, dynamic weight feature fusion calculation is carried out, to obtain comprehensive similarity;Comprehensive similarity is compared with the decision threshold set, whether remote sensing image land use range is over-limit and output result are judged.The application realizes the high-precision detection and abnormal identification of land use range by combining depth similarity, grey level co-occurrence matrix (GLCM) texture feature and dynamic weight adjustment strategy, simplifies the problem of over-limit identification difficulty, and improves the efficiency of identification.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and remote sensing image processing technology, and in particular to a method and system for identifying over-limit land use in remote sensing images based on deep learning. Background Technology

[0002] With the acceleration of urbanization and frequent changes in land use, land resource management faces severe challenges. During the construction process, various types of land use have been found to be irregular, such as encroaching on permanent basic farmland and construction land, which is known as "excessive land use." Traditional land use monitoring relies on manual inspections or ground measurements, which is inefficient and easily affected by subjective factors.

[0003] Current methods for land monitoring using remote sensing imagery mainly rely on traditional computer vision algorithms, such as threshold segmentation, histogram-based texture analysis, or single deep learning models, such as convolutional neural networks. However, these methods still have limitations. For example, remote sensing imagery data is massive, making it difficult for traditional image processing methods to efficiently extract useful information. Existing technologies lack dynamic monitoring capabilities, cannot adapt to large-scale monitoring needs, and have weak generalization abilities for complex scenes. Therefore, how to effectively extract useful information and conduct dynamic monitoring based on the complexity of different scenes has become a crucial technical problem that urgently needs to be solved in the field of land resource management. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method and system for identifying land use exceeding limits based on deep learning in remote sensing images. This invention achieves high-precision detection and anomaly identification of land use areas by combining depth similarity, gray-level co-occurrence matrix (GLCM) texture features, and a dynamic weight adjustment strategy.

[0005] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a method for identifying excessive land use in remote sensing images based on deep learning, comprising the following steps:

[0006] S1. Acquire land use remote sensing images and preprocess the land use remote sensing images to obtain the image pairs to be identified;

[0007] S2. Use the pre-trained Siamese-Resnet50 model and gray-level co-occurrence matrix to extract and match features of the image pairs to be identified in order to obtain depth similarity and texture similarity.

[0008] S3. Based on the depth similarity and texture similarity of image pairs, perform dynamic weighted feature fusion calculation to obtain the comprehensive similarity.

[0009] S4. Compare the overall similarity with the set decision threshold to determine whether the land use area of ​​the remote sensing image exceeds the limit and output the result.

[0010] Furthermore, the preprocessing of the land use remote sensing imagery specifically includes the following sub-steps:

[0011] S1.1. Based on the vector boundaries provided by the reference data, perform registration and orthorectification, and use the HSV color space to accurately extract the land use vector outline on the current time-phase remote sensing image after registration and orthorectification.

[0012] S1.2 Based on the land use area vector contour, generate inner and outer expansion regions through morphological operations, use polygons to approximate the inner and outer expansion contours to obtain vertex information of the inner and outer expansion regions, extract vertex coordinates for cropping, and obtain image blocks A and B;

[0013] S1.3. Perform size normalization, data type conversion, and pixel value standardization on image blocks A and B to meet the input requirements of the Siamese-Resnet50 model, and use the standardized image blocks A and B as the image pairs to be recognized.

[0014] Furthermore, the Siamese-Resnet50 model is constructed based on the Siamese network and deep learning. The Siamese-Resnet50 model includes an input layer, a backbone network, and an output layer. The input layer is used to receive the image pairs to be identified. The backbone network uses two parallel Resnet50 networks with shared weight parameters as encoders to extract features from the image pairs and obtain the corresponding high-dimensional feature vectors. The output layer is used to calculate and output the deep semantic similarity between the high-dimensional feature vectors of the image pairs. The output layer includes a normalization layer, an inner product layer, and a linear mapping layer. The high-dimensional feature vectors of the image pairs enter the output layer, first pass through the normalization layer to obtain the corresponding normalized feature vectors, then pass through the inner product layer to obtain the preliminary deep similarity, and then pass through the linear mapping layer to shift and scale the original interval of the preliminary deep similarity to [0,1] to obtain the deep similarity.

[0015] Furthermore, the training process of the Siamese-Resnet50 model specifically includes the following steps:

[0016] S2.1 Obtain remote sensing image pairs and label them using expert knowledge. Construct a training dataset based on the remote sensing image pairs and their labels. The training dataset includes similar and dissimilar remote sensing image pairs.

[0017] S2.2. Multiple enhancement methods are used to enhance the remote sensing image pairs in the training dataset to expand the training dataset; among them, the enhancement methods include data augmentation methods and radiometric enhancement methods; the data augmentation methods include rotation, flipping, random cropping, and translation transformation;

[0018] S2.3. Pre-train the Siamese-Resnet50 model using the expanded training dataset. During the training process, calculate the contrastive loss function based on the deep similarity of sample pairs and their corresponding labels. Adjust the parameters of the Siamese-Resnet50 model with the goal of minimizing the contrastive loss function until the preset training rounds are reached to obtain the pre-trained Siamese-Resnet50 model.

[0019] Furthermore, the formula for calculating the contrastive loss function is as follows:

[0020]

[0021] In the formula, L contrast This represents the contrastive loss function. Let y represent the depth similarity of the i-th sample pair. i Let m represent the label of the i-th sample pair, m be the marginal hyperparameter, and N be the number of sample pairs in a training batch.

[0022] Furthermore, the method for obtaining texture similarity specifically includes:

[0023] First, the image pair A and B to be identified are converted into grayscale images; then, the texture feature vector of the image pair is extracted using the gray-level co-occurrence matrix, where the texture feature vector is represented as T = [E, C, H, ρ, γ]. T E, C, H, ρ, and γ represent energy, contrast, entropy, correlation, and homogeneity, respectively. Next, the cosine similarity between two texture feature vectors is calculated as the preceding texture similarity. Finally, a linear mapping is used to shift and scale the original interval of the preceding texture similarity to [0,1] to obtain the texture similarity.

[0024] Furthermore, step S3 specifically includes the following sub-steps:

[0025] S3.1 Calculate the texture quality score using the contrast in the texture feature vector of the image pair, and obtain the depth quality score by calculating the absolute value of the preceding depth similarity of the image pair.

[0026] S3.2 Calculate dynamic temperature parameters based on texture quality score and depth quality score;

[0027] S3.3. Based on dynamic temperature parameters, texture quality scores, and depth quality scores, the weights corresponding to texture similarity and depth similarity are obtained through soft maximum normalization, and then normalized to obtain the normalized dynamic weights α and β corresponding to texture similarity and depth similarity.

[0028] S3.4. Use the normalized dynamic weights α and β corresponding to texture similarity and depth similarity to perform feature fusion calculation on texture similarity and depth similarity to obtain the comprehensive similarity.

[0029] A second aspect of the present invention provides a deep learning-based remote sensing image land use over-limit identification system, comprising: a processor and a memory for storing a computer program capable of running on the processor; wherein, when the processor runs the computer program, it executes the above-described deep learning-based remote sensing image land use over-limit identification method.

[0030] A third aspect of the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described method for identifying land use exceeding limits based on deep learning in remote sensing images.

[0031] A fourth aspect of the present invention provides a computer program product, the computer program product including computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the above-described deep learning-based remote sensing image land use over-limit identification method.

[0032] The beneficial effects of this invention are as follows: This invention utilizes the inconsistency between the inner and outer features of land use boundaries exceeding limits for edge segmentation and comparison, simplifying the problem of difficult boundary recognition and improving recognition efficiency; This invention comprehensively utilizes deep semantic information and texture appearance information, fusing multimodal information to more comprehensively depict land use status and improve recognition accuracy; This invention introduces a dynamic weight adjustment mechanism based on feature quality, enabling the model to adaptively allocate weights according to the reliability of each feature under different scenarios, enhancing the model's robustness and generalization ability to complex situations such as illumination changes, local occlusion, and texture differences; This invention separates training and inference, with deep model training focusing on optimizing semantic feature extraction, while complex fusion logic is performed during the inference phase, simplifying the training process. This invention improves the stability and trainability of the model; it enhances interpretability, as the output of dynamic weights can provide a certain degree of explanation for the judgment results of specific samples, revealing which modality(s) features play a dominant role in the final decision; its applications are wide-ranging, applicable not only to operational scenarios such as land use monitoring, planning verification, and illegal land use detection, but also providing new technical ideas for related fields such as remote sensing image change detection and target recognition; through image preprocessing, automatic image segmentation, and over-limit recognition, this invention effectively solves the problems of high precision and scene adaptation difficulties in land use scope monitoring, not only improving the accuracy of land use monitoring but also significantly enhancing the timeliness of dynamic monitoring and identification, making it suitable for various application scenarios such as land monitoring, land management, and academic research. Attached Figure Description

[0033] Figure 1 This is a flowchart of the remote sensing image land use over-limit identification method based on deep learning according to the present invention;

[0034] Figure 2 This is a schematic diagram illustrating the internal and external features of a certain land use according to an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the segmentation of a remote sensing image preprocessing section according to an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the feature extraction and fusion architecture in the dual-branch neural network of the present invention;

[0037] Figure 5 This is a schematic diagram illustrating the loss changes during the training process of the Siamese-Resnet50 model of this invention;

[0038] Figure 6 This is a schematic diagram of the PR curve of the Siamese-Resnet50 model in an embodiment of the present invention;

[0039] Figure 7 This is an example diagram of the internal structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0042] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0043] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0044] This invention presents a deep learning-based remote sensing image land use over-limit identification method. It extracts both deep semantic and textural features in parallel and designs a dynamic weighting mechanism to adaptively adjust the contribution of each feature to the final decision based on its reliability (quality) in the current sample, thereby improving the accuracy and robustness of land use over-limit identification in complex and variable scenarios. In this invention, "land use over-limit" is defined as: a significant discrepancy between the current actual land use status of the target plot (Region A) (interpreted from recent remote sensing imagery) and its corresponding reference benchmark (Region B) in terms of spatial extent, land cover type, or both. Figure 1 As shown, the method specifically includes the following steps:

[0045] S1. Data Acquisition and Preprocessing: Acquire land use remote sensing images and preprocess them to obtain image pairs that need to be identified.

[0046] It should be understood that preprocessing land use remote sensing images and automatically identifying and extracting the required information can efficiently achieve the identification of land use exceeding limits in remote sensing images, which has significant technical value and application prospects.

[0047] Further, the land use remote sensing imagery is preprocessed, specifically including the following sub-steps:

[0048] S1.1. Based on the vector boundaries provided by the reference benchmark data (such as Shapefile, GeoJSON), specifically, when conducting land use over-limit detection, a standard land use range vector will be provided for reference. Figure 2 This vector boundary serves as the limit for detecting whether land use exceeds the limits. After obtaining the vector boundary, registration and orthorectification are performed based on it. The land use vector outline is then accurately extracted using the HSV (Hue (H), Saturation (S), Brightness (V)) color space on the current time-phase remote sensing image that has been registered and orthorectified.

[0049] S1.2. Based on the land use area vector contour, generate inner and outer expansion regions through morphological operations, use polygons to approximate the inner and outer expansion contours to obtain vertex information of the inner and outer expansion regions, extract vertex coordinates for cropping, and obtain image blocks A and B.

[0050] Specifically, due to the unique characteristics of the boundary of land use exceeding limits, the land use types inside and outside the boundary often differ significantly. Figure 2 As shown. Therefore, based on the extracted land use vector contour, dilation and erosion operations can be used for internal and external expansion. The expansion size is 2-10m internally and externally, where the specific expansion size can be changed according to the actual project requirements. The expansion distance can be calculated based on the image resolution to obtain the internal and external expansion pixel distance. Polygon approximation is used to approximate the internal and external expansion contours, obtaining the vertex information of the internal and external expansion regions. The vertex coordinates are extracted and cropped to obtain image blocks A and B, as shown. Figure 3 As shown.

[0051] S1.3. Image patches A and B are normalized in size, converted in data type, and standardized in pixel value to meet the hard input requirements of the downstream module's deep learning-based Siamese-ResNet50 model. This simultaneously improves the training efficiency and stability of the deep learning-based Siamese-ResNet50 model, ensuring its effectiveness when using the Siamese-ResNet50 model and enhancing the comparability of different features (such as depth and texture) across different samples. The preprocessed image patches A and B are then used as the image pair to be identified.

[0052] S2. Using the pre-trained Siamese-ResNet50 model and Gray-Level Co-occurrence Matrix (GLCM), feature extraction and matching are performed on the image pairs to be identified to obtain depth similarity and texture similarity, such as... Figure 4 As shown.

[0053] In this embodiment, the Siamese-ResNet50 model is built based on the Siamese network (a twin network) and deep learning. The Siamese network is a unique neural network architecture whose core consists of two or more identical sub-networks that share the same weights and parameters. Each sub-network can be a multilayer perceptron, convolutional neural network, or recurrent neural network, etc. In this embodiment, the Siamese network includes two sub-networks, which use ResNet50. The Siamese-ResNet50 model includes an input layer, a backbone network, and an output layer, as follows: Figure 4 As shown.

[0054] The input layer is used to receive the image pair A and B that need to be identified.

[0055] The backbone network employs two parallel ResNet50 networks with shared weight parameters as encoders to extract features from image pairs A and B respectively, obtaining high-dimensional feature vectors f corresponding to image pairs A and B. A ,f B ∈R D Where D represents the high-dimensional feature vector f A ,f B The dimension is typically D = 2048 (the number of output channels of the last convolutional block in the ResNet50 network). The two backbone networks have identical structures and parameters; after inputting different data samples, they extract corresponding feature vectors. Further explanation: ImageA and ImageB are processed by two ResNet50 networks with shared weights. Since the shapes and sizes of land use images differ, and different images typically have different sizes, the adaptive nature of convolution and pooling does not mandate a fixed input size. In Siamese-ResNet50 model applications, ResNet50 is primarily used as a feature encoder. Instead of using its original fully connected layers for 1000 ImageNet classification, the output of deeper layers is extracted as a high-dimensional feature vector f. A and f BGlobal average pooling processes variable input sizes and produces a fixed-size output vector. Global average pooling averages the spatial dimensions (height H and width W) of each feature map to a single value. Therefore, as long as the number of channels is the same (e.g., 2048), the output of global average pooling will always be a fixed-length vector, resulting in a high-dimensional feature vector f. A and f B The lengths are fixed and the same.

[0056] Taking ImageA1 and ImageB1 as examples, both have an image size of 25x106 pixels. In ResNet50, the initial input size is 25×106×3. In the convolutional and pooling layers, the output size of conv1 is 13×53×64, obtained through a convolution operation with kernel=7, stride=2, and padding=3. Then, pool1 (max pooling, kernel=3, stride=2) downsamples it to 7×27×64. Next, the output size of conv2_x is 7×27×256; the output size of conv3_x is 4×14×512, downsampled through a convolution with stride=2 in the first residual block of this stage; the output size of conv4_x is 2×7×1024, downsampled in the same way; and the output size of conv5_x is 1×4×2048. In the global average pooling stage, the input size is 1×4×2048. The operation is to average the 1×4 spatial dimension of each channel, and the final output size is 1×1×2048, that is, a 2048-dimensional vector. Therefore, the high-dimensional feature vectors obtained by ImageA1 and ImageB1 through this model are f respectively. A and f B For ImageA2 and ImageB2, whose image size is 50x31 pixels, the same calculation process will be performed to obtain the high-dimensional feature vector f. A and f B .

[0057] The output layer is used to calculate and output the high-dimensional feature vector f corresponding to image pairs A and B. A ,f B The deep semantic similarity S between deep The output layer includes a normalization layer, an inner product layer, and a linear mapping layer. The high-dimensional feature vector f corresponding to image pairs A and B is... A ,f B Upon entering the output layer, the vector first passes through a normalization layer to obtain the corresponding normalized feature vector, represented as:

[0058]

[0059] In the formula, f represents the normalized feature vector corresponding to image A. A This represents the high-dimensional feature vector corresponding to image A; f represents the normalized feature vector corresponding to image B. B represents the high-dimensional feature vector corresponding to image B; |·| represents the L2 norm, also known as the Euclidean norm, which is used to measure the length of the high-dimensional feature vector in 2048-dimensional space. a d f A The d-th dimension feature, b d f B The d-th dimension of the feature vector is given, where D represents the dimension of the high-dimensional feature vector. The normalized feature vectors corresponding to images A and B are then multiplied by an inner product layer to obtain the initial depth similarity, expressed as:

[0060]

[0061] In the formula, S pre-deep This represents the initial depth similarity, with values ​​falling within the range [-1, 1]. Then, a linear mapping layer shifts and scales the original range of the initial depth similarity to [0, 1] to obtain the depth similarity, expressed as:

[0062]

[0063] In the formula, S deep This indicates depth similarity.

[0064] In this embodiment, when extracting high-dimensional feature vectors of image pairs, each ResNet50 network maps the input data to a feature space (embedding space), ensuring that similar input data are closer together and dissimilar input data are farther apart. In the feature space, the similarity of input data pairs is measured by calculating the distance between two feature vectors. Therefore, the Siamese-ResNet50 model can be trained based on this characteristic, typically using a contrastive loss function. This function adjusts the network parameters based on the true labels of the input image pairs (i.e., similarity or dissimilarity) and the distance between the high-dimensional feature vectors, making the feature vectors of similar image pairs closer and the feature vectors of dissimilar image pairs farther apart. This effectively learns the similarity features of images and is highly adaptable to tasks with limited sample sizes (such as few-sample learning). Furthermore, due to its parameter-sharing characteristic, the training process is relatively efficient. The specific training process includes the following steps:

[0065] S2.1 Obtain remote sensing image pairs. Specifically, you can choose to use or not use the image pairs obtained in step S1 as the remote sensing image pairs required in the current step, depending on actual needs. Label them using expert knowledge, and construct a training dataset based on the remote sensing image pairs and their labels. The training dataset includes similar remote sensing image pairs (such as the same type of land cover or compliant changes, with a label of 1) and dissimilar remote sensing image pairs (such as different types of land cover or excessive changes, with a label of 0).

[0066] S2.2. Due to the scarcity of similar land parcel datasets in remote sensing imagery, in order to simulate real-world land use exceeding limits to the greatest extent possible, it is necessary to enhance the remote sensing image pairs in the training dataset. Various enhancement methods are employed to augment the remote sensing image pairs in the training dataset to expand the training dataset. These enhancement methods include, but are not limited to, data augmentation methods and radiometric enhancement methods; data augmentation methods include, but are not limited to, rotation, flipping, random cropping, and translation transformations.

[0067] For example, in one embodiment, the data augmentation strategy's geometric augmentation method includes the following operations in order of priority: first, random rotation with an angle range of -15° to +15° and an application probability of 80%; second, horizontal flipping with an application probability of 50%; then, random cropping, retaining 80% to 100% of the central region with an application probability of 70%; and finally, translation transformation with a maximum offset of 10% in both the X and Y directions and an application probability of 60%. In the radiometric augmentation method, brightness is changed by linear adjustment within a range of ±20%, and Gaussian noise (standard deviation σ = 0.02) is added to simulate cloud noise. Regarding the class balancing strategy, an oversampling method is used to perform a combination of rotation and translation augmentation on out-of-limit samples (Label = 0) to achieve a ratio of 1:1.2 between their number and compliant samples; simultaneously, during training, misclassified difficult samples (e.g., highly similar negative sample pairs) are dynamically screened and added to the next round of training. It should be noted that when performing training dataset augmentation, the operation is performed simultaneously on a pair of similar or dissimilar images to align with normal detection scenarios. Through these operations, the remote sensing out-of-limit image dataset was significantly expanded, successfully completing the construction of the model training dataset.

[0068] S2.3. Pre-train the Siamese-Resnet50 model using the expanded training dataset. During the training process, calculate the contrastive loss function based on the deep similarity of sample pairs and their corresponding labels. Adjust the parameters of the Siamese-Resnet50 model with the goal of minimizing the contrastive loss function until the preset training rounds are reached to obtain the pre-trained Siamese-Resnet50 model.

[0069] Furthermore, the formula for calculating the contrast loss function is as follows:

[0070]

[0071] In the formula, L contrast This represents the contrastive loss function. Let y represent the depth similarity of the i-th sample pair. i This represents the label of the i-th sample pair (label 0 indicates that the sample pair has dissimilar land types, label 1 indicates that the sample pair has similar land types, and the training objective is S). deep The value of m is close to 1, which means the contrastive loss function value is minimized. m is the marginal hyperparameter (its value is 0.2 or 0.5 or other values, which can be set according to actual needs), and N is the number of sample pairs in a training batch.

[0072] Specifically, N = 108 image pairs are randomly selected from the expanded training dataset to form a training batch, with similar and dissimilar remote sensing image pairs each accounting for 50%. Two pairs are used as examples: Sample pair 1 contains ImageA1 and ImageB1, with label y1 = 1 (similar); Sample pair 2 contains ImageA2 and ImageB2, with label y2 = 0 (dissimilar). This training batch is then input into the Siamese-ResNet50 model, and the final result is the depth similarity S of sample pair 1. deep 1 = 0.9; the depth similarity S of sample pair 2 deep 2 = 0.3. The contrastive loss function for this training batch is calculated using the formula for the contrastive loss function, where the marginal hyperparameter m = 0.5. For sample pair 1, y1 = 1, S deep 1 = 0.9, L contrast Since y1 = 0.01, and because it's a similar pair, the Siamese-ResNet50 model predicts a depth similarity of 0.9, which is close to 1, resulting in a relatively small contrastive loss function value. For sample pair 2, y2 = 0, S... deep 2 = 0.3, L contrast 2 = 0 because they are dissimilar pairs. The Siamese-ResNet50 model predicts a depth similarity of 0.3, which is below the marginal threshold of m = 0.5. Therefore, the Siamese-ResNet50 model considers them sufficiently dissimilar, and this loss is 0. If a dissimilar sample pair (y) in this batch... i S = 0) deepWhen i > 0.5 (marginal hyperparameter m), the Siamese-ResNet50 model needs to be penalized with a loss to encourage it to push the feature vectors of the image pair further. After training this batch, the loss terms of all sample pairs in the batch are summed and then divided by the batch size N = 10⁸ to obtain the total batch loss L. contrast_batch .get Then, backpropagation and parameter updates are performed, with weight updates using the Adam (AdaptiveMoment Estimation) method.

[0073] It should be noted that the Adam method is an adaptive learning rate optimization algorithm for training deep learning models. Its core idea combines the advantages of momentum gradient descent and RMSprop algorithms. It dynamically adjusts the learning rate of each parameter by calculating the first and second moments of the gradient, thereby achieving efficient weight updates. The update process of the Adam method is based on a given loss function f(θ), which is the L mentioned above. contrast This is minimized by iteratively updating the parameter θ. In each iteration, the update gradient is first calculated:

[0074]

[0075] In the formula, g t Let be the gradient value for the t-th update. To differentiate with respect to the parameter θ, f t (θ t-1 0 represents the loss function value used in the (t-1)th update at the t-th update, θ t-1 These are the model parameters updated for the (t-1)th time. Then, the first-order moment estimate is calculated:

[0076] m t =β1·m t-1 +(1-β1)·g t

[0077] In the formula, m t Let β1 be the first-order moment estimate at the t-th update (m0 = 0); β1 is the decay coefficient of the first-order moment estimate, β1∈[0,1), which is used to prevent the initial estimate from being too small, typically 0.9. Then calculate the second-order moment estimate:

[0078]

[0079] In the formula, v t The second moment estimate (v) at the t-th update. t =0); β2 is the attenuation coefficient of the second moment estimate, β2∈[0,1), the purpose is to prevent the initial estimate from being too small, the typical value is 0.999; This indicates squaring each element, i.e., operating element-wise. Next, bias correction is performed:

[0080]

[0081] In the formula, This is the first-order moment estimate after bias correction at the t-th update. This is the second-order moment estimate after bias correction at the t-th update. and Let β1 and β2 be the t-th powers. Last updated parameters:

[0082]

[0083] In the formula, θ t Model parameters at the t-th update; ε is the learning rate, which controls the speed at which parameters are updated; ε is a small constant to prevent division by zero.

[0084] Since the two ResNet50 branches of the Siamese network share the same set of weights, during backpropagation, the path from image A... The gradient and the path from image B The gradients of each parameter in the Siamese-ResNet50 model are ultimately accumulated onto the same set of shared weights W. In other words, for any shared parameter w in the Siamese-ResNet50 model, the total gradient it receives is... It is the sum of gradients propagated back from the two branches; more precisely, it is the average of the sum of gradients propagated back from the two branches for all sample pairs in the batch. This ensures that the two branches learn exactly the same feature extraction capabilities.

[0085] Repeat the batch training described above, using different batches of training data for multiple epochs, until the model's performance on the validation set converges. For details on loss changes and PR curves during the training process, please refer to [link to relevant documentation]. Figure 5 and Figure 6 .

[0086] The inference process of the Siamese-ResNet50 model is as follows: The image pairs A and B to be recognized obtained in step S1 are input into the pre-trained Siamese-ResNet50 model, and finally the deep similarity S can be obtained. deep .

[0087] In this embodiment, the method for obtaining texture similarity specifically includes: first, converting the image pair A and B to be identified into grayscale images; then, extracting the texture feature vectors T = [E, C, H, ρ, γ] of image pair A and B using a Gray-Level Co-occurrence Matrix (GLCM). T Where E, C, H, ρ, and γ represent energy, contrast, entropy, correlation, and homogeneity, respectively. Next, the cosine similarity between two texture feature vectors is calculated as the initial texture similarity. Finally, a linear mapping is used to shift and scale the original interval of the initial texture similarity to [0,1] to obtain the final texture similarity.

[0088] Specifically, the image pairs A and B to be identified are first converted into grayscale images, and the grayscale levels, pixel spacing, and orientation angle parameters are defined. Then, the grayscale co-occurrence matrix (GCM) is calculated for image pairs A and B respectively. The GCM is a statistical method used for image texture analysis; it describes the spatial relationship of grayscale values ​​in an image, that is, the frequency with which a pixel of a certain grayscale value and a pixel of another grayscale value appear simultaneously under a certain spatial relationship. The GCM is a two-dimensional matrix, denoted as P(i,j,d,θ), where each element represents the number of times or probability that a pixel with grayscale value i and a pixel with grayscale value j appear simultaneously at a relative position θ distance from d. Here, i,j∈{0,1,…,G-1}, G is the grayscale level in the image; d∈Z + The preset pixel spacing is θ; θ∈{0°,45°,90°,135°} is the preset direction angle. Each pixel (x,y) in the image is traversed, and the corresponding neighboring pixel (x',y') is determined based on the preset co-occurrence direction θ and spatial distance d. Let g1 and g2 be the gray values ​​of the current pixel (x,y) and the neighboring pixel (x',y'), respectively; then the pixel pair can be represented as a gray-level pair (g1,g2). The reference pixel (x,y) is moved sequentially throughout the entire image region, and the joint occurrence frequency of all gray-level pairs (g1,g2) that satisfy the conditions is counted, thereby constructing the initial gray-level co-occurrence matrix P. After normalization, the final gray-level co-occurrence matrix P representing the spatial structure features of the image texture is obtained. n :

[0089]

[0090] Based on the aforementioned gray-level co-occurrence matrix P n The energy, contrast, entropy, correlation, and homogeneity features are extracted using the following formula to construct the texture feature vector T. A and T B :

[0091] ① The formula for the energy characteristic is:

[0092]

[0093] ②The formula for the contrast feature Contrast is:

[0094]

[0095] ③ The formula for the entropy feature is:

[0096]

[0097] In the formula, ε is a small constant to prevent log(0) from occurring.

[0098] ④ The formula for the correlation feature is:

[0099]

[0100] In the formula, μ i ,μ j The mean values ​​of i and j are σ and j, respectively. i ,σ j Let i and j be the standard deviations, respectively.

[0101] ⑤ The formula for homogeneity is:

[0102]

[0103] Extract a set of standard texture feature descriptors from GLCM to construct the texture feature vector T. A and T B T = [E, C, H, ρ, γ] TIn the above formula, E, C, H, ρ, and γ represent energy, contrast, entropy, correlation, and homogeneity, respectively. Energy (E) reflects the uniformity and consistency of the texture; a higher energy value indicates a more uniform image texture and a more concentrated distribution of elements in the gray-level co-occurrence matrix. Energy helps distinguish between regions with relatively uniform textures and regions with complex and varied textures when detecting texture similarity. Contrast (C) measures the sharpness of the texture and the magnitude of variation in texture elements; textures with high contrast typically have significant gray-level variations, while textures with low contrast are smoother. Contrast can effectively distinguish the coarseness of textures and is very sensitive to detecting subtle changes in texture details. Entropy (H) represents the complexity and information content of the texture; a higher entropy value indicates a more complex texture containing richer information. Entropy helps identify the complexity and diversity of textures and is important for distinguishing different types of textures. Correlation (ρ) reflects the degree of linear correlation between pixels in an image and their neighboring pixels; textures with high correlation indicate a strong linear relationship between pixels, while textures with low correlation have a looser relationship between pixels. Correlation can be used to detect the directionality and consistency of textures. Homogeneity (γ) measures the similarity of pixel pairs in an image; high homogeneity indicates smaller gray-level differences between pixel pairs, suggesting a more uniform texture. Homogeneity helps identify the uniformity and consistency of textures, making it very helpful in detecting texture similarity. Next, two texture feature vectors T are calculated. A and T B The cosine similarity between them is used as the similarity of the preceding texture:

[0104]

[0105] In the formula, S pre-GLCM This represents the similarity of the preceding texture, with values ​​falling within the range [-1, 1]. Then, a linear mapping is used to shift and scale the original range of the preceding texture similarity to [0, 1], yielding the texture similarity, expressed as:

[0106]

[0107] In the formula, S GLCM This represents texture similarity; a higher texture similarity value indicates that the texture patterns are more similar.

[0108] S3. Based on the depth similarity and texture similarity of image pairs, perform dynamic weighted feature fusion calculation to obtain a comprehensive similarity.

[0109] In one embodiment, the dynamic weight feature fusion calculation process occurs during the inference phase and is dynamically controlled using a temperature-controlled soft maximum method. The parameter control method specifically includes the following sub-steps:

[0110] S3.1 For both texture and depth similarity, calculate their "quality scores"—scalars reflecting the reliability of the current sample pair in that modality. Specifically: first, calculate the texture quality score Q using the contrast in the texture feature vector of the image pair. tex First, based on the texture feature vectors T of images A and B... A and T B Contrast C in A and C B Using the theoretical maximum contrast C max_theoretical Normalization is performed to obtain the normalized values ​​of the image for pairs A and B. and

[0111]

[0112] In the formula, and C represents the normalized contrast feature vectors corresponding to image pairs A and B, respectively. The normalized contrast feature vectors are used to score texture quality. A and C B These represent the contrast feature values ​​corresponding to image pairs A and B, respectively; C max_theoretical This represents the theoretical maximum contrast of the current image. Its value depends on the gray levels and the window size. In the GLCM calculation of this embodiment, the number of gray levels is compressed to 64 to reduce computational complexity. At this time, the theoretical maximum contrast is (64-1). 2 =3969. Then, using the normalized contrast feature vectors corresponding to images A and B... and Calculate texture quality score Q tex The calculation formula is as follows:

[0113]

[0114] Then, the depth similarity S between the image pairs is calculated. pre-deep The absolute value of the depth quality score Q is obtained. deep , is represented as:

[0115] Q deep =|S pre-deep |∈[0,1]

[0116] In the formula, Q deep The closer the value is to 1, the more confident the Siamese-Resnet50 model is in judging whether they are similar.

[0117] S3.2, Based on texture quality score Q tex And depth quality score Q deep Calculate the dynamic temperature parameter τ to adjust the smoothness of the weight allocation:

[0118] τ=τ0×(1+κVar(Q tex Q deep ))

[0119] In the formula, τ0>0 represents the base temperature, κ≥0 represents the variance sensitivity coefficient, and both τ0 and κ are hyperparameters, specifically determined by taking the best-performing values ​​through multiple tests; Var() represents the texture quality score Q. tex And depth quality score Q deep The variance between them. As the variance Var increases, τ increases, and the weight exp(τQ) i ), i∈{tex,deep} for Q i The variance Var is more sensitive to differences, and the modality with the highest score will receive greater weight. As the variance Var decreases (i.e., the texture quality score Q...), the texture quality score Q... tex And depth quality score Q deep When τ approaches τ0, the weight distribution becomes more even.

[0120] S3.3, subsequently based on the dynamic temperature parameter τ and texture quality score Q tex And depth quality score Q deep The weights corresponding to texture similarity and depth similarity are obtained through soft maximum normalization, and then normalized to ensure that α + β = 1. The normalized dynamic weights α and β are calculated using the Softmax function. The Softmax function is a commonly used activation function, particularly suitable for converting a set of values ​​into a probability distribution so that the sum of the output values ​​is 1, thereby avoiding bias caused by weight imbalance during multimodal feature fusion. At the same time, the Softmax function emphasizes the relative differences between input values ​​through exponential operations, making the weight allocation more biased towards modalities with higher quality scores, thus improving the sensitivity of the Siamese-Resnet50 model to dominant features.

[0121] w tex =exp(τQ tex )

[0122] w deep =exp(τQ deep )

[0123]

[0124] In the formula, w tex w represents the non-normalized weight value corresponding to the texture mode. deep α represents the non-normalized weight value corresponding to the deep learning modality, α represents the normalized dynamic weight corresponding to the texture similarity, and β represents the normalized dynamic weight corresponding to the depth similarity.

[0125] S3.4. Utilize the normalized dynamic weights α and β corresponding to texture similarity and depth similarity to adjust the texture similarity S. GLCM and depth similarity S deep Feature fusion calculation is performed to obtain the comprehensive similarity, and the calculation formula is as follows:

[0126] S total (A,B)=α·S GLCM +β·S deep

[0127] In the formula, S total (A,B) represents the overall similarity between image A (target area) and image B (standard feature area).

[0128] S4. Compare the overall similarity with the set decision threshold to determine whether the land use area of ​​the remote sensing image exceeds the limit and output the result.

[0129] Specifically, in one embodiment, the comprehensive similarity S total Compared with the preset decision threshold T, if the overall similarity S total If the land use area in the remote sensing image exceeds a preset threshold T, it is considered that the land use exceeds the limit; otherwise, it is considered that the land use area does not exceed the limit, and the final over-limit identification result (normal land use or suspected over-limit) is output. The threshold T is set and optimized by maximizing the F1-Score. In addition, the evaluation metrics during the training process of the Siamese-ResNet50 model also include precision and recall, calculated as follows:

[0130]

[0131] In the formula, TP is the number of samples correctly identified as exceeding the land use limit; FP is the number of normal land use samples incorrectly identified as exceeding the land use limit; and FN is the number of excessive land use samples incorrectly identified as normal land use.

[0132] To implement the method of the embodiments of the present invention, the embodiments of the present invention also provide a remote sensing image land use over-limit identification system based on deep learning, including: a processor and a memory for storing a computer program that can run on the processor; wherein, when the processor runs the computer program, it executes the steps of the method described above.

[0133] The system provided in this embodiment and the method embodiment described above belong to the same concept. For details of its implementation process, please refer to the method embodiment, which will not be repeated here.

[0134] To implement the method of the embodiments of the present invention, the present invention also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the above-described method.

[0135] Based on the hardware implementation of the above-described program modules, and in order to implement the method of this embodiment of the invention, this embodiment also provides an electronic device (computer device). Specifically, in one embodiment, the computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor A01, a network interface A02, a display screen A04, an input device A05, and a memory connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A06. The non-volatile storage medium A06 stores an operating system B01 and a computer program B02. The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A06. The network interface A02 is used for communication with external terminals via a network connection. When the computer program is executed by the processor A01, it implements the method of any of the above embodiments. The display screen A04 can be a liquid crystal display or an electronic ink display. The input device A05 can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse, etc.

[0136] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0137] The device provided in the embodiments of the present invention includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the method of any of the above embodiments.

[0138] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0143] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0144] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0145] It is understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or both. Specifically, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying excessive land use in remote sensing images based on deep learning, characterized in that, Includes the following steps: S1. Acquire land use remote sensing images and preprocess the land use remote sensing images to obtain the image pairs to be identified; S2. Use the pre-trained Siamese-Resnet50 model and gray-level co-occurrence matrix to extract and match features of the image pairs to be identified in order to obtain depth similarity and texture similarity. The Siamese-Resnet50 model is built based on the Siamese network and deep learning. It includes an input layer, a backbone network, and an output layer. The input layer receives the image pairs to be identified. The backbone network uses two parallel Resnet50 networks with shared weight parameters as encoders to extract features from the image pairs and obtain corresponding high-dimensional feature vectors. The output layer calculates and outputs the deep semantic similarity between the high-dimensional feature vectors of the image pairs. The output layer includes a normalization layer, an inner product layer, and a linear mapping layer. The high-dimensional feature vectors of the image pairs enter the output layer, first passing through a normalization layer to obtain the corresponding normalized feature vectors, then through an inner product layer to obtain the initial deep similarity, and finally through a linear mapping layer to shift and scale the original interval of the initial deep similarity to [0,1] to obtain the final deep similarity. The method for obtaining texture similarity specifically includes: First, the image pair A and B to be identified are converted into grayscale images; then, the texture feature vector of the image pair is extracted using the grayscale co-occurrence matrix, where the texture feature vector is represented as... , Let energy, contrast, entropy, correlation, and homogeneity be represented respectively. Then, the cosine similarity between two texture feature vectors is calculated as the preceding texture similarity. Finally, a linear mapping is used to shift and scale the original interval of the preceding texture similarity to [0,1] to obtain the texture similarity. S3. Based on the depth similarity and texture similarity of image pairs, perform dynamic weighted feature fusion calculation to obtain the comprehensive similarity. S4. Compare the overall similarity with the set decision threshold to determine whether the land use area of ​​the remote sensing image exceeds the limit and output the result.

2. The method for identifying excessive land use in remote sensing images based on deep learning according to claim 1, characterized in that, The preprocessing of land use remote sensing images specifically includes the following sub-steps: S1.

1. Based on the vector boundaries provided by the reference data, perform registration and orthorectification, and use the HSV color space to accurately extract the land use vector outline on the current time-phase remote sensing image after registration and orthorectification. S1.2 Based on the land use area vector contour, generate inner and outer expansion regions through morphological operations, use polygons to approximate the inner and outer expansion contours to obtain vertex information of the inner and outer expansion regions, extract vertex coordinates for cropping, and obtain image blocks A and B; S1.

3. Perform size normalization, data type conversion, and pixel value standardization on image blocks A and B to meet the input requirements of the Siamese-Resnet50 model, and use the standardized image blocks A and B as the image pairs to be recognized.

3. The method for identifying excessive land use in remote sensing images based on deep learning according to claim 1, characterized in that, The training process of the Siamese-Resnet50 model specifically includes the following steps: S2.1 Obtain remote sensing image pairs and label them using expert knowledge. Construct a training dataset based on the remote sensing image pairs and their labels. The training dataset includes similar and dissimilar remote sensing image pairs. S2.

2. Multiple enhancement methods are used to enhance the remote sensing image pairs in the training dataset to expand the training dataset; among them, the enhancement methods include data augmentation methods and radiometric enhancement methods; the data augmentation methods include rotation, flipping, random cropping, and translation transformation; S2.

3. Pre-train the Siamese-Resnet50 model using the expanded training dataset. During the training process, calculate the contrastive loss function based on the deep similarity of sample pairs and their corresponding labels. Adjust the parameters of the Siamese-Resnet50 model with the goal of minimizing the contrastive loss function until the preset training rounds are reached to obtain the pre-trained Siamese-Resnet50 model.

4. The method for identifying excessive land use in remote sensing images based on deep learning according to claim 3, characterized in that, The formula for calculating the contrastive loss function is as follows: ； In the formula, This represents the contrastive loss function. Let represent the depth similarity of the i-th sample pair. Let m represent the label of the i-th sample pair, m be the marginal hyperparameter, and N be the number of sample pairs in a training batch.

5. The method for identifying excessive land use in remote sensing images based on deep learning according to claim 1, characterized in that, Step S3 specifically includes the following sub-steps: S3.1 Calculate the texture quality score using the contrast in the texture feature vector of the image pair, and obtain the depth quality score by calculating the absolute value of the preceding depth similarity of the image pair. S3.2 Calculate dynamic temperature parameters based on texture quality score and depth quality score; S3.

3. Based on dynamic temperature parameters, texture quality scores, and depth quality scores, the weights corresponding to texture similarity and depth similarity are obtained through soft maximum normalization, and then normalized to obtain the normalized dynamic weights α and β corresponding to texture similarity and depth similarity. S3.

4. Use the normalized dynamic weights α and β corresponding to texture similarity and depth similarity to perform feature fusion calculation on texture similarity and depth similarity to obtain the comprehensive similarity.

6. A remote sensing image land use over-limit identification system based on deep learning, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor; wherein, when the processor runs the computer program, it performs the deep learning-based remote sensing image land use over-limit identification method according to any one of claims 1-5.

7. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method for identifying over-limit land use based on deep learning in any one of claims 1-5.

8. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the remote sensing image land use over-limit identification method based on deep learning as described in any one of claims 1-5.

Images