Urban and rural change intelligent detection method based on visual large model and live-action three-dimensional data
By combining a large visual model with real-world 3D data and employing SAMLoRA fine-tuning technology and elevation change information fusion, the problem of identifying 'different heights with the same texture' in remote sensing change detection was solved. This enabled high-precision urban and rural change detection and semantic analysis, meeting the needs of refined management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- QINGDAO INST OF SURVEYING & MAPPING SURVEY
- Filing Date
- 2026-02-24
- Publication Date
- 2026-05-12
AI Technical Summary
Existing remote sensing change detection methods are susceptible to interference from ground cover shadows and seasonal changes, making it difficult to effectively distinguish between changes with the same texture but different heights. Furthermore, they lack the ability to automatically identify the semantics of changes, resulting in insufficient detection accuracy and reliability, and failing to meet the needs of refined management.
By combining a large visual model with real-world 3D data, and through preprocessing, resampling, and spatial registration of multi-source data, semantic segmentation is performed using SAMLoRA fine-tuning technology. Combined with elevation change information, a multimodal change detection model is constructed, fusing DOM and DSM information and introducing an elevation constraint mechanism to improve the ability to identify three-dimensional changes.
It significantly improves the ability to identify changes in 'same texture but different height', increases detection accuracy by 12.8%, reduces the false negative rate, enhances the ability to detect targets such as newly added artificial facilities, and provides a complete land use conversion matrix and driving factors, improving the interpretability and credibility of the results and reducing computational and data requirements.
Smart Images
Figure CN122020128A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image recognition technology, and in particular to an intelligent detection method for urban and rural changes using a large visual model and real-world 3D data. Background Technology
[0002] Currently, the use of space-based remote sensing technology for multi-temporal observation and automatic change detection has been widely applied in various fields such as farmland protection, ecological monitoring, natural resource management, and urban governance. Traditional remote sensing change detection methods are mainly divided into pixel-based, feature-based, and object-based methods. However, these methods are easily affected by factors such as ground cover shadows and seasonal changes, leading to frequent "false changes," insufficient detection accuracy and reliability, and difficulty in adapting to complex and ever-changing real-world scenarios.
[0003] With the development of artificial intelligence technology, change detection methods based on deep learning—especially fusion models such as convolutional neural networks (CNN) and Transformers—have achieved significant improvements in detection accuracy and have become the mainstream direction for remote sensing change detection. However, as the spatial resolution of remote sensing images continues to improve and the imaging features of ground objects become increasingly complex, AI-based texture representation and local semantic analysis methods still struggle to effectively distinguish between changes of the same texture but different heights, such as changes in ground objects with similar spectral features but different three-dimensional forms, like construction sites and building renovations. Furthermore, existing methods primarily focus on locating the changed area and lack the ability to automatically identify the semantics of the change (i.e., "what changed to what"), failing to meet the needs of refined management. Summary of the Invention
[0004] To address the problems existing in the prior art, this invention provides an intelligent detection method for urban-rural changes using a large visual model and real-world 3D data, comprising:
[0005] Digital orthophotos DOM1 and DOM2 and digital surface models DSM1 and DSM2 of the target area were acquired in two time phases, and the digital orthophotos and digital surface models in the two time phases were preprocessed.
[0006] A pre-trained visual-based segmentation model is used to perform semantic segmentation on two temporal DOM data, and output two temporal land cover classification maps containing semantic basis respectively.
[0007] The elevation change information is extracted by processing the two time-phase DSM data.
[0008] By integrating the land cover classification map with the elevation change information, the output includes land cover transformation patches and artificial facility areas with elevation changes.
[0009] Specifically, the preprocessing includes resampling and spatial registration, so that the DOM and DSM data of the two time phases have a unified spatial reference and matched resolution.
[0010] Specifically, the pre-trained visual base segmentation model keeps the weights W of the image encoder backbone network frozen, and inserts low-rank adaptation matrices in parallel next to the key linear layers of each Transformer module of the image encoder. Fine-tune the parameters and input the remote sensing image X. h During forward propagation, image features are transformed both by the original frozen weights W and by the low-rank increments. The two weights are transformed and added together to form a new weight that adapts to the characteristics of remote sensing. ,
[0011] ,
[0012] Where A is the dimensionality reduction matrix. The input k-dimensional features are projected onto an r-dimensional space; B is the up-dimensional matrix, which reprojects the r-dimensional features back into the output d-dimensional space; r is the rank, which determines the number of new parameters and the model's adaptability. .
[0013] Specifically, the training process of the visual basic segmentation model aims to minimize the segmentation loss on remote sensing samples as the optimization objective L.
[0014] ,
[0015] in, For fine-tuning the semantic segmentation model, L CE The loss function is defined for remote sensing samples (X). h Y h The cross-entropy loss, X, is calculated. h It is remote sensing imagery, Y h These are the corresponding land cover category labels. Here are the parameters of the frozen SAM backbone, and h is the sample index;
[0016] By updating only the A and B matrices in all LoRA modules, and using the cross-entropy loss function L... CE Measuring the model segmentation output relative to the actual land cover label Y h The difference between them is minimized to reduce the overall loss.
[0017] Specifically, the processing of the two time-phase DSM data involves:
[0018] Pixel-level difference calculations were performed on two temporal DSM data to obtain an initial elevation change intensity map. :
[0019] ;
[0020] Initial elevation change intensity map The data is denoised to obtain a smooth elevation change intensity map. Based on the preset threshold for changes in the height of artificial facilities, areas with significant elevation changes are extracted from the smooth elevation change intensity map.
[0021] Specifically, the noise reduction process employs a 3×3 moving window mean filtering algorithm to calculate the mean of the height difference between each pixel and its eight neighboring pixels. This average value is then used as the final height difference value for the center pixel.
[0022] ,
[0023] In the formula, The image shows the intensity of the new elevation change after smoothing, where i and j are the horizontal and vertical coordinates, respectively, and m and n are the increments of the horizontal and vertical coordinates.
[0024] Specifically, the preset threshold τ for the change in height of the artificial facility is 3m-4m.
[0025] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0026] 1. This invention provides an intelligent detection method for urban and rural changes using a large visual model and real-world 3D data. It integrates high-resolution DOM and DSM information to construct a multimodal change detection model based on SAMLoRA fine-tuning and elevation constraints. It employs efficient parameter fine-tuning technology based on the visual basic model to construct a dedicated semantic segmentation model for geographic feature recognition and introduces an elevation change constraint mechanism to improve the ability to identify three-dimensional changes such as artificial facilities.
[0027] 2. This invention fundamentally reduces the uncertainty of a single data source by using a dual-branch independent analysis and mutual verification mechanism of "DOM semantic classification" and "DSM elevation change". It improves the ability to identify changes in "same texture but different height" and increases the change detection accuracy by 12.8% compared to a single method that relies solely on images. This method significantly reduces the false negative rate and greatly enhances the ability to detect typical targets such as newly added artificial facilities.
[0028] 3. This invention adopts the SAMLoRA fine-tuning paradigm, which requires training less than 1% of the parameters of the original model to enable a powerful general-purpose visual model to accurately adapt to remote sensing ground object classification tasks, greatly reducing computational costs, time costs and data requirements, and solving the problem of scarce labeled samples in professional fields.
[0029] 4. This invention not only provides the location of changes, but also automatically outputs a complete land use transformation matrix (such as "cultivated land → construction land"). All changed patches are accompanied by their driving factors (spectral changes, elevation changes, or both), which enhances the interpretability and credibility of the results. Attached Figure Description
[0030] Figure 1 This is a flowchart of the present invention;
[0031] Figure 2 This is a schematic diagram of the SAMLoRA model structure of the present invention;
[0032] Figure 3 This is a rendering of the invention. Detailed Implementation
[0033] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0034] like Figure 1 As shown, this invention provides an intelligent detection method for urban and rural changes using a large visual model and real-world 3D data. This method constructs a multimodal change detection model based on SAMLoRA fine-tuning and elevation constraints by deeply fusing multidimensional information from a high-resolution DOM and a DSM. This enables intelligent recognition of urban and rural change information covering both inter-land type and intra-land type changes, significantly improving the detection capability for complex changes with "same texture but different heights." The specific implementation includes the following steps:
[0035] S1: Collaborative preprocessing of multi-source data and construction of sample library.
[0036] Acquire digital orthophotos DOM1 and DOM2 and digital surface models DSM1 and DSM2 of the target region at two time phases t1 and t2.
[0037] Preprocessing of the two temporal DOM and DSM data, including resampling and spatial registration, is performed to ensure that the DOM and DSM data of the two temporal phases have a unified spatial reference and matched resolution.
[0038] Based on prior knowledge from remote sensing, a classification system was established that includes cultivated land, garden land, artificial facilities, roads, other land surfaces and water bodies. High-quality, multi-period land cover sample datasets were produced through manual visual interpretation.
[0039] S2: The semantic segmentation module based on SAMLoRA parameter efficient fine-tuning outputs a land cover classification map.
[0040] To address the issues of domain differences and high training costs associated with directly applying general-purpose large-scale visual models in professional remote sensing scenarios, this invention uses a large-scale pre-trained visual basic segmentation model (Segment Anything Model, SAM) as its backbone, keeping the weights W of its image encoder and other backbone networks frozen, and fine-tuning it by inserting a lightweight low-rank adaptation matrix ∆W (LoRA) into the key feature interaction layer of the Transformer module.
[0041] like Figure 2 As shown, the pre-trained visual basic segmentation model includes an upsampler, a patch embedding module, and multiple stacked Transformer modules. A low-rank adaptation matrix ∆W is inserted in parallel next to the key linear layers of the Transformer modules.
[0042] Two temporal digital orthophotos, DOM1 and DOM2, are input into the upsampler, where the image resolution is adjusted to the standard input size set during model pre-training. These are then input into the image patch embedding module. The image patch embedding module flattens each image patch through a convolutional layer and maps it to a high-dimensional patch embedding vector, adding positional encoding to preserve the spatial location information of each patch in the original image, thus converting the image data into a Transformer processing sequence. This Transformer processing sequence sequentially passes through a series of frozen Transformer modules with the same structure, performing global context modeling and nonlinear transformation on the input features. The original weight parameters of the Transformer modules are frozen and remain unchanged during fine-tuning, not participating in gradient updates. Fine-tuning is performed by inserting a low-rank adaptation matrix ∆W (LoRA) in parallel next to the key linear layers of the frozen Transformer modules; these multi-layer linear layers are specifically multi-head attention layers.
[0043] ,
[0044] Where A is the dimensionality reduction matrix. The input k-dimensional features are projected onto an r-dimensional space; B is the up-dimensional matrix, which reprojects the r-dimensional features back into the output d-dimensional space; r is the rank, which determines the number of new parameters and the model's adaptability. Insert a lightweight low-rank adaptation matrix (LoRA, Low-Rank Adaptation) is used for fine-tuning. The smaller the r value, the fewer the parameters and the higher the efficiency; the larger the r value, the more parameters and the lower the efficiency.
[0045] The model training process minimizes the remote sensing samples (X). h Y h The segmentation loss on the algorithm is the optimization objective L:
[0046] ,
[0047] in, For fine-tuning the semantic segmentation model, L CE The loss function is defined for remote sensing samples (X). h Y h The cross-entropy loss, X, is calculated. h It is remote sensing imagery, Y h These are the corresponding land cover category labels. These are the parameters of the frozen SAM backbone.
[0048] While preserving all original weight parameters of the SAM image encoder backbone network Under the premise of freezing, by optimizing the low-rank adaptation matrices A and B inserted into the Transformer module, the model is made... Accurately adapt to urban and rural land feature classification tasks.
[0049] Input remote sensing image X h During forward propagation, image features are transformed both by the original frozen weights W and by the low-rank increments. The two weights are transformed and added together to form a new weight that adapts to the characteristics of remote sensing. By updating only the A and B matrices in all LoRA modules, and using the cross-entropy loss function L... CE Measuring the model segmentation output relative to the actual land cover label Y h By minimizing the overall loss, the difference between the parameters is minimized, thereby enabling a general-purpose base model to be efficiently adapted to professional domain classification targets with a very small number of trainable parameters.
[0050] Employing the SAMLoRA fine-tuning paradigm, less than 1% of the parameters of the original model are required for training, enabling a powerful, general-purpose visual model to accurately adapt to remote sensing land cover classification tasks. This significantly reduces computational, time, and data requirements, and solves the problem of scarce labeled samples in specialized fields. After module processing, the input image is transformed into a land cover classification map containing low-level visual features and high-level semantic information, outputting high-confidence land cover classification maps C1 and C2. These maps are then fed into the multi-dimensional change information fusion and refined decision-making module, forming the semantic foundation for subsequent change analysis.
[0051] S3: Elevation Change Extraction Based on DSM Differential and Threshold Analysis
[0052] To obtain the three-dimensional geometric changes caused by the construction and demolition of buildings, roads, etc., pixel-level difference calculations were performed on the two registered temporal DSM data to obtain the initial elevation change intensity map. .
[0053] ;
[0054] To suppress this type of noise, The data undergoes noise reduction processing using a 3×3 moving window mean filtering algorithm. The mean of the elevation difference between each pixel and its eight neighboring pixels is calculated, and this average value is used as the final elevation difference value for the center pixel, resulting in a smoothed elevation change intensity map. :
[0055] ,
[0056] This is the new elevation change intensity map obtained after smoothing; i and j are the horizontal and vertical coordinates, respectively, and m and n are the horizontal and vertical coordinate increments, respectively. The summation subscripts m and n both traverse from -1 to 1, representing all pixels within the 3×3 window centered at (i,j); the denominator 9 corresponds to the total number of pixels within the window (1 central pixel + 8 neighboring pixels).
[0057] A threshold τ for the height change of artificial facilities (such as single-story houses) is set, preferably 3m-4m, and more preferably 3.5m, to match the height of single-story houses and to extract elevation change information.
[0058] ,
[0059] The output of this module As an important indicator reflecting three-dimensional changes such as the construction, demolition or renovation of artificial facilities.
[0060] S4, Multi-dimensional change information fusion and refined decision-making
[0061] This invention designs a multi-dimensional change information fusion and refined decision-making module, which integrates the aforementioned land cover classification map and elevation change information, and outputs the final change patches and their types.
[0062] (1) Analysis of land cover transition trajectory: Spatial analysis of land cover classification data C1 and C2 from two time periods is performed to automatically generate a land cover transition matrix. It accurately identifies land use transformation patches such as "arable land → artificial facilities" and "water bodies → other land surfaces," directly supporting the identification of change information such as "non-agriculturalization" and "non-grainization."
[0063] ,
[0064] Where a represents the land category at the coordinates (i,j) of the first time phase, b represents the land category at the coordinates (i,j) of the second time phase, and E represents the overall classification system.
[0065] (2) Enhanced identification of changes within land use types: Extract land use types such as artificial facilities that have not changed, and perform spatial analysis with areas of significant elevation changes to identify changes of "same texture but different height", so as to accurately identify changes such as construction, reconstruction, and demolition.
[0066] ;
[0067] Through two steps (1) and (2), intelligent identification of urban and rural change information covering both between and within land categories is achieved, and the detection capability for complex changes of "same texture but different height" is significantly improved.
[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention. The above are merely preferred embodiments of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the protection scope of the present invention.
Claims
1. A method for intelligent detection of urban-rural changes using a large visual model and real-world 3D data, characterized in that, include: Digital orthophotos DOM1 and DOM2 and digital surface models DSM1 and DSM2 of the target area were acquired in two time phases, and the digital orthophotos and digital surface models in the two time phases were preprocessed. A pre-trained visual-based segmentation model is used to perform semantic segmentation on two temporal DOM data, and output two temporal land cover classification maps containing semantic basis respectively. The elevation change information is extracted by processing the two time-phase DSM data. By integrating the land cover classification map with the elevation change information, the output includes land cover transformation patches and artificial facility areas with elevation changes.
2. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data as described in claim 1, characterized in that, The preprocessing includes resampling and spatial registration, so that the DOM and DSM data of the two time phases have a unified spatial reference and matched resolution.
3. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data as described in claim 1, characterized in that, The pre-trained visual base segmentation model keeps the weights W of the image encoder backbone network frozen, and inserts low-rank adaptation matrices in parallel next to the key linear layers of each Transformer module of the image encoder. Fine-tune the parameters and input the remote sensing image X. h During forward propagation, image features are transformed both by the original frozen weights W and by the low-rank increments. The two weights are transformed and added together to form a new weight that adapts to the characteristics of remote sensing. , ， Where A is the dimensionality reduction matrix. The input k-dimensional features are projected onto an r-dimensional space; B is the up-dimensional matrix, which reprojects the r-dimensional features back into the output d-dimensional space; r is the rank, which determines the number of new parameters and the model's adaptability. .
4. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data as described in claim 3, characterized in that, The training process of the visual basic segmentation model aims to minimize the segmentation loss on remote sensing samples. ， in, For fine-tuning the semantic segmentation model, L CE The loss function is defined for remote sensing samples (X). h Y h The cross-entropy loss, X, is calculated. h It is remote sensing imagery, Y h These are the corresponding land cover category labels. Here are the parameters of the frozen SAM backbone, and h is the sample index; By updating only the A and B matrices in all LoRA modules, and using the cross-entropy loss function L... CE Measuring the model segmentation output relative to the actual land cover label Y h The difference between them is minimized to reduce the overall loss.
5. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data according to claim 1, characterized in that, The specific processing of the two temporal DSM data is as follows: Pixel-level difference calculations were performed on two temporal DSM data to obtain an initial elevation change intensity map. : ； Initial elevation change intensity map The data is denoised to obtain a smooth elevation change intensity map. Based on the preset threshold for changes in the height of artificial facilities, areas with significant elevation changes are extracted from the smooth elevation change intensity map.
6. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data according to claim 5, characterized in that, The noise reduction process employs a 3×3 moving window mean filtering algorithm, which calculates the mean of the height difference between each pixel and its eight neighboring pixels, and uses this mean as the final height difference value for the center pixel. ， In the formula, The image shows the intensity of the new elevation change after smoothing, where i and j are the horizontal and vertical coordinates, respectively, and m and n are the increments of the horizontal and vertical coordinates.
7. The intelligent detection method for urban-rural changes using a large visual model and real-world 3D data as described in claim 5, characterized in that, The preset threshold τ for the change in height of artificial facilities is 3m-4m.