A dual-stream mamba island three-dimensional change detection method and system fusing LiDAR and multispectral

Abstract

This invention provides a method and system for detecting three-dimensional changes on the Mamba Island using a dual-stream system integrating LiDAR and multispectral imaging. The method employs an unmanned aerial vehicle (UAV)-based integrated system to simultaneously acquire raw data of the island's monitoring area in the first and second time phases, including airborne LiDAR point cloud data and multispectral image data. It performs true surface reconstruction and benchmark unification to construct a benchmark base for three-dimensional detection, generating an elevation difference map characterizing the absolute deformation of the island's surface in the vertical dimension. A parallel dual-tower network structure based on a state-space model is constructed for feature extraction and fusion, outputting a high-dimensional feature map. A logic discriminator integrating multi-dimensional attribute constraints is used to classify and quantify three-dimensional parameters related to illegal sand mining, unauthorized dumping, illegal construction, and natural evolution. This invention can eliminate interference from vegetation growth and water level changes, accurately pinpointing human-caused destructive activities and providing high-precision three-dimensional data support for water conservancy cleanup and rectification.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing data processing, specifically relating to a method for detecting three-dimensional changes in the Mamba Island using a dual-stream system that integrates LiDAR and multispectral imaging. Background Technology

[0002] Dynamic monitoring of islands is crucial for marine resource protection and the regulation of illegal activities. Traditional two-dimensional change detection methods based on optical imagery rely primarily on spectral information, making it difficult to capture vertical deformation in the elevation dimension and accurately quantify behaviors exhibiting "three-dimensional deformation" characteristics, such as illegal sand mining and unauthorized landfilling. Furthermore, relying solely on elevation data makes it difficult to distinguish between natural vegetation evolution and human-caused surface damage, easily leading to misjudgments.

[0003] On the other hand, island regions are often densely covered with vegetation, making it difficult for traditional optical remote sensing methods to penetrate the vegetation layer and obtain accurate surface elevation information. Existing change detection methods lack feature extraction capabilities when dealing with areas of abrupt topographic changes, making it difficult to accurately capture the edge features of key features such as steep slopes and cones. Meanwhile, the surge in high-resolution remote sensing data places higher demands on computational efficiency, and the traditional Transformer architecture suffers from high computational complexity when processing large-scale imagery.

[0004] Therefore, how to achieve high-precision, multi-dimensional three-dimensional change detection and parameter quantification in complex vegetation-covered island environments, while also taking into account computational efficiency, has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to address the problems of existing remote sensing monitoring methods, which struggle to simultaneously consider geometric deformation and semantic attributes and are insufficient in capturing abrupt changes in terrain. This invention proposes a dual-stream Mamba Island three-dimensional change detection method that integrates LiDAR and multispectral imaging to achieve accurate identification and quantification of the island's "four disorder" behaviors.

[0006] The technical solution of the present invention: In a first aspect, the present invention provides a method for detecting three-dimensional changes in the Mamba Island using a dual-stream system that integrates LiDAR and multispectral methods, comprising the following steps: Step S1: Using an unmanned aerial vehicle (UAV) integrated system, the raw data of the island monitoring area are acquired simultaneously in the first and second time phases. The raw data includes airborne LiDAR point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue and near-infrared bands. Step S2: Reconstruct the true surface and unify the reference surface on the original data to construct a reference base for three-dimensional detection and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension. Step S3: Construct a parallel dual-tower network structure based on a state-space model, fuse multispectral image data and elevation difference map data into a five-channel tensor input, use spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features respectively, and introduce an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain change. Use a cross-attention mechanism to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, including spectral flow features and elevation flow features. Step S4: Receive the high-dimensional feature map output by the parallel dual-tower network structure, and use the logic discriminator with integrated multi-dimensional attribute constraints to perform classification and three-dimensional parameter quantization calculation of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior.

[0007] Furthermore, in step S1, the UAV-borne integrated system employs a UAV simultaneously equipped with a lidar and a multispectral sensor; When acquiring raw data, the drone should be controlled to fly at an altitude of 150 to 200 meters, with a forward overlap of no less than 80% and a lateral overlap of no less than 70%. The point cloud density of the airborne LiDAR point cloud data is no less than 50 points / square meter, the ranging accuracy is better than 0.05 meters, and the ground surface information under the forest is obtained by penetrating the vegetation cover layer of the island through multiple echo characteristics; the ground sampling interval of the multispectral image data is no greater than 0.05 meters, and the red, green, blue and near-infrared bands are synchronized with the airborne LiDAR point cloud data in the temporal and spatial dimensions.

[0008] Furthermore, in step S2, the true surface reconstruction performs a progressive encryption triangular mesh filtering step on the airborne LiDAR point cloud data, specifically including: The original point cloud is gridded, and the local lowest point in each grid is extracted as the initial ground seed point to construct an initial irregular triangular network. Perform an iterative search process, calculate the vertical distance from the point to be classified to the plane of its triangle and the slope angle of the line connecting the point to the nearest vertex relative to the horizontal plane. When the slope angle is within the slope threshold range of 30 degrees to 45 degrees and the vertical distance is within the distance threshold range of 0.1 meters to 0.3 meters, the point to be classified is determined as a ground point and the irregular triangular mesh is updated until no new points are added. Using all extracted ground points, an inverse distance weighted interpolation or kriging interpolation method is employed to generate a bare soil digital elevation model with a spatial resolution of 0.1 meters.

[0009] Furthermore, in step S2, the unification of the reference surface includes: Using the trajectory data recorded by the UAV, the digital elevation models of bare soil in the first and second time phases are mapped to the same coordinate system CGCS2000 and the 1985 National Elevation Datum used in the UAV operation. After spatial alignment, pixel-level difference operations are performed on the two phases of bare soil digital elevation models to generate the elevation difference map. Each pixel value in the elevation difference map represents the elevation difference amplitude of the island surface at that spatial location. That is, the vertical deformation value.

[0010] Further, in step S3, the five-channel tensor is defined as Where B is the batch size, X represents the input data, and H and W are the height and width of the input data; the first four channels of the tensor contain red light (R), green light (G), blue light (B), and near-infrared bands. Multispectral image data, the fifth channel is the elevation difference amplitude corresponding to the elevation difference map. The parallel dual-tower network structure includes: Spectral semantic stream: Receives multispectral image data from the first four channels, constructs a dual-branch structure to obtain spectral stream features; one is... The branch calculates the normalized vegetation index. Identify changes in vegetation cover; among which To prevent constants with a denominator of zero; the second is the near-infrared anti-spoofing branch, which uses the reflectance difference between the near-infrared band and the green light band to identify the red edge effect, thereby distinguishing vegetation, water bodies, bare soil, debris and hard ground. Elevation geometric flow involves receiving elevation difference map data from the fifth channel and processing it to obtain elevation flow characteristics, including: extracting the first derivative of the elevation difference map to characterize the elevation gradient magnitude. The local variance of the elevation difference map is calculated using a sliding window of a preset size to obtain a local roughness index that characterizes the smoothness of the surface texture. The edge shape features of the elevation difference map are extracted using a convolution operator to generate an edge regularity index that characterizes the rectangularity of the ground features. The elevation geometry flow identifies local depressions left by illegal sand mining or local protrusions formed by illegal accumulation by capturing vertical abrupt changes in the terrain.

[0011] Furthermore, the parallel dual-tower network structure employs an elevation gradient-guided scanning strategy when performing feature extraction; First, within the linear recursive framework of the Mamba algorithm in the state-space model, the state-space equation is defined as follows: ; in, In hidden state, Given the input sequence, For output, Indicates the sequence number; A, B, C, and D are system parameters; Secondly, the scanning strategy uses elevation gradient magnitude. Dynamically constructing the gating weight factor for selective scanning mechanism The formula is: ;

[0012] in, and These are the learnable weight matrices for the elevation gradient magnitude branch and the elevation difference map data input sequence branch, respectively. Use the Sigmoid activation function; When the elevation gradient magnitude When the preset first geometric transition threshold is exceeded, the gating weights Increase, improve the hidden state of the model in this region. Increase the update frequency to enhance the extraction strength of terrain vertical deformation features; When the elevation gradient magnitude When the value is below the preset second geometric transition threshold, the model skips redundant data and hides the state through linear projection. Keep it unchanged to improve the processing efficiency of long sequence data.

[0013] Furthermore, the first geometric jump threshold is set to 0.5 meters; the second geometric jump threshold is set in the range of 0.001-0.01.

[0014] Furthermore, in step S4, the logic discrimination module integrates a surface texture flatness regression head and a ground feature rectangularity detection head, which respectively obtain the local roughness index and edge regularity index in the elevation geometry flow, and output the normalized surface regularity value and edge regularity value, and perform logic discrimination in combination with the high-dimensional feature map to distinguish between illegal sand mining behavior, illegal accumulation behavior, illegal buildings and natural evolution.

[0015] Furthermore, the execution logic judgment specifically includes: The steps for identifying illegal sand mining activities include: When the spectral flow characteristics of the target area show "vegetation or water body turning into bare soil" and the elevation flow characteristics show "elevation drop" and the average drop depth of the target area exceeds 0.5 meters, it is determined to be illegal sand mining; After determination, the area of ​​the pit and the volume of earthwork in the region are calculated using a spatial integration algorithm based on the elevation difference map. :

[0016] in, Indicates the pixel number. , The first and second time phases are respectively in the first and second time phases at the 1st and 2nd time phases. Elevation value of each pixel This represents the actual ground area corresponding to a single pixel. This represents the total number of pixels within the changing area. The steps for identifying the illegal stacking behavior include: When the spectral flow characteristics of the target area show "vegetation to debris or bare soil" and the elevation flow characteristics show "elevation rise" with an average rise height of more than 0.8 meters in the target area, it is determined to be illegal accumulation. Simultaneously, the peak value of the accumulation height is output, and the volume of waste residue is calculated based on the elevation difference map. :

[0017] The steps for identifying illegal buildings include: when the spectral flow feature is detected as vegetation turning into hard ground, and the elevation flow shows an increase in elevation, and the average surface regularity index is greater than 0.6 and the average edge regularity index is greater than 0.7, it is determined to be an illegal building, and the building outline is vectorized using the edge detection operator to calculate its footprint. The discrimination steps of natural evolution include: when a spectral index is detected. The elevation difference amplitude at the corresponding location changes. When the mean absolute value is less than the measurement error threshold of 0.1 meters, it is determined to be natural evolution.

[0018] Secondly, this invention provides a system for a dual-stream Mamba Island three-dimensional change detection method that integrates LiDAR and multispectral imaging, comprising: The data acquisition module is configured to use an unmanned aerial vehicle (UAV) integrated system to simultaneously acquire raw data of the island monitoring area in the first and second time phases. The raw data includes airborne LiDAR point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue and near-infrared bands. The preprocessing module is configured to reconstruct the true surface and unify the reference surface on the original data, construct a reference base for three-dimensional detection, and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension. The dual-stream Mamba feature extraction module is configured to construct a parallel dual-tower network structure based on a state-space model. It fuses multispectral image data and elevation difference map data into a five-channel tensor input. It uses spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features, respectively. It also introduces an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain changes. A cross-attention mechanism is used to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, which includes spectral flow features and elevation flow features. The logic discrimination module is configured to receive the high-dimensional feature map output by the parallel dual-tower network structure, and use a logic discriminator with integrated multi-dimensional attribute constraints to perform classification and discrimination of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior and three-dimensional parameter quantization calculation.

[0019] The proposed method and system for detecting three-dimensional changes in the Mamba Island using a dual-stream dual-spectral approach, which integrates LiDAR and multispectral methods, has the following advantages: 1. Strong vertical deformation sensing capability: By using LiDAR multiple echoes to penetrate the vegetation cover layer of the island to obtain real surface elevation information, combined with progressive densification triangular network filtering and benchmark unification, a high-precision elevation difference map is generated, realizing the leap from "planar monitoring" to "three-dimensional quantization", which can accurately capture local depressions formed by illegal sand mining and local protrusions formed by illegal accumulation.

[0020] 2. Excellent anti-counterfeiting and anti-interference capabilities: The near-infrared anti-counterfeiting branch in the spectral semantic stream uses the reflectance difference feature between the near-infrared band and the green band to identify the red edge effect. It can effectively distinguish real vegetation from visual deception behaviors such as green dust control nets and camouflage nets, solving the problem that it is difficult to identify other construction site objects by simply relying on spectral indices.

[0021] 3. Accurate capture of terrain abrupt changes: By using an elevation gradient-guided scanning strategy, the gating weight factor of the selective scanning mechanism is dynamically constructed using the elevation gradient amplitude. This increases the update frequency of hidden states in key terrain features such as steep slopes and cones, enhancing the model's ability to extract vertical deformation features. In flat areas, redundant data is skipped, improving processing efficiency.

[0022] 4. High computational efficiency: The Mamba linear recursive framework based on the state-space model is used to replace the traditional Transformer architecture. When processing high-resolution large-scale remote sensing images, linear computational complexity is maintained, avoiding the quadratic complexity problem of Transformer. While ensuring detection accuracy, computational resource consumption is significantly reduced.

[0023] 5. Strong logical discrimination capability of multi-dimensional attribute constraints: Integrating surface texture flatness regression head and land feature rectangularity detection head, and combining spectral flow characteristics and elevation flow characteristics, it establishes logical discrimination rules for multi-dimensional attribute constraints. It can accurately distinguish four different types of island surface changes: illegal sand mining, illegal dumping, illegal construction, and natural evolution. It can also realize the quantitative calculation of three-dimensional parameters such as pit area, earthwork volume, dumping height, and land area, providing accurate data support for law enforcement and supervision.

[0024] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0025] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the invention.

[0026] Figure 1 The overall architecture diagram of the dual-stream Mamba Island three-dimensional change detection system integrating LiDAR and multispectral imaging of the present invention is shown.

[0027] Figure 2 A logic flowchart of the preprocessing module in an embodiment of the present invention is shown.

[0028] Figure 3 A schematic diagram illustrating the principle of the elevation gradient-guided scanning strategy in an embodiment of the present invention is shown. Detailed Implementation

[0029] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0030] This invention provides a dual-stream Mamba Island 3D change detection method integrating LiDAR and multispectral imaging. It primarily targets the precise monitoring of surface changes in island regions, particularly demonstrating significant application value in the identification and quantitative analysis of illegal sand mining, unauthorized dumping, illegal construction, and natural evolution. This method acquires multi-source data through an integrated system mounted on a UAV and combines advanced feature extraction and logical discrimination techniques to achieve high-precision detection of 3D surface changes on the island. Specifically, it includes the following steps: Step S1: Using an unmanned aerial vehicle (UAV) integrated system, raw data of the island monitoring area is simultaneously acquired in the first and second time phases. The raw data includes airborne lidar point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue, and near-infrared bands.

[0031] Specifically, during data acquisition, the integrated system carried by the UAV includes high-precision lidar equipment and multispectral sensors. The UAV's flight altitude is controlled between 150 and 200 meters to ensure the resolution and coverage of the data acquisition. The forward overlap is set to be no less than 80%, and the lateral overlap to be no less than 70%, to ensure the spatial continuity and integrity of the data. The lidar point cloud data has a point cloud density of no less than 50 points / square meter and a ranging accuracy better than 0.05 meters. Through multiple echo characteristics, it can penetrate the vegetation cover layer of the island area to obtain information about the forest floor. The ground sampling interval of the multispectral image data is no greater than 0.05 meters, and the red, green, blue, and near-infrared bands included are synchronized with the lidar point cloud data in both time and space. This synchronization ensures the accuracy of subsequent data fusion and lays a solid foundation for three-dimensional change detection.

[0032] In this step, since island regions typically have complex vegetation cover and topographic features, a single data source is insufficient to comprehensively characterize surface changes. Therefore, a multi-source data synchronous acquisition method is adopted to improve the comprehensiveness and accuracy of monitoring.

[0033] Step S2: Reconstruct the true surface and unify the reference surface on the original data to construct a reference base for three-dimensional detection and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension.

[0034] Specifically, the true surface reconstruction performs a progressively denser triangulation filtering step on the airborne lidar point cloud data. First, the original point cloud is gridded, dividing the data into regular grid cells, each 1 meter × 1 meter in size. Local minimum points are extracted within each grid cell as initial ground seed points, and these seed points are used to construct an initial irregular triangulation. Then, an iterative search process is executed, analyzing the vertical distance from each point to be classified to its corresponding triangle plane, as well as the slope angle of the line connecting that point to the nearest vertex relative to the horizontal plane. When the slope angle is within a slope threshold range of 30 to 45 degrees, and the vertical distance is within a distance threshold range of 0.1 to 0.3 meters, the point to be classified is determined as a ground point and added to the triangulation for updating. This iterative process continues until no new points are determined as ground points. Finally, using all extracted ground points, an inverse distance weighted interpolation method is used to generate a bare soil digital elevation model with a spatial resolution of 0.1 meters.

[0035] The unified reference surface includes: mapping the bare soil digital elevation models of the first and second time phases to the same coordinate system CGCS2000 and the 1985 National Elevation Datum using trajectory data recorded by UAVs; after spatial alignment, performing pixel-level difference operations on the two bare soil digital elevation models to generate the elevation difference map, where each pixel value in the elevation difference map represents the elevation difference amplitude of the island surface at that spatial location. That is, the vertical deformation value.

[0036] In this step, since the surface of island areas usually contains various non-ground elements such as vegetation and buildings, it is difficult to accurately reflect the true surface elevation by directly using the raw point cloud data. Therefore, it is necessary to extract ground information through true surface reconstruction and eliminate systematic errors between data from different time periods by using a reference surface.

[0037] Step S3: Construct a parallel dual-tower network structure based on a state-space model, fuse multispectral image data and elevation difference map data into a five-channel tensor input, use spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features respectively, and introduce an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain changes. Use a cross-attention mechanism to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, including spectral flow features and elevation flow features.

[0038] Specifically, the five-channel tensor is defined as Where B is the batch size, X represents the input data, and H and W are the height and width of the input data; the first four channels of the tensor contain red light (R), green light (G), blue light (B), and near-infrared bands. Multispectral image data, the fifth channel is the elevation difference amplitude corresponding to the elevation difference map. .

[0039] The parallel dual-tower network structure is divided into two parts: spectral semantic flow and elevation geometric flow.

[0040] The spectral semantic stream is responsible for processing multispectral image data and extracting spectral stream features through a two-branch structure. One branch identifies vegetation cover changes by calculating the normalized vegetation index, while the other branch uses the difference in reflectance between the near-infrared band and the green band to identify the red edge effect, thus distinguishing different land surface types such as vegetation, water bodies, bare soil, debris, and hard surfaces.

[0041] The elevation geometry flow process handles elevation difference map data and captures vertical abrupt changes and shape features of the terrain by extracting features such as elevation gradient magnitude, local roughness index, and edge regularity index. This is used to identify local depressions left by illegal sand mining or local protrusions formed by illegal accumulation.

[0042] The parallel dual-tower network structure incorporates an elevation gradient-guided scanning strategy to refine feature capture in areas of abrupt terrain changes during feature extraction. Within this structure, a selective scanning mechanism is constructed using a linear recursive framework based on a state-space model, guided by the elevation gradient magnitude. Dynamically adjust the gating weight factor ; ;

[0043] in, and These are the learnable weight matrices for the elevation gradient magnitude branch and the elevation difference map data input sequence branch, respectively. Use the Sigmoid activation function; When the elevation gradient magnitude exceeds a preset threshold, the model increases the update frequency of the hidden state for that region, enhancing the extraction strength of the terrain's vertical deformation features. When the elevation gradient magnitude is below the preset threshold, the model skips redundant data and keeps the hidden state unchanged, thereby improving the processing efficiency of long-sequence data. This strategy can focus on areas of abrupt terrain change and improve the targeting of feature extraction.

[0044] In this step, since the changing characteristics of island regions are usually manifested as dual changes in spectral properties and geometric shape, a single feature is difficult to fully represent. Therefore, a dual-stream network structure is used to process spectral and geometric information separately, and an elevation gradient-guided strategy is used to enhance attention to areas with abrupt topographic changes.

[0045] Step S4: Receive the high-dimensional feature map output by the parallel dual-tower network structure, and use the logic discriminator with integrated multi-dimensional attribute constraints to perform classification and three-dimensional parameter quantization calculation of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior.

[0046] Specifically, the logic discrimination module integrates a surface texture flatness regression head and a ground feature rectangularity detection head, which respectively obtain the local roughness index and edge regularity index in the elevation geometry flow, and output the normalized surface regularity value and edge regularity value. The logic discrimination is performed in combination with the high-dimensional feature map to distinguish between illegal sand mining, illegal accumulation, illegal buildings and natural evolution.

[0047] The steps for identifying illegal sand mining include: when the spectral flow characteristics of the target area show "vegetation or water body turning into bare soil", and the geocurrent characteristics show "elevation decrease" with an average decrease depth of more than 0.5 meters, it is determined to be illegal sand mining; after determination, the area of ​​the pit and the volume of earthwork in the area are calculated based on the elevation difference map using a spatial integration algorithm. :

[0048] in, Indicates the pixel number. , The first and second time phases are respectively in the first and second time phases at the 1st and 2nd time phases. Elevation value of each pixel This represents the actual ground area corresponding to a single pixel. This represents the total number of pixels within the changing area. The steps for identifying illegal dumping include: when the spectral flow characteristics of the target area show "vegetation to debris or bare soil" and the elevation flow characteristics show "elevation rise" with an average rise height exceeding 0.8 meters, it is determined to be illegal dumping; simultaneously, the peak dumping height is output, and the volume of waste is calculated based on the elevation difference map. :

[0049] The steps for identifying illegal buildings include: when the spectral flow feature is detected as vegetation turning into hard ground, and the elevation flow shows an increase in elevation, and the average surface regularity index is greater than 0.6 and the average edge regularity index is greater than 0.7, it is determined to be an illegal building, and the building outline is vectorized using the edge detection operator to calculate its footprint. The discrimination steps of natural evolution include: when a spectral index is detected. The elevation difference amplitude at the corresponding location changes. When the mean absolute value is less than the measurement error threshold of 0.1 meters, it is determined to be natural evolution.

[0050] This invention also provides a dual-stream Mamba Island three-dimensional change detection system integrating LiDAR and multispectral imaging. The system includes modules that correspond one-to-one with the steps of the above-described method, used to implement all or part of the process. Each module can be implemented through hardware, software, or a combination of both, including: The data acquisition module is configured to use an unmanned aerial vehicle (UAV) integrated system to simultaneously acquire raw data of the island monitoring area in the first and second time phases. The raw data includes airborne LiDAR point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue and near-infrared bands. The preprocessing module is configured to reconstruct the true surface and unify the reference surface on the original data, construct a reference base for three-dimensional detection, and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension. The dual-stream Mamba feature extraction module is configured to construct a parallel dual-tower network structure based on a state-space model. It fuses multispectral image data and elevation difference map data into a five-channel tensor input. It uses spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features, respectively. It also introduces an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain changes. A cross-attention mechanism is used to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, which includes spectral flow features and elevation flow features. The logic discrimination module is configured to receive the high-dimensional feature map output by the parallel dual-tower network structure, and use a logic discriminator with integrated multi-dimensional attribute constraints to perform classification and discrimination of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior and three-dimensional parameter quantization calculation.

[0051] In practice: like Figures 1-3 As shown, this embodiment provides a method for detecting three-dimensional changes in the Mamba Island using a dual-stream system that integrates LiDAR and multispectral methods. Specifically, it may include: Step 1: Data Acquisition and True Surface Reconstruction: This step begins by acquiring raw data of the island monitoring area using an unmanned aerial vehicle (UAV) integrated system. To ensure the accuracy of the 3D detection, the UAV's flight altitude is controlled between 150m and 200m, and the onboard LiDAR point cloud density is set to 50 points / square meter.

[0052] 1. True Surface Reconstruction (PTD Filtering): For LiDAR point clouds, a progressively encrypted triangular mesh filtering algorithm is used to remove vegetation cover.

[0053] First, extract the lowest point within the 0.5m grid as the initial ground seed point to construct the initial triangular mesh.

[0054] During the iteration process, the perpendicular distance from the point to be classified to the triangular face is calculated. With slope angle When satisfied and At that time, it was determined to be a ground point.

[0055] 2. Datum Unification and Difference Map Generation: A digital elevation model (DEM) of bare soil with a spatial resolution of 0.1m is generated using inverse distance weighted (IDW) interpolation. After unifying the two DEMs to the CGCS2000 coordinate system and the 1985 National Elevation Datum, pixel-level differencing is performed to generate an elevation difference map. Each pixel value in the difference map represents the vertical deformation at that location.

[0056] Step 2, Dual-stream Mamba Coupling Feature Extraction: This step constructs a parallel dual-tower architecture that fuses multispectral (R, G, B, NIR) data with elevation difference maps into a five-channel tensor input.

[0057] 1. Spectral semantic flow and near-infrared spoofing detection: In addition to calculating the normalized vegetation index This invention introduces near-infrared radiation. Falsehood detection branch. Utilizing the unique red-edge effect of true vegetation in the near-infrared band, through calculation... The difference characteristics between the green light band and the ground surface effectively distinguish green dust nets, camouflage paint and real vegetation, solving the problem of visual deception in traditional optical detection.

[0058] 2. Elevation geometric flow and EGG-SSM scan: Elevation geometric flow is achieved by calculating the elevation gradient magnitude and the local roughness index. Capturing abrupt terrain changes. In Mamba sequence modeling, the elevation gradient-guided scanning strategy EGG-SSM is introduced. Its gating weighting factor... Determined by the following formula:

[0059] when At that time, the model automatically increases the gating weight, thereby increasing the frequency of updating the hidden state of areas with terrain abrupt changes, such as the edges of illegal sand mining pits or illegally piled cones.

[0060] 3. Cross-attention coupling fusion: To achieve deep integration of attributes and geometry, this invention employs a cross-attention mechanism. It utilizes spectral flow features as... Elevation flow characteristics as and Attention weights guide the model to focus on the deformation of vegetation-damaged areas; conversely, elevation flow guides spectral flow to focus on the attribute changes in deformed areas, achieving complementary advantages of features.

[0061] Step 3, Hybrid Decision Making and Parameter Quantization: This step achieves accurate identification of the "four disorderly" behaviors through a hybrid approach of multi-task learning and physical rules.

[0062] 1. Multi-task loss monitoring: During the model training phase, a comprehensive loss function is used for supervision:

[0063] in, Indicates the category of loss. Force the model to learn the regular edges of buildings. By learning the smooth texture of hardened ground, the accuracy of identifying illegal structures (with regular edges and smooth surfaces) is significantly improved.

[0064] 2.7:3 Hybrid Decision Logic: The system's final judgment score It is composed of a weighted average of neural network probabilities and expert rule scores:

[0065] Expert Rules Strict physical barriers were set up. This represents the class response probability output by the classification branch of the neural network. Illegal sand mining: Furthermore, the spectrum shifts from vegetation to bare soil; Illegal stockpiling: Furthermore, the spectral analysis showed the presence of waste residue and impurities; Natural evolution: if ,even if Any changes are also considered natural evolution.

[0066] 3. Calculation of three-dimensional parameters: After identifying the area as an illegal area, the volume is calculated using a spatial integration algorithm:

[0067] Among them, based on a resolution of 0.1m, the area of ​​a single pixel is... .

[0068] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A dual-stream Mamba Island 3D change detection method fusing LiDAR and multispectral, characterized in that, Includes the following steps: Step S1: Using an unmanned aerial vehicle (UAV) integrated system, the raw data of the island monitoring area are acquired simultaneously in the first and second time phases. The raw data includes airborne LiDAR point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue and near-infrared bands. Step S2: Reconstruct the true surface and unify the reference surface on the original data to construct a reference base for three-dimensional detection and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension. Step S3: Construct a parallel dual-tower network structure based on a state-space model, fuse multispectral image data and elevation difference map data into a five-channel tensor input, use spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features respectively, and introduce an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain change. Use a cross-attention mechanism to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, including spectral flow features and elevation flow features. Step S4: Receive the high-dimensional feature map output by the parallel dual-tower network structure, and use the logic discriminator with integrated multi-dimensional attribute constraints to perform classification and three-dimensional parameter quantization calculation of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior.

2. The dual-stream Mamba island three-dimensional change detection method fusing LiDAR and multispectral according to claim 1, characterized in that, In step S1, the UAV-borne integrated system adopts a UAV that simultaneously carries a lidar and a multispectral sensor. When acquiring raw data, the drone's flight altitude should be controlled between 150 and 200 meters, with a forward overlap of no less than 80% and a lateral overlap of no less than 70%. The point cloud density of the airborne LiDAR point cloud data is no less than 50 points / square meter, the ranging accuracy is better than 0.05 meters, and the ground surface information under the forest is obtained by penetrating the vegetation cover layer of the island through multiple echo characteristics; the ground sampling interval of the multispectral image data is no greater than 0.05 meters, and the red, green, blue and near-infrared bands are synchronized with the airborne LiDAR point cloud data in the temporal and spatial dimensions.

3. The dual-stream Mamba island three-dimensional change detection method fusing LiDAR and multispectral according to claim 1, characterized in that, In step S2, the true surface reconstruction performs a progressive encryption triangulation filtering step on the airborne LiDAR point cloud data, specifically including: The original point cloud is gridded, and the local lowest point in each grid is extracted as the initial ground seed point to construct an initial irregular triangular network. Perform an iterative search process, calculate the vertical distance from the point to be classified to the plane of its triangle and the slope angle of the line connecting the point to the nearest vertex relative to the horizontal plane. When the slope angle is within the slope threshold range of 30 degrees to 45 degrees and the vertical distance is within the distance threshold range of 0.1 meters to 0.3 meters, the point to be classified is determined as a ground point and the irregular triangular mesh is updated until no new points are added. Using all extracted ground points, an inverse distance weighted interpolation or kriging interpolation method is employed to generate a bare soil digital elevation model with a spatial resolution of 0.1 meters.

4. The dual-stream Mamba island three-dimensional change detection method fusing LiDAR and multispectral according to claim 3, characterized in that, In step S2, the unification of the reference surface includes: Using the trajectory data recorded by the UAV, the digital elevation models of bare soil in the first and second time phases are mapped to the same coordinate system CGCS2000 and the 1985 National Elevation Datum used in the UAV operation. After the spatial alignment is completed, a pixel-level difference operation is performed on the two-period bare soil digital elevation models to generate the elevation difference map, each pixel value in the elevation difference map corresponding to an elevation difference amplitude value of the island surface at the spatial position i.e. the vertical deformation value.

5. The method for detecting three-dimensional changes in the Mamba Island using a dual-stream system integrating LiDAR and multispectral imaging as described in claim 1, characterized in that... The five-channel tensor is defined in the step S3 as , where B is a batch size, X represents input data, H and W are height and width of the input data; the first four channels of the tensor are multispectral image data containing red light R, green light G, blue light B, and near-infrared waveband , and the fifth channel is an elevation difference amplitude corresponding to an elevation difference graph , and the parallel double-tower network structure comprises: The spectral semantic stream receives multispectral image data of the first four channels, constructs a double-branch structure to obtain spectral stream features, one of which is an NDVI branch that identifies vegetation coverage changes by calculating a normalized difference vegetation index recognizes vegetation coverage changes; wherein To prevent the denominator from being zero; the second is a near-infrared identification branch that uses the reflectance difference between the near-infrared band and the green band to identify the red edge effect, thereby distinguishing between vegetation, water, bare soil, debris, and hard ground. Elevation geometric flow involves receiving elevation difference map data from the fifth channel and processing it to obtain elevation flow characteristics, including: extracting the first derivative of the elevation difference map to characterize the elevation gradient magnitude. The local variance of the elevation difference map is calculated using a sliding window of a preset size to obtain a local roughness index that characterizes the smoothness of the surface texture. The edge shape features of the elevation difference map are extracted using a convolution operator to generate an edge regularity index that characterizes the rectangularity of the ground features. The elevation geometry flow identifies local depressions left by illegal sand mining or local protrusions formed by illegal accumulation by capturing vertical abrupt changes in the terrain.

6. The method for detecting three-dimensional changes in the Mamba Island using a dual-stream system combining LiDAR and multispectral imaging as described in claim 5, characterized in that... The parallel dual-tower network structure employs an elevation gradient-guided scanning strategy when performing feature extraction. First, within the linear recursive framework of the Mamba algorithm in the state-space model, the state-space equation is defined as follows: ； in, In hidden state, Given the input sequence, For output, Indicates the sequence number; A, B, C, and D are system parameters; Secondly, the scanning strategy uses elevation gradient magnitude. Dynamically constructing the gating weight factor for selective scanning mechanism The formula is: ； in, and These are the learnable weight matrices for the elevation gradient magnitude branch and the elevation difference map data input sequence branch, respectively. The Sigmoid activation function is used; when the elevation gradient magnitude is... When the preset first geometric transition threshold is exceeded, the gating weights Increase, improve the hidden state of the model in this region. Increase the update frequency to enhance the extraction strength of terrain vertical deformation features; When the elevation gradient magnitude When the value is below the preset second geometric transition threshold, the model skips redundant data and hides the state through linear projection. Keep it unchanged to improve the processing efficiency of long sequence data.

7. The method for detecting three-dimensional changes in the Mamba Island using a dual-stream system combining LiDAR and multispectral imaging as described in claim 6, characterized in that... The first geometric jump threshold is set to 0.5 meters; the second geometric jump threshold is set in the range of 0.001-0.

01.

8. The method for detecting three-dimensional changes in the Mamba Island using a dual-stream system combining LiDAR and multispectral imaging as described in claim 1, characterized in that... In step S4, the logic discrimination module integrates a surface texture flatness regression head and a ground feature rectangularity detection head, which respectively obtain the local roughness index and edge regularity index in the elevation geometry flow, and output the normalized surface regularity value and edge regularity value. The logic discrimination is performed in combination with the high-dimensional feature map to distinguish between illegal sand mining, illegal accumulation, illegal buildings and natural evolution.

9. The method for detecting three-dimensional changes in the Mamba Island using a dual-stream system combining LiDAR and multispectral imaging as described in claim 8, characterized in that... The execution logic judgment is specifically as follows: The steps for identifying illegal sand mining activities include: When the spectral flow characteristics of the target area show "vegetation or water body turning into bare soil" and the elevation flow characteristics show "elevation drop" and the average drop depth of the target area exceeds 0.5 meters, it is determined to be illegal sand mining; After determination, the area of ​​the pit and the volume of earthwork in the region are calculated using a spatial integration algorithm based on the elevation difference map. : ； in, Indicates the pixel number. , The first and second time phases are respectively in the first and second time phases at the 1st and 2nd time phases. Elevation value of each pixel This represents the actual ground area corresponding to a single pixel. This represents the total number of pixels within the changing area. The steps for identifying the illegal stacking behavior include: When the spectral flow characteristics of the target area show "vegetation to debris or bare soil" and the elevation flow characteristics show "elevation rise" with an average rise height of more than 0.8 meters in the target area, it is determined to be illegal accumulation; Simultaneously, the peak value of the accumulation height is output, and the volume of the waste residue is calculated based on the elevation difference map. : ； The steps for identifying illegal buildings include: when the spectral flow feature is detected as vegetation turning into hard ground, and the elevation flow shows an increase in elevation, and the average surface regularity index is greater than 0.6 and the average edge regularity index is greater than 0.7, it is determined to be an illegal building, and the building outline is vectorized using the edge detection operator to calculate its footprint. The discrimination step of natural evolution includes: when a change in the spectral index NDVI is detected, but the elevation difference amplitude at the corresponding location is not... When the mean absolute value is less than the measurement error threshold of 0.1 meters, it is determined to be natural evolution.

10. A system used in the dual-stream Mamba Island three-dimensional change detection method integrating LiDAR and multispectral imaging as described in any one of claims 1-9, characterized in that, include: The data acquisition module is configured to use an unmanned aerial vehicle (UAV) integrated system to simultaneously acquire raw data of the island monitoring area in the first and second time phases. The raw data includes airborne LiDAR point cloud data with multiple echo characteristics and multispectral image data containing red, green, blue and near-infrared bands. The preprocessing module is configured to reconstruct the true surface and unify the reference surface on the original data, construct a reference base for three-dimensional detection, and generate an elevation difference map that characterizes the absolute deformation of the island surface in the vertical dimension. The dual-stream Mamba feature extraction module is configured to construct a parallel dual-tower network structure based on a state-space model. It fuses multispectral image data and elevation difference map data into a five-channel tensor input. It uses spectral semantic flow and elevation geometric flow to extract surface attribute change features and geometric deformation features, respectively. It also introduces an elevation gradient-guided scanning strategy to capture refined features in areas of abrupt terrain changes. A cross-attention mechanism is used to deeply fuse attribute change features and geometric deformation features to obtain a high-dimensional feature map, which includes spectral flow features and elevation flow features. The logic discrimination module is configured to receive the high-dimensional feature map output by the parallel dual-tower network structure, and use a logic discriminator with integrated multi-dimensional attribute constraints to perform classification and discrimination of illegal sand mining, illegal accumulation, illegal construction and natural evolution behavior and three-dimensional parameter quantization calculation.

Images