A sand mining ship identification method for large rivers, a sand mining activity dynamic monitoring method and system

Abstract

The application discloses a sand mining ship identification method, a sand mining activity dynamic monitoring method and a system for large rivers, and the sand mining ship identification method comprises the following steps: S100, processing low-resolution remote sensing images of a target river basin at multiple time sequences into low-resolution slice images; S200, performing super-resolution image reconstruction on the low-resolution slice images by using a super network to obtain high-resolution reconstructed slice images at multiple time sequences; and S300, performing sand mining ship identification on the high-resolution reconstructed slice images by using a sand mining ship instance segmentation model, and outputting multiple time sequence images containing sand mining ship identification results; the sand mining ship identification results comprise a boundary box, a target category, a confidence degree and a sand mining ship instance mask. The application can accurately identify sand mining ships with various sizes and dynamic multi-directional characteristics in large rivers, and can meet the complex scene requirements of sand mining ship monitoring in large rivers.

Description

Technical Field

[0001] This invention relates to the field of water conservancy remote sensing technology, and in particular to a method for identifying sand dredgers in large rivers, a method and system for dynamically monitoring sand dredging activities. Background Technology

[0002] With the rapid development of global urbanization, the demand for sand and gravel from construction, industry, and other sectors is constantly increasing. Sand mining activities in large rivers worldwide are widespread and dispersed; however, in practice, the amount of sand mined far exceeds the natural regeneration rate, leading to risks such as riverbed erosion, riverbank instability, saltwater intrusion, and changes in biodiversity, thus impacting the aquatic environment and ecosystems. Monitoring and assessment of sand mining activities often employ traditional methods such as manual inspections, infrared identification, and BeiDou positioning. However, these traditional methods are insufficient for long-term dynamic monitoring of large-scale sand mining activities. Summary of the Invention

[0003] To address or partially address the problems existing in related technologies, this invention provides a method for identifying sand dredgers in large rivers, a method and system for dynamically monitoring sand dredging activities, which can monitor sand dredging activities over a wide area and over a long period of time, especially the accurate identification of sand dredgers in dynamic situations.

[0004] This invention provides a method for identifying sand dredgers in large rivers, comprising:

[0005] S100: Process low-resolution remote sensing images of the target watershed from multiple time series into low-resolution slice images;

[0006] S200: A super-resolution network is used to reconstruct the low-resolution slice image to obtain a high-resolution reconstructed slice image with multiple time series.

[0007] The super-resolution network is built based on the SwinIR model and includes a shallow feature extraction module, a deep feature extraction module, and a super-resolution image reconstruction module. The deep feature extraction module includes: sequentially connected multi-layer RSTB residual blocks, and a parallel multi-scale dilated convolution connected to the last RSTB residual block. The RSTB residual block further includes: sequentially connected multi-layer self-attention STL layers, a parallel multi-scale dilated convolution connected to the last STL layer, and residual connections that directly add the input of the RSTB residual block to the output of the parallel multi-scale dilated convolution.

[0008] S300: The sand dredger instance segmentation model is used to identify sand dredgers in the high-resolution reconstructed slice image, and outputs a multi-temporal image containing the sand dredger identification results; the sand dredger identification results include bounding boxes, target categories, confidence scores, and sand dredger instance masks;

[0009] The sand dredger instance segmentation model is built based on the YOLOv11 model, including a backbone network, a neck network, and a head network. The convolutional units in the backbone network, neck network, and head network all adopt deformable convolutional layers. The deformable convolutional layers are configured as follows: first, ordinary convolutional layers are used to predict the offset and modulation weights of all sampling points in the input feature map; then, deformable convolution calculations are performed on each position of the input feature map based on the offset and modulation weights.

[0010] In some specific implementations, the super-resolution network is trained by performing transfer learning on a pre-constructed first training set; wherein, the method for constructing the first training set includes:

[0011] Acquire high-resolution and low-resolution remote sensing images of the target watershed over multiple time periods;

[0012] High-resolution remote sensing images and low-resolution remote sensing images are spatiotemporally aligned to obtain spatiotemporally aligned image pairs.

[0013] The high-resolution remote sensing images in the image pair are resampled to make the resolution of the high-resolution remote sensing images and the low-resolution remote sensing images a multiple of each other; then the high-resolution remote sensing images and the low-resolution remote sensing images are sliced, and the high-resolution slice images and the low-resolution slice images are matched based on spatial location, thereby constructing a super-resolution dataset composed of spatiotemporally aligned high-resolution slice images and low-resolution slice images.

[0014] Use the super-resolution dataset or a portion thereof as the first training set.

[0015] In some specific implementations, the backbone network includes alternating deformable convolutional modules DCN / CBS and C3k2 modules, and an SPPF module connected to the output of the last C3k2 module;

[0016] The deformable convolutional module DCN / CBS includes:

[0017] Deformable convolutional layer DCNv2;

[0018] The batch normalization layer BatchNorm2d is used to normalize the feature map output by the deformable convolutional layer DCNv2, and to scale and shift the normalized feature map.

[0019] The activation layer silu is used to perform non-linear processing on the feature map output by the batch normalization layer BatchNorm2d.

[0020] The C3k2 module includes:

[0021] The first channel number dimensionality reduction unit is used to reduce the number of channels in the input feature map using the deformable convolution module DCN / CBS;

[0022] The first path segmentation unit is used to utilize the channel segmentation function. The dimensionality-reduced feature map is divided into shallow and deep paths along the channel dimension;

[0023] The deep semantic feature extraction unit is used to extract deep semantic features from deep paths using multi-layer stacked C3K modules.

[0024] Feature splicing units are used to utilize splicing functions The extracted deep semantic features and the shallow detail features preserved in the shallow path are concatenated, and the number of channels of the concatenated feature map is adjusted to the target number of channels using the deformable convolution module DCN / CBS and then output.

[0025] The C3K module includes a multi-layer stacked standard residual bottleneck block, Bottleneck, which in turn includes a multi-layer stacked deformable convolutional module DCN / CBS and residual connections. The residual connections directly add the original input to the output of the last layer of deformable convolutional module DCN / CBS.

[0026] In some specific implementations, the neck network includes a deformable convolutional module DCN / CBS, a C3k2 module, a C2PSA module, and an UpSample module;

[0027] The C3k2 module includes:

[0028] The first channel number dimensionality reduction unit is used to reduce the number of channels in the input feature map using the deformable convolution module DCN / CBS;

[0029] The first path segmentation unit is used to segment the dimensionality-reduced feature map into shallow and deep paths along the channel dimension.

[0030] The deep semantic feature extraction unit is used to extract deep semantic features from deep paths using multi-layer stacked C3K modules.

[0031] Feature splicing units are used to utilize splicing functions The extracted deep semantic features and the shallow detail features preserved in the shallow path are concatenated, and the number of channels of the concatenated feature map is adjusted to the target number of channels using the deformable convolution module DCN / CBS and then output.

[0032] The C3K module includes a multi-layer stacked standard residual bottleneck block Bottleneck, which in turn includes a multi-layer stacked deformable convolutional module DCN / CBS and residual connections. The residual connections directly add the original input to the output of the last layer of deformable convolutional module DCN / CBS.

[0033] The C2PSA module includes:

[0034] The second channel number dimensionality reduction unit is used to reduce the number of channels in the input feature map using the deformable convolution module DCN / CBS;

[0035] The second path segmentation unit is used to split the dimensionality-reduced feature map into two branches along the channel dimension. The main branch embeds a PSA spatial attention stacking sequence at its end, while the branch is preserved.

[0036] PSA spatial attention stacking sequence is used to handle global spatial weight redistribution;

[0037] The attention weighting unit is used to concatenate the main branch and the branch, and then dynamically weight the feature map using global spatial weights. Finally, the deformable convolution module DCN / CBS is used to adjust the number of channels of the dynamically weighted feature map to the target channel and output it.

[0038] In some specific implementations, in the neck network, firstly, deformable convolutional modules DCN / CBS are used to project the number of feature channels of feature maps at different scales output by the neck network to a uniform number of channels; then, the Segment module is used to perform regression and classification decoupled branches on feature maps at different scales, including: predicting the bounding box position for the regression branch; and predicting the target class probability and confidence for the classification branch; finally, the prediction results at different scales are fused and output.

[0039] In some specific implementations, a pre-constructed second training set is used to train the sand dredger instance segmentation model; wherein, the method for constructing the second training set includes:

[0040] The super-resolution network is used to perform super-resolution image reconstruction on low-resolution slice images of the target watershed over multiple time periods, resulting in a high-resolution reconstructed slice image database over multiple time periods; the low-resolution slice images are obtained by processing low-resolution remote sensing images of the target watershed over multiple time periods.

[0041] Reconstructed tile images containing ships were selected from a high-resolution reconstructed tile image database, and the ships were manually labeled to form a labeled dataset.

[0042] The reconstructed slice images containing ships and their labeled datasets are used as the second training set.

[0043] Another aspect of the present invention provides a method for dynamic monitoring of sand mining activities in large rivers, comprising:

[0044] Based on the multi-temporal images containing sand dredger identification results obtained by the above sand dredger identification method, the number distribution of sand dredgers during the high-water season and low-water season in the same year can be statistically analyzed, or the number distribution of sand dredgers in different years can be statistically analyzed, so as to realize the temporal change monitoring of sand mining activities in the target watershed.

[0045] Combining the multi-temporal images containing sand dredger identification results obtained by the above-mentioned sand dredger identification method with remote sensing images of the target watershed, a visualized sand dredger hotspot area is drawn and overlaid on the remote sensing images to realize the spatial change monitoring of sand mining activities in the target watershed.

[0046] Based on the multi-temporal images containing the sand dredger identification results obtained by the above-mentioned sand dredger identification method, the total sand extraction volume of the target watershed is estimated using a sand extraction estimation model.

[0047] In another aspect, the present invention provides a sand dredging vessel identification system for large rivers, comprising:

[0048] The first module is used to process low-resolution remote sensing images of the target watershed from multiple time series into low-resolution slice images.

[0049] The second module is used to perform super-resolution image reconstruction on the low-resolution slice image using a super-resolution network to obtain high-resolution reconstructed slice images in multiple time series.

[0050] The super-resolution network is built based on the SwinIR model and includes a shallow feature extraction module, a deep feature extraction module, and a super-resolution image reconstruction module. The deep feature extraction module includes: sequentially connected multi-layer RSTB residual blocks, and a parallel multi-scale dilated convolution connected to the last RSTB residual block. The RSTB residual block further includes: sequentially connected multi-layer self-attention STL layers, a parallel multi-scale dilated convolution connected to the last STL layer, and residual connections that directly add the input of the RSTB residual block to the output of the parallel multi-scale dilated convolution.

[0051] The third module is used to identify sand dredgers in the high-resolution reconstructed slice image using a sand dredger instance segmentation model, and outputs multi-temporal images containing sand dredger identification results; the sand dredger identification results include bounding boxes, target categories, confidence scores, and sand dredger instance masks.

[0052] The sand dredger instance segmentation model is built based on the YOLOv11 model, including a backbone network, a neck network, and a head network. The convolutional units in the backbone network, neck network, and head network all adopt deformable convolutional layers. The deformable convolutional layers are configured as follows: first, ordinary convolutional layers are used to predict the offset and modulation weights of all sampling points in the input feature map; then, deformable convolution calculations are performed on each position of the input feature map based on the offset and modulation weights.

[0053] Compared with the prior art, the present invention may include the following beneficial effects:

[0054] This invention can effectively solve problems such as low spatiotemporal coverage of high-resolution images over large areas, insufficient detail in medium and low-resolution images, and limitations of single-phase monitoring. It can monitor large-scale, long-term sand mining activities, especially accurately identify sand dredgers in dynamics, thereby capturing the spatiotemporal migration patterns of sand dredgers, improving the timeliness of supervision, reducing supervision costs, and providing technical support for large-scale river sand mining planning and ecological protection. Attached Figure Description

[0055] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same parts in the exemplary embodiments of the invention.

[0056] Figure 1 This is a flowchart illustrating the sand dredger identification method according to an embodiment of the present invention;

[0057] Figure 2 This is an architectural diagram of the SCon-SwinIR model shown in an embodiment of the present invention;

[0058] Figure 3 This is an architectural diagram of a sand dredger instance segmentation model shown in an embodiment of the present invention;

[0059] Figure 4 This is a map showing the nuclear density distribution of sand dredgers in the middle and lower reaches of the Mekong River in 2016.

[0060] Figure 5 This is a map showing the nuclear density distribution of sand dredgers in the middle and lower reaches of the Mekong River in 2025.

[0061] Figure 6 This is a map showing the distribution of sand dredger core density in different years in the Xinzhou section of the middle and lower reaches of the Mekong River.

[0062] Figure 7 This is a map showing the nuclear density distribution of sand dredgers in the Vinh Long section of the middle and lower reaches of the Mekong River in different years. Detailed Implementation

[0063] Embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be more thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0064] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0065] See Figure 1 The present invention provides a method for identifying sand dredgers in large rivers, including steps S110 to S130.

[0066] S100: Process low-resolution remote sensing images of the target watershed from multiple time series into low-resolution slice images;

[0067] S200: A super-resolution network based on statistical constraints is used to reconstruct the low-resolution slice image to obtain a high-resolution reconstructed slice image with multiple time series.

[0068] S300: The sand dredger instance segmentation model is used to identify sand dredgers in the high-resolution reconstructed slice image, and outputs multi-temporal images containing sand dredger identification results; the sand dredger identification results include bounding boxes, target categories, confidence scores, and sand dredger instance masks.

[0069] Multi-temporal imagery refers to a sequence of images of the same scene or target acquired at different time points. In this invention, multi-temporal low-resolution remote sensing images are image sequences of the target watershed at different time points, or image sequences of the target watershed in different seasons and / or different years.

[0070] In some embodiments, the low-resolution remote sensing image can be a Sentinel-2 low-resolution remote sensing image (hereinafter referred to as Sentinel-2 image), which has a resolution of 10m and is suitable for large-scale remote sensing monitoring. Therefore, the target watershed can be a large river.

[0071] In some embodiments, the method further includes: preprocessing the low-resolution remote sensing image before processing it into a low-resolution tile image to eliminate interference from the atmosphere, sensors, etc. Preprocessing may include atmospheric correction, radiometric calibration, etc. Depending on the requirements of the super-resolution image reconstruction task of the super-resolution network, preprocessing may also include data screening, which is mainly used for cloud removal, noise reduction, and removal of abnormal images from the remote sensing image.

[0072] If the multi-time series low-resolution remote sensing image is a Sentinel-2 / Level-2A product, no preprocessing is required since the product has already undergone atmospheric correction and radiometric calibration.

[0073] In step S200, a statistically constrained super-resolution network is used to perform super-resolution image reconstruction on the low-resolution slice image (e.g., multi-temporal Sentinel-2 slice image) quarterly or yearly, to obtain multi-temporal high-resolution reconstructed slice images at different time scales. Here, Sentinel-2 slice image refers to the slice image obtained by processing Sentinel-2 images.

[0074] In this invention, the super-resolution network based on statistical constraints is constructed based on the SwinIR model, and the super-resolution network based on statistical constraints needs to be trained in advance.

[0075] In this embodiment, the training method of the super-resolution network may include:

[0076] S210: Construct the first training set;

[0077] S220: A super-resolution network based on statistical constraints is used to perform transfer learning on the first training set, thereby completing the training.

[0078] To avoid training from scratch, statistically constrained super-resolution networks can use pre-trained models, such as the SwinIR-M_x8.pth model.

[0079] The first training set consists of spatiotemporally aligned LR-HR slice image pairs, where LR and HR slice images represent low-resolution and high-resolution slice images of the target watershed, respectively.

[0080] Furthermore, the specific method for constructing the first training set is as follows:

[0081] S211: Acquire high-resolution and low-resolution remote sensing images of the target watershed over multiple time periods;

[0082] In this embodiment, the high-resolution remote sensing image is a Worldview image with a resolution of 0.3 meters; the low-resolution remote sensing image is a Sentinel-2 image with a resolution of 10 meters. Worldview images have advantages such as clear details and prominent targets, enabling the precise identification of the number and spatial distribution of sand dredgers. However, due to limitations in geographical location and acquisition methods, they suffer from problems such as small swath width, sparse coverage, and spatiotemporal fragmentation over large river areas. This makes it difficult to achieve large-scale, temporal dynamic monitoring using high-resolution images. Sentinel-2 images, on the other hand, have a short revisit period, diverse spectra, and high accessibility, making them suitable for large-scale remote sensing monitoring.

[0083] S212: Preprocess high-resolution and low-resolution remote sensing images to eliminate interference;

[0084] In this embodiment, preprocessing is performed on Worldview images, including atmospheric correction, radiometric calibration, and data screening, while only data screening is performed on Sentinel-2 images.

[0085] S213: Spatiotemporally align high-resolution remote sensing images and low-resolution remote sensing images to obtain spatiotemporally aligned image pairs, denoted as LR-HR image pairs;

[0086] In this embodiment, to ensure the consistency of target alignment between LR-HR images, the time difference between the acquisition of high-resolution remote sensing images and low-resolution remote sensing images should not exceed 3 days.

[0087] Spatiotemporal alignment further includes:

[0088] ① Using high-resolution remote sensing images as a reference, spatial registration is performed on low-resolution remote sensing images; specifically, the RPC (Rational Polynomial Coefficients) registration method is used to perform geometric registration on low-resolution images.

[0089] ② Using high-resolution remote sensing imagery as a benchmark, perform radiometric normalization on low-resolution remote sensing imagery; specifically, perform histogram matching and linear regression calibration on low-resolution remote sensing imagery to ensure that the reflectance difference between high-resolution and low-resolution remote sensing imagery after calibration is not greater than a preset threshold, for example, not greater than 5%;

[0090] ③ Perform bit depth conversion on high-resolution and low-resolution remote sensing images; specifically, map the data range of high-resolution and low-resolution remote sensing images to the compatible range of SCon-SwinIR, such as 16 bits to 8 bits (uint8, value range of 0 to 255), to avoid problems such as numerical overflow or distribution mismatch that lead to over-resolution distortion.

[0091] ④ Using the boundary of the high-resolution remote sensing image as a reference, crop the low-resolution remote sensing image so that the coverage area of ​​the cropped low-resolution remote sensing image is consistent with that of the high-resolution remote sensing image.

[0092] S214: Resample the high-resolution remote sensing imagery to achieve a resolution multiple mapping between the high-resolution and low-resolution images. Then, tile both the high-resolution and low-resolution images. Assign a spatial location and name-based mapping between the high-resolution and low-resolution tiled images to construct a super-resolution dataset. This super-resolution dataset consists of spatiotemporally aligned high-resolution and low-resolution tiled images. Use the super-resolution dataset, or a portion thereof, as the first training set. It should be noted that the aforementioned names refer to the tile sequence number, i.e., the unique identifier of the tile.

[0093] For example, the Worldview image can be downsampled to a resolution of 1.25m using the average resampling method, so that the Worldview image and the Sentinel-2 image are mapped by a factor of 8.

[0094] When tiling high-resolution and low-resolution remote sensing images, a first cropping window is used to slice the high-resolution remote sensing image into blocks, resulting in a high-resolution sliced ​​image; a second cropping window is used to slice the low-resolution remote sensing image into blocks, resulting in a low-resolution sliced ​​image; wherein the sizes of the first cropping window and the second cropping window are mapped to each other by the same multiple.

[0095] For example, Worldview images are sliced ​​using a 512×512 cropping window to obtain the HR dataset; Sentinel-2 images are sliced ​​using a 64×64 cropping window to obtain the LR dataset; the spatial locations and names of the HR slice images in the HR dataset and the LR slice images in the LR dataset are mapped one-to-one to form the LR-HR image pair dataset, i.e., the super-resolution dataset; the LR-HR image pair dataset is divided into the first training set, the first validation set, and the first test set in a 7:2:1 ratio, and the dataset is managed according to the common file format of the SwinIR model.

[0096] When training the SwinIR model, the first training set is used to train the SwinIR model, and the first validation set and the first test set are used to verify the super-resolution reconstruction performance of the trained SwinIR model.

[0097] To make the super-resolution reconstructed images more closely resemble the real scene, this invention introduces statistical constraints into the loss function of the SwinIR model, thereby constructing a statistically constrained super-resolution network, denoted as the SCon-SwinIR model, the architecture of which can be found in [link to architecture]. Figure 2 .

[0098] The SCon-SwinIR model comprises three modules: Shallow Feature Extraction, Deep Feature Extraction, and Super-Resolution Image Reconstruction (SRImage Reconstruction). The Shallow Feature Extraction module extracts shallow features from the input low-resolution slice image and outputs a shallow feature map. The Deep Feature Extraction module extracts deep features from the shallow feature map and outputs a deep feature map. The Super-Resolution Image Reconstruction module fuses the shallow and deep feature maps and performs super-resolution image reconstruction based on the fused feature map.

[0099] The shallow feature extraction module typically uses a 3×3 convolutional layer to map low-resolution slice images to a high-dimensional feature space and extract shallow features such as edges and textures to obtain a shallow feature map. .

[0100] The deep feature extraction module typically includes K sequentially connected RSTB residual blocks and a convolutional layer (Conv, e.g., a 3×3 convolutional layer) connected to the last RSTB residual block. The RSTB residual block is the core foundational module of the SwinIR model, which includes a Swin Transformer layer (i.e., STL) used to deeply mine spatial features through a multi-head self-attention mechanism.

[0101] Because the receptive field of the convolutional layer Conv is limited, it is prone to local artifacts and cannot meet the requirements of multi-scale global modeling scenarios. Therefore, in this invention, a parallel multi-scale dilated convolution AConv is used to replace the convolutional layer Conv connected to K RSTB modules. That is, in this invention, the deep feature extraction module includes K sequentially connected RSTB residual blocks and a parallel multi-scale dilated convolution AConv connected to the last RSTB residual block.

[0102] In this embodiment, the RSTB residual block consists of a self-attention layer STL, a parallel multi-scale dilated convolution AConv, and residual connections. STL is the most basic unit in the SwinIR model, and the RSTB residual block is composed of multiple stacked STLs. Specifically, the multiple STLs are connected sequentially, and the output of the last STL is connected to AConv. The residual connection adds the output of the previous RSTB residual block directly to the output of AConv.

[0103] STL stands for Self-Attention Layer, and its mathematical expression is as follows:

[0104] (1)

[0105] In formula (1): , These represent the input and output of STL, respectively. This represents the feature normalization function; This indicates multi-head self-attention in the window; This refers to a multilayer perceptron. Used to perform nonlinear transformations and information fusion on features extracted by the self-attention mechanism, the multilayer perceptron is essentially a two-layer feedforward neural network. Use later.

[0106] Multi-head self-attention window The corresponding mathematical representation is as follows:

[0107] (2)

[0108] In formula (2): This represents the i-th single-head self-attention, where i = 1, 2, ..., n. h n h For the number of heads; Represents the self-attention function; Q i K i V i The three core vectors of the self-attention mechanism represent the query vector, key vector, and value vector, respectively. The processed feature map is obtained through linear transformation and then segmented into... The input feature map has 100 channels (head); C represents the number of channels in the input feature map. Represents the i-th single-head dimension; This represents the relative position offset matrix of the i-th single head; Responsible for calculating the query vector of the i-th single head. With each key vector Similarity, superscript This represents the transpose operator; Used to convert similarity values ​​into attention weights; Used to integrate multiple single-head self-attention splicing and fusion; This indicates a 1×1 convolutional layer used to calibrate the spliced ​​feature dimensions.

[0109] In the parallel multi-scale dilated convolution AConv, each dilated convolution branch independently extracts features at a single scale, and then all extracted features are concatenated into a multi-scale feature map. By fusing receptive field features at different scales, parallel multi-scale dilated convolution can solve the problem of insufficient global modeling capability of the SwinIR model without increasing the number of parameters.

[0110] The mathematical expression for parallel multi-scale dilated convolution is as follows:

[0111] (3)

[0112] In formula (3): This represents the feature map output by AConv; Let t represent the features extracted by the t-th dilated convolution branch, where t = 1, 2, ..., MT; MT represents the number of parallel branches. This indicates that the t-th dilated convolution branch is at the th... line, number The output pixel value of the c-th output channel; The kernel weights of the t-th dilated convolutional branch are learned by the model.

[0113] For the t-th dilated convolution branch at the th... line, number The convolution kernel weights for the d-th input channel, where c is used to label the output channels; convolution kernel weights Used with feature maps Multiplication and addition;

[0114] Represents the input feature map The value of the original pixel position sampled by dilated convolution in the d-th input channel; input feature map. That is, the output of the last RSTB residual block or the output of the last STL;

[0115] , These represent the row and column positions sampled by the dilated convolution kernel on the input feature map Ψ, respectively; where, , These represent the row and column positions of the dilated convolution kernel, respectively. , represents the fill value and the dilation rate of the t-th dilated convolution branch, respectively;

[0116] d represents the input channel index; c represents the output channel index; Indicates the size of the dilated convolution kernel; Represents the input feature map The channel dimension.

[0117] The super-resolution image reconstruction module reconstructs high-resolution reconstructed slice images by fusing shallow and deep feature maps and then performing upsampling and convolution operations on the fused feature maps in sequence, so as to ensure that image details are not lost.

[0118] The mathematical expression for the super-resolution image reconstruction module is as follows:

[0119] (4)

[0120] In equation (4): Represents the fused feature map. Represents deep feature maps. Represents a shallow feature map; Indicates the upsampling function; This indicates the sampling factor, which is the magnification factor by which a low-resolution slice image is enlarged to a high-resolution reconstructed slice image; This represents the upsampling result of the fused feature map; This indicates a high-resolution reconstructed slice image.

[0121] When training the SwinIR model, the L1 pixel loss function is generally used as a constraint. The core of the L1 pixel loss function is to calculate the absolute error between the predicted value and the true value pixel by pixel. Since the L1 pixel loss function only focuses on pixel-level error, it is prone to local artifacts, affecting the representation of target details. This embodiment introduces mean and variance constraint losses on the basis of the L1 pixel loss function, taking into account both pixel-level accuracy and global statistical features, so that the reconstructed image is closer to the real scene.

[0122] In this embodiment, the loss function is used when training the SwinIR model. L1 pixel loss function and statistical constraint loss function The composition is detailed in the following formula:

[0123] (5)

[0124] In equation (5): This represents the j-th high-resolution reconstructed slice image output by the model; Indicates and The corresponding j-th real high-resolution slice image; N represents the batch number; Represents the L1 norm, used to calculate and The sum of absolute pixel errors; , Let represent the pixel mean of the j-th high-resolution reconstructed slice image and the j-th real high-resolution slice image, respectively. , Let represent the pixel variances of the j-th high-resolution reconstructed slice image and the j-th real high-resolution slice image, respectively. , represent the loss weights for mean constraint and variance constraint, respectively, which are the learnable parameters of the super-resolution network; This represents the loss weight of the statistical constraint.

[0125] Among them, the high-resolution reconstructed slice image and the real high-resolution slice image are a set of corresponding reconstructed images and real images.

[0126] Statistical constraint loss function The loss is a weighted sum of mean loss and variance loss, where the mean loss is the mean loss of the high-resolution reconstructed slice image and the real high-resolution slice image, and the variance loss is the variance loss of the high-resolution reconstructed slice image and the real high-resolution slice image.

[0127] In step S300, the sand dredger instance segmentation model is constructed based on the YOLOv11 model. Considering the diverse sizes and dynamic multi-directional characteristics of sand dredgers, this invention improves the existing YOLOv11 model to enable accurate identification of sand dredgers. Specifically, improvements are made to the feature extraction performance. The architecture of the improved YOLOv11 model is shown below. Figure 3 .

[0128] Similar to the YOLOv11 model in the prior art, in this embodiment, the sand dredger instance segmentation model includes a backbone network (denoted as Backbone), a neck network (denoted as Neck), and a head network (denoted as Head). The backbone network is used to extract multi-scale features from the input high-resolution reconstructed slice image. The neck network is used to fuse the multi-scale features, achieving efficient multi-scale feature fusion through a bidirectional path from top to bottom and bottom to top. The head network is used to identify sand dredgers from the multi-scale fused feature map output by the neck network, and outputs bounding boxes, target categories, confidence scores, and sand dredger instance masks.

[0129] The backbone network typically consists of Conv / CBS modules, C3k2 modules, and SPPF modules (Spatial Pyramid Pooling Fusion). Among them, the Conv / CBS module is the most basic and core convolutional unit in the YOLOv11 model. Its receptive field is fixed and relatively small, making it unable to capture the overall outline of medium to large ship hulls in one go. It also lacks directional robustness and is poorly adapted to dynamically multi-directional sand dredgers.

[0130] To address the shortcomings of poor adaptability to sand dredgers, this invention introduces a deformable convolutional layer DCNv2 based on the existing YOLOv11 model. Specifically, the original Conv / CBS module in the YOLOv11 model is replaced with a deformable convolutional module DCN / CBS. The core of the deformable convolutional layer DCNv2 is to add learnable offsets and weights to the sampling points of the convolution kernel, making it suitable for identifying sand dredger targets with varying orientations.

[0131] Furthermore, the deformable convolutional module DCN / CBS consists of three parts: the deformable convolutional layer DCNv2, the batch normalization layer BatchNorm2d, and the activation layer silu.

[0132] The calculation principle of the deformable convolutional layer DCNv2 is as follows: First, a regular convolutional layer is used to predict the offset and modulation weight of all sampling points on the input feature map X; then, deformable convolutional kernels are calculated at each position of the feature map X.

[0133] The mathematical expressions corresponding to the offset of the above predicted sampling points and the modulation weights are as follows:

[0134] (6)

[0135] In formula (6): Indicates the offset of the sampling point; Indicates the modulation weight of the sampling point; This represents a 1×1 offset convolutional layer used to predict the offset of sampling points; This represents a 1×1 modulation convolutional layer used to predict the modulation weights of the sampling points; This represents the Sigmoid activation function.

[0136] The mathematical expression corresponding to the above deformable convolution kernel calculation is as follows:

[0137] (7)

[0138] In equation (7): Indicates the position on the output feature map The output value; position The position of the sampling point z can be expressed as , , Indicates the variable kernel size; This represents the modulation weight of sampling point z; The convolution weights at sampling point z are learnable parameters. Represents the eigenvalues ​​of the sampling point z; , indicating the standard reference position of the convolution kernel; In standard convolution, the sampling point z is relative to... Fixed offset; This represents the offset of the sampling point z; This indicates the position of the sampling point after the z-off; It is a bilinear interpolation function.

[0139] Therefore, the deformable convolutional layer DCNv2 is used to predict the offset and modulation weights of all sampling points in the input feature map, and to perform deformable convolutional kernel calculations on each sampling point using the offset and modulation weights.

[0140] The batch normalization layer BatchNorm2d is used to normalize the feature map output by the deformable convolutional layer DCNv2, and to scale and shift the normalized feature map to solve the internal covariate offset problem.

[0141] The activation layer silu is used to perform non-linear processing on the feature maps output by the batch normalization layer BatchNorm2d to improve the model's ability to learn complex patterns.

[0142] In the deformable convolutional module DCN / CBS, the deformable convolutional layer DCNv2 is the core unit, which is used to extract feature maps of adaptive targets more accurately. Then, the feature maps output by the deformable convolutional layer DCNv2 are passed through batch normalization layers and activation layers in sequence to enhance feature representation.

[0143] In the backbone network, the DCN / CBS module and the C3k2 module are connected alternately, and the output of the last C3k2 module is connected to the SPPF module. The C3k2 module has a dual-path branch structure, which achieves efficient deep feature extraction through different kernel sizes and channel separation strategies; the SPPF module expands the receptive field and captures features at different scales by introducing multi-scale feature map pooling.

[0144] In this embodiment, the C3k2 module further includes:

[0145] The first channel number dimensionality reduction unit is used to reduce the number of channels in the input feature map using the deformable convolution module DCN / CBS, so as to reduce the amount of subsequent computation.

[0146] The first path segmentation unit is used to utilize the channel segmentation function. The dimensionality-reduced feature map is divided into shallow and deep paths along the channel dimension;

[0147] The deep semantic feature extraction unit is used to extract deep semantic features from deep paths using multi-layer stacked C3K modules.

[0148] Feature splicing units are used to utilize splicing functions The extracted deep semantic features and the shallow detail features preserved in the shallow path are concatenated, and the number of channels of the concatenated feature map is adjusted to the target number of channels using the deformable convolution module DCN / CBS before output.

[0149] Furthermore, the C3K module is the basic residual module of the YOLOv11 model. In this invention, the convolutional units in the C3K module all adopt deformable convolutional modules DCN / CBS. The C3K module is mainly composed of multiple layers of standard residual bottleneck blocks, and the standard residual bottleneck block includes multiple layers of deformable convolutional modules DCN / CBS and residual connections. The multiple layers of deformable convolutional modules DCN / CBS are connected sequentially, and the residual connections directly add the original input to the output of the last layer of deformable convolutional modules DCN / CBS.

[0150] The core of the SPPF module is a nested pooling branch structure. In this embodiment, the mathematical expression of the nested pooling branch is as follows:

[0151] (8)

[0152] In equation (8): Indicates the input feature map; For branch 1, the identity mapping; For branch 2, perform one 5×5 pooling operation; For branch 3, perform 2 rounds of 5×5 pooling; For branch 4, perform 3 5×5 pooling operations; Indicates the size of the deformable convolution kernel; Indicates the stride of the deformable convolution kernel; This indicates the number of padding pixels at the top and bottom of the input feature map; This is the pooling function.

[0153] The neck network typically consists of Conv / CBS modules, C3k2 modules, C2PSA modules, and UpSample modules. Because the Conv / CBS modules have a fixed and relatively small receptive field, deformable convolutional modules (DCN / CBS) are used to replace the original Conv / CBS modules in the neck network. The C2PSA module receives the output of the SPPF module in the backbone network.

[0154] In the neck network, the C2PSA module integrates the dual-channel branching of C2f with the PSA spatial attention mechanism. In this embodiment, the C2PSA module further includes:

[0155] The second channel number dimensionality reduction unit is used to reduce the number of channels in the input feature map using the deformable convolution module DCN / CBS, so as to reduce the amount of subsequent computation.

[0156] The second path segmentation unit is used to utilize the channel segmentation function. The dimensionality-reduced feature map is split into two branches along the channel dimension. The main branch is embedded with a PSA spatial attention stacking sequence at its end, while the branch is preserved.

[0157] PSA spatial attention stacking sequence is used to handle global spatial weight redistribution in order to adaptively enhance the target region and suppress invalid background.

[0158] The attention weighting unit is used to concatenate the main branch and the branch, and then use global spatial weights to dynamically weight the feature map to enhance the response of important channels. Finally, the deformable convolution module DCN / CBS is used to adjust the number of channels of the dynamically weighted feature map to the target channel and output it.

[0159] In the neck network, the UpSample module is the core module. It is used to upsample the feature map size from H×W to rH×rW (r is the upsampling factor, for example, r=2) without changing the number of channels. Then, the newly added pixel values ​​are filled by the bilinear interpolation algorithm. Upsampling only changes the size, while the number of channels remains unchanged. Moreover, the UpSample module is a pure interpolation module with no trainable parameters such as weights or biases.

[0160] In the head network, firstly, the feature channel number of the multi-scale fusion feature map (e.g., 3-scale fusion feature map) output by the neck network is projected to a uniform channel number through the deformable convolutional module DCN / CBS, so as to facilitate subsequent decoupled branch calculation.

[0161] Then, using the Segment module, the regression and classification branches are decoupled. The regression and classification decoupling branches include: for the regression branch (see...) Figure 3 The left branch of the Segment module shown in the diagram predicts the offset of the bounding box. And determine the bounding box coordinates based on the offset, where, , , , These represent the x-coordinate offset, y-coordinate offset, width offset, and height offset, respectively; for the classification branch (see... Figure 3 (The right branch of the Segment module shown in the diagram) predicts the class probability and confidence.

[0162] The mathematical expression for determining the bounding box coordinates is as follows:

[0163] (9)

[0164] In equation (9): Indicates the first The bounding box coordinates (x1, y1, x2, y2) are used in this embodiment. ; Indicates the first Sampling magnification of scale; H f W f Indicates the first The height and width are input for the scale. This represents the Sigmoid activation function.

[0165] Finally, the prediction results from each scale are fused and output, and their mathematical expression is as follows:

[0166] (10)

[0167] In formula (10): This indicates a channel dimension concatenation operation; This indicates a spatial dimension concatenation operation; Indicates the first The scale-based classification branch output includes the target class probability and confidence level.

[0168] The sand dredger instance segmentation model needs to be trained in advance. In this embodiment, to avoid training from scratch, the pre-trained model yolo11n-seg.pt is selected for the sand dredger instance segmentation model; the pre-built second training set is used to train the pre-trained model yolo11n-seg.pt.

[0169] The method for constructing the second training set is as follows:

[0170] A super-resolution network based on statistical constraints is used to perform super-resolution image reconstruction on multi-temporal low-resolution slice images (such as multi-temporal Sentinel-2 slice images) quarterly and yearly to obtain a multi-temporal high-resolution reconstructed slice image database, denoted as the M_SRI database.

[0171] 5000 reconstructed slice images containing ships were selected from the M_SRI database. The ships in the slice images were manually labeled using Labelme software to form a labeled dataset and then the format was converted.

[0172] To enrich the data, data augmentation processing (including but not limited to random flipping, rotation, elastic transformation, etc.) was performed on the reconstructed slice images and their labels in the label dataset; the label dataset was divided into a second training set, a second validation set, and a second test set in a ratio of 7:2:1.

[0173] Sand dredgers can be categorized into chain bucket dredgers and suction pump dredgers. The bow shows a dredging arm or dredging pipe, and the deck is equipped with a conveyor belt and screening equipment. Operation is accompanied by loud mechanical noise and water churning. Since sand dredgers themselves do not have the capacity to transport sand and gravel, they are often deployed alongside multiple sand transport vessels. Therefore, in some embodiments, the following three indicators are used to comprehensively determine the type of sand dredger. Specifically, these three indicators are used to distinguish between sand dredgers and sand transport vessels, and further to differentiate between chain bucket dredgers and suction pump dredgers:

[0174] One aspect is the shape and texture. Sand dredgers are mostly composed of multiple colors, which is significantly different from the single color exposed at the bottom of transport ships, giving them a mottled texture in images.

[0175] Secondly, water transparency. When sand dredgers operate, they disturb the surrounding water, making it turbid. The difference in water transparency around the dredger can be used as one of the distinguishing indicators between sand dredgers and transport ships.

[0176] Thirdly, there is the positional relationship. Suction pump dredgers generally operate within 100m of the riverbank, chain bucket dredgers operate outside the waterway, and transport ships usually travel within the waterway. Therefore, the distance between the ship and the riverbank and the positional relationship between the ship and the electronic waterway map can be used to help determine the ship type.

[0177] When training a YOLOv11 model, its loss function is generally a weighted sum of bounding box regression loss, confidence loss, and classification loss. The bounding box regression loss of the YOLOv11 model typically uses the CIoU Loss function. However, the CIoU Loss function is sensitive to the shape of the anchor boxes and non-overlapping boxes, and it has poor adaptability to dynamically unoriented targets. Therefore, in this embodiment, the Normalized Cosine Loss function (NCIoU Loss) is used instead of the CIoU Loss function. The Normalized Cosine Loss function takes into account factors such as overlap, aspect ratio, center distance, and direction vector, making it more suitable for dynamically unoriented sand dredgers.

[0178] The normalized cosine loss function (NCIoU Loss) represents the mathematical expression of the bounding box regression loss as follows:

[0179] (11)

[0180] In equation (11): Indicates the first The regression loss of each bounding box takes into account overlap, aspect ratio, orientation, and center distance. Indicates the first The intersection-union ratio of bounding boxes is used to characterize the degree of overlap between the predicted boxes and the ground truth boxes; Indicates the first The normalized cosine distance of each bounding box, which is the normalized cosine distance between the center of the predicted box and the center of the ground truth box, takes into account both the center distance and direction, and is suitable for dynamic and non-directional targets of sand dredging vessels. Indicates that it contains the first The diagonal length of the minimum bounding rectangle of each predicted bounding box and the ground truth bounding box; Indicates the first Aspect ratio consistency coefficient between predicted bounding boxes and ground truth bounding boxes; This represents the total loss of the bounding box, i.e., the bounding box regression loss. , represents an indicator function, where 1 represents the first... The first bounding box is a positive sample, and 0 represents the first bounding box. Each bounding box is a negative sample; Indicates the total number of samples in the batch; This represents the total number of positive samples within the batch.

[0181] In the CIoU Loss loss function, Euclidean distance is used to measure the distance between the center of the predicted bounding box and the center of the ground truth bounding box; this distance is the center distance. The center distance measured by Euclidean distance is sensitive to the shape of the anchor box and non-overlapping boxes. During training, the model indiscriminately treats offset errors on the x and y axes, leading to a decrease in the accuracy of bounding box regression for non-square targets such as sand dredgers. Furthermore, Euclidean distance does not consider the orientation and angle of the bounding box, resulting in poor adaptation to rotated targets. This leads to a significant decrease in the accuracy of bounding box regression for dynamically non-directional targets such as sand dredgers, and the target loss accounts for an excessively high proportion in dense scenes. Therefore, normalized cosine distance is used to measure the center distance between the predicted bounding box and the ground truth bounding box.

[0182] Among them, the normalized cosine distance measures the center distance. The mathematical expression is as follows:

[0183] (12)

[0184] (13)

[0185] In equations (12) to (13): This represents the center distance based on the normalized cosine distance metric. Indicates the coordinates of the center of the l-th prediction box; Indicates the coordinates of the center of the l-th ground truth box; , These represent the height and width of the image, respectively. , They represent axis, Axial direction vector; , These represent the length and width of the actual bounding box, respectively. , These represent the length and width of the prediction box, respectively; , These represent the rotation angles of the predicted bounding box and the ground truth bounding box relative to the horizontal axis, respectively. This represents the operation of vector magnitude.

[0186] The invention employs normalized cosine distance instead of traditional Euclidean distance, which yields the following beneficial effects: 1. Normalizing the offset first maps offsets of different scales to the same scale space, eliminating scale sensitivity and overcoming gradient imbalance problems for targets of different sizes; 2. By calculating the cosine similarity between the offset vector and the direction basis vector, the principal axis directions of the predicted box and the ground truth box are forced to align, avoiding the problem that Euclidean distance only considers coordinate differences and ignores the orientation and shape of the bounding box. Therefore, compared to using traditional Euclidean distance, normalized cosine distance is more suitable for slender targets with high aspect ratios and multiple tilt angles, such as sand dredgers.

[0187] Furthermore, in this embodiment, the confidence loss uses the binary cross-entropy loss function (BCE Loss) to distinguish between positive samples (with a target) and negative samples (without a target), and Focal Loss weights are introduced to alleviate the imbalance between positive and negative samples. Also, in this embodiment, the classification loss is multi-label classification, using the binary cross-entropy loss function (BCE Loss), and only positive samples (with a target) are calculated.

[0188] In this embodiment, the total loss function of the YOLOv11 model Configured as bounding box regression loss Confidence loss Classification loss The weighted sum.

[0189] Total loss function The mathematical expression is as follows:

[0190] (16)

[0191] In equation (16): The bounding box regression loss is respectively Confidence loss Classification loss The weighting coefficients are used to balance the loss magnitudes of different tasks by adjusting the weighting coefficients.

[0192] In the YOLOv11 model, The default values ​​are 7.5, 1.0, and 0.5.

[0193] By inputting multi-time-series high-resolution reconstructed slice images into the trained YOLOv11 model, intelligent identification of sand dredgers is performed quarterly and / or yearly, resulting in large-scale, long-time-series intelligent identification results of sand dredgers. These results can reflect the spatiotemporal variation patterns of sand dredgers at different time and spatial scales.

[0194] Clearly, common low-to-medium resolution remote sensing images offer wide swaths, high timeliness, and low data acquisition barriers, making them suitable for large-scale dynamic monitoring. However, their limited spatial resolution restricts target precision and accuracy, resulting in existing sand mining monitoring largely being confined to macroscopic monitoring levels such as suspended sediment concentration inversion. Furthermore, considering the diverse sizes and dynamic, multi-directional characteristics of sand dredgers in rivers, there are significant shortcomings in key aspects such as conventional super-resolution reconstruction and sand dredger target identification, which could meet the complex scenario requirements for monitoring large river sand dredgers.

[0195] In this embodiment of the invention, a statistically constrained super-resolution network is first used to reconstruct multi-temporal low-resolution slice images of the target watershed, resulting in corresponding multi-temporal high-resolution reconstructed slice images. By introducing statistical constraint loss into the super-resolution network, pixel-level accuracy and global statistical features can be balanced, making the reconstructed images more realistic. This effectively overcomes the limitations of low resolution on the fineness and accuracy of the target, and is especially suitable for large rivers. Then, a sand dredger instance segmentation model based on the YOLOv11 model is used to perform dynamic target identification of sand dredgers, resulting in multi-temporal images containing sand dredger identification results. The obtained multi-temporal images containing sand dredger identification results can provide a basis for monitoring sand mining activities in the target watershed.

[0196] This invention also provides a method for dynamic monitoring of sand mining activities in large rivers, comprising:

[0197] S400: Based on multi-temporal images of the target watershed containing sand dredger identification results, it statistically analyzes the number distribution of sand dredgers during the high-water and low-water seasons of the same year, or the number distribution of sand dredgers in different years, to achieve temporal variation monitoring of sand mining activities in the target watershed.

[0198] In addition, the above-mentioned methods for dynamic monitoring of sand mining activities also include:

[0199] S500: Combining multi-temporal images and remote sensing images containing sand dredger identification results of the target watershed, a visualized sand dredger hotspot area is drawn and overlaid on the remote sensing image to realize spatial change monitoring of sand mining activities in the target watershed.

[0200] In addition, the above-mentioned methods for dynamic monitoring of sand mining activities also include:

[0201] S600: Estimate the total sand extraction volume of the target watershed based on multi-temporal images containing sand dredger identification results.

[0202] In this embodiment, a sand mining estimation model is used to estimate the total sand mining volume. The mathematical expression of the sand mining estimation model is as follows:

[0203] (17)

[0204] In equation (17): This represents the total sand extraction volume of the target watershed; Indicates the serial number of the sand dredger. This indicates the total number of sand dredgers; Indicates sand dredger The capacity conversion factor is an empirical value and is related to the ship type. For sand dredgers The length of the ship is calculated using the masking method of the sand dredging vessel example; Indicates sand dredger The ship type correction factor is an empirical value used to correct for differences in sand mining efficiency among different ship types. Indicates sand dredger Daily effective working hours; Indicates sand dredger The work efficiency coefficient is an empirical value. This indicates the number of days the sand dredger operates.

[0205] In practical applications, the capacity conversion coefficients for small, medium, and large chain bucket dredgers range from 0.8 to 1.0 t / (m·h), 1.0 to 1.2 t / (m·h), and 1.2 to 1.5 t / (m·h), respectively; while the capacity conversion coefficients for small, medium, and large suction pump dredgers range from 1.2 to 1.5 t / (m·h), 1.5 to 1.8 t / (m·h), and 1.8 to 2.2 t / (m·h), respectively.

[0206] In this embodiment, the vessel size is distinguished according to the conventions of engineering vessels. The lengths of small, medium and large chain bucket dredgers are as follows: no more than 20m, 20~35m, and no less than 35m. The lengths of small, medium and large suction pump dredgers are as follows: no more than 15m, 15~30m, and no less than 30m.

[0207] In this embodiment, the hull form correction coefficients for small, medium, and large chain bucket dredgers are 0.8, 1.0, and 1.15, respectively, and are dimensionless; the hull form correction coefficients for small, medium, and large suction pump dredgers are 0.85, 1.0, and 1.25, respectively, and are dimensionless.

[0208] Based on the general values ​​for dredging / sand excavation machinery in the industry, the operating efficiency coefficients for small, medium, and large chain bucket dredgers are generally 0.6, 0.78, and 0.9, respectively; while the operating efficiency coefficients for small, medium, and large suction pump dredgers are generally 0.58, 0.72, and 0.85, respectively.

[0209] To verify the accuracy of this invention in identifying large river sand dredgers and estimating sand dredging volume, experiments and demonstrations were conducted using the middle and lower reaches of the Mekong River as an example.

[0210] High-resolution Worldview imagery of the middle and lower reaches of the Mekong River was acquired, covering 76 Worldview image fragments from 2014 to 2025. Low-resolution Sentinel-2 imagery was also acquired, covering 203 Sentinel-2 image fragments from the dry and wet seasons of 2016 to 2025. First, a SwinIR-M_x8.pth model was trained based on a subset of Sentinel-2 image fragments and their corresponding Worldview image fragments. This trained model was then used to perform super-resolution image reconstruction on the remaining Sentinel-2 image fragments. Next, a yolo11n-seg.pt model was trained based on the reconstructed images from a subset of the super-resolution images. This trained yolo11n-seg.pt model was then used to extract sand dredger instance masks from the remaining reconstructed image fragments.

[0211] Based on the sand dredger identification results, kernel density distribution maps of sand dredgers in different years were drawn for the middle and lower reaches of the Mekong River. (See attached maps.) Figures 4-5 ,in, Figure 4 This is a map showing the nuclear density distribution of sand dredgers in the middle and lower reaches of the Mekong River in 2016. Figure 5 This is a map showing the nuclear density distribution of sand dredgers in the middle and lower reaches of the Mekong River in 2025.

[0212] Based on the sand dredger identification results, a kernel density distribution map of sand dredgers in different years was drawn for the Xinzhou section of the Qianjiang River in the middle and lower reaches of the Mekong River. Figure 6 ,in, Figure 6 (a) ~ Figure 6 (j) are the distribution maps of the core density of sand dredgers in 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024 and 2025, respectively.

[0213] Based on the sand dredger identification results, a kernel density distribution map of sand dredgers in different years was drawn for the Vinh Long section of the Tien River in the middle and lower reaches of the Mekong River. Figure 7 ,in, Figure 7 (a) ~ Figure 7 (i) are the distribution maps of the core density of sand dredgers in 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2024 and 2025, respectively.

[0214] Based on the results of satellite remote sensing identification of sand dredgers, and taking into account factors such as the number, size and working capacity of sand dredgers, the model shown in Equation (17) is used to estimate the average annual sand dredging volume in the middle and lower reaches of the Mekong River from 2016 to 2025 to be about 73 million tons, of which about 18 million tons are from Kampong Cham to Xinzhou and Zhu Du, and about 55 million tons are from Xinzhou and Zhu Du downstream.

[0215] Clearly, this invention solves the technical bottleneck of tracking large-scale, long-term sand mining activities and can provide technical support for global sand resource sustainability assessment and illegal sand mining supervision.

[0216] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for identifying sand dredgers in large rivers, characterized in that, include: S100: Process low-resolution remote sensing images of the target watershed from multiple time series into low-resolution slice images; S200: A super-resolution network is used to reconstruct the low-resolution slice image to obtain a high-resolution reconstructed slice image with multiple time series. The super-resolution network is built based on the SwinIR model and includes a shallow feature extraction module, a deep feature extraction module, and a super-resolution image reconstruction module. The deep feature extraction module includes: sequentially connected multi-layer RSTB residual blocks, and a parallel multi-scale dilated convolution connected to the last RSTB residual block. The RSTB residual block further includes: sequentially connected multi-layer self-attention STL layers, a parallel multi-scale dilated convolution connected to the last STL layer, and residual connections that directly add the input of the RSTB residual block to the output of the parallel multi-scale dilated convolution. S300: The sand dredger instance segmentation model is used to identify sand dredgers in the high-resolution reconstructed slice image, and outputs a multi-temporal image containing the sand dredger identification results; the sand dredger identification results include bounding boxes, target categories, confidence scores, and sand dredger instance masks; The sand dredger instance segmentation model is built based on the YOLOv11 model, including a backbone network, a neck network, and a head network. The convolutional units in the backbone network, neck network, and head network all adopt deformable convolutional layers. The deformable convolutional layers are configured as follows: first, ordinary convolutional layers are used to predict the offset and modulation weights of all sampling points in the input feature map; then, deformable convolution calculations are performed on each position of the input feature map based on the offset and modulation weights.

2. The sand dredger identification method as described in claim 1, characterized in that: The super-resolution network is trained by performing transfer learning on a pre-constructed first training set; wherein, the method for constructing the first training set includes: Acquire high-resolution and low-resolution remote sensing images of the target watershed over multiple time periods; High-resolution remote sensing images and low-resolution remote sensing images are spatiotemporally aligned to obtain spatiotemporally aligned image pairs. The high-resolution remote sensing images in the image pair are resampled to make the resolution of the high-resolution remote sensing images and the low-resolution remote sensing images a multiple of each other; then the high-resolution remote sensing images and the low-resolution remote sensing images are sliced, and the high-resolution slice images and the low-resolution slice images are matched based on spatial location, thereby constructing a super-resolution dataset composed of spatiotemporally aligned high-resolution slice images and low-resolution slice images. Use the super-resolution dataset or a portion thereof as the first training set.

3. The sand dredger identification method as described in claim 2, characterized in that: When training the super-resolution network, a loss function is used. As a constraint; among which... , where is the statistical constraint loss function; above, L1 pixel loss function; N represents batch number; , Let represent the pixel mean of the j-th high-resolution reconstructed slice image and the j-th real high-resolution slice image, respectively. , Let represent the pixel variances of the j-th high-resolution reconstructed slice image and the j-th real high-resolution slice image, respectively. , These represent the loss weights for mean constraint and variance constraint, respectively, and are learnable parameters; The loss weights representing statistical constraints are: and sum.

4. The sand dredger identification method as described in claim 1, characterized in that: The backbone network includes alternating deformable convolutional modules DCN / CBS and C3k2 modules, as well as an SPPF module connected to the output of the last C3k2 module; The neck network includes a deformable convolutional module DCN / CBS, a C3k2 module, a C2PSA module, and an UpSample module; In the head network, firstly, deformable convolutional modules DCN / CBS are used to project the feature channel number of the feature maps of different scales output by the neck network to a uniform channel number; Then, the Segment module is used to decouple the regression and classification branches for feature maps at different scales, including: predicting the bounding box position for the regression branch; and predicting the target class probability and confidence for the classification branch; finally, the prediction results at different scales are fused and output. Among them, the convolutional units in the C3k2 module, SPPF module, and C2PSA module all adopt deformable convolutional modules DCN / CBS; The deformable convolutional module DCN / CBS includes: Deformable convolutional layer DCNv2; The batch normalization layer BatchNorm2d is used to normalize the feature map output by the deformable convolutional layer DCNv2, and to scale and shift the normalized feature map. The activation layer silu is used to perform non-linear processing on the feature maps output by the batch normalization layer BatchNorm2d.

5. The sand dredger identification method as described in claim 1, characterized in that: The sand dredger instance segmentation model is trained using a pre-constructed second training set; wherein, the method for constructing the second training set includes: The super-resolution network is used to perform super-resolution image reconstruction on low-resolution slice images of the target watershed over multiple time periods, resulting in a high-resolution reconstructed slice image database over multiple time periods; the low-resolution slice images are obtained by processing low-resolution remote sensing images of the target watershed over multiple time periods. Reconstructed tile images containing ships were selected from a high-resolution reconstructed tile image database, and the ships were manually labeled to form a labeled dataset. The reconstructed slice images containing ships and their labeled datasets are used as the second training set.

6. The sand dredger identification method as described in claim 5, characterized in that: When training the sand dredger instance segmentation model, the total loss function is configured as a weighted sum of bounding box regression loss, confidence loss, and classification loss; in the bounding box regression loss function, normalized cosine distance is used to measure the center distance between the predicted box and the ground truth box. Among them, the normalized cosine distance The calculation formula is as follows: ； The above, Indicates the coordinates of the center of the prediction box; Represents the coordinates of the center of the ground truth box corresponding to the predicted box; , These represent the height and width of the image, respectively. , They represent axis, Axial direction vector, , ; , These represent the length and width of the actual bounding box, respectively. , These represent the length and width of the prediction box corresponding to the prediction box, respectively; , These represent the rotation angles of the predicted bounding box and the ground truth bounding box relative to the horizontal axis, respectively. This represents the operation of vector magnitude.

7. A method for dynamic monitoring of sand mining activities in large rivers, characterized in that, include: Based on the multi-temporal images containing sand dredger identification results obtained by any one of the sand dredger identification methods in claims 1 to 6, the number distribution of sand dredgers during the high-water season and low-water season in the same year is statistically analyzed, or the number distribution of sand dredgers in different years is statistically analyzed, so as to realize the temporal change monitoring of sand mining activities in the target watershed.

8. A method for dynamic monitoring of sand mining activities in large rivers, characterized in that, include: Combining the multi-temporal images containing sand dredger identification results obtained by any one of the sand dredger identification methods in claims 1 to 6, and the remote sensing images of the target watershed, a visualized sand dredger hotspot area is drawn and overlaid on the remote sensing images to realize the spatial change monitoring of sand mining activities in the target watershed.

9. A method for dynamic monitoring of sand mining activities in large rivers, characterized in that, include: Based on the multi-temporal images containing sand dredger identification results obtained by any one of the sand dredger identification methods in claims 1 to 6, the total sand extraction volume of the target watershed is estimated using a sand extraction estimation model. The mathematical expression of the sand mining estimation model is as follows: ； in: Indicates the total sand extraction volume; Indicates the serial number of the sand dredger. This indicates the total number of sand dredgers; Indicates sand dredger The capacity conversion factor is an empirical value and is related to the ship type. For sand dredgers The length of the ship is calculated using the masking method of the sand dredging vessel example; Indicates sand dredger The ship type correction factor is an empirical value used to correct for differences in sand mining efficiency among different ship types. Indicates sand dredger The water depth correction factor is an empirical value that is related to water depth and is used to reflect the impact of river water depth on sand mining operations. Indicates sand dredger Daily effective working hours; Indicates sand dredger The work efficiency coefficient; This indicates the number of days the sand dredger operates.

10. A sand dredging vessel identification system for large rivers, characterized in that, include: The first module is used to process low-resolution remote sensing images of the target watershed from multiple time series into low-resolution slice images. The second module is used to perform super-resolution image reconstruction on the low-resolution slice image using a super-resolution network to obtain high-resolution reconstructed slice images in multiple time series. The super-resolution network is built based on the SwinIR model and includes a shallow feature extraction module, a deep feature extraction module, and a super-resolution image reconstruction module. The deep feature extraction module includes: sequentially connected multi-layer RSTB residual blocks, and a parallel multi-scale dilated convolution connected to the last RSTB residual block. The RSTB residual block further includes: sequentially connected multi-layer self-attention STL layers, a parallel multi-scale dilated convolution connected to the last STL layer, and residual connections that directly add the input of the RSTB residual block to the output of the parallel multi-scale dilated convolution. The third module is used to identify sand dredgers in the high-resolution reconstructed slice image using a sand dredger instance segmentation model, and outputs multi-temporal images containing sand dredger identification results; the sand dredger identification results include bounding boxes, target categories, confidence scores, and sand dredger instance masks. The sand dredger instance segmentation model is built based on the YOLOv11 model, including a backbone network, a neck network, and a head network. The convolutional units in the backbone network, neck network, and head network all adopt deformable convolutional layers. The deformable convolutional layers are configured as follows: first, ordinary convolutional layers are used to predict the offset and modulation weights of all sampling points in the input feature map; then, deformable convolution calculations are performed on each position of the input feature map based on the offset and modulation weights.

Images