Pipeline backfilling settlement monitoring method and system based on image recognition

By integrating the multimodal feature fusion and cross-attention mechanism of UAV aerial images and SAR remote sensing images, the problems of insufficient accuracy and limited coverage in pipeline backfill settlement monitoring are solved, and automated monitoring and real-time early warning of settlement progress and compaction degree are achieved, thereby improving safety management efficiency.

CN120747863AActive Publication Date: 2025-10-03湛江市润通水务工程有限公司 +1
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
CN202510909014.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-02
Publication Date
2025-10-03
Estimated Expiration
2045-07-02

AI Technical Summary

Technical Problem

Existing technologies for settlement monitoring in pipeline backfill areas suffer from high costs, limited coverage, untimely response, and insufficient accuracy. In particular, signal interference is severe in vegetation-covered areas, and there is a lack of comprehensive analysis of soil compaction.

Method used

An image recognition-based method is adopted to integrate UAV aerial images and SAR remote sensing images. Through multimodal feature fusion and cross-attention mechanism, elevation changes, ground texture, color features and terrain deformation features are extracted, and a dual-stream multimodal feature fusion architecture is constructed to achieve simultaneous prediction and early warning of settlement progress and compaction degree.

Benefits of technology

It improves the accuracy and convenience of pipeline backfill settlement monitoring, realizes the automated monitoring of settlement progress and compaction degree, breaks through the limitations of traditional technology, and significantly improves safety management efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120747863A_ABST
    Figure CN120747863A_ABST
Patent Text Reader

Abstract

The invention provides a pipeline backfill settlement monitoring method and system based on image recognition. The method comprises the following steps: acquiring a plurality of aerial images and a plurality of SAR remote sensing images of each preset time point of a target area; carrying out image preprocessing on the plurality of SAR remote sensing images based on an interferometry technology to obtain a plurality of differential interferograms; inputting the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features and color features from the aerial images of the unmanned aerial vehicle, extracts landform change features from the differential interferograms, and obtains the landform change features of the unmanned aerial vehicle. And finally, performing feature fusion on each feature based on a cross attention mechanism, and outputting the current settlement progress and compaction degree of the target area. And judging whether the current pipeline backfill settlement is abnormal or not according to the settlement progress and the compaction degree, and performing corresponding early warning based on a judgment result, so that the accuracy of monitoring the pipeline backfill settlement is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical fields of image processing and machine learning, and in particular to a pipeline backfill settlement monitoring method and system based on image recognition. Background Art

[0002] With the expansion of urban underground pipeline construction, settlement in pipeline backfill areas has become a key safety hazard for infrastructure. Traditional monitoring methods primarily rely on manual inspections, GNSS positioning, or single-use InSAR technology, but their limitations are becoming increasingly apparent. For example, the high cost of deploying GNSS monitoring points makes large-scale coverage difficult. Manual inspections are time-consuming and subject to environmental constraints, making it difficult to respond promptly to sudden settlement risks. While InSAR-based monitoring methods offer wide-area observation capabilities, they are susceptible to signal interference in vegetated areas, resulting in reduced accuracy in deformation information extraction. Furthermore, existing technologies typically focus solely on surface deformation data and lack comprehensive analysis of key parameters such as soil compaction, which is strongly correlated with settlement risk (for example, insufficiently compacted backfill soil is susceptible to subsequent settlement due to rainwater infiltration). Therefore, there is an urgent need for a monitoring system that can integrate multi-source heterogeneous data, simultaneously invert three-dimensional settlement and compaction, and provide automated early warning capabilities.

[0003] In recent years, the coordinated development of image recognition and multi-source remote sensing technologies has provided new insights into this issue. With its high resolution and flexible deployment capabilities, drone aerial images can capture subtle changes in surface texture, color, and other aspects, providing a direct basis for compaction analysis. Synthetic aperture radar (SAR) uses interferometric measurement technology (such as SBAS-InSAR) to achieve millimeter-level deformation monitoring, compensating for the lack of environmental shielding in optical images. However, how to efficiently fuse the two types of data features and establish an implicit correlation between settlement progress and compaction still faces technical challenges. Existing research is mostly limited to single-modal feature extraction, lacking a deep learning framework for collaborative modeling of the two, and traditional models are difficult to adapt to complex terrain and dynamic environmental changes. Summary of the Invention

[0004] In response to the above technical problems, the present application provides a pipeline backfill settlement monitoring method and system based on image recognition, which realizes joint recognition based on multimodal data and improves the accuracy and convenience of pipeline backfill settlement monitoring.

[0005] In a first aspect, an embodiment of the present application provides a pipeline backfill settlement monitoring method based on image recognition, comprising: Acquire several aerial images and several SAR remote sensing images of the target area at each preset time point; performing image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; Inputting the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the UAV aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch, and finally performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area based on the fused features; Determine whether there is an abnormality in the current pipeline backfill settlement based on the settlement progress and compaction degree, and issue a corresponding warning based on the judgment result; Among them, the subsidence monitoring model is obtained by training the initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module and a multi-task output head.

[0006] The embodiment of the present application provides a pipeline backfill settlement monitoring method based on image recognition, which obtains aerial images and SAR remote sensing images of the target area at each preset time point and inputs them into a trained settlement monitoring model, obtains the current settlement progress and compaction degree of the target area through image recognition technology, and realizes automated settlement monitoring of the target area. Specifically, the embodiment of the present application integrates the multimodal data of drone aerial images and SAR remote sensing images, and uses multimodal feature fusion and cross-attention mechanisms to realize deep correlation analysis of elevation changes, ground texture, color features and terrain deformation features, solving the problem of insufficient accuracy of settlement monitoring with a single data source, and effectively improving the accuracy of the model for pipeline backfill settlement monitoring. At the same time, the embodiment of the present application designs a dual-stream multimodal feature fusion architecture including an optical image branch and a SAR image branch, and integrates the image processing process for aerial images and remote sensing images into the model, thereby improving the convenience of pipeline backfill settlement monitoring. Finally, by using a multi-task output head to simultaneously predict the settlement progress and compaction degree, the limitation of traditional technology that only monitors surface deformation is broken through, a comprehensive assessment of backfill quality is achieved, and combined with a real-time early warning mechanism, the safety management and control efficiency of pipeline backfill projects is significantly improved.

[0007] In one possible implementation, performing image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms includes: Extracting each of the SAR remote sensing images sequentially based on time sequence; If the current SAR remote sensing image extracted is not a SAR remote sensing image at the last moment, the current SAR remote sensing image is used as a main image, the next moment SAR remote sensing image is used as a secondary image, and a corresponding differential interferogram is generated according to the main image and the secondary image; If the extracted current SAR remote sensing image is the SAR remote sensing image at the last moment, the image preprocessing is terminated.

[0008] The embodiment of the present application provides a method for generating a differential interferogram, which uses a time-series sliding window method to process each SAR remote sensing image in sequence and generate a corresponding differential interferogram, avoiding nonlinear deformation interference between multi-phase SAR data, effectively suppressing atmospheric noise through a primary and auxiliary image registration strategy, ensuring the temporal continuity and deformation information integrity of the differential interferogram sequence, and providing a high-quality data foundation for subsequent deformation feature extraction and image recognition.

[0009] Furthermore, generating a corresponding differential interference pattern according to the primary image and the auxiliary image includes: According to the preset orbital parameters and pixel-level registration algorithm, the auxiliary image is registered to the main image coordinate system to obtain a registered image; performing a complex conjugate product operation on the registration image and the main image to generate an original interference pattern; Phase unwrapping is performed on the original interference pattern based on a minimum cost flow algorithm to obtain the differential interference pattern.

[0010] In the process of generating differential interferograms based on primary and auxiliary images, the embodiment of the present application uses a phase unwrapping method based on the minimum cost flow algorithm, which significantly reduces the residual phase noise in the traditional unwrapping process while ensuring computational efficiency. At the same time, it combines the complex conjugate product operation to enhance the sensitivity of microwave signals to tiny surface deformations, thereby improving the SAR data's ability to resolve millimeter-level deformations and improving the accuracy of subsequent models for image recognition.

[0011] In one possible implementation, extracting elevation change features, ground texture features, and color features from each of the drone aerial images based on the optical image branch includes: According to a preset computer vision algorithm, three-dimensional reconstruction is performed on each of the drone aerial images by feature point detection and matching to obtain corresponding digital elevation models; Comparing each of the digital elevation models with a preset digital elevation model before settlement, and arranging the comparison results in chronological order to obtain the elevation change characteristics; Inputting the latest drone aerial image from each of the drone aerial images into a preset residual network, so that the residual network extracts the ground texture features by edge detection, wherein the residual network has frozen the bottom convolution layer; Performing image segmentation on the latest drone aerial image to determine a subsidence area in the latest drone aerial image; Based on a preset HSV mapping network, the subsidence area in the latest drone aerial image is converted into a corresponding HSV histogram as the color feature.

[0012] The embodiment of the present application provides a method for extracting features from drone aerial images. In the process of extracting elevation change features, a digital elevation model (DEM) is reconstructed and compared based on time series to achieve sub-centimeter-level elevation change quantitative analysis. In the process of extracting ground texture features, a residual network with a frozen bottom convolution layer is used to focus on the detection of subtle surface texture changes to avoid shallow noise interference. When extracting color features, the corresponding HSV histogram is extracted by combining image segmentation technology and HSV mapping technology, which can accurately capture visual representations of soil moisture content, degree of exposure, and other factors that are strongly related to compaction, breaking through the traditional spatiotemporal limitations of relying solely on density meter detection. The embodiment of the present application improves the accuracy and comprehensiveness of feature extraction from drone aerial images through the above three feature extraction methods, thereby improving the accuracy of subsequent pipeline backfill settlement monitoring.

[0013] In one possible implementation, extracting terrain deformation features from each of the differential interferograms based on the SAR image branch includes: Extracting a corresponding original amplitude matrix and an original phase matrix from each of the differential interferograms, and performing standardization processing on the original amplitude matrix and the original phase matrix to obtain an amplitude matrix and a phase matrix of the same coordinate system and the same size; After performing multi-layer downsampling encoding and multi-layer upsampling decoding on each of the amplitude matrices and each of the phase matrices through a plurality of preset complex convolutional layers, a multi-scale fusion feature corresponding to each of the differential interferograms is generated through an attention gating mechanism, wherein skip connections are used between each layer of downsampling encoding and upsampling decoding; Performing 1×1 convolution processing on each of the multi-scale fusion features to generate a deformation feature map corresponding to each of the differential interference maps; The deformation feature maps are integrated based on the time sequence to obtain the deformation feature of the ground.

[0014] The present application provides a method for feature extraction from differential interferograms. The method extracts amplitude and phase matrices from the differential interferograms, processes them through complex convolutional layers and an attention gating mechanism, generates corresponding multi-scale fusion features, and then obtains terrain deformation features based on each multi-scale fusion feature. The complex convolutional layers simultaneously process amplitude and phase information, preserving the electromagnetic wave phase characteristics of SAR data. The encoding-decoding structure with jump connections enhances the scale invariance of the multi-scale deformation features. The attention gating mechanism dynamically enhances the feature expression of key deformation regions, significantly improving the modeling accuracy of deformation distribution under complex terrain conditions, thereby improving the accuracy of subsequent pipeline backfill settlement monitoring.

[0015] In one possible implementation, the cross-attention mechanism is used to fuse the features and output the current settlement progress and compaction degree of the target area based on the fused features, including: Based on the cross attention mechanism, the elevation change feature and the terrain deformation feature are fused to obtain a first fused feature; By feature splicing, the ground texture feature and the color feature are fused to obtain a second fused feature; After performing global average pooling processing on the first fusion feature, the feature is input into a preset fully connected layer to generate the current settlement progress of the target area; Based on the preset compaction degree classification, perform 1×1 convolution processing on the second fused features to generate a classification feature map; After the classification feature map is processed by spatial pyramid pooling, it is input into a preset Softmax classifier to generate the current compaction degree of the target area.

[0016] The embodiment of the present application provides a feature fusion method. Unlike the method of directly fusing various features in the prior art, the present application selectively cross-fuses some features in the two branches, that is, it fuses the elevation change features with time series features with the terrain deformation features, and performs feature splicing on the ground texture features and color features without time series features, thereby achieving accurate feature fusion of cross-modal data and improving the accuracy of pipeline backfill settlement monitoring. In the specific feature fusion process, the embodiment of the present application utilizes the cross-attention mechanism to eliminate the spatial alignment deviation between the optical image and the SAR data through cross-modal feature response mapping; spatial pyramid pooling is combined with classification feature map processing to enhance the model's ability to spatially locate local compaction anomalies while ensuring computational efficiency, thereby improving the accuracy of feature fusion.

[0017] Furthermore, the training of the initial settlement monitoring model based on the labeled historical image dataset to obtain the settlement monitoring model includes: Acquire multi-temporal drone aerial images, multi-temporal SAR remote sensing images, and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset. The optical imaging branch is constructed according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; the SAR imaging branch is constructed according to a complex convolutional network; the cross-feature fusion module is constructed based on a cross-attention mechanism; the multi-task output head is constructed based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; the initial settlement monitoring model is constructed according to the optical imaging branch, the SAR imaging branch, the cross-feature fusion module, and the multi-task output head; The initial settlement monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function.

[0018] Furthermore, the initial subsidence monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the subsidence monitoring model, including: Dividing the historical image dataset into a first training dataset and a second training dataset; performing several rounds of single-branch alternating training on the initial subsidence monitoring model according to the first training dataset and the multi-task joint loss function to obtain a first subsidence monitoring model, wherein, during each single-branch alternating training process, model parameters other than the optical imaging branch are frozen, and the model parameters of the optical imaging branch are updated using the first training dataset, or model parameters other than the SAR imaging branch are frozen, and the model parameters of the SAR imaging branch are updated using the first training dataset; Based on the second training data set and the multi-task joint loss function, the first subsidence monitoring model is end-to-end jointly trained to obtain the subsidence monitoring model.

[0019] The embodiment of the present application provides a training method for a settlement monitoring model, which combines a multi-task joint loss function and image data from several backfill projects to perform alternating training on the initial settlement monitoring model in stages. Among them, the multi-task joint loss function constrains the model to simultaneously optimize the spatial deformation perception and material property recognition capabilities through the gradient sharing mechanism of the settlement regression loss and the compaction classification loss, avoiding the feature representation offset caused by single-task training; the staged alternating training strategy effectively alleviates the parameter coupling problem of the dual-stream model: the first stage single-branch pre-training ensures that the optical / SAR branches each converge to the local optimum; the second stage end-to-end fine-tuning promotes cross-modal feature collaborative optimization, significantly reduces the risk of gradient oscillation in the early stage of joint training, improves the model convergence stability, and thus improves the accuracy of pipeline backfill settlement monitoring.

[0020] In a second aspect, an embodiment of the present application provides a pipeline backfill settlement monitoring system based on image recognition, comprising an acquisition module, an image preprocessing module, a settlement monitoring module, and an early warning module; The acquisition module is used to acquire a number of aerial images and a number of SAR remote sensing images of the target area at each preset time point; The image preprocessing module is used to perform image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; The settlement monitoring module is used to input the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the UAV aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch, and finally performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area based on the fused features; The early warning module is used to determine whether there is any abnormality in the current pipeline backfill settlement based on the settlement progress and compaction degree, and to issue a corresponding early warning based on the judgment result; The subsidence monitoring model is obtained by training an initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module, and a multi-task output head.

[0021] Furthermore, the monitoring system further includes a model training module, which includes a data set construction unit, a model construction unit and a training unit; The dataset construction unit is used to obtain multi-temporal UAV aerial images, multi-temporal SAR remote sensing images and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset; The model construction unit is used to construct the optical image branch according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; construct the SAR image branch according to a complex convolutional network; construct the cross-feature fusion module based on a cross-attention mechanism; construct the multi-task output head based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; and construct the initial settlement monitoring model according to the optical image branch, the SAR image branch, the cross-feature fusion module, and the multi-task output head. The training unit is used to train the initial settlement monitoring model according to a preset multi-task joint loss function and the historical image data set to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a process flow of a pipeline backfill settlement monitoring method based on image recognition provided in an embodiment of the present application; Figure 2 A schematic structural diagram of a pipeline backfill settlement monitoring system based on image recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] It should be noted that the step numbers herein are for convenience of explanation of the specific embodiments and do not serve to define the order in which the steps are to be performed. In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature designated "first" or "second" may explicitly or implicitly include one or more of such features.

[0025] Example 1: like Figure 1 As shown, embodiment 1 provides a pipeline backfill settlement monitoring method based on image recognition, including steps S1-S4: Step S1, obtaining a plurality of aerial images and a plurality of SAR remote sensing images of the target area at each preset time point; Step S2: performing image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; Step S3: inputting the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the UAV aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch; finally, performing feature fusion on each feature based on a cross-attention mechanism, and outputting the current settlement progress and compaction degree of the target area based on the fused features; Step S4: judging whether there is any abnormality in the current pipeline backfill settlement according to the settlement progress and compaction degree, and issuing a corresponding warning based on the judgment result; Among them, the subsidence monitoring model is obtained by training the initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module and a multi-task output head.

[0026] The embodiment of the present application provides a pipeline backfill settlement monitoring method based on image recognition, which obtains aerial images and SAR remote sensing images of the target area at each preset time point and inputs them into a trained settlement monitoring model, obtains the current settlement progress and compaction degree of the target area through image recognition technology, and realizes automated settlement monitoring of the target area. Specifically, the embodiment of the present application integrates the multimodal data of drone aerial images and SAR remote sensing images, and uses multimodal feature fusion and cross-attention mechanisms to realize deep correlation analysis of elevation changes, ground texture, color features and terrain deformation features, solving the problem of insufficient accuracy of settlement monitoring with a single data source, and effectively improving the accuracy of the model for pipeline backfill settlement monitoring. At the same time, the embodiment of the present application designs a dual-stream multimodal feature fusion architecture including an optical image branch and a SAR image branch, and integrates the image processing process for aerial images and remote sensing images into the model, thereby improving the convenience of pipeline backfill settlement monitoring. Finally, by using a multi-task output head to simultaneously predict the settlement progress and compaction degree, the limitation of traditional technology that only monitors surface deformation is broken through, a comprehensive assessment of backfill quality is achieved, and combined with a real-time early warning mechanism, the safety management and control efficiency of pipeline backfill projects is significantly improved.

[0027] In a preferred embodiment, in step S1, a drone equipped with a high-precision RGB camera (such as the DJI Phantom series) conducts multi-temporal aerial photography of the target area at an altitude of 50-100 meters with a ground resolution better than 5 cm / pixel, covering the pipeline backfill area and surrounding buffer zone, to obtain the plurality of aerial images. SAR remote sensing images of the target area are acquired based on Sentinel-1 satellite C-band data (wavelength 5.6 cm, angle of incidence 20°-45°) at preset temporal and spatial resolutions.

[0028] In one possible implementation, in step S2, performing image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms includes: Extracting each of the SAR remote sensing images sequentially based on time sequence; If the current SAR remote sensing image extracted is not a SAR remote sensing image at the last moment, the current SAR remote sensing image is used as a main image, the next moment SAR remote sensing image is used as a secondary image, and a corresponding differential interferogram is generated according to the main image and the secondary image; If the extracted current SAR remote sensing image is the SAR remote sensing image at the last moment, the image preprocessing is terminated.

[0029] The embodiment of the present application provides a method for generating a differential interferogram, which uses a time-series sliding window method to process each SAR remote sensing image in sequence and generate a corresponding differential interferogram, avoiding nonlinear deformation interference between multi-phase SAR data, effectively suppressing atmospheric noise through a primary and auxiliary image registration strategy, ensuring the temporal continuity and deformation information integrity of the differential interferogram sequence, and providing a high-quality data foundation for subsequent deformation feature extraction and image recognition.

[0030] Furthermore, generating a corresponding differential interference pattern according to the primary image and the auxiliary image includes: According to the preset orbital parameters and pixel-level registration algorithm, the auxiliary image is registered to the main image coordinate system to obtain a registered image; performing a complex conjugate product operation on the registration image and the main image to generate an original interference pattern; Phase unwrapping is performed on the original interference pattern based on a minimum cost flow algorithm to obtain the differential interference pattern.

[0031] In the process of generating differential interferograms based on primary and auxiliary images, the embodiment of the present application uses a phase unwrapping method based on the minimum cost flow algorithm, which significantly reduces the residual phase noise in the traditional unwrapping process while ensuring computational efficiency. At the same time, it combines the complex conjugate product operation to enhance the sensitivity of microwave signals to tiny surface deformations, thereby improving the SAR data's ability to resolve millimeter-level deformations and improving the accuracy of subsequent models for image recognition.

[0032] In one possible implementation, in step S3, extracting elevation change features, ground texture features, and color features from each of the drone aerial images based on the optical image branch includes: According to a preset computer vision algorithm, three-dimensional reconstruction is performed on each of the drone aerial images by feature point detection and matching to obtain corresponding digital elevation models; Comparing each of the digital elevation models with a preset digital elevation model before settlement, and arranging the comparison results in chronological order to obtain the elevation change characteristics; Inputting the latest drone aerial image from each of the drone aerial images into a preset residual network, so that the residual network extracts the ground texture features by edge detection, wherein the residual network has frozen the bottom convolution layer; Performing image segmentation on the latest drone aerial image to determine a subsidence area in the latest drone aerial image; Based on a preset HSV mapping network, the subsidence area in the latest drone aerial image is converted into a corresponding HSV histogram as the color feature.

[0033] The embodiment of the present application provides a method for extracting features from drone aerial images. In the process of extracting elevation change features, a digital elevation model (DEM) is reconstructed and compared based on time series to achieve sub-centimeter-level elevation change quantitative analysis. In the process of extracting ground texture features, a residual network with a frozen bottom convolution layer is used to focus on the detection of subtle surface texture changes to avoid shallow noise interference. When extracting color features, the corresponding HSV histogram is extracted by combining image segmentation technology and HSV mapping technology, which can accurately capture visual representations of soil moisture content, degree of exposure, and other factors that are strongly related to compaction, breaking through the traditional spatiotemporal limitations of relying solely on density meter detection. The embodiment of the present application improves the accuracy and comprehensiveness of feature extraction from drone aerial images through the above three feature extraction methods, thereby improving the accuracy of subsequent pipeline backfill settlement monitoring.

[0034] In a preferred embodiment, the SFM (Structure from Motion) algorithm is used to perform three-dimensional reconstruction on each aerial image to generate corresponding digital elevation models (DEMs). Then, each DEM is subjected to difference analysis with the pre-backfill baseline DEM to generate a time-series elevation change matrix as the elevation change feature. A ResNet-50 network with fixed parameters for the first five layers is used to extract features from the latest aerial image to obtain an edge texture feature map as the ground texture feature. The aerial image subsidence area is segmented using a U-Net model, where the IoU parameter is set to ≥0.85. The RGB image of the subsidence area is then converted to the HSV color space, and the HSV histogram of the subsidence area is calculated as the color feature.

[0035] In one possible implementation, in step S3, extracting the terrain deformation features from each of the differential interferograms based on the SAR image branch includes: Extracting a corresponding original amplitude matrix and an original phase matrix from each of the differential interferograms, and performing standardization processing on the original amplitude matrix and the original phase matrix to obtain an amplitude matrix and a phase matrix of the same coordinate system and the same size; After performing multi-layer downsampling encoding and multi-layer upsampling decoding on each of the amplitude matrices and each of the phase matrices through a plurality of preset complex convolutional layers, a multi-scale fusion feature corresponding to each of the differential interferograms is generated through an attention gating mechanism, wherein skip connections are used between each layer of downsampling encoding and upsampling decoding; Performing 1×1 convolution processing on each of the multi-scale fusion features to generate a deformation feature map corresponding to each of the differential interference maps; The deformation feature maps are integrated based on the time sequence to obtain the deformation feature of the ground.

[0036] The present application provides a method for feature extraction from differential interferograms. The method extracts amplitude and phase matrices from the differential interferograms, processes them through complex convolutional layers and an attention gating mechanism, generates corresponding multi-scale fusion features, and then obtains terrain deformation features based on each multi-scale fusion feature. The complex convolutional layers simultaneously process amplitude and phase information, preserving the electromagnetic wave phase characteristics of SAR data. The encoding-decoding structure with jump connections enhances the scale invariance of the multi-scale deformation features. The attention gating mechanism dynamically enhances the feature expression of key deformation regions, significantly improving the modeling accuracy of deformation distribution under complex terrain conditions, thereby improving the accuracy of subsequent pipeline backfill settlement monitoring.

[0037] In a preferred embodiment, the original amplitude matrix and phase matrix of the differential interferogram are normalized (mean 0, variance 1) and resized to 256×256. A complex convolutional network (CNN) is then used to convolve the amplitude matrix and phase matrix to generate corresponding multi-scale fused features. The complex convolutional network comprises four layers of downsampling (3×3 convolution kernel, stride 2) and four layers of upsampling (3×3 transposed convolution kernel), with an embedded attention gate (SEBlock) in between. The downsampling layer outputs a complex feature map (real and imaginary parts), and the upsampling layer generates multi-scale fused features, preserving multi-scale edge information through skip connections. Finally, a 1×1 convolution is used to compress the channel to 8 dimensions, outputting a deformation feature map corresponding to the differential interferogram.

[0038] In one possible implementation, in step S3, the cross-attention mechanism is used to fuse the features, and the current settlement progress and compaction degree of the target area are output based on the fused features, including: Based on the cross attention mechanism, the elevation change feature and the terrain deformation feature are fused to obtain a first fused feature; By feature splicing, the ground texture feature and the color feature are fused to obtain a second fused feature; After performing global average pooling processing on the first fusion feature, the feature is input into a preset fully connected layer to generate the current settlement progress of the target area; Based on the preset compaction degree classification, perform 1×1 convolution processing on the second fused features to generate a classification feature map; After the classification feature map is processed by spatial pyramid pooling, it is input into a preset Softmax classifier to generate the current compaction degree of the target area.

[0039] The embodiment of the present application provides a feature fusion method. Unlike the method of directly fusing various features in the prior art, the present application selectively cross-fuses some features in the two branches, that is, it fuses the elevation change features with time series features with the terrain deformation features, and performs feature splicing on the ground texture features and color features without time series features, thereby achieving accurate feature fusion of cross-modal data and improving the accuracy of pipeline backfill settlement monitoring. In the specific feature fusion process, the embodiment of the present application utilizes the cross-attention mechanism to eliminate the spatial alignment deviation between the optical image and the SAR data through cross-modal feature response mapping; spatial pyramid pooling is combined with classification feature map processing to enhance the model's ability to spatially locate local compaction anomalies while ensuring computational efficiency, thereby improving the accuracy of feature fusion.

[0040] In a preferred embodiment, the elevation change features output by the optical branch and the terrain deformation features output by the SAR branch are obtained, both with a feature dimension of 512. They are then aligned to a unified time scale T through linear interpolation to ensure consistency in the time dimension. Query (Q), Key (K), and Value (V) are constructed:

[0041] Among them, Q is the query vector, which represents the elevation change feature that needs to be paid attention to at present; K is the key vector, which represents the index of the terrain deformation feature; V is the value vector, which represents the actual information of the terrain deformation feature. is the learnable weight matrix, is the elevation change characteristic, is the deformation characteristic of the earth. Then, by calculating QK T The dot product of is used to obtain the similarity weight between the elevation change feature and the terrain deformation feature. The similarity weight is converted into a probability distribution through the Softmax function, that is, the attention weight. Finally, the attention weight is used to weight the sum of the Value (deformation feature) and the first feature after fusion is output. , containing the correlation information between elevation and deformation.

[0042] Get the ground texture features (channel number 256, size H×W) and color features (channel number 3, size H×W), and splice them in the channel dimension to get the second fusion feature .

[0043] For the first fusion feature, average pooling is performed on the time dimension T to obtain the feature vector , and input it into the preset fully connected network to generate the current settlement progress S of the target area. The fully connected network includes two fully connected layers (512→128→1), and the activation function is ReLU.

[0044] For the second fusion feature, 1×1 convolution is used to compress the number of channels from 259 to 64, and the classification feature map is output. Then, pooling layers with different kernel sizes are applied to the feature map to obtain the corresponding pooling results. Among them, pooling layers with different kernel sizes include pooling layers with kernel sizes of 1×1, 2×2, and 4×4. Each pooling result is flattened and concatenated to output the feature vector Finally, the eigenvector Input into the preset Softmax classifier to generate the current compaction degree of the target area.

[0045] Furthermore, the training of the initial settlement monitoring model based on the labeled historical image dataset to obtain the settlement monitoring model includes: Acquire multi-temporal drone aerial images, multi-temporal SAR remote sensing images, and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset. The optical imaging branch is constructed according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; the SAR imaging branch is constructed according to a complex convolutional network; the cross-feature fusion module is constructed based on a cross-attention mechanism; the multi-task output head is constructed based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; the initial settlement monitoring model is constructed according to the optical imaging branch, the SAR imaging branch, the cross-feature fusion module, and the multi-task output head; The initial settlement monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function.

[0046] Furthermore, the initial subsidence monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the subsidence monitoring model, including: Dividing the historical image dataset into a first training dataset and a second training dataset; performing several rounds of single-branch alternating training on the initial subsidence monitoring model according to the first training dataset and the multi-task joint loss function to obtain a first subsidence monitoring model, wherein, during each single-branch alternating training process, model parameters other than the optical imaging branch are frozen, and the model parameters of the optical imaging branch are updated using the first training dataset, or model parameters other than the SAR imaging branch are frozen, and the model parameters of the SAR imaging branch are updated using the first training dataset; Based on the second training data set and the multi-task joint loss function, the first subsidence monitoring model is end-to-end jointly trained to obtain the subsidence monitoring model.

[0047] The embodiment of the present application provides a training method for a settlement monitoring model, which combines a multi-task joint loss function and image data from several backfill projects to perform alternating training on the initial settlement monitoring model in stages. Among them, the multi-task joint loss function constrains the model to simultaneously optimize the spatial deformation perception and material property recognition capabilities through the gradient sharing mechanism of the settlement regression loss and the compaction classification loss, avoiding the feature representation offset caused by single-task training; the staged alternating training strategy effectively alleviates the parameter coupling problem of the dual-stream model: the first stage single-branch pre-training ensures that the optical / SAR branches each converge to the local optimum; the second stage end-to-end fine-tuning promotes cross-modal feature collaborative optimization, significantly reduces the risk of gradient oscillation in the early stage of joint training, improves the model convergence stability, and thus improves the accuracy of pipeline backfill settlement monitoring.

[0048] In a preferred embodiment, the training process of the settlement monitoring model is: 1. Data collection and annotation. Multi-source data from 10 typical pipeline backfill projects was collected. GNSS measured coordinates (accuracy ±1 cm) were compared with the baseline DEM to generate settlement labels. The knife-ring method was used to generate compaction labels, and a labeled historical image dataset was constructed.

[0049] 2. Data preprocessing: Aerial images: Crop into 224×224 image blocks according to time series, annotate subsidence area masks (interference over union ≥ 0.85); SAR images: Align to a unified coordinate system, generate differential interferograms (deformation accuracy ±2mm), and extract amplitude / phase matrices.

[0050] 3. Dataset Partitioning: The training set, validation set, and test set were divided into a 7:2:1 ratio to ensure time series continuity. Furthermore, within the training set, 50% of the data was separated into single-modal data consisting only of aerial imagery or SAR imagery, serving as the first training set, and the remaining data as the second training set.

[0051] 4. Single-branch alternating training. According to the first training set and the preset loss function, the initial sedimentation monitoring model is subjected to several rounds of single-branch alternating training to obtain a first sedimentation monitoring model. Each round of single-branch alternating training includes optical branch pre-training and SAR branch pre-training. In the optical branch pre-training, the SAR branch parameters are frozen, and only the optical branch is updated. The Adam optimizer is used for training, and the learning rate is set to 1e-4, and batch_size=16; in the SAR branch pre-training, the optical branch parameters are frozen, and only the SAR branch is updated. The optimizer parameters are set as above.

[0052] 5. End-to-end joint training. Based on the second training dataset and the multi-task joint loss function, the first subsidence monitoring model is trained end-to-end to obtain the subsidence monitoring model. All parameters are unfrozen during training, and the learning rate is optimized (from 1e-4 to 1e-5) using cosine annealing. An early stopping mechanism is also introduced, and training is performed for 30 epochs with an early stopping threshold of 3.

[0053] Example 2: like Figure 2 As shown, the second embodiment provides a pipeline backfill settlement monitoring system based on image recognition, including an acquisition module 10, an image preprocessing module 20, a settlement monitoring module 30 and an early warning module 40; The acquisition module 10 is used to acquire a plurality of aerial images and a plurality of SAR remote sensing images of the target area at each preset time point; The image preprocessing module 20 is used to perform image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; The settlement monitoring module 30 is used to input the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the drone aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch, and finally performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area based on the fused features; The early warning module 40 is used to determine whether there is any abnormality in the current pipeline backfill settlement based on the settlement progress and compaction degree, and to issue a corresponding early warning based on the judgment result; The subsidence monitoring model is obtained by training an initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module, and a multi-task output head.

[0054] Furthermore, the image preprocessing module 20 performs image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms, including: Extracting each of the SAR remote sensing images sequentially based on time sequence; If the current SAR remote sensing image extracted is not a SAR remote sensing image at the last moment, the current SAR remote sensing image is used as a main image, the next moment SAR remote sensing image is used as a secondary image, and a corresponding differential interferogram is generated according to the main image and the secondary image; If the extracted current SAR remote sensing image is the SAR remote sensing image at the last moment, the image preprocessing is terminated.

[0055] Furthermore, generating a corresponding differential interference pattern according to the primary image and the auxiliary image includes: According to the preset orbital parameters and pixel-level registration algorithm, the auxiliary image is registered to the main image coordinate system to obtain a registered image; performing a complex conjugate product operation on the registration image and the main image to generate an original interference pattern; Phase unwrapping is performed on the original interference pattern based on a minimum cost flow algorithm to obtain the differential interference pattern.

[0056] In one possible implementation, the settlement monitoring module 30 extracts elevation change features, ground texture features, and color features from each of the drone aerial images based on the optical image branch, including: According to a preset computer vision algorithm, three-dimensional reconstruction is performed on each of the drone aerial images by feature point detection and matching to obtain corresponding digital elevation models; Comparing each of the digital elevation models with a preset digital elevation model before settlement, and arranging the comparison results in chronological order to obtain the elevation change characteristics; Inputting the latest drone aerial image from each of the drone aerial images into a preset residual network, so that the residual network extracts the ground texture features by edge detection, wherein the residual network has frozen the bottom convolution layer; Performing image segmentation on the latest drone aerial image to determine a subsidence area in the latest drone aerial image; Based on a preset HSV mapping network, the subsidence area in the latest drone aerial image is converted into a corresponding HSV histogram as the color feature.

[0057] In one possible implementation, the subsidence monitoring module 30 extracts the terrain deformation features from each of the differential interferograms based on the SAR image branch, including: Extracting a corresponding original amplitude matrix and an original phase matrix from each of the differential interferograms, and performing standardization processing on the original amplitude matrix and the original phase matrix to obtain an amplitude matrix and a phase matrix of the same coordinate system and the same size; After performing multi-layer downsampling encoding and multi-layer upsampling decoding on each of the amplitude matrices and each of the phase matrices through a plurality of preset complex convolutional layers, a multi-scale fusion feature corresponding to each of the differential interferograms is generated through an attention gating mechanism, wherein skip connections are used between each layer of downsampling encoding and upsampling decoding; Performing 1×1 convolution processing on each of the multi-scale fusion features to generate a deformation feature map corresponding to each of the differential interference maps; The deformation feature maps are integrated based on the time sequence to obtain the deformation feature of the ground.

[0058] Furthermore, the settlement monitoring module 30 performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area according to the fused features, including: Based on the cross attention mechanism, the elevation change feature and the terrain deformation feature are fused to obtain a first fused feature; By feature splicing, the ground texture feature and the color feature are fused to obtain a second fused feature; After performing global average pooling processing on the first fusion feature, the feature is input into a preset fully connected layer to generate the current settlement progress of the target area; Based on the preset compaction degree classification, perform 1×1 convolution processing on the second fused features to generate a classification feature map; After the classification feature map is processed by spatial pyramid pooling, it is input into a preset Softmax classifier to generate the current compaction degree of the target area.

[0059] In a possible implementation, the monitoring system further includes a model training module, which includes a data set construction unit, a model construction unit, and a training unit; The dataset construction unit is used to obtain multi-temporal UAV aerial images, multi-temporal SAR remote sensing images and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset; The model construction unit is used to construct the optical image branch according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; construct the SAR image branch according to a complex convolutional network; construct the cross-feature fusion module based on a cross-attention mechanism; construct the multi-task output head based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; and construct the initial settlement monitoring model according to the optical image branch, the SAR image branch, the cross-feature fusion module, and the multi-task output head. The training unit is used to train the initial settlement monitoring model according to a preset multi-task joint loss function and the historical image data set to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function.

[0060] Furthermore, the initial subsidence monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the subsidence monitoring model, including: Dividing the historical image dataset into a first training dataset and a second training dataset; performing several rounds of single-branch alternating training on the initial subsidence monitoring model according to the first training dataset and the multi-task joint loss function to obtain a first subsidence monitoring model, wherein, during each single-branch alternating training process, model parameters other than the optical imaging branch are frozen, and the model parameters of the optical imaging branch are updated using the first training dataset, or model parameters other than the SAR imaging branch are frozen, and the model parameters of the SAR imaging branch are updated using the first training dataset; Based on the second training data set and the multi-task joint loss function, the first subsidence monitoring model is end-to-end jointly trained to obtain the subsidence monitoring model.

[0061] The more detailed working principle and process flow of this embodiment can be referred to, but not limited to, the relevant records of the first embodiment.

[0062] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application by those skilled in the art should be included within the scope of protection of this application.

Claims

1. A pipeline backfill settlement monitoring method based on image recognition, characterized in that: include: Acquire several aerial images and several SAR remote sensing images of the target area at each preset time point; performing image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; Inputting the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the UAV aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch, and finally performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area based on the fused features; Determine whether there is an abnormality in the current pipeline backfill settlement based on the settlement progress and compaction degree, and issue a corresponding warning based on the judgment result; Among them, the subsidence monitoring model is obtained by training the initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module and a multi-task output head.

2. A pipeline backfill settlement monitoring method based on image recognition according to claim 1, characterized in that: The performing image preprocessing on the plurality of SAR remote sensing images based on the interferometry technology to obtain a plurality of differential interferograms includes: Extracting each of the SAR remote sensing images sequentially based on time sequence; If the current SAR remote sensing image extracted is not a SAR remote sensing image at the last moment, the current SAR remote sensing image is used as a main image, the next moment SAR remote sensing image is used as a secondary image, and a corresponding differential interferogram is generated according to the main image and the secondary image; If the extracted current SAR remote sensing image is the SAR remote sensing image at the last moment, the image preprocessing is terminated.

3. A pipeline backfill settlement monitoring method based on image recognition according to claim 2, characterized in that: Generating a corresponding differential interference map according to the primary image and the auxiliary image includes: According to the preset orbital parameters and pixel-level registration algorithm, the auxiliary image is registered to the main image coordinate system to obtain a registered image; performing a complex conjugate product operation on the registration image and the main image to generate an original interference pattern; Phase unwrapping is performed on the original interference pattern based on a minimum cost flow algorithm to obtain the differential interference pattern.

4. The pipeline backfill settlement monitoring method based on image recognition according to claim 1, characterized in that: The extracting of elevation change features, ground texture features, and color features from each of the drone aerial images based on the optical image branch includes: According to a preset computer vision algorithm, three-dimensional reconstruction is performed on each of the drone aerial images by feature point detection and matching to obtain corresponding digital elevation models; Comparing each of the digital elevation models with a preset digital elevation model before settlement, and arranging the comparison results in chronological order to obtain the elevation change characteristics; Inputting the latest drone aerial image from each of the drone aerial images into a preset residual network, so that the residual network extracts the ground texture features by edge detection, wherein the residual network has frozen the bottom convolution layer; Performing image segmentation on the latest drone aerial image to determine a subsidence area in the latest drone aerial image; Based on a preset HSV mapping network, the subsidence area in the latest drone aerial image is converted into a corresponding HSV histogram as the color feature.

5. The pipeline backfill settlement monitoring method based on image recognition according to claim 1, characterized in that: The extracting of terrain deformation features from each of the differential interferograms based on the SAR image branch includes: Extracting a corresponding original amplitude matrix and an original phase matrix from each of the differential interferograms, and performing standardization processing on the original amplitude matrix and the original phase matrix to obtain an amplitude matrix and a phase matrix of the same coordinate system and the same size; After performing multi-layer downsampling encoding and multi-layer upsampling decoding on each of the amplitude matrices and each of the phase matrices through a plurality of preset complex convolutional layers, a multi-scale fusion feature corresponding to each of the differential interferograms is generated through an attention gating mechanism, wherein skip connections are used between each layer of downsampling encoding and upsampling decoding; Performing 1×1 convolution processing on each of the multi-scale fusion features to generate a deformation feature map corresponding to each of the differential interference maps; The deformation feature maps are integrated based on the time sequence to obtain the deformation feature of the ground.

6. The pipeline backfill settlement monitoring method based on image recognition according to claim 1, characterized in that: The cross-attention mechanism is used to fuse the features and output the current settlement progress and compaction degree of the target area based on the fused features, including: Based on the cross attention mechanism, the elevation change feature and the terrain deformation feature are fused to obtain a first fused feature; By feature splicing, the ground texture feature and the color feature are fused to obtain a second fused feature; After performing global average pooling processing on the first fusion feature, the feature is input into a preset fully connected layer to generate the current settlement progress of the target area; Based on the preset compaction degree classification, perform 1×1 convolution processing on the second fused features to generate a classification feature map; After the classification feature map is processed by spatial pyramid pooling, it is input into a preset Softmax classifier to generate the current compaction degree of the target area.

7. A pipeline backfill settlement monitoring method based on image recognition according to any one of claims 1 to 6, characterized in that: The training of the initial settlement monitoring model based on the labeled historical image dataset to obtain the settlement monitoring model includes: Acquire multi-temporal drone aerial images, multi-temporal SAR remote sensing images, and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset. The optical imaging branch is constructed according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; the SAR imaging branch is constructed according to a complex convolutional network; the cross-feature fusion module is constructed based on a cross-attention mechanism; the multi-task output head is constructed based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; the initial settlement monitoring model is constructed according to the optical imaging branch, the SAR imaging branch, the cross-feature fusion module, and the multi-task output head; The initial settlement monitoring model is trained according to a preset multi-task joint loss function and the historical image dataset to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function.

8. The pipeline backfill settlement monitoring method based on image recognition according to claim 7, characterized in that: The training of the initial settlement monitoring model according to the preset multi-task joint loss function and the historical image dataset to obtain the settlement monitoring model includes: Dividing the historical image dataset into a first training dataset and a second training dataset; performing several rounds of single-branch alternating training on the initial subsidence monitoring model according to the first training dataset and the multi-task joint loss function to obtain a first subsidence monitoring model, wherein, during each single-branch alternating training process, model parameters other than the optical imaging branch are frozen, and the model parameters of the optical imaging branch are updated using the first training dataset, or model parameters other than the SAR imaging branch are frozen, and the model parameters of the SAR imaging branch are updated using the first training dataset; Based on the second training data set and the multi-task joint loss function, the first subsidence monitoring model is end-to-end jointly trained to obtain the subsidence monitoring model.

9. A pipeline backfill settlement monitoring system based on image recognition, characterized in that: It includes acquisition module, image pre-processing module, settlement monitoring module and early warning module; The acquisition module is used to acquire a number of aerial images and a number of SAR remote sensing images of the target area at each preset time point; The image preprocessing module is used to perform image preprocessing on the plurality of SAR remote sensing images based on interferometry technology to obtain a plurality of differential interferograms; The settlement monitoring module is used to input the plurality of aerial images and the plurality of differential interferograms into a preset settlement monitoring model, so that the settlement monitoring model extracts elevation change features, ground texture features, and color features from each of the UAV aerial images based on the optical image branch, and extracts terrain deformation features from each of the differential interferograms based on the SAR image branch, and finally performs feature fusion on each feature based on a cross-attention mechanism, and outputs the current settlement progress and compaction degree of the target area based on the fused features; The early warning module is used to determine whether there is any abnormality in the current pipeline backfill settlement based on the settlement progress and compaction degree, and to issue a corresponding early warning based on the judgment result; The subsidence monitoring model is obtained by training an initial subsidence monitoring model based on a labeled historical image dataset. The initial subsidence monitoring model is a dual-stream multimodal feature fusion architecture, including an optical image branch, a SAR image branch, a cross-feature fusion module, and a multi-task output head.

10. The pipeline backfill settlement monitoring system based on image recognition according to claim 9, characterized in that: The monitoring system further includes a model training module, which includes a data set construction unit, a model construction unit and a training unit; The dataset construction unit is used to obtain multi-temporal UAV aerial images, multi-temporal SAR remote sensing images and monitoring results corresponding to each moment in several backfill projects to construct a labeled historical image dataset; The model construction unit is used to construct the optical image branch according to a computer vision algorithm, an image segmentation algorithm, a residual network, and an HSV mapping network; construct the SAR image branch according to a complex convolutional network; construct the cross-feature fusion module based on a cross-attention mechanism; construct the multi-task output head based on a global average pooling layer, a fully connected layer, a convolutional layer, a pyramid pooling layer, and a Softmax classifier; and construct the initial settlement monitoring model according to the optical image branch, the SAR image branch, the cross-feature fusion module, and the multi-task output head. The training unit is used to train the initial settlement monitoring model according to a preset multi-task joint loss function and the historical image data set to obtain the settlement monitoring model, wherein the multi-task joint loss function is composed of a settlement regression loss function and a compaction classification loss function.

Citation Information

Patent Citations

  • Dam settlement monitoring method based on high-score image and lifting / lowering orbit InSAR

    CN109029344A

  • Deformation image and DEM fusion method and electronic equipment

    CN114627171A

  • Landslide hidden danger identification method, system, medium and equipment

    CN115631162A

  • Multi-modal remote sensing data change detection method and system based on twin U-Net neural network

    CN117372885A

  • Road surface subsidence detection method based on unmanned aerial vehicle DOM and satellite-borne SAR images

    CN117968631A