A method and system for detecting abnormal deformation of a hydraulic engineering

Abstract

The application relates to the technical field of water conservancy project safety monitoring and intelligent remote sensing image analysis, in particular to a water conservancy project abnormal deformation detection method and system; the method obtains multi-period remote sensing images of a drone, matches the SuperPoint and LightGlue models, and calculates deep feature distances to generate a difference map; the core is to adopt multi-modal large model preliminary screening and CLIP semantic alignment fine screening to eliminate false abnormalities and determine the final abnormal area; when an abnormality is detected, the system will automatically label, generate a report and output a warning; the application overcomes the non-precise alignment problem and realizes robust abnormality detection under complex shooting conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water conservancy safety monitoring and intelligent remote sensing image analysis, in particular to a water conservancy abnormal deformation detection method and system. BACKGROUND

[0002] In the field of water conservancy safety monitoring, unmanned aerial vehicles are often used to obtain remote sensing images at different periods for slope deformation comparison and analysis. The existing technology mainly relies on pixel or low-level feature comparison between images to detect differences. However, when unmanned aerial vehicles take pictures at different periods, their flight attitude, lighting conditions, and weather are inconsistent, making it difficult to accurately align the images and reducing matching accuracy. At the same time, environmental factors such as vegetation growth, water surface reflection, and cloud shadow movement can also cause visual changes, causing serious pseudo-exceptional interference. Traditional methods lack high-level semantic understanding ability and cannot distinguish between real geological disasters and non-disaster changes caused by environmental interference, resulting in missed and false detections in the detection results, making it difficult to meet the demand for high-reliability early warning. Therefore, how to accurately identify real abnormal deformation through deep semantic understanding under the conditions of non-precise alignment and strong environmental interference has become a technical problem that needs to be solved. SUMMARY

[0003] To solve the above technical problems, the present application provides a water conservancy abnormal deformation detection method and system. Specifically, the technical solution of the present application is as follows:

[0004] A water conservancy abnormal deformation detection method, comprising:

[0005] Step 1: Obtain the reference remote sensing image and the to-be-detected remote sensing image of the same slope area taken by the unmanned aerial vehicle at different periods. Based on the reference remote sensing image and the to-be-detected remote sensing image, perform key point detection and descriptor extraction through the SuperPoint model, and perform descriptor matching using the LightGlue model to obtain the cross-period corresponding point pair.

[0006] Step 2: Extract the deep visual features of the local neighborhood window in combination with the cross-period corresponding point pair.

[0007] Step 3: Calculate the feature distance based on the deep visual features.

[0008] Step 4: Generate an initial difference response map based on the feature distance.

[0009] Step 5: Perform dynamic threshold segmentation on the initial difference response map to extract the difference point set.

[0010] Step 6: Perform density clustering processing on the difference point set to determine the abnormal candidate region.

[0011] Step 7, multi-modal large model preliminary screening of abnormal candidate regions, eliminating the first type of false anomaly;

[0012] Step 8, CLIP semantic alignment fine screening of abnormal candidate regions that have not been eliminated, to obtain the final abnormal region;

[0013] Step 9, according to the final abnormal region, label the bounding box on the remote sensing image to be detected;

[0014] Step 10, based on the final abnormal region, generate a structured detection report;

[0015] Step 11, based on the structured detection report, output an early warning signal.

[0016] Preferably, the deep visual features of the local neighborhood window are extracted, including:

[0017] A local neighborhood window is constructed around the matching points in the cross-period corresponding point pair;

[0018] Deep visual features within the local neighborhood window are extracted.

[0019] Preferably, the initial difference response map is dynamically threshold segmented to extract the difference point set, including:

[0020] Adaptively set the threshold according to the local texture complexity and noise level;

[0021] The initial difference response map is binarized to extract the difference point set.

[0022] Preferably, the difference point set is subjected to density clustering processing to determine the abnormal candidate region, including:

[0023] A density-based spatial clustering algorithm is applied to cluster the difference point set, and spatially adjacent difference points are aggregated into clusters;

[0024] A minimum bounding rectangle is generated for each cluster to determine the abnormal candidate region.

[0025] Preferably, the multi-modal large model preliminary screening of abnormal candidate regions, eliminating the first type of false anomaly, including:

[0026] Splicing the corresponding sub-images of each abnormal candidate region in the reference remote sensing image and the remote sensing image to be detected to generate a spliced sub-image;

[0027] The spliced sub-image is input into the pre-trained multi-modal large model;

[0028] The pre-trained multi-modal large model determines whether the spliced sub-image belongs to a real geological anomaly to eliminate the first type of false anomaly.

[0029] Preferably, the CLIP semantic alignment fine screening is performed on the un-rejected abnormal candidate region, including:

[0030] A semantic description library is constructed, and the semantic description library includes positive sample texts and negative sample texts;

[0031] The CLIP model is used to calculate the semantic similarity between the image of the abnormal candidate region and the positive sample texts and the negative sample texts;

[0032] When the semantic similarity between the image of the abnormal candidate region and the positive sample texts is high, and the semantic similarity between the image of the abnormal candidate region and the negative sample texts is low, the abnormal candidate region is retained as a final abnormal region;

[0033] Otherwise, the abnormal candidate region is rejected.

[0034] Preferably, a structured detection report is generated, including:

[0035] The abnormal type, geographic coordinates and time change information of the final abnormal region are compiled into the structured detection report.

[0036] A water conservancy project abnormal deformation detection system, comprising:

[0037] An image input module is configured to acquire reference remote sensing images and to-be-detected remote sensing images of the same slope region taken by a UAV at different periods;

[0038] An abnormality detection module, comprising:

[0039] A robust matching unit is configured to perform key point detection and descriptor extraction based on the reference remote sensing images and the to-be-detected remote sensing images by using a SuperPoint model, and to perform descriptor matching by using a LightGlue model to obtain cross-period corresponding point pairs;

[0040] A feature extraction unit is configured to extract deep visual features of a local neighborhood window in combination with the cross-period corresponding point pairs;

[0041] A feature calculation unit is configured to calculate a feature distance based on the deep visual features;

[0042] A response map generation unit is configured to generate an initial difference response map based on the feature distance;

[0043] A segmentation and clustering unit, comprising:

[0044] A segmentation subunit is configured to perform dynamic threshold segmentation on the initial difference response map to extract a difference point set;

[0045] A clustering subunit is configured to perform density clustering processing on the difference point set to determine an abnormal candidate region;

[0046] A semantic screening unit, comprising:

[0047] The preliminary screening sub-unit is configured to perform multimodal large model preliminary screening on the abnormal candidate region, and eliminate the first type of false abnormality.

[0048] The fine screening sub-unit is configured to perform CLIP semantic alignment fine screening on the abnormal candidate region that is not eliminated, and obtain a final abnormal region.

[0049] The result generation unit comprises:

[0050] The labeling unit is configured to label a bounding box on the to-be-detected remote sensing image according to the final abnormal region.

[0051] The report generation unit is configured to generate a structured detection report based on the final abnormal region.

[0052] The early warning pushing unit is configured to output an early warning signal based on the structured detection report.

[0053] Preferably, the fine screening sub-unit is specifically configured to:

[0054] The semantic description library comprises positive sample texts and negative sample texts.

[0055] The CLIP model is used to calculate the semantic similarity between the image of the abnormal candidate region and the positive sample texts and the negative sample texts.

[0056] When the semantic similarity between the image of the abnormal candidate region and the positive sample texts is high, and the semantic similarity between the image of the abnormal candidate region and the negative sample texts is low, the abnormal candidate region is retained as the final abnormal region.

[0057] Otherwise, the abnormal candidate region is eliminated.

[0058] Compared with the prior art, the present application has the following beneficial effects:

[0059] 1. The present application uses SuperPoint and LightGlue models for robust matching, which overcomes the problem of non-accurate image alignment caused by inconsistent unmanned aerial vehicle shooting posture, light and weather in different periods, and ensures that reliable corresponding point pairs can be obtained under complex shooting conditions.

[0060] 2. The present application can adaptively extract a difference point set according to local texture and noise level through dynamic threshold segmentation and density clustering processing, and aggregate discrete points into a spatially continuous candidate region, effectively filtering environmental noise such as wind-blown leaves, and improving the detection sensitivity to small changes in low-texture areas.

[0061] 3. The present application introduces a multi-modal large model for preliminary screening, which can efficiently filter out a large number of non-disaster false anomalies caused by vegetation growth, cloud shadow movement, water surface reflection, etc. by using its general image understanding and common sense reasoning ability, significantly reducing the data volume of subsequent fine screening and improving the detection efficiency;

[0062] 4. The present application realizes detection based on deep semantic understanding by fine screening through CLIP semantic alignment, using a semantic description library containing positive sample texts and negative sample texts. This method can essentially distinguish between real geological disasters and environmental disturbances, solving the problem of missed detection and false detection caused by the lack of advanced semantic understanding ability in traditional methods, ensuring high precision and high reliability of the detection results. BRIEF DESCRIPTION OF DRAWINGS

[0063] The present application will be further explained in conjunction with the accompanying drawings and examples:

[0064] Figure 1 is a flowchart of the method of the present application;

[0065] Figure 2 is a structural diagram of the system of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in conjunction with specific examples.

[0067] Example 1:

[0068] Please refer to Figure 1 A water conservancy project abnormal deformation detection method, comprising:

[0069] Step 1, obtain the reference remote sensing image and the to-be-detected remote sensing image of the same side slope region taken by the unmanned aerial vehicle at different periods, based on the reference remote sensing image and the to-be-detected remote sensing image, perform key point detection and descriptor extraction through the SuperPoint model, and perform descriptor matching through the LightGlue model to obtain the cross-period corresponding point pair;

[0070] Step 2, in combination with the cross-period corresponding point pair, extract the deep visual features of the local neighborhood window;

[0071] Step 3, based on the deep visual features, calculate the feature distance;

[0072] Step 4, based on the feature distance, generate an initial difference response map;

[0073] Step 5, perform dynamic threshold segmentation on the initial difference response map to extract the difference point set;

[0074] Step 6, perform density clustering processing on the difference point set to determine the abnormal candidate region;

[0075] Step 7, multi-modal large model preliminary screening of abnormal candidate regions, eliminating the first type of false anomaly;

[0076] Step 8, CLIP semantic alignment fine screening of abnormal candidate regions that have not been eliminated, to obtain the final abnormal region;

[0077] Step 9, according to the final abnormal region, label the bounding box on the to-be-detected remote sensing image;

[0078] Step 10, based on the final abnormal region, generate a structured detection report;

[0079] Step 11, based on the structured detection report, output an early warning signal.

[0080] The embodiment provides a water conservancy project abnormal deformation detection method, which aims to solve the problems of low image alignment accuracy, serious environmental interference and lack of semantic understanding ability in the prior art; as shown in the figure, Figure 1 The method is a complete technical closed loop including data input, algorithm processing, result analysis and warning;

[0081] Obtain the reference remote sensing image and the to-be-detected remote sensing image of the same slope region taken by the unmanned aerial vehicle at different periods; the reference remote sensing image refers to the image collected in the initial or known normal state, which is used as the pattern for subsequent comparison; the to-be-detected remote sensing image refers to the image collected at a subsequent monitoring time; an unmanned aerial vehicle is used to obtain high-resolution RGB images for fine analysis;

[0082] Based on the reference remote sensing image and the to-be-detected remote sensing image, key point detection and descriptor extraction are performed through the SuperPoint model, and descriptor matching is performed through the LightGlue model to obtain the cross-period corresponding point pair; the LightGlue model is a lightweight but robust neural network matching model; the purpose of using this model is to overcome the non-rigid deformation or slight displacement caused by inconsistent conditions such as unmanned aerial vehicle posture, light, weather, etc. when T1 and T2 are taken, to ensure that reliable homonymous points, i.e. cross-period corresponding point pairs, can be found even under non-precise alignment conditions;

[0083] Deep visual features of the local neighborhood window are extracted in combination with the cross-period corresponding point pairs; the purpose of this step is to switch from pixel-level point correspondence to region-level feature description;

[0084] Based on the deep visual features, the feature distance is calculated; for example, the Euclidean distance or cosine similarity between the feature vectors of the corresponding windows of T1 and T2 extracted in step 2, such as SIFT or SuperPoint features, is calculated to quantify the degree of visual difference between the two regions;

[0085] Based on the feature distance, an initial difference response map is generated; this step maps all the feature distance values calculated in step 3 back to the two-dimensional spatial coordinates of the image, forming a grayscale map; the brightness or value in the map represents the feature distance at the corresponding position, and the greater the difference, the higher the response value;

[0086] The initial difference response map is dynamically threshold segmented to extract a difference point set; wherein the dynamic threshold segmentation specifically includes: setting a threshold according to the local texture complexity and noise level, and binarizing the initial difference response map to extract the difference point set; the purpose of this step is to filter out the regions with significant differences from the response map;

[0087] The difference point set is subjected to density clustering processing to determine an abnormal candidate region; the purpose of this step is to aggregate the discrete difference point set obtained in the previous step into meaningful, spatially continuous patches;

[0088] The abnormal candidate region is subjected to multi-modal large model preliminary screening to eliminate the first type of pseudo-abnormalities; wherein the first type of pseudo-abnormalities refers to changes caused by non-geological disaster factors; the preliminary screening specifically includes: splicing the corresponding sub-images of the candidate region in the reference remote sensing image and the remote sensing image to be detected, inputting a pre-trained multi-modal large model (such as LLaVA), and asking the model, the model uses its common sense reasoning ability to filter non-disaster changes; the first type of pseudo-abnormalities refers to changes caused by non-geological disaster factors, such as vegetation growth, cloud shadow movement, water surface reflection, etc.; although these changes exist visually, they are not the safety hazards concerned by the present application; this step uses the common sense understanding ability of multi-modal large models such as BLIP and LLaVA for rapid screening;

[0089] The abnormal candidate region that has not been eliminated is subjected to CLIP semantic alignment fine screening to obtain the final abnormal region; wherein the fine screening specifically includes: constructing a semantic description library containing positive sample texts and negative sample texts; using the CLIP model to calculate the semantic similarity of the candidate region image and all texts in the library; only when the image has high semantic similarity with the positive sample text and low semantic similarity with the negative sample text, the region is retained as the final abnormal region; this step aims to distinguish between landslide and leaf shaking through deep semantic understanding;

[0090] According to the final abnormal region, a bounding box is labeled on the remote sensing image to be detected; this is to visualize the detection results, and the operator can intuitively see the location and range of the abnormality;

[0091] Based on the final abnormal region, a structured detection report is generated; the purpose of this step is to convert the detection results into operable and archivable data;

[0092] Based on the structured detection report, an early warning signal is output; this is the final output of the application, and the early warning signal can be automatically pushed to the water conservancy project operation management platform or the mobile terminal of the relevant personnel, realizing early warning of safety hazards;

[0093] The method of the embodiment fuses robust matching, dynamic difference perception, density clustering and multi-modal semantic filtering to construct a full-process automatic detection chain from image input to early warning output; the method can effectively overcome environmental interference such as light, vegetation and water body under the condition of non-precise alignment of unmanned aerial vehicle images, and utilize the semantic understanding ability of the multi-modal and CLIP models to accurately distinguish real geological disasters such as landslides and rolling stones from non-disaster changes, thereby realizing high-robustness, low-false-alarm-rate abnormal deformation detection.

[0094] Embodiment 2:

[0095] Deep visual features of the local neighborhood window are extracted, including:

[0096] A local neighborhood window is constructed with the matching point in the cross-period corresponding point pair as the center;

[0097] Deep visual features within the local neighborhood window are extracted.

[0098] The embodiment is a specific description of step 2 in embodiment 1; the specific way of extracting deep visual features of the local neighborhood window includes:

[0099] A local neighborhood window is constructed with the matching point in the cross-period corresponding point pair as the center; the local neighborhood window refers to a rectangular region of a preset size, for example, 32x32 pixels, around the matching point; the purpose is to change from comparing points to comparing surfaces, and to use the neighborhood information of the points to enhance the robustness of the features;

[0100] Deep visual features within the local neighborhood window are extracted; deep visual features refer to feature descriptors extracted by a deep learning model such as VGG, ResNet or a classic feature extraction algorithm such as SIFT, SuperPoint; these features have stronger invariance to light, rotation and slight scale changes than original pixels;

[0101] Through the method of the embodiment, i.e., from point matching to neighborhood window feature matching, the stability and anti-interference ability of the feature description can be significantly improved; this makes the subsequent feature distance calculation not sensitive to local non-rigid deformation caused by shooting angle or light, laying a foundation for generating high-quality initial difference response graphs.

[0102] Embodiment 3:

[0103] The initial difference response graph is dynamically threshold segmented to extract a difference point set, including:

[0104] The threshold is adaptively set according to the local texture complexity and noise level.

[0105] The initial difference response map is binarized to extract the difference point set.

[0106] This embodiment is a specific description of step 5 in embodiment 1; the specific way of dynamic threshold segmentation of the initial difference response map includes:

[0107] The threshold is adaptively set according to the local texture complexity and noise level; the adaptive threshold refers to that the threshold is not a global fixed value, but dynamically changes according to the local characteristics of the image or the difference response map; for example, in a texture complex area such as tree crown, its inherent noise level is high, the threshold should be automatically adjusted to avoid misjudgment; while in a texture flat area such as dam body, the threshold should be automatically adjusted to improve the detection sensitivity;

[0108] The initial difference response map is binarized to extract the difference point set; that is, at each pixel position of the response map, its value is compared with the adaptive threshold of the position, the points greater than the threshold are marked as 1, i.e. difference points, and the points less than or equal to the threshold are marked as 0, i.e. normal points, so as to obtain the difference point set;

[0109] Compared with the traditional global threshold segmentation, the dynamic threshold segmentation method adopted in this embodiment can intelligently adapt to the difference characteristics of different land cover types such as vegetation, rock and water body; this greatly reduces the false detection caused by environmental noise such as wind blowing leaves, while ensuring the sensitivity to small changes in low texture areas, significantly improving the signal-to-noise ratio of the difference point set.

[0110] Embodiment 4:

[0111] The difference point set is subjected to density clustering processing to determine the abnormal candidate area, including:

[0112] A density-based spatial clustering algorithm is applied to cluster the difference point set, and spatially adjacent difference points are aggregated into clusters;

[0113] A minimum circumscribed rectangle is generated for each cluster to determine the abnormal candidate area.

[0114] This embodiment is a specific description of step 6 in embodiment 1; the specific way of density clustering processing of the difference point set includes:

[0115] A density-based spatial clustering algorithm such as DBSCAN is applied to cluster the difference point set, and spatially adjacent difference points are aggregated into clusters; DBSCAN is a classic algorithm that can find clusters of arbitrary shape; its core parameters neighborhood radius and The minimum number of points can be set according to historical experimental data statistics or prior knowledge to match the spatial scale of typical geological disasters such as rockfall; the purpose is to automatically identify statistically significant and spatially concentrated difference point sets;

[0116] A minimum bounding rectangle is generated for each cluster to determine the abnormal candidate region; this step converts the irregularly shaped point clusters output by DBSCAN into regular rectangular bounding boxes, i.e., abnormal candidate regions;

[0117] In this embodiment, the discrete and noisy difference points generated in the previous step are converted into spatially explicit and semantically concentrated abnormal candidate regions; this realizes the conversion from points to planes, greatly compresses the data volume, and provides clear and analyzable image sub-regions for subsequent multi-modal large models and CLIP models.

[0118] Embodiment 5:

[0119] The multi-modal large model is used to preliminarily screen the abnormal candidate regions to eliminate the first type of false anomalies, including:

[0120] The sub-images corresponding to each abnormal candidate region in the reference remote sensing image and the to-be-detected remote sensing image are spliced to generate a spliced sub-image;

[0121] The spliced sub-image is input into the pre-trained multi-modal large model;

[0122] The pre-trained multi-modal large model determines whether the spliced sub-image belongs to a real geological anomaly to eliminate the first type of false anomaly.

[0123] This embodiment is a specific description of step 7 in embodiment 1; the specific way of preliminarily screening the multi-modal large model for the abnormal candidate region includes:

[0124] The sub-images corresponding to each abnormal candidate region in the reference remote sensing image and the to-be-detected remote sensing image are spliced to generate a spliced sub-image; for example, the candidate region image before the change at T1 and the same region image after the change at T2 are placed side by side to form a comparison chart;

[0125] The spliced sub-image is input into the pre-trained multi-modal large model; the pre-trained multi-modal large model refers to models such as LLaVA or BLIP-2, which can understand image content and answer natural language questions about images;

[0126] The pre-trained multi-modal large model determines whether the spliced sub-image belongs to a real geological anomaly to eliminate the first type of pseudo-anomaly; this step is realized by asking the model questions such as whether the image shows a landslide, rockfall or soil displacement, or only shows vegetation, shadow or water surface change; the model uses its common sense reasoning ability to quickly filter out the first type of pseudo-anomaly that is non-disastrous and common environmental changes;

[0127] The present embodiment introduces a multi-modal large model for preliminary screening, which uses its general image understanding and common sense reasoning ability to efficiently filter out a large number of pseudo-anomalies caused by vegetation, shadow, water surface fluctuations, etc. without relying on specific field training data; this greatly reduces the number of candidates entering the next stage of fine screening, significantly improving the computational efficiency and accuracy of the entire detection process.

[0128] Embodiment 6:

[0129] The CLIP semantic alignment fine screening of the uneliminated abnormal candidate region includes:

[0130] A semantic description library is constructed, which contains positive sample texts and negative sample texts;

[0131] The CLIP model is used to calculate the semantic similarity of the image of the abnormal candidate region with the positive sample texts and the negative sample texts;

[0132] When the image of the abnormal candidate region has high semantic similarity with the positive sample texts and low semantic similarity with the negative sample texts, the abnormal candidate region is retained as the final abnormal region;

[0133] Otherwise, the abnormal candidate region is eliminated.

[0134] The present embodiment is a specific description of step 8 in embodiment 1; the specific way of CLIP semantic alignment fine screening of the uneliminated abnormal candidate region includes:

[0135] A semantic description library is constructed, which contains positive sample texts and negative sample texts; the semantic description library is one of the core components of the present application, which is used to define the target to be detected; wherein the positive sample text is a text describing the target disaster, such as slope sliding, rock accumulation, soil displacement; the negative sample text is a text describing common interference, such as leaf shaking, cloud shadow interference, water surface reflection;

[0136] The CLIP model is used to calculate the semantic similarity of the image of the abnormal candidate region with the positive sample texts and the negative sample texts; the CLIP model is a model that can calculate the similarity between any image and any text; this step will calculate the similarity score of the candidate image, such as the image at T2, with all positive and negative examples in the library;

[0137] Decision-making according to the similarity score: when the image of the abnormal candidate region has high semantic similarity with the positive sample text and low semantic similarity with the negative sample text, the abnormal candidate region is retained as the final abnormal region; otherwise, the abnormal candidate region is excluded; for example, a region is confirmed as the final abnormal region only when the rock accumulation score is high, while the leaf shaking and cloud shadow interference scores are low;

[0138] The embodiment realizes detection based on semantic understanding by the semantic alignment fine screening of the CLIP model, rather than the traditional difference detection based on pixels or low-level features; it can accurately distinguish visual changes with different semantic connotations, such as landslides and shadows, solves the problem of coexistence of missed detection and false detection in the prior art, and ensures high precision and high reliability of the detection result.

[0139] Embodiment 7:

[0140] Generating a structured detection report, including:

[0141] Compiling the abnormal type, geographic coordinates and time change information of the final abnormal region into a structured detection report.

[0142] The embodiment is a specific description of step 10 in embodiment 1; the specific way of generating a structured detection report includes:

[0143] Compiling the abnormal type, geographic coordinates and time change information of the final abnormal region into a structured detection report;

[0144] The abnormal type is derived from the positive sample text with the highest matching degree in step 8, for example, slope sliding; the geographic coordinates are obtained by calculating the boundary box pixel coordinates on the image and the geographic information such as GPS data in EXIF provided by the unmanned aerial vehicle remote sensing image, to obtain the latitude and longitude range of the abnormal region; the time change information is the shooting time stamp of the reference remote sensing image and the remote sensing image to be detected;

[0145] The structured detection report is in a machine-readable format such as JSON, XML or database record;

[0146] The embodiment ensures that the detection result is not only visualized such as the boundary box, but also structured and informatized; this report provides standardized data input for subsequent early warning, historical review and quantitative analysis of disaster development trends, greatly improving the practicality and integrability of the invention in actual water conservancy engineering management.

[0147] Embodiment 8:

[0148] Please refer to Figure 2 A water conservancy engineering abnormal deformation detection system, comprising:

[0149] The image input module is used to acquire baseline remote sensing images and remote sensing images to be detected of the same slope area taken by UAVs at different times;

[0150] The anomaly detection module includes:

[0151] The robust matching unit is used to detect key points and extract descriptors based on the benchmark remote sensing image and the remote sensing image to be detected, and to perform descriptor matching using the SuperPoint model and the LightGlue model to obtain corresponding point pairs across time periods.

[0152] The feature extraction unit is used to extract deep visual features of local neighborhood windows by combining corresponding point pairs across time periods.

[0153] The feature calculation unit is used to calculate feature distance based on deep visual features;

[0154] The response map generation unit is used to generate an initial differential response map based on the feature distance;

[0155] Segmenting clustering units, including:

[0156] The segmentation sub-unit is used to perform dynamic threshold segmentation on the initial difference response map and extract the set of difference points;

[0157] Clustering subunits are used to perform density clustering on the set of difference points to identify candidate regions of anomalies.

[0158] Semantic screening unit, including:

[0159] The initial screening subunit is used to perform multimodal large model initial screening on anomaly candidate regions and eliminate the first type of pseudo-anomalies;

[0160] The fine screening sub-unit is used to perform CLIP semantic alignment fine screening on the non-removed abnormal candidate regions and calculate the final abnormal regions.

[0161] The result generation unit includes:

[0162] The annotation unit is used to annotate bounding boxes on the remote sensing image to be detected based on the final anomaly region;

[0163] The report generation unit is used to generate a structured detection report based on the final anomaly areas;

[0164] The early warning push unit is used to output early warning signals based on structured detection reports.

[0165] This embodiment provides a system for detecting abnormal deformation in hydraulic engineering projects, such as... Figure 2 As shown, the system is a hardware or software entity that implements the methods of Examples 1-7; the system can be deployed on a cloud server, edge computing device, or local workstation; the system includes:

[0166] The image input module is used to acquire the reference remote sensing image and the remote sensing image to be detected taken by the UAV; this module corresponds to the data acquisition step in the method and may include the UAV flight control, image receiving and storage unit;

[0167] The anomaly detection module, the core computational unit of the system, is responsible for executing the main detection algorithms. It is further divided into: a robust matching unit, responsible for executing the SuperPoint and LightGlue models to acquire corresponding point pairs across time periods (step 1 in the method); a feature extraction unit, responsible for extracting deep visual features from local neighborhood windows (step 2 in the method); a feature calculation unit, responsible for calculating feature distances (step 3 in the method); a response map generation unit, responsible for generating initial differential response maps (step 4 in the method); and a segmentation and clustering unit, which includes a segmentation subunit responsible for performing dynamic thresholding. The method includes: a segmentation unit to extract the difference point set (corresponding to step 5 in the method); a clustering subunit responsible for performing density clustering to determine candidate anomaly regions (corresponding to step 6 in the method); a semantic screening unit including a preliminary screening subunit responsible for performing multimodal large model preliminary screening to remove first-type pseudo-anomalies (corresponding to step 7 in the method); and a fine screening subunit responsible for performing CLIP semantic alignment fine screening to solve for the final anomaly region (corresponding to step 8 in the method); and a result generation unit including a labeling unit responsible for labeling bounding boxes on the image (corresponding to step 9 in the method); and a report generation unit responsible for generating a structured detection report (corresponding to step 10 in the method).

[0168] The early warning push unit is responsible for outputting early warning signals based on the structured detection report; this module corresponds to step 11 in the method and may include a message queue, SMS gateway, mail server or API interface;

[0169] This embodiment provides a modular and fully functional detection system. Through the collaborative work of its various units, the system realizes all the functions of Embodiment 1, from data input to early warning output, providing a reliable, automatic and intelligent end-to-end solution for the safety monitoring of water conservancy projects.

[0170] Example 9:

[0171] The fine sieve unit is specifically used for:

[0172] Construct a semantic description library, which contains positive and negative sample texts;

[0173] The CLIP model is used to calculate the semantic similarity between images of anomaly candidate regions and positive and negative sample texts;

[0174] When the image of the abnormal candidate region has high semantic similarity with the positive sample text and low semantic similarity with the negative sample text, the abnormal candidate region is retained as the final abnormal region.

[0175] Otherwise, the abnormal candidate region is eliminated.

[0176] The embodiment is a specific description of the fine sifter unit in embodiment 8; the function of the unit corresponds to the method of embodiment 6, and the internal implementation logic is as follows:

[0177] The fine sifter unit is configured to build a semantic description library, and the unit will hold a storage such as a configuration file or a database table, in which the positive sample text such as landslide and the negative sample text such as shadow are pre-defined or allowed to be defined by the user; using the CLIP model, the unit will load a pre-trained CLIP model for computing tasks;

[0178] At runtime, the fine sifter unit receives the image of the candidate region from the primary sifter unit, and calculates the semantic similarity of the image with all positive sample texts and negative sample texts in the library;

[0179] The unit executes the decision logic: when the image of the abnormal candidate region has high semantic similarity with the positive sample text and low semantic similarity with the negative sample text, for example, the similarity with landslide is greater than 0.8, and the similarity with shadow, leaves and construction vehicles is less than 0.2, the abnormal candidate region is retained as the final abnormal region; otherwise, the abnormal candidate region is eliminated;

[0180] By configuring the fine sifter unit as in the embodiment, the core semantic discrimination function of the system is ensured to be accurately implemented; this double semantic constraint based on positive confirmation and reverse exclusion gives the system strong anti-interference ability, which is the key technical guarantee for the system to achieve high precision and low false alarm;

[0181] The above is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application; any modification, equivalent replacement, improvement, etc. made by any person skilled in the art without departing from the spirit and principles of the present application shall be included in the protection scope of the present application.

[0182] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting; although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for detecting abnormal deformation of hydraulic engineering, characterized in that, The method comprises the following steps: Step 1, obtaining the reference remote sensing image and the to-be-detected remote sensing image of the same slope region taken by the unmanned aerial vehicle at different time periods, based on the reference remote sensing image and the to-be-detected remote sensing image, performing key point detection and descriptor extraction through a SuperPoint model, and performing descriptor matching through a LightGlue model to obtain a cross-period corresponding point pair; Step 2, combining the cross-period corresponding point pair, extracting deep visual features of a local neighborhood window; Step 3, calculating a feature distance based on the deep visual features; Step 4, generating an initial difference response map based on the feature distance; Step 5, performing dynamic threshold segmentation on the initial difference response map to extract a difference point set; Step 6, performing density clustering processing on the difference point set to determine an abnormal candidate region; Step 7, performing multi-modal large model preliminary screening on the abnormal candidate region to eliminate a first type of pseudo anomaly; Step 8, performing CLIP semantic alignment fine screening on the abnormal candidate region that is not eliminated to obtain a final abnormal region; Step 9, labeling a bounding box on the to-be-detected remote sensing image according to the final abnormal region; Step 10, generating a structured detection report based on the final abnormal region; Step 11, outputting an early warning signal based on the structured detection report.

2. The method according to claim 1, characterized in that, The method for extracting deep visual features of a local neighborhood window comprises the following steps: Taking a matching point in the cross-period corresponding point pair as the center, a local neighborhood window is constructed; Deep visual features in the local neighborhood window are extracted.

3. The method according to claim 1, characterized in that, The method for performing dynamic threshold segmentation on the initial difference response map to extract a difference point set comprises the following steps: Adaptively setting a threshold according to local texture complexity and noise level; Performing binaryzation processing on the initial difference response map to extract the difference point set.

4. The hydraulic engineering abnormal deformation detection method according to claim 1, characterized in that, The method for performing density clustering processing on the difference point set to determine an abnormal candidate region comprises the following steps: Applying a density-based spatial clustering algorithm to cluster the difference point set, and aggregating spatially adjacent difference points into clusters; Generating a minimum bounding rectangle for each cluster to determine the abnormal candidate region.

5. The hydraulic engineering abnormal deformation detection method according to claim 1, characterized in that, The method for performing multi-modal large model preliminary screening on the abnormal candidate region to eliminate a first type of pseudo anomaly comprises the following steps: Splicing sub-images corresponding to each abnormal candidate region in the reference remote sensing image and the to-be-detected remote sensing image to generate a spliced sub-image; Inputting the spliced sub-image into a pre-trained multi-modal large model; The pre-trained multi-modal large model judges whether the spliced sub-image belongs to a real geological anomaly to eliminate the first type of pseudo anomaly.

6. The hydraulic engineering abnormal deformation detection method according to claim 1, characterized in that, The method for performing CLIP semantic alignment fine screening on the abnormal candidate region that is not eliminated comprises the following steps: Constructing a semantic description library, the semantic description library containing positive sample texts and negative sample texts; Calculating semantic similarities between an image of the abnormal candidate region and the positive sample texts and the negative sample texts by using a CLIP model; When the semantic similarity between the image of the abnormal candidate region and the positive sample texts is high and the semantic similarity between the image of the abnormal candidate region and the negative sample texts is low, the abnormal candidate region is retained as the final abnormal region; Otherwise, the abnormal candidate region is eliminated.

7. The hydraulic engineering abnormal deformation detection method according to claim 1, characterized in that, The method for generating a structured detection report comprises the following steps: Compiling an abnormal type, geographic coordinates and time variation information of the final abnormal region into a structured detection report.

8. A hydraulic engineering abnormal deformation detection system applied to the hydraulic engineering abnormal deformation detection method of any one of claims 1-7, characterized in that, The method comprises the following steps: An image input module is configured to acquire a reference remote sensing image and a to-be-detected remote sensing image of a same slope region captured by a UAV at different time periods; The anomaly detection module comprises: A robust matching unit is configured to perform key point detection and descriptor extraction based on the reference remote sensing image and the to-be-detected remote sensing image by using a SuperPoint model, and perform descriptor matching by using a LightGlue model, to obtain a cross-period corresponding point pair; A feature extraction unit is configured to extract deep visual features of a local neighborhood window in combination with the cross-period corresponding point pair; A feature calculation unit is configured to calculate a feature distance based on the deep visual features; A response map generation unit is configured to generate an initial difference response map based on the feature distance; The segmentation and clustering unit comprises: A segmentation subunit is configured to perform dynamic threshold segmentation on the initial difference response map to extract a difference point set; A clustering subunit is configured to perform density clustering processing on the difference point set to determine an abnormal candidate region; The semantic screening unit comprises: A preliminary screening subunit is configured to perform preliminary screening on the abnormal candidate region by using a multi-modal large model to eliminate a first type of false anomaly; A fine screening subunit is configured to perform fine screening on the abnormal candidate region that is not eliminated by using CLIP semantic alignment to obtain a final abnormal region; The result generation unit comprises: A labeling unit is configured to label a bounding box on the to-be-detected remote sensing image according to the final abnormal region; A report generation unit is configured to generate a structured detection report based on the final abnormal region; An early warning pushing unit is configured to output an early warning signal based on the structured detection report.

9. The hydraulic engineering abnormal deformation detection system according to claim 8, characterized in that, The fine screening subunit is specifically configured to: Construct a semantic description library, wherein the semantic description library comprises positive sample texts and negative sample texts; Calculate semantic similarity between an image of the abnormal candidate region and the positive sample texts and the negative sample texts by using a CLIP model; When the image of the abnormal candidate region has high semantic similarity with the positive sample texts and low semantic similarity with the negative sample texts, the abnormal candidate region is retained as the final abnormal region; Otherwise, the abnormal candidate region is eliminated.

Images