A method for intelligent change detection of unaligned two-phase images from an inspection drone

Abstract

This invention discloses an intelligent change detection method for non-aligned two-phase images from an inspection drone, relating to the field of drone image processing and change detection technology. This method acquires two images from a low-altitude drone under a fixed flight path and identical sensor parameters; constructs and trains a lightweight registration model, uses feature point matching to predict matching point pairs and calculates the homography matrix to achieve accurate registration of the non-aligned images; constructs and trains a change detection model based on image pair interaction feature fusion, analyzes the registered aligned images, and outputs a binary map of change information. This method effectively solves the non-alignment problem caused by position and angle differences during two-phase image acquisition by drones, as well as the technical problems of low accuracy, poor robustness, and insufficient computational efficiency in traditional change detection methods. It effectively improves the automation level of low-altitude safety monitoring for ground highways and railways and reduces maintenance costs, demonstrating significant application value.

Description

Technical Field

[0001] This invention relates to the field of UAV image processing and change detection technology, specifically to an intelligent change detection method for non-aligned two-phase images of an inspection UAV. Background Technology

[0002] With the rapid development of highway and railway transportation and infrastructure in my country, the monitoring and maintenance of routes and their surrounding environment have become crucial for ensuring transportation safety. Traditional monitoring methods rely on manual inspections or fixed sensors, which suffer from limited coverage, low efficiency, and high costs. In recent years, low-altitude unmanned aerial vehicles (UAVs) have been widely used in remote sensing monitoring of highway and railway environments due to their flexibility, high-resolution image acquisition capabilities, and high efficiency. By comparing UAV images from two different time periods, changes in route infrastructure, environmental hazards, and geological disasters can be extracted, providing a basis for safety management decisions.

[0003] However, due to factors such as terrain differences, UAV flight attitude, ambient lighting, and viewpoint changes, images from different time periods often exhibit geometric distortions, scale differences, and viewpoint deviations (hereinafter referred to as "unaligned images"). This leads to high false detection rates and insufficient robustness in traditional change detection methods based on pixel-level comparison. Existing change detection methods for unaligned images mainly include single-stage detection based on region feature alignment and two-stage detection based on image registration. Single-stage methods detect changes by extracting local semantic features, but struggle to quickly capture complex linear structures and edge information. Two-stage methods perform change detection after high-precision registration, but are limited by the dynamic environment of the scene (such as vegetation changes and weather interference) and the diversity of unaligned images, making it difficult for traditional registration algorithms to meet practical needs in terms of computational complexity and robustness. Although deep learning technology has made significant progress in image processing, research on change detection for unaligned images from low-altitude inspection UAVs remains insufficient. Existing methods struggle to simultaneously address image unalignment, complex environmental interference, and lightweight requirements.

[0004] Therefore, how to solve the above problems and overcome the limitations of traditional methods in terms of accuracy, robustness and computational efficiency has become a technical problem that urgently needs to be solved by people in this field. Summary of the Invention

[0005] This invention aims to at least solve one of the technical problems existing in the prior art. Therefore, one objective of this invention is to propose an intelligent change detection method for unaligned two-phase images from an inspection drone. This method, through an innovative lightweight intelligent registration model, image alignment processing, and an efficient intelligent change detection model, effectively overcomes the limitations of traditional methods in terms of accuracy, robustness, and computational efficiency. It aims to improve the automation level of low-altitude safety monitoring for ground-based highways and railways and reduce maintenance costs, thus possessing significant application value.

[0006] To address the aforementioned problems, this invention provides an intelligent change detection method for non-aligned two-phase images from an inspection drone, comprising the following steps:

[0007] Step 1: Acquire two images from a low-altitude UAV along a fixed route and with the same sensor parameters.

[0008] Step 2: Select unaligned images, perform normalization processing and registration point annotation to form a registration sample set;

[0009] Step 3: Select aligned images with changed areas, perform normalization processing, label change information, and form a change detection sample set;

[0010] Step 4: Construct a lightweight registration model based on feature point matching, and train it based on the registration point sample set in Step 2 to obtain the optimal training weight parameters of the lightweight registration model.

[0011] Step 5: Construct a change detection model based on the fusion of image interaction features, and train it based on the change sample set in Step 3 to obtain the optimal training weight parameters of the change detection model.

[0012] Step 6: After normalizing a pair of unaligned images, input them into the registration model constructed in Step 4, extract features and predict matching point pairs, calculate the homography matrix, and generate aligned images.

[0013] Step 7: Based on the aligned image obtained in Step 6, input it into the change detection model trained in Step 5, output a binary map of change information, and overlay the binary map onto the second image of the two images to mark the change area.

[0014] Preferably, in step 1, multiple pairs of low-altitude image sets S of two phases of UAV acquisition need to be obtained. The image sets include first phase images and second phase images acquired based on fixed waypoints and at certain time intervals.

[0015] Preferably, in step 2, it is necessary to select two corresponding image sets from periods with different perspectives. Normalization and registration point labeling are performed to form a registration sample set. The specific steps are as follows:

[0016] Step 2-1: Perform scale normalization on the acquired image using adaptive scaling and padding. Assume the size of the acquired image is... a × b， in a ≥ b, a Length of the longer side ，b The shorter side length is defined as the adaptive scaling factor. a With normalized scale d reciprocal of the ratio The length of the longer side after scaling is The length of the shorter side is To convert to a normalized scale, double-sided short-side zero-filling is used, resulting in single-sided short-side fill pixels. c The calculation formula is:

[0017] ;

[0018] Step 2-2: Manually label the key feature pixels in the normalized image pairs; form a sample set containing at least 1000 pairs of registration points, covering two image scenes with changes in illumination and viewpoint shifts, and the sample labeling information includes the coordinate labels of each pair of points.

[0019] Preferably, in step 3, two datasets that do not have perspective differences but exhibit different scale variations need to be selected. The image scale is normalized to 1024×1024×3 using the method in step 2-1. Then, the change information in each pair of images is labeled to generate a binary image, which serves as the label for the change detection dataset.

[0020] Preferably, step 4 requires constructing a lightweight convolutional neural network model for two-stage image registration. The model is trained using the dataset constructed in step 2 to obtain the registration model. Optimal weight parameter set The specific steps are as follows:

[0021] Step 4-1: Input the pre- and post-registration temporal images A and B from the registration sample set into the lightweight feature extraction network of the registration model to obtain refined feature maps. , and semantic feature maps , ;

[0022] Step 4-1-1: Use depthwise separable convolution, batch normalization, and SiLU activation to perform shallow feature extraction on the input image;

[0023] Step 4-1-2: Then, the GhostELA-DWC2f module is stacked multiple times for multi-scale feature extraction. First, the GhostELA-DWC2f module is stacked to process the shallow features and obtain a fine feature map. , The semantic feature map is obtained by overlaying the GhostELA-DWC2f module twice. , ;

[0024] Step 4-2: Process the semantic feature map , The matching confidence matrix is ​​calculated using local cross-attention encoding.

[0025] Step 4-2-1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation , Divide the space into 8x8 windows, and iterate through each window. m At that time, its features are converted into query vectors. Key vector Sum value vector The formula is as follows:

[0026] ;

[0027] in, , , These are the embedding matrices used to transform queries, key vectors, and value vectors, respectively.

[0028] Step 4-2-2: Calculate from arrive And from arrive Window cross-attention score:

[0029] ;

[0030] ;

[0031] In the formula h For the number of attention heads, and h =8, C For the number of channels, C =256, then the output feature after cross-attention calculation is represented as:

[0032] ;

[0033] ;

[0034] Step 4-2-3: The confidence score of the match within the window is represented as follows:

[0035] ;

[0036] The global matching confidence matrix, formed by concatenating all windows, is represented as follows:

[0037] ;

[0038] In the formula: R represents the real number field;

[0039] Step 4-3: Based on the mutual proximity calculation matching rule, use a lightweight method. MLP Predict and filter high-quality matching pairs;

[0040] Step 4-3-1: Match the global confidence matrixP Extracting nearest neighbor matching information ,set up Get the matching set M, where The threshold for mutual nearest neighbor matching and The specific calculation method is as follows:

[0041]

[0042]

[0043]

[0044] ;

[0045] Step 4-3-2: For each pair of matches Arrange to obtain splicing features ,use MLP Confidence level of prediction The calculation method is as follows:

[0046] ;

[0047] in , For learnable weight values, retain Get high-quality matching sets from the matching pairs. ,in The confidence threshold for high-quality matching pairs;

[0048] Step 4-4: Fuse the cross-attention feature map with the features extracted by the backbone network; the specific steps are as follows:

[0049] Step 4-4-1: First, generate the cross-attention feature map. , The GhostELA-DWC2f-x module was used for calculation, and channel splicing and fusion were employed to obtain the fusion features. , ;

[0050] Step 4-4-2: Continue with the fusion features as follows , The GhostELA-DWC2f-x module was used for computation, and then combined with fine features. , Channel stitching and fusion to obtain fine features after fusion. , ;

[0051] Step 4-5: Select the high-quality matching set filtered in Step 4-3 Points in the map are mapped to the fused fine feature map. , In the middle, calculation , Pixel correlation of the corresponding window and correlation probability The expected coordinates are calculated as the sub-pixel position values ​​of the fused fine feature map mapping region corresponding to the matching point pair. ;

[0052] Steps 4-6: Using correlation probabilities as weights, more precise sub-pixel displacements can be calculated further:

[0053] ;

[0054] ;

[0055] in The weighted function is generated based on Gaussian weights. For the weighted intensity of the control center, take ;

[0056] Steps 4-7: Update the offsets of effective matching points during the registration model training process using this method to obtain the optimal set of weight parameters for the registration model. .

[0057] Preferably, step 5 requires constructing a convolutional neural network model for extracting changing regions. It consists of three parts: a lightweight backbone network, an efficient Transformer neck network, and a boundary-aware decoding network; the location and category labels of all targets in the segmentation map are input to train the neural network to obtain... , For change detection model The optimal set of weight parameters is determined by the following steps:

[0058] Step 5-1: Combine the before and after time-phase images of the change detection sample set. and The input is fed into a lightweight backbone network, which uses the same backbone network as the image registration model to obtain semantic features. , ;

[0059] Step 5-2: Perform channel feature exchange and spatial feature exchange on the semantic features sequentially; , The input is fed into the channel switching module to obtain switching features, and then the GhostELA-DWC2f module is used to extract deep switching features. , Furthermore, the half-channel features are input into the spatial exchange module for exchange at corresponding positions in the spatial dimension. Then, GhostELA-DWC2f is used to extract deep exchange features to obtain the interaction information features. , ;

[0060] Step 5-3: Input the interactive features into the feature fusion network to obtain fused features with deep semantic and spatial information. ; Use convolution and 2x upsampling structure to extract dimensionality-reduced features , The initial fused features are obtained by using concatenation and fusion. This feature will be channel-wise concatenated and fused with the third layer of the feature extraction network, and then the GhostELA-DWC2f-x module will be used to obtain the first-layer enhanced fusion feature. , ; Continue fusing twice in this manner to obtain the final enhanced fusion feature. ;

[0061] Step 5-4: Input the fused features into the decoding network for binary mask prediction of changed and unchanged features; first, use bilinear interpolation to align the spatial scale of the input image, then use 1×1 Conv to map the fused features to the number of target categories. N ,in N=2 The dataset includes two categories: changed and unchanged. Then, 3×3Conv is used to enhance local features and improve spatial details. Finally, the SiLU activation function is used to output a probability map of changed pixels.

[0062] Step 5-5: Construct an effective loss function during model training;

[0063] Step 5-5-1: Use binary cross-entropy loss for pixel-level classification to measure the predicted change mask. With the real mask The differences between them are explained below, and the pixel classification loss is calculated as follows:

[0064] ;

[0065] in, This represents the true label, where 0 indicates no change and 1 indicates change. This represents the probability predicted by the model, and the value is based on the SiLU function predicted in step 5-4.

[0066] Step 5-5-2: Construct boundary loss to enhance edge accuracy; explicitly optimize the edges of changing regions using the Sobel operator; the boundary loss is defined as:

[0067] ;

[0068] in This represents the confidence level of the true edges generated using the Sobel operator; This represents the Sobel output value of the model's predicted edge map;

[0069] Step 5-5-3: Obtain the total loss, which is a weighted combination of the boundary loss and the pixel classification loss. This total loss is used for tuning the weight parameters during model training and is expressed as:

[0070] ;

[0071] in Weights for pixel classification loss, The weights represent the boundary loss; the optimal set of weight parameters for the change detection model is obtained after training. .

[0072] Preferably, during step 6, for the unaligned preceding and following phase test images... Input it into the registration model Using a set of weight parameters Extracting images Compared to The registration point information is obtained, and the aligned image after registration is acquired; the specific steps are as follows:

[0073] Step 6-1: Assume the image representing the first phase of the image is The second phase image is Input to configuration model Output the set of matching point coordinate pairs in the unaligned image;

[0074] Step 6-2: Map the set of matching point coordinate pairs to the unaligned before and after temporal test images. In the grayscale image, to calculate the homography transformation matrix. Select 4 pairs of matching points Each pair of points satisfies the following homogeneous coordinate relationship:

[0075]

[0076] ;

[0077] Will Transform into an unknown parameter vector Then solve the matrix using the least squares method;

[0078] Step 6-3: Based on the homography transformation matrix Perspective transformation is used to transform the image Mapping to Image Coordinate system; Obtaining a registered image with irregular black borders. ;

[0079] Step 6-4: Extract the irregular black border areas and fill them into the image. To obtain the current period image Corresponding registration image Overlapping registration images Irregular black border areas are extracted from the locations where the pixel value is 0 in the detected image.

[0080] Preferably, step 7 requires aligning the images. , Input to change detection model Using the optimal set of weight parameters for the change detection model Extract the information of the binarized change regions and label them onto the original image. .

[0081] The advantages of this invention compared to the prior art are:

[0082] The present invention provides an intelligent change detection method for non-aligned two-phase images from an inspection drone, based on two non-aligned images acquired by a low-altitude drone. The specific advantages are as follows:

[0083] 1. Compared with the direct change detection in the prior art, the present invention introduces image registration and image alignment processing, which effectively addresses the complex geometric distortion and viewpoint deviation of low-altitude UAV image pairs.

[0084] 2. This invention constructs a lightweight registration model that combines cross-attention, effectively reducing computational complexity; it adopts a two-stage matching point filtering mechanism to achieve accurate and fast feature point matching.

[0085] 3. This invention constructs a multi-temporal high-resolution image change detection network, which can effectively capture medium and small-scale change areas in traffic routes and improve the detection capability of minor safety hazards.

[0086] 4. This invention, based on two-stage learning using dual-temporal data, provides a novel approach to extracting change information from unaligned UAV imagery. Specifically, by pre-training image registration and change detection models for the "matching-alignment-change detection" inference process of unaligned images, it effectively reduces false detection rates and improves accuracy. This provides an efficient and practical technical solution for identifying potential hazards along transportation routes and for disaster prevention and maintenance. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1 This is a general framework diagram of the non-aligned low-altitude remote sensing image change detection method of the present invention;

[0089] Figure 2 This is a structural diagram of the lightweight image registration network model of the present invention;

[0090] Figure 3 This is a schematic diagram of the lightweight layer aggregation feature extraction module of the present invention;

[0091] Figure 4 This is a schematic diagram showing the visualization results of the unaligned image registration process of the present invention;

[0092] Figure 5 This is a structural diagram of the change detection network model of the present invention;

[0093] Figure 6 This is a diagram illustrating the change detection results of the present invention. Detailed Implementation

[0094] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0095] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0096] The present invention will now be described in further detail with reference to the accompanying drawings.

[0097] Combination Figure 1 The present invention provides an intelligent change detection method for non-aligned two-phase images of an inspection drone, comprising the following steps:

[0098] Step 1: Obtain multiple pairs of low-altitude image sets S from two phases of UAV data acquisition. The image set contains two phases of images acquired based on fixed waypoints and at certain time intervals. Due to environmental influences during UAV flight image acquisition, especially interference from wind and rain, some image pairs acquired at fixed waypoints may have offsets and distortions.

[0099] Step 2: Select two corresponding image sets from periods with different perspectives. Normalization and registration point labeling are performed to form a registration sample set.

[0100] Step 2-1: Perform scale normalization on the acquired image using adaptive scaling and padding. Assume the size of the acquired image is... a × b， in a ≥ b, a Length of the longer side ，b The shorter side length is defined as the adaptive scaling factor. a With normalized scale d reciprocal of the ratio The length of the longer side after scaling is The length of the shorter side is To convert to a normalized scale, double-sided short-side zero-filling is used, resulting in single-sided short-side fill pixels. c The calculation formula is:

[0101] ;

[0102] Step 2-2: Manually label the key feature pixels in the normalized image pairs; form a sample set containing at least 1000 pairs of registration points, covering two image scenes such as illumination changes and viewpoint shifts, and the sample labeling information includes the coordinate labels of each pair of points.

[0103] Step 3: Select two datasets that do not have perspective differences but exhibit different scale variations. The image scale is normalized to 1024×1024×3 using the method in step 2-1. Then, the change information in each pair of images is labeled to generate a binary image, which serves as the label for the change detection dataset.

[0104] Step 4: Construct a lightweight convolutional neural network model for two-stage image registration. ,like Figure 2 As shown. The dataset constructed in step 2 is then used for model training to obtain the optimal set of weight parameters for the registration model. .

[0105] Step 4-1: Input the preprocessed image pairs (size 1024×1024×3) of the registration point sample set into the lightweight feature extraction network of the registration model to obtain fine feature maps. , and semantic feature maps , ;

[0106] Step 4-1-1: Perform shallow feature extraction on the input image using depthwise separable convolution, batch normalization, and SiLU activation (DW-CBS module);

[0107] Step 4-1-2: Then, the GhostELA-DWC2f module is stacked multiple times for multi-scale feature extraction. GhostELA-DWC2f is an improved lightweight deep feature extraction module, and its structure is as follows: Figure 3 As shown. Specifically, the GhostELA-DWC2f module is overlaid to process shallow features and obtain fine feature maps. , Then, the semantic feature maps are obtained by superimposing them twice. , ;

[0108] Step 4-2: Process the semantic feature map , The matching confidence matrix is ​​calculated using local cross-attention encoding.

[0109] Step 4-2-1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation , Divide the space into 8x8 windows, and iterate through each window. m At that time, its features are converted into query vectors. Key vector Sum value vector The formula is as follows:

[0110] ;

[0111] in, , , These are the embedding matrices used to transform queries, key vectors, and value vectors, respectively.

[0112] Step 4-2-2: Calculate from arrive And from arrive Window cross-attention score:

[0113] ;

[0114] ;

[0115] In the formula h For the number of attention heads, and h =8, CFor the number of channels, C =256, then the output feature is represented as:

[0116] ;

[0117] ;

[0118] Step 4-2-3: The confidence score of the match within the window is represented as follows:

[0119] ;

[0120] The global matching confidence matrix, formed by concatenating all windows, is represented as follows:

[0121] ;

[0122] This method can significantly reduce the computational load and roughly obtain the matching confidence value between semantic feature maps.

[0123] Step 4-3: Based on the mutual proximity calculation matching rule, use a lightweight method. MLP Predict and filter high-quality matching pairs;

[0124] Step 4-3-1: Match the global confidence matrix P Extracting nearest neighbor matching information ,set up M in The threshold for mutual nearest neighbor matching and The specific calculation method is as follows:

[0125]

[0126]

[0127]

[0128] ;

[0129] Step 4-3-2: For each pair of matches Arrange to obtain splicing features ,use MLP Prediction confidence The calculation method is as follows:

[0130] ;

[0131] in , For learnable weight values, retain Get high-quality matching sets from the matching pairs. .

[0132] Step 4-4: Fuse the cross-attention feature map with the features extracted by the backbone network;

[0133] Step 4-4-1: First, generate the cross-attention feature map. , Use the GhostELA-DWC2f-x module (an improvement on the GhostELA-DWC2f module). Figure 3 The calculation is performed (as shown), and channel splicing and fusion are used to obtain the fusion features. , ;

[0134] Step 4-4-2: Continue with the fusion features as follows , The GhostELA-DWC2f-x module was used for computation, and then combined with fine features. , Channel stitching and fusion to obtain fine features after fusion. , .

[0135] Step 4-5: Select the high-quality matching set filtered in Step 4-3 Points in the map are mapped to the fused fine feature map. , In, a certain matching point pair For example, calculate , Pixel correlation of the corresponding window and correlation probability The expected coordinates are calculated as the sub-pixel position values ​​of the fused fine feature map mapping region corresponding to the matching point pair. The specific calculation method is as follows:

[0136]

[0137]

[0138] ;

[0139] Steps 4-6: Using correlation probabilities as weights, more precise sub-pixel displacements can be calculated further:

[0140] ;

[0141] ;

[0142] in The weighting function can be generated based on Gaussian weights. Control center weighted strength, generally taken .

[0143] Steps 4-7: Update the offsets of effective matching points during the registration model training process using this method to obtain the optimal set of weight parameters for the registration model. .

[0144] Step 5: Construct a convolutional neural network model for extracting changing regions. ,like Figure 4 As shown, it consists of three parts: a lightweight backbone network, an efficient Transformer neck network, and a boundary-aware decoding network. The location and category labels of all objects in the segmentation map are input to train the neural network and obtain the weight parameters. ;

[0145] Step 5-1: Convert the dual-temporal images and The input is fed into a lightweight backbone network, which uses the same backbone network as the image registration model to obtain semantic features. , ;

[0146] Step 5-2: Perform channel feature exchange and spatial feature exchange on the semantic features sequentially. Specifically, the semantic features will be... , Input to Channel Exchange module Figure 4 As shown in the figure, the exchange features are obtained, and then the GhostELA-DWC2f module is used to extract the deep exchange features. , Furthermore, half of the channel features are input to the Spatial Exchange module. Figure 4 As shown, the elements are swapped at corresponding positions in the spatial dimension. Then, GhostELA-DWC2f is used to extract deep swap features, yielding the interaction information features. , ;

[0147] Step 5-3: Input the interactive features into the feature fusion network to obtain fused features with deep semantic and spatial information. Specifically, convolution and a 2x upsampling structure are used to extract dimensionality-reduced features. , The initial fused features are obtained by using concatenation and fusion. This feature will be channel-wise concatenated and fused with the third layer of the feature extraction network, and then the GhostELA-DWC2f-x module will be used to obtain the first-layer enhanced fusion feature. , This process is repeated twice to obtain the final enhanced fusion feature. .

[0148] Step 5-4: Input the fused features into the decoding network for binary mask prediction of both changed and unchanged features. The decoding network is as follows: Figure 4 The CD-Decoder Head structure is shown below. First, bilinear interpolation is used to align the spatial scale of the input image, then a 1×1 Conv is used to map the fused features to the number of target classes. N ( N=2 The model is divided into two categories: changed and unchanged. Then, 3×3Conv is used to enhance local features and improve spatial details. Finally, the SiLU activation function is used to output a probability map of changed pixels.

[0149] Step 5-5: Construct an effective loss function during model training.

[0150] Step 5-5-1: Use binary cross-entropy loss for pixel-level classification to measure the predicted change mask. With the real mask The difference between them is calculated as follows:

[0151] ;

[0152] in, This represents the true label, where 0 indicates no change and 1 indicates change. This represents the probability predicted by the model, which is based on the SiLU function predicted in step 5-4.

[0153] Step 5-5-2: Construct boundary loss to enhance edge accuracy. Specifically, the edges of the changing regions are explicitly optimized using the Sobel operator. The boundary loss is defined as:

[0154] ;

[0155] in This represents the actual edge labels generated using the Sobel operator; This represents the Sobel output value of the model's predicted edge map.

[0156] Step 5-5-3: The total loss is a weighted combination of the boundary loss and the pixel classification loss, used for tuning the weight parameters during model training, and is expressed as:

[0157] ;

[0158] in , This represents the loss weights. The optimal set of weight parameters for the change detection model is obtained after training. .

[0159] Step 6: Using the original unaligned image For example, input it into the registration model Using a set of weight parameters Extracting images Compared to The registration point information is obtained, and the aligned image after registration is acquired.

[0160] Step 6-1: As Figure 4 As shown, assuming the image representing the first phase of the imagery is... The corresponding second phase image is Input to configuration model Outputs a set of matching point coordinate pairs in the unaligned image, and its visualization result is as follows. Figure 4 As shown in (c);

[0161] Step 6-2: Map the set of matching point coordinate pairs to the unaligned image. In the grayscale image, to calculate the homography transformation matrix. Select 4 pairs of matching points Each pair of points satisfies the following homogeneous coordinate relationship:

[0162]

[0163] ;

[0164] Will Transform into an unknown parameter vector The matrix is ​​solved using the least squares method.

[0165] Step 6-3: Based on the homography transformation matrix Perspective transformation is used to transform the image Mapping to Image Coordinate system. Obtain a registered image with irregular black borders. ( Figure 4 (d));

[0166] Step 6-4: Extract the irregular black border areas and fill them into the image. That is, to obtain the current period image Corresponding registration image Overlapping registration images ( Figure 4 (e) Irregular black border areas are extracted by detecting the locations of pixels with a value of 0 in the image.

[0167] Step 7: Align the image , Input to change detection model Using a set of weight parameters Extract the information of the binarized change regions and label them onto the original image. ,like Figure 5 As shown.

[0168] To more clearly illustrate the specific embodiments of the present invention, two examples are provided below:

[0169] Example 1: Highway Inspection Application:

[0170] Two sets of UAV imagery were collected for a section of a highway, with a time interval of 6 months. The image resolution for both images is 0.1m, and there are significant differences in rotation and scale.

[0171] First, the image is preprocessed by scaling and padding the original 4000×3000 pixel image to 1024×1024 pixels. Then, a pre-trained registration network is used to extract feature points and calculate the homography matrix to achieve image alignment.

[0172] The aligned images are input into a change detection network to identify areas of change such as newly added cracks and road surface damage.

[0173] Example 2: Railway Inspection Application:

[0174] A routine inspection of a railway section was conducted, and two sets of drone images were collected. Due to high wind speeds during the data collection, there was a significant misalignment issue between the two sets of images.

[0175] The method of this invention successfully identified safety hazards such as track deviation and slope slippage. Compared with traditional methods, the detection accuracy and efficiency are significantly improved.

[0176] Finally, for any aspects of this invention not fully described, mature technical means from the prior art are employed.

[0177] The following is a list of definitions for formula characters in this invention:

[0178]

[0179]

[0180]

[0181]

[0182] This invention is not only applicable to road and railway inspection, but can also be extended to power line inspection, pipeline monitoring, and other fields. Any equivalent substitutions or modifications made based on the essential spirit of this invention should be covered within the scope of protection of this invention.

[0183] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for intelligent change detection of non-aligned two-temporal images from an inspection drone, characterized in that, Includes the following steps: Step 1: Acquire two images from a low-altitude UAV along a fixed route and with the same sensor parameters. Step 2: Select unaligned images, perform normalization processing and registration point annotation to form a registration sample set; Step 3: Select aligned images with changed areas, perform normalization processing, label change information, and form a change detection sample set; Step 4: Construct a lightweight registration model based on feature point matching, and train it based on the registration point sample set in Step 2 to obtain the optimal training weight parameters of the lightweight registration model. In step 4, a lightweight convolutional neural network model for two-stage image registration needs to be constructed, and the model is trained using the dataset constructed in step 2 to obtain the registration model. Optimal weight parameter set The specific steps are as follows: Step 4-1: Input the pre- and post-registration temporal images A and B from the registration sample set into the lightweight feature extraction network of the registration model to obtain refined feature maps. , and semantic feature maps , ; Step 4-2: Process the semantic feature map , The matching confidence matrix is ​​calculated using local cross-attention encoding. Step 4-3: Based on the mutual proximity calculation matching rule, use a lightweight method. MLP Predict and filter high-quality matching pairs; Step 4-4: Fuse the cross-attention feature map with the features extracted by the backbone network; Step 4-5: Select the high-quality matching set filtered in Step 4-3 Points in the map are mapped to the fused fine feature map. , In the middle, calculation , Pixel correlation of the corresponding window and correlation probability The expected coordinates are calculated as the sub-pixel position values ​​of the fused fine feature map mapping region corresponding to the matching point pair. ; Steps 4-6: Using correlation probabilities as weights, further calculate more precise sub-pixel displacements: ； ； in The weighted function is generated based on Gaussian weights. For the weighted intensity of the control center, take ; Steps 4-7: Update the offsets of effective matching points during the registration model training process using this method to obtain the optimal set of weight parameters for the registration model. ; Step 5: Construct a change detection model based on the fusion of image interaction features, and train it based on the change sample set in Step 3 to obtain the optimal training weight parameters of the change detection model. Step 5 requires constructing a convolutional neural network model for extracting change regions, comprising a lightweight backbone network, an efficient Transformer neck network, and a boundary-aware decoding network. The location and category labels of all targets in the segmentation map are input to train the neural network. , For change detection model Optimal set of weight parameters; Step 6: After normalizing a pair of unaligned images, input them into the registration model constructed in Step 4, extract features and predict matching point pairs, calculate the homography matrix, and generate aligned images. Step 7: Based on the aligned image obtained in Step 6, input it into the change detection model trained in Step 5, output a binary map of change information, and overlay the binary map onto the second image of the two images to mark the change area.

2. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: Step 1 requires acquiring multiple pairs of low-altitude image sets S from two phases of UAV acquisition. The image sets include first-phase images and second-phase images acquired based on fixed waypoints and at certain time intervals.

3. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: Step 2 requires selecting two corresponding image sets from periods with different perspectives. Normalization and registration point labeling are performed to form a registration sample set. The specific steps are as follows: Step 2-1: Perform scale normalization on the acquired image using adaptive scaling and padding. Assume the size of the acquired image is... a × b， in a ≥ b, a Length of the longer side ，b For the shorter side length, the adaptive scaling factor is defined as 'a' and the normalized scale. d reciprocal of the ratio The length of the longer side after scaling is The length of the shorter side is To convert to a normalized scale, double-sided short-side zero-filling is used, resulting in single-sided short-side fill pixels. c The calculation formula is: ； Step 2-2: Manually label the key feature pixels in the normalized image pairs; form a sample set containing at least 1000 pairs of registration points, covering two image scenes with changes in illumination and viewpoint shifts, and the sample labeling information includes the coordinate labels of each pair of points.

4. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 3, characterized in that: Step 3 requires selecting two datasets that do not have perspective differences but exhibit different scale variations. The image scale is normalized to 1024×1024×3 using the method in step 2-1. Then, the change information in each pair of images is labeled to generate a binary image, which serves as the label for the change detection dataset.

5. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: Step 4-1 includes the following sub-steps: Step 4-1-1: Use depthwise separable convolution, batch normalization, and SiLU activation to perform shallow feature extraction on the input image; Step 4-1-2: Then, the GhostELA-DWC2f module is stacked multiple times for multi-scale feature extraction. First, the GhostELA-DWC2f module is stacked to process the shallow features and obtain a fine feature map. , The semantic feature map is obtained by overlaying the GhostELA-DWC2f module twice. , ; Step 4-2 includes the following sub-steps: Step 4-2-1: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] , Divide the space into 8x8 windows, and iterate through each window. m At that time, its features are converted into query vectors. Key vector Sum value vector The formula is as follows: ； in, , , These are the embedding matrices used to transform queries, key vectors, and value vectors, respectively. Step 4-2-2: Calculate from arrive And from arrive Window cross-attention score: ； ； In the formula h For the number of attention heads, and h =8, C For the number of channels, C =256, then the output feature after cross-attention calculation is represented as: ； ； Step 4-2-3: The confidence score of the match within the window is represented as follows: ； The global matching confidence matrix, formed by concatenating all windows, is represented as follows: ； In the formula: R represents the real number field; Step 4-3 includes the following sub-steps: Step 4-3-1: Match the global confidence matrix P Extracting nearest neighbor matching information ,set up Get the matching set M, where The threshold for mutual nearest neighbor matching and The specific calculation method is as follows: ; ; ; ； Step 4-3-2: For each pair of matches Arrange to obtain splicing features ,use MLP Confidence level of prediction The calculation method is as follows: ； in , For learnable weight values, retain Get high-quality matching sets from the matching pairs. ,in The confidence threshold for high-quality matching pairs; Step 4-4 includes the following sub-steps: Step 4-4-1: First, generate the cross-attention feature map. , The GhostELA-DWC2f-x module was used for calculation, and channel splicing and fusion were employed to obtain the fusion features. , ; Step 4-4-2: Continue with the fusion features as follows , The GhostELA-DWC2f-x module was used for computation, and then combined with fine features. , Channel stitching and fusion to obtain fine features after fusion. , .

6. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: Step 5 includes the following sub-steps: Step 5-1: Combine the before and after time-phase images of the change detection sample set. and The input is fed into a lightweight backbone network, which uses the same backbone network as the image registration model to obtain semantic features. , ; Step 5-2: Perform channel feature exchange and spatial feature exchange on the semantic features sequentially; , The input is fed into the channel switching module to obtain switching features, and then the GhostELA-DWC2f module is used to extract deep switching features. , Furthermore, the half-channel features are input into the spatial exchange module for exchange at corresponding positions in the spatial dimension. Then, GhostELA-DWC2f is used to extract deep exchange features to obtain the interaction information features. , ; Step 5-3: Input the interactive features into the feature fusion network to obtain fused features with deep semantic and spatial information. ; Use convolution and 2x upsampling structure to extract dimensionality-reduced features , The initial fused features are obtained by using concatenation and fusion. This feature will be channel-wise concatenated and fused with the third layer of the feature extraction network, and then the GhostELA-DWC2f-x module will be used to obtain the first-layer enhanced fusion feature. , ; Continue fusing twice in this manner to obtain the final enhanced fusion feature. ; Step 5-4: Input the fused features into the decoding network for binary mask prediction of changed and unchanged features; first, use bilinear interpolation to align the spatial scale of the input image, then use 1×1 Conv to map the fused features to the number of target categories. N ,in N= 2 Includes two categories: those that have changed and those that have not. Then, 3×3Conv is used to enhance local features and improve spatial details, and the SiLU activation function is used to output a probability map of changing pixels. Step 5-5: Construct an effective loss function during model training; Step 5-5-1: Use binary cross-entropy loss for pixel-level classification to measure the predicted change mask. With the real mask The differences between them are explained below, and the pixel classification loss is calculated as follows: ； in, This represents the true label, where 0 indicates no change and 1 indicates change. This represents the probability predicted by the model, and the value is based on the SiLU function predicted in step 5-4. Step 5-5-2: Construct boundary loss to enhance edge accuracy; explicitly optimize the edges of changing regions using the Sobel operator; the boundary loss is defined as: ； in This represents the confidence level of the true edges generated using the Sobel operator; This represents the Sobel output value of the model's predicted edge map; Step 5-5-3: Obtain the total loss, which is a weighted combination of the boundary loss and the pixel classification loss. This total loss is used for tuning the weight parameters during model training and is expressed as: ； in Weights for pixel classification loss, The weights represent the boundary loss; the optimal set of weight parameters for the change detection model is obtained after training. .

7. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: When step 6 is performed, for the unaligned preceding and following phase test images Input it into the registration model Using a set of weight parameters Extracting images Compared to The registration point information is obtained, and the aligned image after registration is acquired; the specific steps are as follows: Step 6-1: Assume the image representing the first phase of the image is The second phase image is Input to the registration model Output the set of matching point coordinate pairs in the unaligned image; Step 6-2: Map the set of matching point coordinate pairs to the unaligned before and after temporal test images. In the grayscale image, to calculate the homography transformation matrix. Select 4 pairs of matching points Each pair of points satisfies the following homogeneous coordinate relationship: ; ； Will Transform into an unknown parameter vector Then solve the matrix using the least squares method; Step 6-3: Based on the homography transformation matrix Perspective transformation is used to transform the image Mapping to Image Coordinate system; Obtaining a registered image with irregular black borders. ; Step 6-4: Extract the irregular black border areas and fill them into the image. To obtain the current period image Corresponding registration image Overlapping registration images The irregular black border areas are extracted from the locations where the pixel value is 0 in the detected image.

8. The intelligent change detection method for non-aligned two-phase images of an inspection drone according to claim 1, characterized in that: In step 7, the image needs to be aligned. , Input to change detection model Using the optimal set of weight parameters for the change detection model Extract the information of the binarized change regions and label them onto the original image. .

Images