Image change detection method and device, electronic equipment and storage medium

Abstract

This application provides an image change detection method, apparatus, electronic device, and storage medium. The method includes: acquiring a first image captured by a drone at a first moment and a second image captured at a second moment for the same area; determining a set of feature point matching pairs based on the first and second images; determining a target homography matrix based on each feature point matching pair in the feature point matching pair set; converting the pixel coordinates of each pixel in the second image to target pixel coordinates based on the target homography matrix, and aligning the second image with the first image based on each target pixel coordinate to obtain an aligned second image; determining multiple difference maps based on the first image and the aligned second image, and determining the difference result between the first and second images based on each difference map. This achieves accurate alignment between the old and new images and improves the accuracy of the difference detection results.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image change detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of low-altitude intelligent sensing technology, unmanned aerial vehicle (UAV) inspection systems have been widely applied in critical infrastructure management scenarios such as power line inspection, oil pipeline monitoring, forest fire prevention patrols, and urban infrastructure operation and maintenance. In these applications, image change detection is the core analytical task, aiming to automatically identify the dynamic evolution of ground features by comparing remote sensing images of the same geographical area acquired at different times.

[0003] Existing image detection methods directly subtract the pixel values ​​of two images taken at different times, and then use thresholding to identify the difference regions. Existing methods mainly rely on pixel-level differences, resulting in poor robustness and inaccurate detection results. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing an image change detection method, apparatus, electronic device, and storage medium to improve the accuracy of image change detection.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide an image change detection method, the method comprising: Acquire a first image of the same area captured by the drone at a first moment and a second image captured at a second moment, the second moment being later than the first moment; Based on the first image and the second image, a set of feature point matching pairs is determined. The set of feature point matching pairs includes at least one feature point matching pair. Each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point. Based on each feature point matching pair in the feature point matching pair set, a target homography matrix is ​​determined, which is used to map each pixel in the second image to the first image; The pixel coordinates of each pixel in the second image are converted into target pixel coordinates according to the target homography matrix, and the second image is aligned with the first image according to the target pixel coordinates to obtain the aligned second image. Based on the first image and the aligned second image, a plurality of difference maps are determined, and the difference result between the first image and the second image is determined based on each of the difference maps.

[0006] Optionally, determining the feature point matching pair set based on the first image and the second image includes: The first image and the second image are respectively subjected to illumination correction using a histogram equalization algorithm to obtain the corrected first image and the corrected second image; The ORB algorithm is used to extract feature points from the corrected first image and the corrected second image respectively, resulting in multiple first feature points and multiple second feature points. The first feature points are feature points on the first image, and the second feature points are feature points on the second image. Feature point matching is performed on the plurality of first feature points and the plurality of second feature points to obtain the feature point matching pair set.

[0007] Optionally, determining the target homography matrix based on each feature point matching pair in the feature point matching pair set includes: A: Select at least one target feature point matching pair from the set of feature point matching pairs, and obtain the initial homography matrix based on each target feature point matching pair using the direct linear transformation algorithm; B: Based on the initial homography matrix, transform the second feature points in the remaining feature point matching pairs (excluding the target feature point matching pairs) by projecting coordinates to obtain the transformed coordinates of each second feature point; C: Determine the distance between the transformed coordinates of each second feature point and the coordinates of the corresponding first feature point, and obtain multiple distances; D: Determine an initial set of interior points based on the multiple distances and distance thresholds, wherein the initial set of interior points includes at least one second feature point; E: Determine whether the iteration termination condition is met; If not, select a new target feature point matching pair and repeat steps A to E until the iteration termination condition is met; if yes, end the iteration and determine the target homography matrix based on the obtained multiple interior point sets.

[0008] Optionally, converting the pixel coordinates of each pixel in the second image into target pixel coordinates based on the target homography matrix includes: Multiply the pixel coordinates of the pixel point by the target homography matrix to obtain the homogeneous coordinates of the pixel point; The homogeneous coordinates are normalized to obtain non-homogeneous coordinates, and the target pixel coordinates of the pixel are determined by the bilinear interpolation algorithm.

[0009] Optionally, determining multiple difference maps based on the first image and the aligned second image includes: The brightness similarity, contrast similarity, and structural similarity between the first image and the aligned second image are calculated using a sliding local window method, resulting in a brightness difference map, a contrast difference map, and a structural difference map.

[0010] Optionally, determining the difference between the first image and the second image based on each of the difference maps includes: An adaptive threshold segmentation process is performed on each of the difference maps to obtain a binary transformation mask corresponding to each of the difference maps. Morphological filtering, dilation, and opening operations are sequentially performed on each of the binary transformation masks to obtain the difference results corresponding to each difference map.

[0011] Optionally, it also includes: In the HSV color space, semantic color exclusion processing is performed on each of the aforementioned difference maps to obtain a color mask; For the changing regions in the color mask, calculate the first average pixel intensity of the changing regions in the first image and the second average pixel intensity of the changing regions in the second image; The change type of the changed region is determined based on the first average pixel intensity and the second average pixel intensity.

[0012] Secondly, embodiments of this application also provide an image change detection device, the device comprising: The acquisition module is used to acquire a first image of the same area captured by the UAV at a first moment and a second image captured at a second moment, wherein the second moment is later than the first moment; The determining module is configured to determine a set of feature point matching pairs based on the first image and the second image, wherein the set of feature point matching pairs includes at least one feature point matching pair, and each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point; The determination module is used to determine a target homography matrix based on each feature point matching pair in the feature point matching pair set, wherein the target homography matrix is ​​used to map each pixel in the second image to the first image; The alignment module is used to convert the pixel coordinates of each pixel in the second image into target pixel coordinates according to the target homography matrix, and to align the second image with the first image according to the target pixel coordinates to obtain an aligned second image that is aligned with the first image. The determining module is configured to determine multiple difference maps based on the first image and the aligned second image, and determine the difference result between the first image and the second image based on each of the difference maps.

[0013] Optionally, the determining module is specifically used for: The first image and the second image are respectively subjected to illumination correction using a histogram equalization algorithm to obtain the corrected first image and the corrected second image; The ORB algorithm is used to extract feature points from the corrected first image and the corrected second image respectively, resulting in multiple first feature points and multiple second feature points. The first feature points are feature points on the first image, and the second feature points are feature points on the second image. Feature point matching is performed on the plurality of first feature points and the plurality of second feature points to obtain the feature point matching pair set.

[0014] Optionally, the determining module is specifically used for: A: Select at least one target feature point matching pair from the set of feature point matching pairs, and obtain the initial homography matrix based on each target feature point matching pair using the direct linear transformation algorithm; B: Based on the initial homography matrix, transform the second feature points in the remaining feature point matching pairs (excluding the target feature point matching pairs) by projecting coordinates to obtain the transformed coordinates of each second feature point; C: Determine the distance between the transformed coordinates of each second feature point and the coordinates of the corresponding first feature point, and obtain multiple distances; D: Determine an initial set of interior points based on the multiple distances and distance thresholds, wherein the initial set of interior points includes at least one second feature point; E: Determine whether the iteration termination condition is met; If not, select a new target feature point matching pair and repeat steps A to E until the iteration termination condition is met; if yes, end the iteration and determine the target homography matrix based on the obtained multiple interior point sets.

[0015] Optionally, the alignment module is specifically used for: Multiply the pixel coordinates of the pixel point by the target homography matrix to obtain the homogeneous coordinates of the pixel point; The homogeneous coordinates are normalized to obtain non-homogeneous coordinates, and the target pixel coordinates of the pixel are determined by the bilinear interpolation algorithm.

[0016] Optionally, the determining module is specifically used for: The brightness similarity, contrast similarity, and structural similarity between the first image and the aligned second image are calculated using a sliding local window method, resulting in a brightness difference map, a contrast difference map, and a structural difference map.

[0017] Optionally, the determining module is specifically used for: An adaptive threshold segmentation process is performed on each of the difference maps to obtain a binary transformation mask corresponding to each of the difference maps. Morphological filtering, dilation, and opening operations are sequentially performed on each of the binary transformation masks to obtain the difference results corresponding to each difference map.

[0018] Optionally, the determining module is further specifically used for: In the HSV color space, semantic color exclusion processing is performed on each of the aforementioned difference maps to obtain a color mask; For the changing regions in the color mask, calculate the first average pixel intensity of the changing regions in the first image and the second average pixel intensity of the changing regions in the second image; The change type of the changed region is determined based on the first average pixel intensity and the second average pixel intensity.

[0019] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores program instructions executable by the processor. When the application runs, the processor communicates with the memory via the bus, and the processor executes the program instructions to perform the steps of the image change detection method described in the first aspect.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is read and executes the steps of the image change detection method described in the first aspect.

[0021] The beneficial effects of this application are: This application provides an image change detection method, apparatus, electronic device, and storage medium. After acquiring a first image of the same area captured by a UAV at a first moment and a second image captured at a second moment, a set of feature point matching pairs is determined based on the first and second images. A target homography matrix is ​​determined based on each feature point matching pair in the set. The pixel coordinates of each pixel in the second image are converted to target pixel coordinates based on the target homography matrix, and the second image is aligned with the first image based on the target pixel coordinates to obtain an aligned second image. Multiple difference maps are determined based on the first image and the aligned second image, and the difference between the first and second images is determined based on each difference map. Employing the ORB+RANSAC feature matching and homography transformation method, sub-pixel-level accurate image alignment is achieved, robustly handling slight attitude and position differences of the UAV in different flight missions, and improving the stability of the entire system. Using SSIM instead of simple pixel difference in the difference detection stage yields results closer to human visual perception, significantly reducing false alarms caused by non-substantial changes. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic flowchart of an image change detection method provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating the second image change detection method provided in this application embodiment; Figure 3 A schematic flowchart illustrating the third image change detection method provided in this application embodiment; Figure 4 A flowchart illustrating the fourth image change detection method provided in this application embodiment; Figure 5 A schematic diagram of an apparatus for an image change detection method provided in an embodiment of this application; Figure 6 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0025] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0027] Optionally, the image change detection method provided in this application embodiment can be applied to an electronic device, such as a mobile phone, tablet computer, laptop computer, PDA, desktop computer, or other terminal device with computing power and display function, or it can be a server. Specifically, it can be applied to applications in terminal devices, such as mobile phone applications (APP) and computer application systems.

[0028] The following section will explain in detail the specific implementation process of image change detection provided in the embodiments of this application.

[0029] Figure 1 This is a flowchart illustrating an image change detection method provided in an embodiment of this application. The execution subject of this method is as described above: electronic device. Figure 1 As shown, the method includes: S101. Acquire the first image of the same area captured by the UAV at the first moment and the second image captured at the second moment.

[0030] The second moment is later than the first moment. That is, the first image shows the surface condition of the drone at an earlier moment, and the second image shows the surface condition at a later moment.

[0031] Optionally, both the first and second images are acquired using the same drone with positioning and attitude determination capabilities, targeting the same geographic coordinate range. This geographic coordinate range is a preset latitude and longitude bounding box, and the acquired first and second images have partially overlapping common areas with an overlap rate greater than a preset percentage. Due to differences in the drone's flight trajectory, altitude, and attitude, there are ensemble distortions and pixel shifts between the first and second images. The preset percentage is, for example, 85%, to ensure the spatial basis for subsequent feature matching and homography modeling.

[0032] S102. Determine the set of feature point matching pairs based on the first image and the second image.

[0033] The feature point matching pair set includes at least one feature point matching pair, and each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point. Furthermore, each first feature point is obtained by feature point extraction based on a first image, and each second feature point is obtained by feature point extraction based on a second image. Each feature point may include its pixel coordinates and its descriptor. The ORB (Oriented Fast and Rotated BRIEF) feature extraction algorithm can be used for feature extraction.

[0034] For example, the pixel coordinates of the first feature point A1 are (120, 85), the descriptor is scale 3.2, and the orientation is 45°; the pixel coordinates of the first feature point A2 are (450, 230), the descriptor is scale 2.8, and the orientation is 120°; the pixel coordinates of the first feature point A3 are (890, 510), the descriptor is scale 4.1, and the orientation is 200°. The pixel coordinates of the second feature point B1 are (125, 90), the descriptor is scale 3.1, and the orientation is 43°; the pixel coordinates of the second feature point B2 are (455, 235), the descriptor is scale 2.7, and the orientation is 118°; the pixel coordinates of the second feature point B3 are (895, 515), the descriptor is scale 4.0, and the orientation is 198°.

[0035] Optionally, the set of feature point matching pairs can be, for example, P={(A1,B1),(A2,B2),(A3,B3),…,(Ai,Bx)}, where (Ai,Bx) is a feature point matching pair, Ai is the i-th first feature point, Bx is the x-th second feature point, and Bx is the second feature point that corresponds one-to-one with Ai. S103. Determine the target homography matrix based on each feature point matching pair in the feature point matching pair set.

[0036] The target homography matrix is ​​used to map each pixel in the second image to the first image. The target homography matrix describes the projection transformation relationship between the first and second images.

[0037] Optionally, due to the influence of cloud shadow edges, occlusion by newly built facilities, and vegetation swaying, the initial feature point matching pairs contain a large number of correct matches and a certain proportion of incorrect matches. Directly using all feature point matching pairs will cause serious deviations in the calculated homography matrix. Therefore, under the constraints of significant out-point contamination, sparse and spatially heterogeneous in-point distribution, and sub-pixel-level imaging uncertainty, after obtaining the matching results based on the feature point matching pair set, the RANSAC (Random Sample Consensus) algorithm is used to perform RANSAC homography transformation to robustly estimate the target homography transformation model that can optimally describe the projection geometry between two images.

[0038] S104. Convert the pixel coordinates of each pixel in the second image to target pixel coordinates according to the target homography matrix, and align the second image with the first image according to the target pixel coordinates to obtain the aligned second image.

[0039] Here, target pixel coordinates refer to the pixel coordinates of each pixel in the second image mapped to the pixel coordinates in the first image according to the target homography matrix. For example, if the pixel coordinates of pixel B1 in the second image are (x2, y2), the target pixel coordinates obtained by transforming the target homography matrix are (x2, y2). , ).

[0040] Optionally, after obtaining the optimal projective geometry model, the pixel coordinates of each pixel in the second image are redistributed in high-fidelity space according to their semantic correspondence in the first image, thereby generating a remapped image with maximum comparability in both set structure and radiation characteristics. After converting the pixel coordinates of each pixel in the second image into target pixels, the first and second images can be aligned based on the target pixel coordinates of each pixel and the pixel coordinates of each pixel in the first image. The aligned second image needs to have the same size and coordinate system as the first image. For example, if the width of the first image is 1920 and the height is 1080, the width of the aligned second image is 1920 and the height is 1080.

[0041] S105. Based on the first image and the aligned second image, determine multiple difference maps, and determine the difference results between the first image and the second image based on each difference map.

[0042] Optionally, after the first image is aligned with the aligned second image (i.e., after the first image and the aligned second image are superimposed), the similarity between the two images can be detected from three different dimensions using the Structural Similarity Index Measure (SSIM) difference detection method, thereby obtaining difference maps in different dimensions. The difference results between the first image and the second image are then determined based on these difference maps. These difference results can include structural difference results, intensity difference results, etc.

[0043] In this embodiment, after acquiring a first image of the same area captured by the UAV at a first moment and a second image captured at a second moment, a set of feature point matching pairs is determined based on the first and second images. A target homography matrix is ​​determined based on each feature point matching pair in the set. The pixel coordinates of each pixel in the second image are converted to target pixel coordinates based on the target homography matrix, and the second image is aligned with the first image based on the target pixel coordinates to obtain an aligned second image. Multiple difference maps are determined based on the first image and the aligned second image, and the difference between the first and second images is determined based on each difference map. The ORB+RANSAC feature matching and homography transformation method achieves sub-pixel-level accurate image alignment, robustly handling slight attitude and position differences of the UAV in different flight missions, thus improving the stability of the entire system. SSIM is used in the difference detection stage instead of simple pixel difference, and its results are closer to human visual perception, significantly reducing false alarms caused by non-substantial changes.

[0044] Figure 2 This is a flowchart illustrating the second image change detection method provided in the embodiments of this application, as shown below. Figure 2 As shown, in step S102 above, determining the feature point matching pair set based on the first image and the second image may include: S201. Apply histogram equalization algorithm to the first image and the second image respectively for illumination correction to obtain the corrected first image and the corrected second image.

[0045] Specifically, the first and second images can be divided into multiple small regions, and histogram equalization can be performed independently in each small region. Over-enhancement is prevented by cropping restrictions, and finally, the boundaries are smoothed by bilinear interpolation to effectively eliminate unstructured changes caused by weather and lighting differences, thereby obtaining the corrected first image and the corrected second image.

[0046] S202. The ORB algorithm is used to extract feature points from the corrected first image and the corrected second image respectively, resulting in multiple first feature points and multiple second feature points.

[0047] Wherein, each first feature point is a feature point on the first image, and each second feature point is a feature point on the second image.

[0048] Specifically, the corrected first image is input into the ORB feature detection module, where keypoint detection and descriptor generation are performed to quickly extract rotationally robust first feature points from the corrected first image. Similarly, the corrected second image is input into the ORB feature detection module, where keypoint detection and descriptor generation are performed to quickly extract rotationally robust second feature points from the corrected second image.

[0049] S203. Perform feature point matching on multiple first feature points and multiple second feature points to obtain a set of feature point matching pairs.

[0050] Specifically, a brute-force matcher and Hamming distance can be used to determine high-similarity feature point matching pairs based on the descriptors of each first feature point and each second feature point, and the combination of each feature point matching pair is taken as the feature point matching pair set.

[0051] Figure 3 A flowchart illustrating the third image change detection method provided in this application embodiment is shown below. Figure 3 As shown, in step S103 above, determining the target homography matrix based on each feature point matching pair in the feature point matching pair set may include: S301. Select at least one target feature point matching pair from the feature point matching pair set, and obtain the initial homography matrix based on each target feature point matching pair and using the direct linear transformation algorithm.

[0052] Optionally, four non-collinear feature point matching pairs can be randomly selected from the feature point matching pair set as the target feature point matching pairs. The initial homography matrix is ​​then calculated using the direct linear transformation algorithm to solve the linear method set.

[0053] As mentioned above, P={(A1,B1),(A2,B2),(A3,B3),…,(Ai,Bx)}, and the feature point matching pairs (A1,B1),(A2,B2),(A3,B3) and (A4,B4) are four non-collinear feature point matching pairs. Then, (A1,B1),(A2,B2),(A3,B3) and (A4,B4) are taken as each target feature point matching pair, and the initial homography matrix H1 is obtained by using the direct linear transformation algorithm based on these four target feature point matching pairs.

[0054] S302. Based on the initial homography matrix, transform the projection coordinates of the second feature points in the remaining feature point matching pairs (excluding the target feature point matching pairs) to obtain the transformed coordinates of each second feature point.

[0055] The transformed coordinates of each second feature point refer to the reprojected coordinates obtained by reprojecting each second feature point.

[0056] For example, as mentioned above, the remaining feature point matching pairs include the feature point matching pairs {(A5,B5),(A6,B6),(A7,B7),...,(Ai,Bx)}. The second feature points B5, B6, B7, etc., can be transformed by projecting their coordinates based on the initial homography matrix H1 to obtain the transformed coordinates B51 for B5, B61 for B6, B71 for B7, and so on, to obtain the transformed coordinates of each second feature point.

[0057] S303. Determine the distance between the transformed coordinates of each second feature point and the coordinates of the corresponding first feature point, and obtain multiple distances.

[0058] Here, the first feature point corresponding to each second feature point refers to the first feature point in the feature point matching pair formed with each second feature point. Each distance refers to the reprojection error between the second feature point and the first feature point that matches that second feature point after reprojection.

[0059] For example, the distance between the transformed coordinates B51 of the second feature point B5 and the pixel coordinates of the first feature point A5 in the first image can be calculated to obtain distance 1; the distance between the transformed coordinates B61 of the second feature point B6 and the pixel coordinates of the first feature point A6 in the first image can be calculated to obtain distance 2; the distance between the transformed coordinates B71 of the second feature point B7 and the pixel coordinates of the first feature point A7 in the first image can be calculated to obtain distance 3, and so on, to obtain multiple distances.

[0060] S304. Determine the initial set of interior points based on multiple distances and distance thresholds.

[0061] The initial set of interior points includes at least one second feature point.

[0062] Specifically, if any one of the multiple distances is less than the distance threshold, the feature point matching pair corresponding to that distance is added as an inlier to the initial inlier set. If the distance is greater than or equal to the distance threshold, the feature point matching pair corresponding to that distance is treated as an outlier.

[0063] For example, if the distance 1 mentioned above is less than the distance threshold, then the feature point matching pair (A5, B5) is treated as an inlier.

[0064] S305. Determine whether the iteration termination condition is met.

[0065] The iteration termination condition refers to the number of iterations reaching the preset total number of iterations, or the absence of other target feature point matching pairs in the feature point matching pair set.

[0066] Specifically, if the iteration termination condition is met, then execute step S306; if the iteration termination condition is not met, then select a new target feature point matching pair and return to execute step S301.

[0067] S306. Determine the target homography matrix based on the obtained set of multiple interior points.

[0068] Here, the interior point set among the multiple interior point sets refers to the interior point set obtained at the end of each iteration. That is, at each iteration, a homography matrix calculated in that round and an interior point set determined in that round are obtained.

[0069] Specifically, the number of inliers in the multiple inlier sets is compared, and the inlier set with the largest number of inliers is selected as the target inlier set. The target homography matrix is ​​then obtained by refitting the feature point matching pairs in the target inlier set using the least squares method.

[0070] For example, in the first iteration, a homography matrix H1 and an interior point set 1 are obtained; in the second iteration, a homography matrix H2 and an interior point set 2 are obtained; in the third iteration, a homography matrix H3 and an interior point set 3 are obtained; in the second iteration, a homography matrix H4 and an interior point set 4 are obtained; in the third iteration, a homography matrix H5 and an interior point set 5 are obtained, and so on, until the iteration ends, a homography matrix H1000 and an interior point set 1000 are obtained. Among them, the interior point set 5 has the largest number of interior points, so the target homography matrix is ​​obtained by refitting based on all feature point matching pairs in the interior point set 5.

[0071] Figure 4 This is a flowchart illustrating the fourth image change detection method provided in the embodiments of this application, as shown below. Figure 4 As shown, in step S104 above, converting the pixel coordinates of each pixel in the second image into target pixel coordinates based on the target homography matrix can include: S401. Multiply the pixel coordinates of the pixel with the target homography matrix to obtain the homogeneous coordinates of the pixel.

[0072] Specifically, it can be calculated using the following formula (a).

[0073] Formula (1) Where H is the target homography matrix, ( ) represents the pixel coordinates of a pixel in the second image. , , ) represents the homogeneous coordinates of the pixel.

[0074] S402. Normalize the homogeneous coordinates to obtain non-homogeneous coordinates, and use the bilinear interpolation algorithm to determine the target pixel coordinates of the pixel.

[0075] Specifically, the non-homogeneous coordinates (X, Y) can be obtained using the formulas X = x' / w' and Y = y' / w'. Since non-homogeneous coordinates are generally not integers, a bilinear interpolation algorithm is used to calculate the target pixel coordinates of the non-homogeneous coordinate by taking a weighted average of the four neighboring pixels. This avoids jagged edges or holes after image alignment, ensuring a geometrically accurate match between the first image and the aligned second image.

[0076] Optionally, determining multiple difference maps based on the first image and the aligned second image in step S105 above may include: The brightness similarity, contrast similarity, and structural similarity between the first image and the aligned second image are calculated using a sliding local window method, resulting in a brightness difference map, a contrast difference map, and a structural difference map.

[0077] The comparison process involves several steps: brightness comparison, which compares the mean pixel values ​​of each pixel within a local region of the two images (mean representing the average brightness of the region); and contrast comparison, which compares the standard deviation of pixel values ​​within a local region of the two images (standard deviation representing the degree of variation in pixel intensity). After brightness and contrast comparisons, brightness and contrast factors are removed from both the first and second images. Then, a structural comparison is performed by calculating a structural similarity index between the two images. This structural similarity index reflects structural information such as the edges and shapes of objects in the images.

[0078] The brightness difference map, contrast difference map, and structural difference map are the same size as the first or second image. The pixel value of each pixel in the brightness difference map is used to indicate the similarity between the first and second images in the brightness dimension. The pixel value of each pixel can be in the range of [-1, 1]. The closer the pixel value is to 1, the closer the brightness of the area is in the first and second images, which belongs to the "unchanged area". The closer the pixel value is to 0 or negative, the closer the brightness of the area is to a "suspected changed area".

[0079] The pixel values ​​of each pixel in the contrast difference map are used to indicate the similarity between the first image and the second image in the contrast dimension. The pixel values ​​of each pixel can range from [-1, 1]. The closer the pixel value is to 1, the closer the contrast of the region is in the first image and the second image, and it belongs to the "unchanged region". The closer the pixel value is to 0 or negative, the more significant the difference is in the contrast dimension, and it belongs to the "suspected changed region".

[0080] The pixel values ​​of each pixel in the structural difference map are used to indicate the similarity between the first image and the second image in the structural dimension. The pixel values ​​of each pixel can range from [-1, 1]. The closer the pixel value is to 1, the more similar the structure of the region is in the first image and the second image, and it belongs to the "unchanged region". The closer the pixel value is to 0 or negative, the more significant the difference is in the structural dimension, and it belongs to the "suspected changed region".

[0081] In this embodiment, the structural similarity index difference detection method abandons the traditional difference method which is sensitive to illumination and pixel changes, and instead adopts SSIM to comprehensively evaluate from three aspects: structure, brightness and contrast, which greatly improves the accuracy of detection and resistance to non-substantial changes.

[0082] Optionally, the result of determining the difference between the first image and the second image based on each difference map in S105 above may include: Optionally, adaptive threshold segmentation is performed on each difference map to obtain a binary transformation mask corresponding to each difference map. Then, morphological filtering, dilation, and opening operations are sequentially performed on the binary transformation masks corresponding to each difference map to obtain the difference results corresponding to each difference map. Specifically, the dilation operation connects adjacent regions and fills small holes, and the opening operation removes thin lines or isolated small noise points, further purifying the difference regions.

[0083] In this embodiment, the method better reflects human visual perception, is less sensitive to changes in lighting, and converts each difference map into a visualized difference result.

[0084] Optionally, the method may further include: Semantic color exclusion processing is performed on each difference map in the Hue, Saturation, Value (HSV) color space to obtain a color mask. For the change regions in the color mask, the first average pixel intensity of the change regions in the first image and the second average pixel intensity of the change regions in the second image are calculated. Based on the first average pixel intensity and the second average pixel intensity, the change type of the change region is determined. The change type can be addition or removal; it can also be brightening or darkening.

[0085] In this embodiment, a mask is created in the HSV color space to perform semantic color exclusion, eliminating specific color regions that may generate false alarms due to seasonal or lighting changes, such as green vegetation and water bodies. This makes the change results more accurate. Furthermore, the difference results are intuitive and information-rich. By judging the average pixel intensity of the changed area in the first and second images, the nature of the change—whether it is "added" or "removed"—can be clearly distinguished, and different colors are used for encoding. This intuitive visualization method greatly reduces the difficulty and time cost of manual review.

[0086] This application combines adaptive thresholding, morphological filtering, and semantic color exclusion to transform the SSIM difference map into a clear, accurate, and low-false-positive-rate change mask, improving the usability of the final result. Furthermore, based on directional change judgment according to pixel intensity comparison, it simply and effectively distinguishes whether the change is "added" or "removed," providing crucial information for manual review.

[0087] Figure 5 This is a schematic diagram of an apparatus for an image change detection method provided in an embodiment of this application, as shown below. Figure 5 As shown, the device includes: The acquisition module 501 is used to acquire a first image of the same area captured by the UAV at a first moment and a second image captured at a second moment, wherein the second moment is later than the first moment; The determining module 502 is used to determine a set of feature point matching pairs based on the first image and the second image. The set of feature point matching pairs includes at least one feature point matching pair, and each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point. The determining module 502 is used to determine a target homography matrix based on each feature point matching pair in the feature point matching pair set, wherein the target homography matrix is ​​used to map each pixel in the second image to the first image; Alignment module 503 is used to convert the pixel coordinates of each pixel in the second image into target pixel coordinates according to the target homography matrix, and to align the second image with the first image according to each target pixel coordinate, so as to obtain an aligned second image that is aligned with the first image; The determining module 502 is configured to determine multiple difference maps based on the first image and the aligned second image, and determine the difference result between the first image and the second image based on each of the difference maps.

[0088] Optionally, the determining module 502 is specifically used for: The first image and the second image are respectively subjected to illumination correction using a histogram equalization algorithm to obtain the corrected first image and the corrected second image; The ORB algorithm is used to extract feature points from the corrected first image and the corrected second image respectively, resulting in multiple first feature points and multiple second feature points. The first feature points are feature points on the first image, and the second feature points are feature points on the second image. Feature point matching is performed on the plurality of first feature points and the plurality of second feature points to obtain the feature point matching pair set.

[0089] Optionally, the determining module 502 is specifically used for: A: Select at least one target feature point matching pair from the set of feature point matching pairs, and obtain the initial homography matrix based on each target feature point matching pair using the direct linear transformation algorithm; B: Based on the initial homography matrix, transform the second feature points in the remaining feature point matching pairs (excluding the target feature point matching pairs) by projecting coordinates to obtain the transformed coordinates of each second feature point; C: Determine the distance between the transformed coordinates of each second feature point and the coordinates of the corresponding first feature point, and obtain multiple distances; D: Determine an initial set of interior points based on the multiple distances and distance thresholds, wherein the initial set of interior points includes at least one second feature point; E: Determine whether the iteration termination condition is met; If not, select a new target feature point matching pair and repeat steps A to E until the iteration termination condition is met; if yes, end the iteration and determine the target homography matrix based on the obtained multiple interior point sets.

[0090] Optionally, the alignment module 503 is specifically used for: Multiply the pixel coordinates of the pixel point by the target homography matrix to obtain the homogeneous coordinates of the pixel point; The homogeneous coordinates are normalized to obtain non-homogeneous coordinates, and the target pixel coordinates of the pixel are determined by the bilinear interpolation algorithm.

[0091] Optionally, the determining module 502 is specifically used for: The brightness similarity, contrast similarity, and structural similarity between the first image and the aligned second image are calculated using a sliding local window method, resulting in a brightness difference map, a contrast difference map, and a structural difference map.

[0092] Optionally, the determining module 502 is specifically used for: An adaptive threshold segmentation process is performed on each of the difference maps to obtain a binary transformation mask corresponding to each of the difference maps. Morphological filtering, dilation, and opening operations are sequentially performed on each of the binary transformation masks to obtain the difference results corresponding to each difference map.

[0093] Optionally, the determining module 502 is further specifically used for: In the HSV color space, semantic color exclusion processing is performed on each of the aforementioned difference maps to obtain a color mask; For the changing regions in the color mask, calculate the first average pixel intensity of the changing regions in the first image and the second average pixel intensity of the changing regions in the second image; The change type of the changed region is determined based on the first average pixel intensity and the second average pixel intensity. Figure 6 This is a structural block diagram of an electronic device 600 provided in an embodiment of this application. (See diagram below.) Figure 6 As shown, the electronic device may include: a processor 601 and a memory 602.

[0094] Optionally, a bus 603 may also be included, wherein the memory 602 is used to store machine-readable instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 and the memory 602 communicate via the bus 603, and the processor 601 executes the machine-readable instructions to perform the method steps in the above method embodiments.

[0095] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the method steps described in the above-described image change detection method embodiments.

[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0098] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image change detection method, characterized in that, The method includes: Acquire a first image of the same area captured by the drone at a first moment and a second image captured at a second moment, the second moment being later than the first moment; Based on the first image and the second image, a set of feature point matching pairs is determined. The set of feature point matching pairs includes at least one feature point matching pair. Each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point. Based on each feature point matching pair in the feature point matching pair set, a target homography matrix is ​​determined, which is used to map each pixel in the second image to the first image; The pixel coordinates of each pixel in the second image are converted into target pixel coordinates according to the target homography matrix, and the second image is aligned with the first image according to the target pixel coordinates to obtain the aligned second image. Based on the first image and the aligned second image, a plurality of difference maps are determined, and the difference result between the first image and the second image is determined based on each of the difference maps.

2. The image change detection method according to claim 1, characterized in that, The step of determining the feature point matching pair set based on the first image and the second image includes: The first image and the second image are respectively subjected to illumination correction using a histogram equalization algorithm to obtain the corrected first image and the corrected second image; The ORB algorithm is used to extract feature points from the corrected first image and the corrected second image respectively, resulting in multiple first feature points and multiple second feature points. The first feature points are feature points on the first image, and the second feature points are feature points on the second image. Feature point matching is performed on the plurality of first feature points and the plurality of second feature points to obtain the feature point matching pair set.

3. The image change detection method according to claim 1, characterized in that, The step of determining the target homography matrix based on each feature point matching pair in the feature point matching pair set includes: A: Select at least one target feature point matching pair from the set of feature point matching pairs, and obtain the initial homography matrix based on each target feature point matching pair using the direct linear transformation algorithm; B: Based on the initial homography matrix, transform the second feature points in the remaining feature point matching pairs (excluding the target feature point matching pairs) by projecting coordinates to obtain the transformed coordinates of each second feature point; C: Determine the distance between the transformed coordinates of each second feature point and the coordinates of the corresponding first feature point, and obtain multiple distances; D: Determine an initial set of interior points based on the multiple distances and distance thresholds, wherein the initial set of interior points includes at least one second feature point; E: Determine if the iteration termination condition is met; If not, select a new target feature point matching pair and repeat steps A to E until the iteration termination condition is met; if yes, end the iteration and determine the target homography matrix based on the obtained multiple interior point sets.

4. The image change detection method according to claim 1, characterized in that, The step of converting the pixel coordinates of each pixel in the second image into target pixel coordinates based on the target homography matrix includes: Multiply the pixel coordinates of the pixel point by the target homography matrix to obtain the homogeneous coordinates of the pixel point; The homogeneous coordinates are normalized to obtain non-homogeneous coordinates, and the target pixel coordinates of the pixel are determined by the bilinear interpolation algorithm.

5. The image change detection method according to claim 1, characterized in that, The step of determining multiple difference maps based on the first image and the aligned second image includes: The brightness similarity, contrast similarity, and structural similarity between the first image and the aligned second image are calculated using a sliding local window method, resulting in a brightness difference map, a contrast difference map, and a structural difference map.

6. The image change detection method according to claim 5, characterized in that, The determination of the difference between the first image and the second image based on each of the difference maps includes: An adaptive threshold segmentation process is performed on each of the difference maps to obtain a binary transformation mask corresponding to each of the difference maps. Morphological filtering, dilation, and opening operations are sequentially performed on each of the binary transformation masks to obtain the difference results corresponding to each difference map.

7. The image change detection method according to claim 6, characterized in that, Also includes: In the HSV color space, semantic color exclusion processing is performed on each of the aforementioned difference maps to obtain a color mask; For the changing regions in the color mask, calculate the first average pixel intensity of the changing regions in the first image and the second average pixel intensity of the changing regions in the second image; The change type of the changed region is determined based on the first average pixel intensity and the second average pixel intensity.

8. An image change detection device, characterized in that, include: The acquisition module is used to acquire a first image of the same area captured by the UAV at a first moment and a second image captured at a second moment, wherein the second moment is later than the first moment; The determining module is configured to determine a set of feature point matching pairs based on the first image and the second image, wherein the set of feature point matching pairs includes at least one feature point matching pair, and each feature point matching pair includes a first feature point and a second feature point that corresponds one-to-one with the first feature point; The determination module is used to determine a target homography matrix based on each feature point matching pair in the feature point matching pair set, wherein the target homography matrix is ​​used to map each pixel in the second image to the first image; The alignment module is used to convert the pixel coordinates of each pixel in the second image into target pixel coordinates according to the target homography matrix, and to align the second image with the first image according to the target pixel coordinates to obtain an aligned second image that is aligned with the first image. The determining module is configured to determine multiple difference maps based on the first image and the aligned second image, and determine the difference result between the first image and the second image based on each of the difference maps.

9. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor executes the computer program to implement the steps of the image change detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image change detection method as described in any one of claims 1-7.

Images