Image similarity comparison method, device and medium based on feature matching and segmentation

Abstract

The present application relates to a kind of feature matching and segmentation-based image similarity comparison method, equipment and medium, it includes: S1: obtaining image sample, construct comprehensive target feature;S2: using clustering segmenter to carry out superpixel segmentation to input image, in combination with similarity map generation and area selection mechanism realizes target area segmentation;S3: in target area, using SIFT to extract feature point, using FLANN matcher to carry out feature matching;S4: to feature point matching result removes abnormal matching point;S5: according to matching point set, calculate homography matrix, calculate the overlap area between images by perspective transformation, obtain overlap rate and overlap rate error;S6: according to the preset overlap rate threshold and overlap rate error threshold, the similarity of two images is automatically judged, and the similar image set is output.The accuracy of similarity judgment is significantly improved by the double optimization of target segmentation and abnormal point rejection.Experiments show that the overlap rate calculation accuracy on building image dataset reaches more than 85%.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and in particular to an image similarity comparison method, device, and medium based on feature matching and segmentation. Background Technology

[0002] With the rapid development of drone aerial photography and satellite remote sensing technologies, acquiring large amounts of building images has become increasingly easy. Drones can quickly cover large areas and obtain high-resolution images of building surfaces; satellite remote sensing technology provides more macroscopic information on the distribution of building clusters. Within these massive image datasets, there are many similar or repetitive scenes, such as images of the same building taken consecutively from different angles, or images of the same area collected at adjacent time periods. This phenomenon is particularly prevalent in fields such as building monitoring, urban planning, and infrastructure management.

[0003] Traditional image similarity calculation methods are mainly based on global features such as color histograms and texture features. However, these methods have significant shortcomings in complex scenes: they are not robust to changes in lighting, viewpoint, and local occlusion, and they struggle to handle images with similar backgrounds but different subjects. These problems are even more pronounced in the specific applications of drones and remote sensing images, because aerial images typically present challenges such as large changes in viewpoint, complex lighting conditions, and numerous background disturbances.

[0004] Existing local feature-based methods such as SIFT and SURF have improved the robustness of matching to some extent, but still face many challenges: (1) The feature points are unevenly distributed and tend to be over-concentrated in areas with rich textures, while feature points are scarce in smooth wall areas of architectural images; (2) There are many mismatched points, especially on architectural surfaces with repetitive textures, which affects the accuracy of similarity judgment; (3) Insufficient attention is paid to the main building targets in the image, and they are easily affected by background interference such as sky, trees, and vehicles; (4) There is a lack of effective use of spatial relationships in the image, making it difficult to process structural features in architectural images.

[0005] In recent years, deep learning-based methods have made some progress in image matching. However, these methods typically require large amounts of labeled data for training, resulting in high computational complexity and efficiency issues in practical deployments. In UAV and remote sensing image processing scenarios, the application of deep learning methods is limited due to the massive data volume and high real-time requirements. Therefore, there is an urgent need for an image similarity comparison method that maintains high accuracy while offering good practicality, particularly suitable for processing similar repetitive scenes in UAV aerial photography and satellite remote sensing architectural images. Summary of the Invention

[0006] This invention aims to overcome the shortcomings of existing technologies and provide an image similarity comparison method that is highly accurate, robust, and can effectively handle complex scenes in UAV aerial photography and satellite remote sensing images.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] An image similarity comparison method based on feature matching and segmentation includes the following steps:

[0009] S1: Obtain image samples from the dataset to be processed, construct a target feature learning dataset, extract depth features, handcrafted features and region features respectively, and construct comprehensive target features through clustering optimization;

[0010] S2: A clustering segmenter is used to perform superpixel segmentation on each input image, and the target region segmentation is achieved by combining a similarity map generation and region selection mechanism; the input images are obtained from the image set to be processed;

[0011] S3: Use SIFT to extract feature points in the target area obtained in step S2, use FLANN matcher to perform pairwise feature matching on each input image to obtain a set of matching feature points between input images, and use a depth filtering mechanism to retain only feature points whose depth value is greater than a preset threshold and are located in the target area.

[0012] S4: Remove abnormal matching points from the feature point matching results obtained in step S3. The methods used include concentration analysis, spatial distribution analysis, cluster analysis, and bidirectional unique matching constraints.

[0013] S5: Calculate the homography matrix based on the set of matching points obtained in step S4, and calculate the overlapping area between images through perspective transformation to obtain the overlap rate and overlap rate error;

[0014] S6: Automatically determine the similarity between input images based on the preset overlap rate threshold and overlap rate error threshold, and output a set of similar images.

[0015] A further improvement of the present invention is that, in step S1:

[0016] In the process of deep feature extraction, a pre-trained ResNet50 network model is used to extract deep features of the image, and a fixed-dimensional feature vector is obtained through a global average pooling layer.

[0017] During the extraction of manual features, color histogram features, HSV color space statistical features, and LBP texture features are calculated.

[0018] During the extraction of region features, edge density features and gradient direction features are calculated to describe the local structural properties of the image;

[0019] The features of each image sample are optimized using the K-means clustering algorithm, and the features of the largest cluster are used as representative target features.

[0020] A further improvement of the present invention is that step S2 specifically includes:

[0021] The SLIC algorithm is used to perform superpixel segmentation on the input image;

[0022] Calculate the similarity score between each superpixel region and the target feature to generate a full-image similarity map;

[0023] The region selection mechanism includes a threshold-based strategy and a region sorting strategy;

[0024] High-similarity regions obtained using the region selection mechanism are adjusted through morphological operations, region growing, and edge optimization.

[0025] A further improvement of the present invention is that, in step S3, the process of filtering feature points through a depth-of-field filtering mechanism includes:

[0026] Calculate the depth estimate of the input image;

[0027] Retain feature points whose depth of field value is greater than a set threshold;

[0028] Based on the mask of the target region obtained in step S2, delete feature points outside the target region.

[0029] A further improvement of the present invention is that, in step S4:

[0030] During the concentration analysis process, the input image is divided into grids of a predetermined size, and the concentration of matching points in each grid is calculated. If the concentration of feature points in a grid is greater than the concentration threshold, the feature points in that grid are removed.

[0031] During the spatial distribution analysis, the spatial distribution characteristics of the matching points are analyzed based on variance and entropy, and matching points with abnormal distribution are eliminated.

[0032] During the cluster analysis, the DBSCAN clustering algorithm is used to identify dense clusters and remove outlier matching points that form dense clusters in both of the two input images being compared.

[0033] Bidirectional unique matching constraint: Only one pair of matching relationships is retained for each matching point.

[0034] A further improvement of the present invention is that step S5 specifically includes:

[0035] The homography matrix is ​​estimated from the matching point pairs using the RANSAC algorithm;

[0036] Apply perspective transformation to the corner points of one image to map them to the coordinate system of another image;

[0037] The overlapping mask is generated using the convex polygon filling method;

[0038] Generate overlapping region mask by filling with convex polygons.

[0039] The bidirectional overlap ratio is calculated based on the mask area, and the smaller value is taken as the overlap rate. The difference between the two is taken as the overlap rate error.

[0040] A further improvement of the present invention is that, in step S6, the overlap rate threshold is 0.3 and the overlap rate error threshold is 0.05.

[0041] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0042] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0043] Compared with the prior art, the beneficial effects of the present invention include:

[0044] High accuracy: Through dual optimization of target segmentation and outlier removal, the accuracy of similarity judgment is significantly improved. Experiments show that the accuracy of overlap rate calculation reaches over 85% on architectural image datasets.

[0045] High robustness: Multi-feature fusion and multiple outlier removal strategies make the method more adaptable to complex situations such as changes in lighting, viewpoint, and local occlusion, making it particularly suitable for processing drone aerial images.

[0046] High practicality: The method does not rely on a large amount of labeled data, has high computational efficiency, and can be directly deployed in UAV image processing systems or remote sensing image analysis platforms, making it suitable for practical application scenarios.

[0047] High degree of automation: The entire process from feature learning to similarity judgment is automated, requiring no manual intervention, which greatly improves processing efficiency and is suitable for processing large-scale UAV and remote sensing image data.

[0048] Specifically designed for building image optimization: By learning and segmenting building features, it effectively overcomes the limitations of traditional methods in building image processing and improves its practicality in applications such as building monitoring and urban planning. Attached Figure Description

[0049] Figure 1 This is a simplified flowchart of the method of the present invention;

[0050] Figure 2 This is a detailed flowchart of the method of the present invention;

[0051] Figure 3 This is a schematic diagram of the target feature learning module structure used in this invention.

[0052] Figure 4 This is a schematic diagram of the workflow of the improved clustering segmenter used in this invention.

[0053] Figure 5 This is a schematic diagram of the multi-strategy outlier removal method used in this invention.

[0054] Figure 6 This is a diagram illustrating the architectural segmentation effect of an image captured by a drone, based on the present invention.

[0055] Figure 7 This is a similarity matching graph of building images photographed by UAVs using this method. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0057] The embodiments of the present invention provide an image similarity comparison method based on feature matching and segmentation. When implementing this method, the hardware environment used is an NVIDIA GPU (such as an RTX 4090) or a computing device with equivalent performance, which supports the deployment of UAV image processing platforms. The dependent software environment includes: Python 3.8+, OpenCV 4.5+, PyTorch 1.8+, and scikit-learn.

[0058] like Figure 1 , Figure 2 As shown, the method in this embodiment specifically includes the following steps.

[0059] S1: Obtain image samples from the dataset to be processed and construct a target feature learning dataset. Extract depth features, handcrafted features, and region features respectively, and construct a comprehensive target feature through clustering optimization; specifically:

[0060] like Figure 3 As shown, 5-10 building image samples are randomly selected from the image set to be processed to construct the target feature learning dataset. The process of constructing a comprehensive target feature model involves extracting depth features, handcrafted features, and region features. In this embodiment, the images in the processing folder are all images of the same building from different perspectives.

[0061] In this embodiment, a pre-trained ResNet50 model is used to extract deep features. Global average pooling is used to obtain a fixed-dimensional feature vector to capture high-level semantic information of the image. During the extraction of handcrafted features, color histogram features, HSV color space statistical features, and LBP (Local Binary Pattern) texture features are calculated to provide low-level visual information. During the extraction of region features, edge density features and gradient direction features are calculated to describe the local structural properties of the image, thus describing local structural characteristics.

[0062] The extracted depth features, handcrafted features, and region features are fused together, and the features of each image sample are optimized by K-means clustering algorithm, with the features of the largest cluster being used as the representative target features.

[0063] S2: A clustering segmenter is used to perform superpixel segmentation on the input image, and the target region segmentation is achieved by combining similarity map generation and region selection mechanisms; specifically:

[0064] like Figure 4 , Figure 6 As shown, the SLIC (Superpixel Segmentation) algorithm is used to segment each input image into superpixels. A similarity score is calculated between each superpixel region and the target feature, generating a full-image similarity map. The value of each pixel in the full-image similarity map represents the similarity score of the corresponding superpixel region. High-similarity regions are selected based on a dual strategy of setting a similarity threshold (0.85) and region ranking (top 20%). The segmentation mask is then adjusted through morphological operations, region growing, and edge optimization to obtain the segmentation result of the target region of the input image (the mask of the input image). The purpose of this step is to segment the target region (extract the building region) of each input image for subsequent comparison of the similarity between input images. The input images are also selected from the image set to be processed.

[0065] In one specific embodiment, the input image is an aerial photograph taken by a drone or a remote sensing image. During superpixel segmentation, the number of superpixels, n_segments, is 400, and the parameter compactness is 15.

[0066] S3: Use SIFT to extract feature points within the target area obtained in step S2, use the FLANN matcher to perform pairwise feature matching on the input images, obtain a set of matching feature points (matching points) between each pair of input images, and use a depth filtering mechanism to retain only feature points whose depth values ​​are greater than a preset threshold and are located within the target area.

[0067] In step S3, the process of filtering feature points using the depth-of-field filtering mechanism includes:

[0068] Calculate the depth estimate of the input image; in this embodiment, a depth map is constructed based on edge density, location information, and image sharpness;

[0069] Retain feature points whose depth of field value is greater than a set threshold;

[0070] Based on the mask of the target region obtained in step S2, delete feature points outside the target region.

[0071] In one specific embodiment, the parameters used in the FLANN algorithm include trees=5, checks=50, and distance ratio=0.7.

[0072] S4: Remove abnormal matching points from the feature point matching results (set of matching points) obtained in step S3. The methods used include concentration analysis, spatial distribution analysis, cluster analysis, and bidirectional unique matching constraints.

[0073] like Figure 5 , Figure 6 As shown, specifically: During the concentration analysis, the input image is divided into a 20-pixel * 20-pixel grid, and the concentration of matching points in each grid is calculated. If the concentration of feature points in a grid is greater than the concentration threshold (0.15), the feature points in that grid are removed. Based on variance and entropy analysis, the spatial distribution characteristics of matching points are analyzed, and matching points with abnormal distribution are removed. The distribution threshold is set to 0.4. During the clustering analysis, the DBSCAN clustering algorithm with eps=15 and min_samples=3 is used to remove abnormal matching points that form dense clusters in both of the compared input images. Bidirectional unique matching constraint: only one pair of matching relationships is retained for each matching point.

[0074] S5: Calculate the homography matrix based on the set of matching points obtained in step S4, and calculate the overlapping area between images through perspective transformation to obtain the overlap rate and overlap rate error;

[0075] like Figure 7 As shown, step S5 specifically includes:

[0076] The homography matrix is ​​estimated from the matching point pairs using the RANSAC algorithm;

[0077] Apply perspective transformation to the corner points of one image to map them to the coordinate system of another image;

[0078] The overlapping mask is generated using the convex polygon filling method;

[0079] Generate overlapping region mask by filling with convex polygons.

[0080] The bidirectional overlap ratio is calculated based on the mask area, and the smaller value is taken as the final overlap rate. The difference between the two is taken as the overlap rate error.

[0081] S6: Based on preset overlap rate thresholds and overlap rate error thresholds, automatically determine the similarity between input images and output a set of similar images. In a specific embodiment, the overlap rate threshold is 0.3; the overlap rate error threshold is 0.05. Input images that meet the conditions are marked as similar images and can be further processed or output.

[0082] On a dataset of drone aerial images of buildings (1797 images of 5472×3948 pixels), the method of this invention achieved the following performance metrics:

[0083] False match rejection rate: 97.8%

[0084] Overlap rate calculation accuracy: 87.3%

[0085] Duplicate image removal percentage: 61.3%

[0086] Compared to traditional SIFT matching methods and deep learning-based methods, this invention significantly improves the accuracy of similarity judgment while maintaining high efficiency, especially performing exceptionally well in scenarios with complex backgrounds, changing viewpoints, and varying lighting conditions in UAV aerial images. This method can be effectively applied to image deduplication and similarity analysis tasks in fields such as building monitoring, urban planning, and infrastructure management.

[0087] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An image similarity comparison method based on feature matching and segmentation, comprising the following steps: S1: Obtain image samples from the dataset to be processed and construct a target feature learning dataset. Extract depth features, handcrafted features, and region features respectively, and construct a comprehensive target feature through clustering optimization. S2: A clustering segmenter is used to perform superpixel segmentation on each input image, and the target region segmentation is achieved by combining a similarity map generation and region selection mechanism; the input images are obtained from the image set to be processed; S3: Use SIFT to extract feature points in the target area obtained in step S2, use FLANN matcher to perform pairwise feature matching on each input image to obtain a set of matching feature points between input images, and use a depth filtering mechanism to retain only feature points whose depth value is greater than a preset threshold and are located in the target area. S4: Remove abnormal matching points from the feature point matching results obtained in step S3. The methods used include concentration analysis, spatial distribution analysis, cluster analysis, and bidirectional unique matching constraints. S5: Calculate the homography matrix based on the set of matching points obtained in step S4, and calculate the overlapping area between images through perspective transformation to obtain the overlap rate and overlap rate error; S6: Automatically determine the similarity between input images based on the preset overlap rate threshold and overlap rate error threshold, and output a set of similar images.

2. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, In step S1: In the process of deep feature extraction, a pre-trained ResNet50 network model is used to extract deep features of the image, and a fixed-dimensional feature vector is obtained through a global average pooling layer. During the extraction of manual features, color histogram features, HSV color space statistical features, and LBP texture features are calculated. During the extraction of region features, edge density features and gradient direction features are calculated to describe the local structural properties of the image; The features of each image sample are optimized using the K-means clustering algorithm, and the features of the largest cluster are used as representative target features.

3. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, Step S2 specifically includes: The SLIC algorithm is used to perform superpixel segmentation on the input image; Calculate the similarity score between each superpixel region and the target feature to generate a full-image similarity map; The region selection mechanism includes a threshold-based strategy and a region sorting strategy; High-similarity regions obtained using the region selection mechanism are adjusted through morphological operations, region growing, and edge optimization.

4. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, In step S3, the process of filtering feature points using the depth-of-field filtering mechanism includes: Calculate the depth estimate of the input image; Retain feature points whose depth of field value is greater than a set threshold; Based on the mask of the target region obtained in step S2, delete feature points outside the target region.

5. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, In step S4: During the concentration analysis process, the input image is divided into grids of a predetermined size, and the concentration of matching points in each grid is calculated. If the concentration of feature points in a grid is greater than the concentration threshold, the feature points in that grid are removed. During the spatial distribution analysis, the spatial distribution characteristics of the matching points are analyzed based on variance and entropy, and matching points with abnormal distribution are eliminated. During the cluster analysis, the DBSCAN clustering algorithm is used to identify dense clusters and remove outlier matching points that form dense clusters in both of the two input images being compared. Bidirectional unique matching constraint: Only one pair of matching relationships is retained for each matching point.

6. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, Step S5 specifically includes: The homography matrix is ​​estimated from the matching point pairs using the RANSAC algorithm; Apply perspective transformation to the corner points of one image to map them to the coordinate system of another image; The overlapping mask is generated using the convex polygon filling method; Generate overlapping region mask by filling with convex polygons. The bidirectional overlap ratio is calculated based on the mask area, and the smaller value is taken as the overlap rate. The difference between the two is taken as the overlap rate error.

7. The image similarity comparison method based on feature matching and segmentation according to claim 1, characterized in that, In step S6, the overlap rate threshold is 0.3; The overlap rate error threshold is 0.

05.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-6.

Images