Impact damage assessment method based on feature matching and K-means clustering

Abstract

The invention relates to an attack damage assessment method based on feature matching and K-means clustering, and belongs to the field of unmanned aerial vehicle damage assessment. The feature matching technology is taken as an important tool in the field of image processing, key feature points in images can be effectively extracted and compared, high-precision detection of target area changes is achieved, the damaged area and the damage degree can be accurately recognized by comparing image features before and after striking, and the detection accuracy is improved. And reliable data support is provided for impact damage assessment. Meanwhile, as a common unsupervised learning algorithm, K-means clustering has wide application in the fields of data mining and pattern recognition. According to the method, data is divided into K clusters through iterative computation, so that data points in the same cluster have high similarity, and data points in different clusters have low similarity. In the attack damage assessment, the K-means clustering can be used for classifying the extracted features, so that the accuracy and the refinement degree of the assessment are improved.

Description

Technical Field

[0001] The present invention relates to the field of UAV damage assessment, and specifically to a strike damage assessment method based on feature matching and K-means clustering. Background Art

[0002] Traditional strike damage assessment methods mostly rely on manual observation, on-site inspections or post-analysis. These methods are not only time-consuming and labor-intensive, but also easily affected by environmental, weather and human factors, resulting in greater uncertainty and lag in the assessment results.

[0003] With the rapid development of remote sensing technology, image processing technology and big data analysis technology, automatic assessment methods based on remote sensing images and image features have gradually become a research hotspot in the field of strike damage assessment. This type of method quantitatively assesses the strike effect by capturing and analyzing the surface change characteristics before and after the strike, such as building collapse, vegetation destruction, road damage, etc. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] In order to avoid the shortcomings of the prior art, the present invention provides a strike damage assessment method based on feature matching and K-means clustering, which is used to solve the uncertainty and lag caused by the existing manual assessment.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A strike damage assessment method based on feature matching and K-means clustering, characterized by comprising:

[0008] Obtain two drone aerial images of the same target or area before and after damage;

[0009] Performing grayscale processing on the acquired image to obtain a grayscale image;

[0010] Based on the grayscale image, interpolation and scaling are performed to obtain a uniform size;

[0011] Based on the deep learning SuperPoint / SuperGlue algorithm, feature points are extracted from the scaled grayscale image and the matching relationship of feature points is calculated;

[0012] The homography transformation matrix is ​​calculated based on the feature point matching relationship and the random sampling consensus algorithm; based on the homography transformation matrix, the image before the impact is matched to the image after the impact through perspective transformation to obtain a matching image;

[0013] A batch of sampling points with equal coordinates and equal intervals are set for the matching image and the image after the impact, the feature information of the sampling points is calculated by the SIFT algorithm, and filtering is performed based on the feature information to obtain the difference points of the first damage area;

[0014] Cluster the difference points of the first damaged area based on the K-means clustering algorithm, filter the sampling points far from the sampling center, and obtain the difference points of the second damaged area;

[0015] A labeling rectangle is drawn based on the pixel coordinates of the difference points in the second damaged area, thereby realizing automatic analysis and labeling of the damaged area.

[0016] A further technical solution of the present invention is: the acquisition of two drone aerial images of the same target or area before and after damage is specifically as follows:

[0017] The UAV optoelectronic payload is used to obtain two UAV aerial images of the same target or area before and after damage.

[0018] A further technical solution of the present invention is: grayscale processing of the acquired image to obtain a grayscale image is specifically as follows:

[0019] The acquired image is gray-scaled based on the RGB pixel values ​​of the image to obtain an 8-bit grayscale image.

[0020] A further technical solution of the present invention is: the grayscale images are interpolated and scaled to obtain a uniform size, specifically:

[0021] The bicubic interpolation algorithm is used to interpolate the pixels of the grayscale image to obtain a grayscale image of uniform size before and after damage.

[0022] A further technical solution of the present invention: the method of extracting feature points from the scaled grayscale image and calculating the matching relationship between the feature points based on the deep learning SuperPoint / SuperGlue algorithm comprises:

[0023] The grayscale image before damage and the grayscale image after damage are respectively input into the trained SuperPoint network model to obtain the feature points and feature vectors of the corresponding images;

[0024] Input the feature points and feature vectors into the trained SuperGlue network model to obtain the feature point matching relationship.

[0025] A further technical solution of the present invention: filtering based on feature information includes:

[0026] Set the image difference feature threshold;

[0027] The features of all sampling points are traversed in turn, and the mean square error of the SIFT features of the sampling points at the same position of the image before and after damage is calculated. If the mean square error is greater than the image difference feature threshold, the difference point is retained.

[0028] A further technical solution of the present invention: clustering the difference points of the first damage area based on the K-means clustering algorithm comprises:

[0029] Set the distance threshold between each sampling point and the cluster center point;

[0030] Each cluster center is traversed, and the distance between the sampling point and the cluster center in each category is calculated. When the distance exceeds the distance threshold, the sampling point is filtered.

[0031] A computer system, characterized in that it comprises: one or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0032] A computer-readable storage medium, characterized in that it stores computer-executable instructions, which are used to implement the above method when executed.

[0033] A computer program product, characterized by comprising computer executable instructions, wherein the instructions are used to implement the above method when executed.

[0034] The beneficial effects of the present invention are:

[0035] The present invention provides a strike damage assessment method based on feature matching and K-means clustering. Based on the feature matching of deep learning SuperPoint / SuperGlue, it can accurately capture and match subtle features in the image, and can still maintain high robustness under complex scenes and lighting changes, thereby ensuring the accuracy of image matching before calculating the difference points; the difference points after feature matching are screened using the K-means clustering algorithm, which effectively reduces the influence of mismatching and redundant information, and can automatically distinguish significant changes caused by damage from natural differences in the image, thereby improving the efficiency and accuracy of damage identification. The entire assessment process is highly automated, reducing the need for manual intervention. From feature extraction to cluster analysis, and then to the final damage annotation, it all relies on algorithm processing, which significantly improves the intelligent level of damage assessment. The assessment results are presented in an intuitive and visual way, which is convenient for users to quickly understand the degree and location of damage, and provide strong support for subsequent decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0037] Figure 1 The figure is a schematic diagram of a damage assessment process in an embodiment of the present invention.

[0038] Figure 2 A schematic diagram of automatic annotation of damaged images in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] The present invention provides a strike damage assessment method based on feature matching and K-means clustering. Considering that feature matching technology is an important tool in the field of image processing, it can effectively extract and compare key feature points in the image, thereby realizing high-precision detection of changes in the target area. By comparing the image features before and after the strike, the damaged area and the degree of damage can be accurately identified, providing reliable data support for strike damage assessment. At the same time, K-means clustering, as a commonly used unsupervised learning algorithm, has a wide range of applications in the fields of data mining and pattern recognition. This method divides the data into K clusters through iterative calculation, so that the data points in the same cluster have a higher similarity, while the data points between different clusters have a lower similarity. In strike damage assessment, K-means clustering can be used to classify the extracted features, thereby improving the accuracy and refinement of the assessment.

[0042] The present invention proposes a strike damage assessment method based on feature matching and K-means clustering, comprising:

[0043] Obtain two drone aerial images of the same target or area before and after damage;

[0044] Processing the acquired image to obtain a grayscale image;

[0045] Based on the grayscale image, interpolation and scaling are performed to obtain a uniform size;

[0046] Based on the deep learning SuperPoint / SuperGlue algorithm, feature points are extracted from the scaled grayscale image and the matching relationship between the feature points is calculated;

[0047] The homography transformation matrix is ​​calculated based on the random sampling consensus algorithm, and the image before the impact is matched to the image after the impact through perspective transformation to obtain the matching image;

[0048] A batch of sampling points with equal coordinates and equal intervals are set for the matching image and the image after the impact. The feature description of the sampling points is calculated by the SIFT algorithm. The threshold is set and the sampling points with too small feature differences are filtered to obtain a rough description of the difference points in the damage area.

[0049] The sampling points are clustered based on the K-means clustering algorithm, and the sampling points far from the sampling center are filtered to obtain a more accurate description of the difference points in the damage area;

[0050] On the client interface, the damaged area annotation box and the difference points are drawn according to the difference points, realizing the automatic analysis and annotation of the damaged area.

[0051] Preferably, the step of then acquiring two drone aerial images of the same target or area before and after the damage comprises: acquiring images of the target before and after the damage by using the drone's optoelectronic payload.

[0052] Preferably, the processing of the acquired image to obtain a grayscale image includes: obtaining an 8-bit grayscale image using RGB pixel values ​​of the image.

[0053] Preferably, the interpolation and scaling based on the grayscale image to obtain a uniform size includes: using a bicubic interpolation algorithm to perform interpolation operations on the pixels of the grayscale image to obtain a grayscale image of uniform size before and after damage.

[0054] Preferably, the extracting feature points from the scaled grayscale image based on the deep learning SuperPoint / SuperGlue algorithm and calculating the matching relationship between the feature points includes:

[0055] The grayscale image before damage and the grayscale image after damage are respectively input into the trained SuperPoint network model to obtain the feature points and feature descriptors of the corresponding images;

[0056] The feature points and feature descriptors are input into the trained SuperGlue network model to obtain the feature point matching relationship.

[0057] Preferably, the step of calculating the homography transformation matrix based on a random sampling consensus algorithm and matching the image before the impact to the image after the impact to obtain a matching image by perspective transformation comprises:

[0058] Based on the feature point matching relationship and the random sampling consensus algorithm, the optimal homography matrix H is calculated;

[0059] Based on the optimal homography matrix, the pre-damage image is matched to the post-damage image through perspective transformation to obtain the matched image.

[0060] Preferably, a batch of sampling points with equal coordinates and equal intervals are set for the matching image and the post-strike image, feature descriptions of the sampling points are calculated using a SIFT algorithm, thresholds are set and sampling points with too small feature differences are filtered to obtain a rough description of difference points in the damaged area, including: setting a sampling density of the sampling points, sampling equal positions of the post-strike image and the matching image, and obtaining two groups of sampling points.

[0061] Preferably, the sampling points are clustered based on the K-means clustering algorithm, and the sampling points far from the sampling center are filtered to obtain a more accurate description of the difference points in the damage area, including:

[0062] Clustering is performed based on the K-means clustering algorithm and the filtered sampling points. According to the obtained cluster center, the sampling points with farther locations are filtered.

[0063] Set a threshold for the number of sampling points in the cluster area, and filter cluster centers with too few points based on the threshold to obtain a more accurate description of the difference points in the damage area.

[0064] Preferably, the damaged area annotation box and the difference points are drawn on the client interface according to the difference points, thereby realizing the automatic analysis and annotation of the damaged area, including: drawing the annotation rectangular box based on the pixel coordinates of the difference points of the damaged area; adding an optional box control to control the display of different difference damaged areas.

[0065] In order to enable those skilled in the art to better understand the present invention, the present invention is described in detail below in conjunction with specific embodiments.

[0066] Example 1

[0067] like Figure 1 As shown, the following steps are included:

[0068] Step 1: Obtain two drone aerial images of the same target or area before and after damage. After shooting, check the image quality in time to ensure that the image is clear and not blurred or distorted; compare the images before and after damage to ensure that the shooting targets are consistent and highly comparable; save and back up the captured image data in time.

[0069] Step 2: Process the acquired image to obtain a grayscale image: The acquired image usually includes three components, R, G, and B, which respectively display various colors such as red, green, and blue. Use the cvtColor interface of OpenCV to average the grayscale of the three components of R, G, and B of the image to calculate the grayscale image.

[0070] Step 3: Based on the grayscale image, use the resize interface of OpenCV, select the interpolation method as cv.INTER_CUBIC, and scale both the images before and after the damage to a size of 320x240.

[0071] Step 4: Extract feature points from the scaled grayscale image based on the deep learning SuperPoint / SuperGlue algorithm and calculate the matching relationship between the feature points, including:

[0072] Step 4.1. Set the YAML configuration parameters of the SuperPoint / SuperGlue network model: set the maximum number of feature points to be extracted to 500, the boundary point filtering width to 4, the input feature tensor name to "input", and the output feature tensor name to "scores" and "descriptors".

[0073] Step 4.2, input the grayscale images before and after damage into the SuperPoint network model, and obtain partial feature points and feature vectors of the images before and after damage respectively (the image before damage is passed through the SuperPoint network model to obtain partial feature points and feature vectors before damage, and the image after damage is passed through the SuperPoint network model to obtain partial feature points and feature vectors after damage).

[0074] Step 4.3, post-process some feature points and feature vectors, set the feature point filtering threshold to 0.004, filter the points with low feature point scores, remove the feature points at the image boundary points, and finally set the variable Eigen::Matrix through the third-party library Eigen<double,259,Eigen::Dynamic> , put the processed feature points and feature vectors into this variable.

[0075] Step 4.4: Input the damaged image before and after into Eigen::Matrix<double,259,Eigen::Dynamic> Normalize the image and input it into the SuperGlue network model to obtain the feature point matching relationship between the images. Use std::vector <cv::dmatch>The result is stored.

[0076] Step 5. Set a set of variable thresholds {0.05, 0.1, 0.2, 0.5, 0.9} for the matching relationship distance, starting from 0.05. If the matching relationship with a distance score less than the threshold is greater than 4 under this threshold, then use this set of results, otherwise continue to iterate backward. Use the random sampling consensus algorithm to calculate the homography transformation matrix H through the cv::findHomography interface, set the maximum number of iterations to 2000, and the confidence level to 0.995; select the pre-damage image for perspective transformation, and use the cv::warpPerspective interface to match the image to the post-damage image to obtain the matching image.

[0077] Step 6: Set a batch of sampling points with equal coordinates and equal intervals for the matching image and the image after the impact, calculate the feature description of the sampling points through the SIFT algorithm, set the threshold and filter the sampling points with too small feature differences, and obtain a rough description of the difference points in the damage area. The steps are as follows:

[0078] Step 6.1, set the sampling point pixel size to input image width * 0.01, pixel sampling density DetectDensity = 2, and sampling point pixel interval interval to sampling point pixel size * DetectDensity.

[0079] Step 6.2, use the cv::xfeatures2d::SiftDescriptorExtractor::create method to create the SIFT descriptor extraction class, and use the compute method to extract SIFT features by declaring std::vector <cv::keypoint>Stores the characteristic sampling points at the specified location.

[0080] Step 6.3, set the image difference feature threshold to 400, traverse the feature information of all sampling points in turn, calculate the mean square error of the SIFT features of the sampling points at the same position in the image before and after damage, and choose to retain the difference point if the mean square error is greater than the threshold.

[0081] Step 7: Cluster the sampling points based on the K-means clustering algorithm and filter the sampling points that are far from the sampling center. The marking is as follows: Figure 2 As shown, the specific steps are as follows:

[0082] Step 7.1, set the iterative termination conditions of the K-means clustering algorithm through OpenCV's cv::TermCriteria. The termination conditions are: stop when the maximum number of iterations is reached (cv::TermCriteria::COUNT) and stop when the algorithm converges to the set accuracy (cv::TermCriteria::EPS). The maximum number of iterations is 10, and the algorithm accuracy is 0.1.

[0083] Step 7.2: Based on the sampling points obtained in step 6, use the cv::kmeans interface to perform clustering calculations to obtain the labels and cluster center coordinates of each class.

[0084] Step 7.3, set the distance threshold between each sampling point and the cluster center to 50; traverse each cluster center, calculate the distance between the sampling point and the cluster center in each category, filter the sampling points with a long distance, and obtain a more accurate description of the difference points in the damage area.

[0085] Step 8. Based on the difference sampling points and cluster categories selected in step 7, the client interface is designed using Qt 5.12. QComboBox is used to prompt the user to select the annotation area, and the annotation rectangle is managed and drawn through QGraphicsScene.

[0086] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural transformation made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.< / cv::keypoint> < / cv::dmatch>

Claims

1. A strike damage assessment method based on feature matching and K-means clustering, characterized in that: include: Obtain two drone aerial images of the same target or area before and after damage; Performing grayscale processing on the acquired image to obtain a grayscale image; Based on the grayscale image, interpolation and scaling are performed to obtain a uniform size; Based on the deep learning SuperPoint / SuperGlue algorithm, feature points are extracted from the scaled grayscale image and the matching relationship of feature points is calculated; Calculate the homography transformation matrix based on the feature point matching relationship and random sampling consensus algorithm; Based on the homography transformation matrix, the image before the impact is matched to the image after the impact through perspective transformation to obtain a matching image; A batch of sampling points with equal coordinates and equal intervals are set for the matching image and the image after the impact, the feature information of the sampling points is calculated by the SIFT algorithm, and filtering is performed based on the feature information to obtain the difference points of the first damage area; Cluster the difference points of the first damaged area based on the K-means clustering algorithm, filter the sampling points far from the sampling center, and obtain the difference points of the second damaged area; A labeling rectangle is drawn based on the pixel coordinates of the difference points in the second damaged area, thereby realizing automatic analysis and labeling of the damaged area.

2. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The obtaining of two UAV aerial images of the same target or area before and after damage specifically includes: obtaining two UAV aerial images of the same target or area before and after damage by using the UAV optoelectronic payload.

3. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The grayscale processing of the acquired image to obtain a grayscale image is specifically as follows: The acquired image is gray-scaled based on the RGB pixel values ​​of the image to obtain an 8-bit grayscale image.

4. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The grayscale images are interpolated and scaled to obtain a uniform size, specifically: The bicubic interpolation algorithm is used to interpolate the pixels of the grayscale image to obtain a grayscale image of uniform size before and after damage.

5. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The method of extracting feature points from the scaled grayscale image and calculating the matching relationship between the feature points based on the deep learning SuperPoint / SuperGlue algorithm includes: The grayscale image before damage and the grayscale image after damage are respectively input into the trained SuperPoint network model to obtain the feature points and feature vectors of the corresponding images; Input the feature points and feature vectors into the trained SuperGlue network model to obtain the feature point matching relationship.

6. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The filtering based on the feature information includes: Set the image difference feature threshold; The features of all sampling points are traversed in turn, and the mean square error of the SIFT features of the sampling points at the same position of the image before and after damage is calculated. If the mean square error is greater than the image difference feature threshold, the difference point is retained.

7. The strike damage assessment method based on feature matching and K-means clustering according to claim 1 is characterized in that: The clustering of the difference points of the first damaged area based on the K-means clustering algorithm includes: Set the distance threshold between each sampling point and the cluster center point; Each cluster center is traversed, and the distance between the sampling point and the cluster center in each category is calculated. When the distance exceeds the distance threshold, the sampling point is filtered.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

9. A computer-readable storage medium, characterized in that Computer executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 1.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 1 when being executed.

Images