A method and system for extracting normalized structural features of multimodal remote sensing images
Through bilateral filtering and nonlinear scale spatial processing, UC-KAZE feature points are extracted, which solves the problem of image feature extraction with large nonlinear radiation differences and texture differences in multimodal images, and achieves efficient feature extraction in this case.
Patent Information
- Application Number
- CN202411480529.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-10-23
AI Technical Summary
In the prior art, when processing multimodal images with large nonlinear radiation differences and texture differences, it is difficult to effectively extract structural features, resulting in the impact of feature extraction performance.
A bilateral filter is used to smoothly filter and normalize the multimodal remote sensing image, a nonlinear scale space is constructed, KAZE feature points are detected using Hessian matrix, and UC value is calculated to extract UC-KAZE feature points.
Through bilateral filtering and nonlinear scale spatial processing, a large number of uniformly distributed feature points can be extracted on images with large nonlinear radiation differences and texture differences, improving the feature extraction performance of multimodal image matching.
Smart Images
Figure CN119006839B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image feature extraction, and in particular to a method and system for extracting normalized structural features of a multimodal remote sensing image. Background Art
[0002] In multimodal image matching technology, the extraction of feature targets with large nonlinear radiation differences and texture differences has always been a very challenging task. High-discrimination feature descriptors are usually used to identify, locate and track targets. The mainstream image feature extraction algorithms include: SIFT, Harris, FAST, etc. The above methods are suitable for image feature extraction with small radiation differences and rich texture information. However, when the image radiation difference is large or lacks texture information, the feature extraction performance of these methods will be greatly affected. Therefore, how to detect the spatial structural characteristics of regions with large nonlinear radiation differences and texture differences has become a problem that needs to be solved urgently in the current multimodal image matching technology. Summary of the invention
[0003] In view of the problems existing in the prior art, the present invention proposes a method for extracting normalized structural features of a multimodal remote sensing image, comprising the following steps:
[0004] Step 1, use a bilateral filter to smooth the multimodal remote sensing image, delete the image detail information, and obtain an image processed by bilateral filtering;
[0005] Step 2, normalizing the multimodal remote sensing image using the image processed by bilateral filtering to obtain a normalized image;
[0006] Step 3, using UC-KAZE detector to extract evenly distributed feature points in the normalized image;
[0007] Step 3.1, construct a nonlinear scale space of the normalized image;
[0008] Step 3.2, in the nonlinear scale space, the Hessian matrix is used to detect KAZE feature points;
[0009] Step 3.3, calculate the UC value of each KAZE feature point;
[0010] Step 3.4, extract UC-KAZE features.
[0011] Furthermore, the definition of bilateral filtering in step 1 is as follows:
[0012] (1)
[0013] In the formula, Represents the image after bilateral filtering; is the normalized weight; Represents the original multimodal remote sensing image The pixel value at , , N , M are the number of rows and columns of the original multimodal remote sensing image, respectively; It is a spatial domain kernel function, which is used to calculate the spatial distance weight between pixels; Represents the original multimodal remote sensing image ( x,y ) at the pixel value; It is a pixel value domain kernel function, which is used to calculate the similarity weights between pixel values.
[0014] Furthermore, the normalization calculation method in step 2 is as follows:
[0015] (2)
[0016] In the formula, represents the normalized image; Represents the original multimodal remote sensing image; Represents the image after bilateral filtering; So ( x , y ) is the center of the local window, whose size is , The value is set according to user needs.
[0017] Furthermore, the specific calculation formula of the nonlinear scale space in step 3.1 is as follows:
[0018] (8)
[0019] In the formula, div represents the divergence operator, represents the gradient operator, is the normalized image, is the nonlinear diffusion function, is the gradient of the normalized image after Gaussian smoothing, k is a constant that controls the diffusion level.
[0020] Furthermore, in step 3.2, the second-order derivative of the Hessian matrix is used to calculate the response value of the pixel L H , the calculation formula is as follows:
[0021] (9)
[0022] In the formula, is the scale space factor, , , are the elements of the Hessian matrix, To normalize the image x The second derivative of the direction, To normalize the image y The second derivative of the direction, Indicates that the normalized image is first x After taking the derivative in the direction y Direction derivative.
[0023] The pixel LH and all its neighboring pixels in the image domain and scale domain LH When the pixel LH Greater than all neighboring pixels LH When , the pixel is the local maximum point and is used as the KAZE feature point.
[0024] Furthermore, in step 3.3, the quality metric of the KAZE feature points is calculated, and the specific calculation formula is as follows:
[0025] (10)
[0026] In the formula, Indicates j The quality metric vector i The quality metric value of the KAZE feature point, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j A quality metric vector.
[0027] For each quality metric vector, the rank vector is calculated as follows:
[0028] (11)
[0029] In the formula, Indicates j The quality metric vector i The rank of the KAZE feature points, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j The rank vector of the quality metric vectors.
[0030] The calculation formula of UC value of each KAZE feature point is as follows:
[0031] (12)
[0032] In the formula, Indicates i UC value of KAZE feature points, Indicates j The weight factor of the quality metric vector, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j The quality metric vector i The rank of the KAZE feature points.
[0033] Furthermore, in step 3.4, each layer of the nonlinear scale space is divided into grids of fixed size, the KAZE feature points in each grid unit are sorted according to their UC values, and the KAZE feature point with the highest UC value is selected as the UC-KAZE feature point of the grid unit.
[0034] The present invention also provides a multimodal remote sensing image normalized structural feature extraction system, which is used to implement the multimodal remote sensing image normalized structural feature extraction method as described above.
[0035] Furthermore, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned multi-modal remote sensing image normalized structural feature extraction method.
[0036] Alternatively, it comprises a readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for extracting normalized structural features of a multimodal remote sensing image as described above is implemented.
[0037] Compared with the prior art, the present invention has the following advantages:
[0038] 1) Using bilateral filtering to normalize multimodal images can achieve the purpose of denoising and smoothing the images while extracting the spatial structural features of the images; 2) First, KAZE feature points are detected in the nonlinear scale space, and then the UC detector is used to further extract evenly distributed feature points, thereby extracting a large number of evenly distributed feature points on multimodal images with large nonlinear radiation differences and texture differences. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0040] Figure 1It is a flow chart of normalized structural feature extraction of multimodal remote sensing images according to an embodiment of the present invention.
[0041] Figure 2 1 is a multimodal remote sensing image pair according to an embodiment of the present invention, the left image is an optical image, and the right image is a near infrared (NIR) image.
[0042] Figure 3 1 is a pair of multimodal remote sensing images after normalization filtering according to an embodiment of the present invention. The left image is an optical image after normalization filtering, and the right image is a near-infrared (NIR) image after normalization filtering.
[0043] Figure 4 It is a probability distribution diagram of the structural similarity index increment ΔSSIM after normalization filtering in an embodiment of the present invention.
[0044] Figure 5 It is a cumulative distribution diagram of the structural similarity index increment ΔSSIM after normalization filtering in an embodiment of the present invention.
[0045] Figure 6 (a) - Figure 6 (f) are the feature extraction process of an embodiment of the present invention, wherein Figure 6 (a) is a multimodal remote sensing image, Figure 6 (b) is a normalized image, Figure 6 (c) is a nonlinear scale space, Figure 6 (d) is a KAZE feature point, Figure 6 (e) is a regular grid unit, and Figure 6 (f) is a UC-KAZE feature point.
[0046] Figures 7 (a) to 7 (h) are distribution diagrams of UC-KAZE feature points extracted by an embodiment of the present invention and feature points extracted by other methods; wherein, Figure 7 (a) shows feature points extracted by the method proposed in the present invention, Figure 7 (b) shows feature points extracted by the KAZE method, Figure 7 (c) shows feature points extracted by the UC-SIFT method, Figure 7 (d) shows feature points extracted by the SIFT method, Figure 7 (e) shows feature points extracted by the UC-SURF method, Figure 7 (f) shows feature points extracted by the SURF method, Figure 7 (g) shows feature points extracted by the Harris method, and Figure 7 (h) shows feature points extracted by the FAST method. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in combination with the drawings and embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] Example 1
[0049] like Figure 1 As shown, an embodiment of the present invention provides a method for extracting normalized structural features of a multimodal remote sensing image, comprising the following steps:
[0050] Step 1: Use a bilateral filter to smooth the multimodal remote sensing image, delete the image detail information, and obtain the image after bilateral filtering.
[0051] Bilateral filtering is a nonlinear filter that can reduce noise and smooth the image. The definition of bilateral filtering is as follows:
[0052] (1)
[0053] In the formula, Represents the image after bilateral filtering; is the normalized weight; Represents the original multimodal remote sensing image The pixel value at , , N , M are the number of rows and columns of the original multimodal remote sensing image, respectively; It is a spatial domain kernel function, which is used to calculate the spatial distance weight between pixels; Represents the original multimodal remote sensing image ( x,y ) at the pixel value; It is a pixel value domain kernel function, which is used to calculate the similarity weights between pixel values.
[0054] Step 2: Normalize the multimodal remote sensing image using the image processed by bilateral filtering, retain the structural information of the image and eliminate nonlinear radiation differences and texture differences to obtain a normalized image.
[0055] The normalization calculation method is as follows:
[0056] (2)
[0057] In the formula, represents the normalized image; Represents the original multimodal remote sensing image; Represents the image after bilateral filtering; So ( x , y ) is the center of the local window, whose size is In this embodiment The value is 3.
[0058] Normalization is equivalent to the original multimodal remote sensing image minus the result of bilateral filtering. The bilateral filter can smooth the original multimodal remote sensing image and delete the detail information. Therefore, the image retains a lot of detail structure information after normalization. Figure 2 As shown in Figure 2, taking a pair of multimodal remote sensing images as an example, the left image is an optical image and the right image is a near infrared (NIR) image. They are normalized separately, and the results are as follows: Figure 3 As shown, it can be found that the normalized image retains the structural information of the image and the nonlinear radiation difference is significantly reduced.
[0059] In order to quantitatively evaluate the similarity of multimodal remote sensing images before and after normalization, the structural similarity index (SSIM) is used to evaluate the images before and after normalization. SSIM is a model for measuring the similarity between two images, in which image changes are regarded as changes in the perception of structural information. The calculation formula of the structural similarity index is as follows:
[0060] (3)
[0061] In the formula, I Represents the original multimodal remote sensing image; represents the normalized image; is the brightness contrast function, used to compare the brightness between two images; is the contrast comparison function used to distinguish the range between the brightest and darkest areas of two images; It is a structural contrast function, which is used to measure whether images are similar; , , is a normal number.
[0062] in, , and The calculation formula is as follows:
[0063] (4)
[0064] (5)
[0065] (6)
[0066] In the formula, , are the local mean and standard deviation of the original multimodal remote sensing image, , are the local mean and standard deviation of the normalized image, is the covariance of the original multimodal remote sensing image and the normalized image, , and is a constant.
[0067] set up , , then formula (3) is simplified to:
[0068] (7)
[0069] In theory, normalization can greatly reduce the nonlinear radiation differences between multimodal remote sensing images, and the overlapping area of the normalized image should be closer to the overlapping area of the original multimodal image. Figure 2 As shown in Figure 2, before normalization, there is a large nonlinear radiation difference between the Optical image and the NIR image. , the two images after normalization , the SSIM value increases significantly, indicating that the Optical image and the NIR image become more similar after normalization. To verify this conclusion, 4000 image pairs were selected from the public multimodal dataset OSdataset for experiments. and , first calculate their normalized images and , and then use SSIM to measure the original multimodal remote sensing image pair ( and ) and the normalized image pair ( and ). Using the normalized and before normalization Calculate the increment . Figure 4 , Figure 5 are the structural similarity index increments after normalization filtering The probability distribution diagram and cumulative distribution diagram of The values are concentrated around 0.9; in the cumulative distribution graph, after bilateral filtering The mean value is 0.8715, which further illustrates that the normalized filtering in the present invention can extract highly similar structural features.
[0070] Step 3: Use UC-KAZE detector to extract evenly distributed feature points in the normalized image.
[0071] Step 3.1, construct a nonlinear scale space of the normalized image.
[0072] The specific calculation formula of nonlinear scale space is as follows:
[0073] (8)
[0074] In the formula, div represents the divergence operator, represents the gradient operator, is the normalized image, is the nonlinear diffusion function, is the gradient of the normalized image after Gaussian smoothing, k is a constant that controls the diffusion level.
[0075] In this embodiment, the original multimodal remote sensing image ( FIG. 6 (a) ) is normalized to obtain a normalized remote sensing image ( FIG. 6 (b) ), and then a nonlinear scale space is constructed as shown in FIG. 6 (c) .
[0076] Step 3.2: In the nonlinear scale space, the Hessian matrix is used to detect KAZE feature points.
[0077] The second-order derivative of the Hessian matrix is used to calculate the pixel response value. L H , the calculation formula is as follows:
[0078] (9)
[0079] In the formula, is the scale space factor, , , are the elements of the Hessian matrix, To normalize the image x The second derivative of the direction, To normalize the image y The second derivative of the direction, Indicates that the normalized image is first x After taking the derivative in the direction y Direction derivative.
[0080] The pixel L H and all its neighboring pixels in the image domain and scale domain L H When the pixel L H Greater than all neighboring pixels L H When , the pixel is a local maximum point, and is used as a KAZE feature point. The KAZE feature points extracted in the nonlinear scale space of this embodiment are shown in FIG6 (d).
[0081] Step 3.3, calculate the UC value of each KAZE feature point.
[0082] Calculate the quality metric of KAZE feature points. The specific calculation formula is as follows:
[0083] (10)
[0084] In the formula, Indicates j The quality metric vector i The quality metric value of the KAZE feature point, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j A quality metric vector.
[0085] For each quality metric vector, the rank vector is calculated as follows:
[0086] (11)
[0087] In the formula, Indicates j The quality metric vector i The rank of the KAZE feature points, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j The rank vector of the quality metric vectors.
[0088] The UC (Uniform Competency) value of each KAZE feature point is calculated as follows:
[0089] (12)
[0090] In the formula, Indicates i The UC value of each KAZE feature point. UC can evaluate the robustness and significance of the feature point. The feature corresponding to a higher UC value will have higher robustness and significance. Indicates j The weight factor of the quality metric vector, m is the number of quality metric vectors, n is the number of KAZE feature points, Indicates j The quality metric vector i The rank of the KAZE feature points.
[0091] Step 3.4, extract UC-KAZE features.
[0092] Each layer of the nonlinear scale space is divided into grids of fixed size. In this embodiment, The grid is divided into grids of different sizes (as shown in FIG6 (e)), the KAZE feature points of each grid unit are sorted according to their UC values, and the KAZE feature points with the highest UC value are selected as the UC-KAZE feature points of the grid unit. The UC-KAZE feature points extracted in this embodiment are shown in FIG6 (f).
[0093] Calculate the number of UC-KAZE feature points in each grid cell using the following formula:
[0094] (13)
[0095] In the formula, Indicates k The number of UC-KAZE feature points in a grid cell, It is k The sum of the UC values of all KAZE feature points in a grid cell, It is k The average UC value of all KAZE feature points in the grid cells, It is k The number of KAZE feature points in a grid cell, K represents the number of grid cells, and is the weight factor, n Represents the total number of feature points in the nonlinear scale space.
[0096] by Figure 3 Taking the normalized remote sensing image pair shown in FIG. 7 as an example, the UC-KAZE proposed in the present invention is compared with seven mainstream feature extraction methods (KAZE, UC-SIFT, SIFT, UC-SURF, SURF, Harris, and FAST). The feature extraction results are shown in FIG. 7 (a) to FIG. 7 (h), and the number of extracted feature points and the feature repetition rate are shown in Table 1.
[0097] The feature repetition rate is used to evaluate the extraction effect of each method. The specific calculation formula is as follows:
[0098] (14)
[0099] In the formula, Rep represents the feature repetition rate, cors Refers to the number of points with the same name in two normalized images, and is the number of feature points in the two images.
[0100] Table 1 Comparison results of the number of feature points and feature repetition rate
[0101]
[0102] From Figure 7 (a) and Table 1, it can be found that the feature points extracted by UC-KAZE are more evenly distributed, and the feature repetition rate is higher than that of other feature extraction methods. Although the number of feature points of UC-KAZE (4556 in the left figure and 4725 in the right figure) is lower than that of KAZE (4880 in the left figure and 5392 in the right figure), the feature repetition rate of UC-KAZE is higher than that of KAZE. Therefore, compared with the other 7 comparison methods, UC-KAZE is more suitable for multimodal remote sensing image feature detection.
[0103] Example 2
[0104] Based on the same inventive concept, the present invention also provides a multimodal remote sensing image normalized structural feature extraction system, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute a multimodal remote sensing image normalized structural feature extraction method as described above.
[0105] Example 3
[0106] Based on the same inventive concept, the present invention also provides a multimodal remote sensing image normalized structural feature extraction system, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a multimodal remote sensing image normalized structural feature extraction method as described above is implemented.
[0107] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.
[0108] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific implementation cases described, or replace them with similar methods, without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.
Claims
1. A method for extracting normalized structural features of multimodal remote sensing images, characterized in that: The following steps are involved: Step 1, use a bilateral filter to smooth the multimodal remote sensing image, delete the image detail information, and obtain an image processed by bilateral filtering; The definition of bilateral filtering is as follows: In the formula, Represents the image after bilateral filtering; is the normalized weight; Represents the original multimodal remote sensing image The pixel value at , , , are the number of rows and columns of the original multimodal remote sensing image, respectively; It is a spatial domain kernel function, which is used to calculate the spatial distance weight between pixels; Represents the original multimodal remote sensing image The pixel value at ; It is a pixel value domain kernel function, which is used to calculate the similarity weight between pixel values; Step 2, normalizing the multimodal remote sensing image using the image processed by bilateral filtering to obtain a normalized image; Step 3, use The detector extracts feature points that are evenly distributed in the normalized image; Step 3.1, construct a nonlinear scale space of the normalized image; Step 3.2, in nonlinear scale space, use Matrix detection Feature points; Step 3.3, calculate each Feature Points value; calculate The quality metric of feature points is calculated as follows: In the formula, Indicates The quality metric vector indivual The quality metric of the feature points, is the number of quality metric vectors, for The number of feature points, Indicates A quality metric vector; For each quality metric vector, the rank vector is calculated as follows: In the formula, Indicates The quality metric vector indivual The rank of the feature points, is the number of quality metric vectors, for The number of feature points, Indicates The rank vector of the quality metric vectors; Each Feature Points The value is calculated as follows: In the formula, Indicates indivual Feature Points value, Indicates The weight factor of the quality metric vector, is the number of quality metric vectors, for The number of feature points, Indicates The quality metric vector indivual Rank of feature points; Step 3.4, Extraction feature.
2. A method for extracting normalized structural features of multimodal remote sensing images according to claim 1, characterized in that: The normalization calculation method in step 2 is as follows: In the formula, represents the normalized image; Represents the original multimodal remote sensing image; Represents the image after bilateral filtering; So The local window centered on is , The value is set according to user needs.
3. A method for extracting normalized structural features of multimodal remote sensing images according to claim 1, characterized in that: The specific calculation formula of the nonlinear scale space in step 3.1 is as follows: In the formula, represents the divergence operator, represents the gradient operator, is the normalized image, is the nonlinear diffusion function, is the gradient of the normalized image after Gaussian smoothing, is a constant that controls the diffusion level.
4. A method for extracting normalized structural features of multimodal remote sensing images according to claim 1, characterized in that: Utilized in step 3.2 The second derivative in the matrix calculates the response value of the pixel , the calculation formula is as follows: In the formula, is the scale space factor, , , for The elements of the matrix, To normalize the image The second derivative of the direction, To normalize the image The second derivative of the direction, Indicates that the normalized image is first After taking the derivative in the direction Direction derivation; The pixel and all its neighboring pixels in the image domain and scale domain When the pixel Greater than all neighboring pixels When , the pixel is the local maximum point, and it is taken as Feature points.
5. The method for extracting normalized structural features of multimodal remote sensing images according to claim 1, characterized in that: Step 3.4 includes: Each layer of the nonlinear scale space is divided into a grid of fixed size, and the The feature points are based on To sort the values, select The highest value Feature points as grid cells Feature points.
6. A multimodal remote sensing image normalized structural feature extraction system, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute a multimodal remote sensing image normalized structural feature extraction method as described in any one of claims 1-5.
7. A multimodal remote sensing image normalized structural feature extraction system, characterized in that: It comprises a readable storage medium, on which a computer program is stored. When the computer program is executed, a method for extracting normalized structural features of a multimodal remote sensing image as described in any one of claims 1 to 5 is implemented.