An extended target hartmann wavefront detection method based on multi-feature matching fusion

Abstract

The application provides an extended target Hartmann wavefront detection method based on multi-feature matching fusion. The method obtains multiple feature regions of an extended target image through feature point detection, matching and extraction; then, correlation operation is performed on each feature region and a corresponding feature region of a reference image respectively to calculate the position offset of the feature region relative to the corresponding feature region of the reference image; and the average value of the position offsets of all the feature regions is taken as a sub-aperture offset required for wavefront reconstruction. The application combines the correlation algorithm and image matching to reduce the wavefront reconstruction residual in the extended target Hartmann detection, which can effectively improve the stability and precision of the extended target correlation Hartmann wavefront detection. The method has a simple process and is easy to implement, and provides technical support for better application of the extended target wavefront detection system in the fields of astronomical observation, target identification, target imaging and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of wavefront detection and image processing, and particularly relates to an extended target Hartmann wavefront detection method based on multi-feature matching fusion. BACKGROUND

[0002] With the development of the application field of adaptive optics (AO) systems such as laser high-beam-quality transmission, target identification and target imaging, in some scenarios, the observation target of people is not limited to the initial point target, and these targets are generally referred to as extended targets. For the definition of the extended target, it is generally recognized that the target occupying more than 2 / 3 of the area of the field of view is referred to as an extended target by defining the number of pixels of the imaged target. Compared with astronomical observation, the objects in the fields of target identification and target imaging are mostly moving non-cooperative targets, and the target environment changes in real time. Influenced by camera acquisition, sub-aperture imaging may have some missing, dimness, flickering and the like, and the wavefront detection based on the Hartmann correlation will face the problem of reduced detection accuracy.

[0003] A large number of studies have been conducted on extended target Hartmann wavefront detection. The main achievements are as follows: In 1998, the NSO low-order solar adaptive system was successfully applied, marking the birth of the correlation Hartmann-based extended target wavefront detector (Rao C, Jiang W, Ling N, et al. Correlation tracking algorithms for low-contrast extended object [C] / / Adaptive Optics Systems and Technology II. International Society for Optics and Photonics, 2002, 4494: 245-251); In 2003, L. A. Poyneer et al. conducted a systematic study on the correlation algorithm based on FFT and parabolic interpolation, and proposed the idea of extending the correlation algorithm to wavefront detection of extended targets other than the sun. When the relative offset between sub-images is 0, the interpolation error formula of the correlation function is derived, and a simple result is obtained (Poyneer LA. Scene-based Shack-Hartmann wave-front sensing: analysis and simulation [J]. Applied Optics, 2003, 42(29): 5807-5815); In 2005, the team conducted a distortion wavefront detection experiment on an extended target of 100 meters in the horizontal direction at a height of 1 to 2 meters above the hot asphalt road in California. The experiment proved the feasibility of the correlation Hartmann wavefront detector for horizontal extended target wavefront detection (Poyneer LA, Palmer D W, LaFortune K N, et al. Experimental results for correlation-based wavefront sensing [C] / / Advanced Wavefront Control: Methods, Devices, and Applications III. International Society for Optics and Photonics, 2005, 5894: 58940N); In 2019, Wang Yawen et al. from Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences proposed the gamma transform correlation algorithm and the gradient cross-correlation algorithm, which effectively solved the problems of low wavefront detection accuracy of low-contrast extended targets and low detection accuracy of edge targets in the feature border field (Wang Yawen. Correlation Hartmann wavefront detection method based on extended target [D]. University of the Chinese Academy of Sciences (Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences), 2019).

[0004] The existing Hartmann wavefront detection research of the extended target is analyzed, and how to improve the detection accuracy from the perspective of combining image multi-feature matching fusion and correlation algorithm has not been reported at home and abroad. The application provides an extended target Hartmann wavefront detection method based on multi-feature matching fusion. The method detects, matches and extracts feature points, and then performs correlation operation to obtain the sub-aperture offset required for wavefront reconstruction. The application combines the correlation algorithm and image matching to reduce the wavefront reconstruction residual in the extended target Hartmann detection. The process is simple, stable and easy to implement. SUMMARY

[0005] The technical problem to be solved by the application is that when the Hartmann wavefront detection extended target is detected, the sub-aperture imaging may be partially missing, dark, and flickering, which leads to a decrease in detection accuracy. In view of this problem, a method of using reference and sub-aperture image matching is proposed to ensure higher accuracy and stability in wavefront detection in complex environments.

[0006] The technical solution adopted by the application to solve the technical problem is: an extended target Hartmann wavefront detection method based on multi-feature matching fusion, which comprises the following steps:

[0007] Step 1) Obtain a sub-aperture image array from a Hartmann sensor with a sub-aperture unit of (2P-1)×(2P-1) or 2P×2P, and the size of a single aperture image is N×N pixels. The single-pixel gray value is represented by I , is the offset of the center pixel point of the single sub-aperture image and the center pixel point of the reference image I in the horizontal and vertical directions. The center sub-aperture of the Hartmann sub-aperture array is selected as the reference image I position. The row and column numbers of the sub-aperture correspond to P. The reference image I is usually selected in the center sub-aperture with a 3 / 4 field size area. The ratio and position can be adjusted according to the actual different extended target pixel number in the Hartmann sub-aperture and the sub-aperture image background. The pixel size of the reference image I is M×M, and N≥M. The single-pixel gray value is represented by I r (x,y), and the pixel gray value data format of the reference image and the sub-aperture image is converted to double type, and then normalized, that is, the single-pixel gray value is divided by the maximum gray value of the whole image, so that the gray value range is between [0, 1];

[0008] Step 2) Detect the key points of the reference image and each sub-aperture image, assign directions to the key points, and then construct the key point descriptors to generate key regions. Then, the key regions of the two are matched to generate feature regions.

[0009] Step 3) the normalized correlation coefficient matrix obtained by one-to-one corresponding the feature area obtained after the reference image and the sub-aperture image feature matching is calculated by using a normalized correlation algorithm (NCC in the following text), and the correlation calculation formula is shown as (1), wherein is the average gray value of the reference image feature point pixel set, is the average gray value of the single sub-aperture image pixel set. The maximum value in the correlation coefficient matrix element is the average value of the position where the maximum value of the correlation matrix obtained by each pair of matching is located, that is, the sub-aperture offset.

[0010]

[0011] Further, the key point detection and key area matching method in step 2) is not limited to the scale invariant feature transform matching algorithm (SIFT), but can also be the speeded up robust features algorithm (SURF), the principal component analysis-scale invariant feature transform matching algorithm (PCA-SIFT).

[0012] Further, after obtaining the feature point set in step 3) and obtaining the average value of the offset of each point set, the gray value weight of the feature point set can also be selected to directly obtain the overall offset.

[0013] Compared with the prior art, the present application has the following advantages:

[0014] 1. In the present application, in the extended target wavefront detection, in the background environment of missing, dark, flickering and the like of the sub-aperture image acquisition part, the calculation precision of the peak position of the correlation function of the NCC algorithm can be effectively improved, and the wavefront recovery residual RMS (Root Mean Square) and PV value (Peak to Valley) are improved.

[0015] 2. In the present application, the image matching fusion is combined with the correlation algorithm, the wavefront recovery precision is effectively improved without increasing the system hardware complexity, and a technical selection for the application of image matching in wavefront detection is provided.

[0016] 3. In the present application, the feature point matching and extraction method reduces the requirement of the wavefront detection optical system hardware to a certain extent, and improves the environmental adaptability of the system to the detection of extended targets. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The operation flow of the image matching method in the application for detecting an extended target;

[0018] Figure 2 The extended target is a four-wing unmanned aerial vehicle;

[0019] Figure 3 The detection of the feature points of the reference image and the sub-aperture respectively;

[0020] Figure 4 The extraction and matching results of the feature points of the reference image and the missing content sub-aperture;

[0021] Figure 5 The imaging result of the 13x13 sub-aperture;

[0022] Figure 6 The wavefront reconstruction wave surface diagram, wherein, Figure 6 (a) is a Zernike aberration, Figure 6 (b) is a reconstruction aberration, Figure 6 (c) is a reconstruction residual error;

[0023] Figure 7 The Gaussian pyramid under the image feature extraction;

[0024] Figure 8 The wavefront reconstruction process under the feature extraction of the detected four-wing unmanned aerial vehicle;

[0025] Figure 9 The specific operation flowchart of the image matching method in the application for detecting an extended target. DETAILED DESCRIPTION

[0026] The application will be further described in combination with the accompanying drawings and specific embodiments.

[0027] For an extended target wavefront detection system, the extended target is a six-wing unmanned aerial vehicle, the number of sub-aperture units is 13x13, and the size of a single aperture image is 64x64 pixels.

[0028] As shown in Figure 1 and Figure 8 , the specific implementation method of the extended target Hartmann wavefront detection method based on multi-feature matching fusion of the application is as follows:

[0029] Step 1) detecting a four-wing unmanned aerial vehicle as shown in Figure 2 , obtaining a sub-aperture image array from a Hartmann sensor with a sub-aperture unit of 13x13, taking the sub-aperture at the center of the Hartmann sub-aperture array (the sub-aperture corresponds to 6 rows and 6 columns) as the selection position of the reference image, and representing the single-pixel gray value with , is the offset of the center pixel of the single-aperture image and the center pixel of the reference image in the horizontal and vertical directions, the ratio of the starting position coordinate value of the reference image I in the horizontal and vertical directions within the center sub-aperture to the pixel size of the center sub-aperture in the horizontal and vertical directions is 0.8 / 6, and the ratio of the pixel size of the reference image I in the horizontal and vertical directions to the pixel size of the center sub-aperture in the horizontal and vertical directions is 4.5 / 6, so that the pixel size of the reference image I is 48x48, and the single-pixel gray value is denoted by I r (x,y) represents. Preferably, as shown in Figure 9 The extended target sub-aperture image is obtained by using the correlation Hartmann wavefront sensor, and the size of the center sub-aperture image is taken as 3 / 4 of the reference image.

[0030] Step 2) Here, the SIFT feature extraction matching algorithm is used to convolve the reference image I r (x,y) with a Gaussian convolution kernel to obtain L r (x,y,σ),

[0031] L(x,y,σ) = I(x,y)*G(x,y,σ) (1)

[0032]

[0033] where G(x,y,σ) is a scale-variable Gaussian kernel function, σ is the standard deviation of the normal distribution in the Gaussian function, and * is a convolution operation. By convolving the original image with different scales of Gaussian kernels, the lowermost Octave1 image group is obtained. The upper Octave2 image group of the Gaussian pyramid is obtained by taking every other point and downsampling the Octave1 image group, and then convolving with different scales of Gaussian kernels, as shown in Figure 7 The reference and sub-aperture images are respectively constructed into Gaussian difference pyramids, and the scale space D(x,y,σ) of the two-dimensional image I(x,y) is represented as:

[0034] D(x,y,σ) = DoG*I(x,y) = L(x,y,kσ)-L(x,y,σ) (4)

[0035] where the DoG operator is the difference of the Gaussian function. The scale spaces D r (x,y,σ) and D a (x,y,σ) of the reference and sub-aperture images are obtained.

[0036] Step 3) After finding the extreme points of the DoG function, the curve fitting of the scale space DoG function is performed. The Taylor expansion of the DoG function in the scale space is used, i.e.:

[0037]

[0038] where T is the transpose, D and its derivatives At sample point x0, X = (x, y, σ) T As the offset from X0, this second-order Taylor expansion is shown as an approximation of the DoG function, and the derivative is taken and set to zero, then the offset of the extreme point position can be obtained That is:

[0039]

[0040]

[0041] When the offset is greater than 0.5 in any dimension, change the position of the current key point, and repeatedly interpolate at the new position until convergence. If it exceeds the set number of iterations or exceeds the image boundary range, delete the point. Secondly, The extreme point of D is also discarded.

[0042] Step 4) Eliminate edge effects and improve feature point stability. Obtain the 2x2 Hessian matrix of the principal curvature (second-order directional derivative maximum value):

[0043]

[0044] The value of D is obtained by the difference of the adjacent point pixels. The eigenvalue of H is proportional to the principal curvature of D.

[0045] Let λ be the eigenvalue of the Hessian matrix. Let λ max = α, λ min = β, we can get:

[0046]

[0047] When r = 1, that is, the two eigenvalues are equal, the X point is directly discarded. For the feature points with principal curvature ratio r greater than 10, the feature points are also discarded, and other feature points will be retained;

[0048] Step 5) Direction assignment. A direction reference is obtained according to the local image properties of the detected feature points. The stable direction of the local structure is obtained using the image gradient direction. For the feature points that have been detected, the scale value σ of the feature point is known, according to the scale value, the Gaussian image closest to the scale value is obtained, and for the feature point L(x, y, σ), its gradient is:

[0049]

[0050] The amplitude and argument of the gradient are:

[0051] ​

[0052] θ(x,y) = tan -1 ((L(x,y+1)-L(x,y-1)) / (L(x,y+1)-L(x,y-1))) (12)

[0053] According to the 3σ principle of scale sampling, the amplitude and the argument of the gradient of all points in the 3x1.5σ radius region centered on the feature point are calculated. The gradient amplitude of the feature point neighborhood calculated above is counted into a histogram according to the gradient direction. The horizontal axis of the gradient direction histogram is the gradient direction angle, and the vertical axis is the gradient amplitude cumulative value corresponding to the gradient direction angle. The direction histogram divides the 360° direction into 36 bins, and each bin represents 10°. The peak value of the histogram represents the main direction of the image gradient in the neighborhood of the feature point. The gradient amplitude of each sampling point added to the gradient histogram is weighted. The weighting uses a circular Gaussian weighting function;

[0054] Step 6) The angle between the image pixel coordinate system and the gradient direction of the key point is θ. Then for the point (x, y) in the image coordinate system, it is represented as (x', y') in the key point coordinate system after rotation, which is equivalent to counterclockwise rotation θ or clockwise rotation -θ. The conversion relationship between them is represented by the following formula:

[0055]

[0056] Then the gradient direction histogram is calculated. In each 4x4 window, the gradient amplitude and direction are calculated, and the direction is counted into a histogram with 8 bins. Each bin represents a range direction. The gradient direction between 0-44° is added to the first bin, the gradient direction between 45-89° is added to the second bin, and so on.

[0057] Step 7) Feature matching. The matching of feature points is completed by comparing the key point descriptors in the two point sets. The similarity is described by the Euclidean distance.

[0058] Step 8) After the feature matching of the reference image and the sub-aperture image, the feature points obtained as shown in Figure 3 are one-to-one corresponding to the coefficient matrix calculated by the normalized correlation algorithm. The correlation coefficient calculation formula is shown in formula (14), where is the average gray value of the reference image feature point pixel set, is the average gray value of the single sub-aperture image pixel set. The maximum value in the correlation coefficient matrix element is the average value of the position where the maximum value of the correlation matrix obtained by each pair of matching is located, that is, the sub-aperture offset.

[0059]

[0060] Step 9) obtain the relevant Hartmann sub-aperture centroid offset, restore the wavefront by using the mode method, and calculate the restored residual error between the restored wavefront and the wavefront to be corrected. Figure 6 As shown in the figure, Figure 6 (a) is a Zernike aberration, Figure 6 (b) is a reconstructed aberration, Figure 6 (c) is a reconstructed residual error.

[0061] In the embodiment, for an extended target, the background change of the extension degree affects the wavefront reconstruction accuracy. In the case of missing content as Figure 5 , the background is dark and weak, and the flicker makes the gray value of the target in the sub-aperture image change. When the missing content accounts for 1 / 16 of the sub-aperture field of view, the average values of the restored residual error RMS and PV in the simulation are increased by 44.2% and 50.4%, respectively. By extracting and matching the feature points, the average sub-aperture offset is calculated, and the average values of the restored residual error RMS and PV are reduced by 6.7% and 5.4%, respectively.

[0062] The wavefront reconstruction algorithm in step 9 can also be a region method, etc.

[0063] The contents not described in detail in the specification of the present application belong to the prior art known to those skilled in the art.

Claims

1. An extended target Hartmann wavefront sensing method based on multi-feature matching fusion, characterized in that, The implementation steps are as follows: Step 1) Obtain the sub-aperture image array from the Hartmann sensor with the number of sub-aperture units of (2P-1) x (2P-1) or 2P x 2P, (P, P) being the coordinates of the Hartmann central sub-aperture, the value of P being determined according to the actual number of sub-aperture units of the sensor, the single-aperture image size being N x N pixels, and the single-pixel gray value being represented by , (a, ) being the offset of the center pixel point of the single sub-aperture image and the center pixel point of the reference image I in the horizontal and vertical directions, the central sub-aperture being the selected position of the reference image I, the number of rows and columns corresponding to the sub-aperture being P, the pixel size of the reference image I being M x M, N≥M, the single-pixel gray value being represented by , and then performing normalization processing, i.e., dividing the single-pixel gray value by the maximum gray value of the entire image, so that the gray value range is between [0, 1]; Step 2) key point detection is performed on the reference image and each sub-aperture image respectively, a direction is given to the key point, a descriptor of the key point is constructed to generate a key region, and then key region matching of the two is performed to generate a feature region; Step 3) After the feature matching of the reference image and the sub-aperture image, the normalized correlation coefficient matrix is calculated using the normalized cross-correlation algorithm, and the normalized correlation coefficient calculation formula is shown as formula (14), wherein is the average gray value of the pixel set of the reference image feature region, is the average gray value of the pixel set of the single sub-aperture image feature region, the maximum value in the correlation coefficient matrix element, the average value of the position of the maximum value of each pair of matched correlation matrix is taken, and the difference between the position of the maximum value of the correlation coefficient matrix element and the initial calibration position is the offset of the pair of matched feature regions. The average value of the offset of each pair of matched feature regions is obtained, that is, the sub-aperture offset required for wavefront reconstruction. （14） Step 4) the relevant Hartmann sub-aperture centroid offset is obtained, a mode method is used to restore the wave front, and a restored residual error of the wave front and the wave front to be corrected is calculated.

2. The extended target Hartmann wavefront sensing method based on multi-feature matching fusion according to claim 1, characterized in that, Step 2) the key point detection and key region matching method is a scale invariant feature transform matching algorithm (SIFT), or a speeded up robust features algorithm (SURF), or a principal component analysis-scale invariant feature transform matching algorithm (PCA-SIFT).

3. The method of claim 1, wherein the method is a multi-feature matching fusion based extended target Hartmann wavefront sensing method. Step 3) after the feature point set is obtained in step 3, the average value of the offset of each point set is calculated, or the gray value weight of the feature point set is increased to directly calculate the overall offset.

Images