Method for realizing composite stable imaging of optical system based on target angular point characteristics

Abstract

The invention relates to a method for realizing composite stable imaging of an optical system based on target angular point characteristics. The method comprises the following steps of: 1, adding a fast reflector on an optical path; secondly, a high-frame-frequency detector is matched; thirdly, target feature points are judged based on angular points; 4, calculating a tracking miss distance; and 5, fast reflector closed-loop control. The optical system composite stable imaging implementation method based on the target angular point features is provided, and the method has good image stable tracking performance and high application value especially for a target which is clear in edge, clear in corner angle and obvious in angular point features.

Description

Technical Field

[0001] The present invention relates to a method for realizing composite stable imaging of an optical system based on target corner point features, and belongs to the field of optical measurement. Background Art

[0002] In modern optical applications, stable target imaging is crucial. When an optical system experiences image instabilities such as jitter, offset, and blur, observations become blurred and difficult to discern, leading to deviations in target information acquisition. Image stability directly determines the performance of the optical system and influences subsequent decisions that rely on imaging analysis. Therefore, developing effective image stabilization methods is both a key and a challenge in advancing optical technology and expanding the application of optical systems.

[0003] Currently, the commonly used methods for optical system imaging stabilization control are as follows: first, mechanical stabilization. This method is low-cost, but has limited ability to suppress high-frequency vibrations, and is bulky and heavy, affecting the system's maneuverability; second, optical stabilization. This method has a fast response speed, but the system design is complex, the cost is high, and there is a lot of energy loss; third, electronic stabilization. This method can process multiple targets simultaneously, but its implementation requires strong computing power, and the image resolution and frame rate will be affected. Summary of the Invention

[0004] The purpose of the present invention is to make up for the deficiencies of the prior art and propose a method for realizing composite stable imaging of an optical system based on target corner point features.

[0005] The technical solution adopted by the present invention to solve the above-mentioned problem is: a method for realizing composite stable imaging of an optical system based on target corner point features. The method is based on the idea of ​​composite image stabilization. First, an imaging fast reflector is added to the optical system design and an adaptive high frame rate detector is selected to realize high-precision tracking. Secondly, based on the target corner point features, the target feature points are acquired on the image sensor. Thirdly, the tracking point coordinate values ​​are calculated based on the target feature point coordinate values, and the difference between the tracking point coordinate values ​​and the reference point coordinate values ​​is used to form the tracking miss distance. Finally, the tracking miss distance is used to control the deflection of the optical system imaging fast reflector in a closed loop to realize target imaging stabilization. The method comprises the following steps:

[0006] Step 1: Add a fast reflector to the optical path

[0007] 1. Add a fast reflector to the main optical path of the optical system to provide an optical hardware foundation for the realization of composite stable imaging of the optical system.

[0008] Step 2: Matching high frame rate detector

[0009] 1. Estimate the image size based on the focal length f of the optical system, the number of sensor pixels n, and the target size S and the distance L from the target to the optical system, without considering target dispersion.

[0010]

[0011] S——sensor pixel size;

[0012] f——focal length of optical system;

[0013] L——the distance between the target and the optical system;

[0014] n——the number of sensor pixels.

[0015] 2. The target imaging area on the sensor occupies at least 5×5 pixels, and a matching high frame rate detector is selected accordingly.

[0016] Step 3: Determine target feature points based on corner points

[0017] 1. Basic principles of target corner feature determination

[0018] For corner feature extraction, we use the characteristic that the grayscale change of the window image containing corner points is large when it moves in any direction. The grayscale change is expressed as:

[0019]

[0020] Where:

[0021] E(x,y)——change in grayscale value of corner point;

[0022] w u,v ——Gaussian window function;

[0023] I u,v ——The grayscale value of the image at the coordinate (u, v);

[0024] (u,v) image coordinates;

[0025] x,y are coordinate offsets;

[0026] 2. Target corner feature determination process

[0027] Will I u+x，v+y Using the first-order Taylor series approximation and ignoring the higher-order terms, we have:

[0028]

[0029] Where:

[0030] E (x，y) ——The change of gray value of corner points;

[0031] w u，v ——Gaussian window function;

[0032] x, y——coordinate offset;

[0033] I——image gray value;

[0034] ——find derivatives;

[0035] ——Find the tensor product.

[0036] Define matrices A, B, C and autocorrelation matrix M:

[0037]

[0038]

[0039] Where:

[0040] w——Gaussian window function;

[0041] X and Y are defined by Equation 3 and Equation 4.

[0042] Then we have:

[0043] E(x,y)=(x,y)M(x,y) T

[0044] E(x, y)——change in grayscale value of corner point;

[0045] x, y——coordinate offset;

[0046] M- defined by Formula 8.

[0047] It can be seen that the grayscale change E is determined by the autocorrelation matrix M.

[0048] Define the corner response function R(x, y):

[0049] R(x,y)=det(M)-k×trace(M) 2

[0050] det(M)=AB-C 2

[0051] trace(M)=A+B

[0052] det——determinant symbol;

[0053] trace - Matrix trace algorithm symbol.

[0054] If R(x, y) is greater than a certain threshold and is a local maximum point, the point (x, y) is considered to be a corner point.

[0055] Step 4: Tracking off-target distance calculation

[0056] In order to make as many corner points as possible in the field of view, the reference point is set to the center of the image. The initial tracking point is the center of the corner point group. There are N corner points with coordinates (xi, yi). The center of the corner point group (x c ,y c )for:

[0057]

[0058]

[0059] (xi, yi)——coordinates of the i-th corner point

[0060] N——number of corner points;

[0061] (x c ,y c )——the center position of the corner point group.

[0062] The initial tracking miss distance is:

[0063] Δx0=x c -x

[0064] Δy0=y c -y

[0065] Δx0, Δy0——Initial tracking miss distance

[0066] 3. Select 3 non-collinear corner points from N corner points to form the constraint coordinate system poq. Then the coordinates of any point in the image coordinate system can be converted to the coordinate values ​​in the poq coordinate system. The same conversion can be done in reverse, such as Figure 2 As shown in Figure 2, it can be seen that when the inter-frame image of the rigid target undergoes translation, scaling, and rotation changes, the coordinate values ​​of the same image points in the image coordinate system will change. However, the coordinate values ​​of the corresponding points in the constrained coordinate system poq remain unchanged.

[0067] 4. Using this characteristic, the target tracking miss distance is solved as follows:

[0068] a) Calculate the initial tracking point coordinates (x c ,y c ) and tracking miss distance (△x0, △y0);

[0069] b) Choose any three non-collinear corner points as p, o, and q to form a constraint coordinate system. Calculate the coordinate value of the tracking point in the constraint coordinate system (p) based on their coordinate values ​​and the coordinates of the tracking point. c ,q c );

[0070] c) The next frame is based on the coordinate values ​​of the matching points in the p, o, q corner area and (p c ,qc ) Calculate the tracking point coordinates (xc1, yc1) of the frame image, and the difference between the tracking point coordinates and the image center point is used to obtain the tracking miss distance (△x1, △y1) of the frame;

[0071] d) Repeat steps b) and c).

[0072] Step 5: Fast mirror closed-loop control

[0073] 1. Selection of fast mirror drive platform

[0074] Piezoelectric ceramic-driven fast mirrors have fast response speeds and high resonant frequencies. Therefore, a piezoelectric ceramic-driven platform is the ideal platform for driving fast mirrors. When selecting a platform, consider factors such as travel range, accuracy requirements, load capacity, response speed, and stability and reliability.

[0075] 2. Servo system design

[0076] The fast mirror servo system uses a first-order zero-difference system, which boasts high steady-state accuracy, excellent dynamic response characteristics, and relatively easy physical implementation. It also offers strong anti-interference capabilities and a certain degree of suppression of low-frequency signals. The servo system maximizes bandwidth while ensuring closed-loop control system stability.

[0077] Compared with the prior art, the present invention has the following advantages:

[0078] 1) A fast reflector was added to the optical path system, matched with a high-frequency frame detector, and a piezoelectric ceramic drive platform was configured, providing a foundation for composite stable imaging of the optical system from the system hardware level.

[0079] 2) The present invention proposes a method for determining target feature points based on corner points, and uses this method to calculate the target tracking miss distance. The servo system is designed as a first-order zero-degree system. Finally, the servo system drives the piezoelectric ceramics to achieve precise control of the fast reflection mirror, and finally realizes composite stable imaging.

[0080] 3) This method has good stable imaging performance and strong application value, especially for targets with clear edges, sharp corners and obvious corner features such as aircraft and ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 Reference point position and initial tracking point position.

[0082] Figure 2 The corner points form the constraint coordinate system.

[0083] Figure 3 Schematic diagram of the optical system with the addition of a fast reflector.

[0084] Figure 4 Schematic diagram of the hexagonal search method.

[0085] Figure 5 Schematic diagram of the diamond search method.

[0086] Figure 6 Target feature extraction simulation results.

[0087] Figure 7 Fast mirror closed-loop servo system test results. DETAILED DESCRIPTION

[0088] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0089] Example:

[0090] 1. Add a fast reflector to the optical path

[0091] Add a fast reflector to the optical path. Figure 3

[0092] 2. Matching high frame rate detector

[0093] The focal length of the optical system is f = 1.92 m, the pixel size of the fine tracking image sensor is s = 10 μm × 10 μm, and the target dispersion is not considered. The estimated imaging size for targets of different sizes and distances is shown in Table 1.

[0094] Table 1 Image size estimation table (number of pixels)

[0095]

[0096] For high frame rate imaging, the detector selected was the VSP800 detector, with a pixel count of 800 × 600 and a pixel size of 10 μm.

[0097] 3. Determine target feature points based on corner points

[0098] The determined corner points (x, y) cannot achieve pixel-level accuracy.

[0099] Furthermore, the inter-frame matching method is used to select a template area with the corner point as the center. The matching point is obtained by the template matching method in the next frame. The matching point is the inter-frame displacement change of the same corner point, reflecting the inter-frame displacement of the target area. Considering the high frame rate characteristics and real-time requirements of the precision tracking image sensor, the template matching criterion selects the absolute error sum function, that is:

[0100]

[0101] Where:

[0102] (x,y) corner coordinates;

[0103] (i, j) matching point coordinates;

[0104] I(x, y)——image grayscale value;

[0105] IM(x, y)——template grayscale value;

[0106] SAD(i, j) – sum of absolute differences function.

[0107] Calculate SAD one by one within the search range, and find the coordinate point (i, j) corresponding to the minimum SAD (i, j) as the template matching point. In order to efficiently complete the template matching search process, this solution adopts the hexagonal search method, such as Figure 4 shown.

[0108] First follow Figure 4 The hexagonal search template is used to search a large area, and the SAD of the center point and the six vertices of the hexagon is calculated. If the minimum SAD is not at the center point, the hexagonal search is continued with the vertex with the minimum SAD as the center point; otherwise, it is converted to Figure 5 The diamond search template is searched in a small range, and the SAD of the center point and the four vertices of the diamond is calculated. If the minimum SAD is at the center point, this point is used as the template matching point; otherwise, another diamond search is performed with the vertex with the minimum SAD as the center point, and the minimum SAD point is used as the template matching point.

[0109] Similarly, the target feature extraction method is simulated using the actual target tracking video as input data, and the results are as follows: Figure 6 The white boxes in the figure represent the locations of the corner points extracted from the current frame, and the cross marks indicate the locations of the matching points in the previous frame's corner region in the current frame. The simulation results show that the number and locations of corner points extracted from each frame vary, but the inter-frame matching accuracy of the corner points is very high. The locations of the corner points in the previous frame and the matching points in the corner region of the current frame accurately reflect the inter-frame displacement of the target.

[0110] In summary, for the target, the extracted features are the target corner point positions and the inter-frame matching point positions in the corner point area.

[0111] 4. Tracking miss distance calculation

[0112] a) Calculate the initial tracking point coordinates (xc, yc) and the tracking miss distance (△x0, △y0);

[0113] b) Arbitrarily select three non-collinear corner points as p, o, and q to form a constrained coordinate system. Based on their coordinate values ​​and the coordinates of the tracking point, the coordinate values ​​(pc, qc) of the tracking point in the constrained coordinate system are calculated;

[0114] c) In the next frame, the tracking point coordinates (xc1, yc1) of the image frame are calculated based on the coordinate values ​​of the matching points in the p, o, and q corner points and (pc, qc), and the difference between the coordinates and the center point of the image is used to obtain the tracking miss distance (△x1, △y1) of the frame.

[0115] d) Repeat steps b) and c).

[0116] 5. Fast mirror closed-loop control

[0117] A PI piezoelectric ceramic drive platform was selected. The fast imaging mirror is approximately 40 mm in size and 5 mm thick. Considering the resonant frequency and aiming field of view requirements, the S330.8 was chosen as the driver for the fast imaging mirror.

[0118] Table 2 PI fast mirror drive platform parameters

[0119]

[0120] The optical closed-loop control test was conducted using the PI company's P-T04K003 fast reflector driver. The driver resonant frequency was 1.5kHz±20% (Φ27mm×3mm glass mirror). The simulated target was an LED indicator light that rotated at a set period. The image frame rate and image processing time were changed, and the maximum bandwidth servo system was designed accordingly. The ratio of the target miss distance before and after the closed loop was calculated as the input error suppression value. The input error suppression values ​​of different frequencies were obtained by changing the simulated target rotation period and compared with the design results. The relevant test results are shown in Figure 1. Figure 7 As shown in the figure, the input error suppression result data has a high degree of matching with the theoretical design curve, which shows that the servo system design result is accurate and can achieve stable tracking of corner feature target images.

[0121] In addition to the above embodiments, the present invention also includes other implementation methods. Any technical solutions formed by equivalent transformation or equivalent replacement should fall within the scope of protection of the claims of the present invention.

Claims

1. A method for realizing composite stable imaging of an optical system based on target corner point features, characterized by: The method includes the following steps: first, adding a fast reflector to the optical path; second, matching a high frame rate detector; third, determining target feature points based on corner points; fourth, calculating and tracking miss distance based on the corner feature points; and fifth, implementing closed-loop control of the fast reflector based on the miss distance.

2. The method for realizing composite stable imaging of an optical system based on target corner features according to claim 1, characterized in that: The method specifically comprises the following steps: Step 1: Add a fast reflector to the optical path Add a fast reflecting mirror to the main optical path of the optical system; Step 2: Matching high frame rate detector 1) Based on the focal length f of the optical system, the number of sensor pixels n, and without considering the target dispersion, the image size is estimated by combining the target size S and the distance L from the target to the optical system. S——sensor pixel size; f——focal length of optical system; L——the distance between the target and the optical system; n——number of sensor pixels; 2) The target image area on the sensor occupies at least 5 × 5 pixels, and a matching high frame rate detector is selected accordingly; Step 3: Determine target feature points based on corner points 1) Basic principles of target corner feature determination For corner feature extraction, we use the characteristic that the grayscale change of the window image containing corner points is large when it moves in any direction. The grayscale change is expressed as: Where: E(x,y)——change in grayscale value of corner point; w u,v ——Gaussian window function; I u,v ——The grayscale value of the image at the coordinate (u, v); (u,v)——image coordinates; x,y——coordinate offset; 2) Target corner feature determination process Will I u+x,v+y Using the first-order Taylor series approximation and ignoring the higher-order terms, we have: I——image gray value; ——find derivatives; ——Find the tensor product; Define matrices A, B, C and autocorrelation matrix M: Where: w——Gaussian window function; Then we have: E(x,y)=(x,y)M(x,y) T It can be seen that the grayscale change E is determined by the autocorrelation matrix M, and the corner response function R(x, y) is defined as: R(x,y)=det(M)-k×trace(M) 2 det(M)=AB-C 2 trace(M)=A+B det——determinant symbol; trace——matrix trace algorithm symbol; If R(x, y) is greater than a certain threshold and is a local maximum point, the point (x, y) is considered to be a corner point; Step 4: Tracking off-target distance calculation 1) In order to make as many corner points as possible in the field of view, the reference point is set as the center of the image, the initial tracking point is the center of the corner point group, and the coordinates of N corner points are (xi, yi), the center position of the corner point group (x c ,y c )for: (xi, yi)——coordinates of the i-th corner point; N——number of corner points; The initial tracking miss distance is: Δx0=x c -x Δy0=y c -and Δx0, Δy0——initial tracking miss distance; 2) From the N corner points, three non-collinear corner points are randomly selected to form the constrained coordinate system poq. Then, the coordinates of any point in the image coordinate system can be converted to the coordinate values ​​in the poq coordinate system, and vice versa. When the inter-frame image of the rigid target undergoes translation, scaling, and rotation changes, the coordinate values ​​of the same image point in the image coordinate system will change. However, the coordinate values ​​of the corresponding points in the constrained coordinate system poq remain unchanged. Using this characteristic, the target tracking miss distance is solved as follows: a) Calculate the initial tracking point coordinates (x c ,y c ) and tracking miss distance (△x0, △y0); b) Choose any three non-collinear corner points as p, o, and q to form a constraint coordinate system. Calculate the coordinate value of the tracking point in the constraint coordinate system (p) based on their coordinate values ​​and the coordinates of the tracking point. c ,q c ); c) The next frame is based on the coordinate values ​​of the matching points in the p, o, q corner area and (p c ,q c ) Calculate the tracking point coordinates (xc1, yc1) of the frame image, and the difference between the tracking point coordinates and the image center point is used to obtain the tracking miss distance (△x1, △y1) of the frame; d) Repeat steps b) and c); Step 5: Fast mirror closed-loop control A piezoelectric ceramic drive platform is selected, and a first-order zero-difference system is used for the fast mirror servo system. The servo system maximizes the bandwidth while ensuring the stability of the closed-loop control system.

Images