Image self-registration TDI imaging method and system based on kernel density estimation

Abstract

The invention relates to an image self-registration TDI imaging method and system based on kernel density estimation, and the method comprises the following steps: feature point extraction of a reference frame area array image, feature point extraction of a to-be-registered frame area array image, feature point matching, calculation of a feature point pair coordinate difference value, and calculation of a feature point pair displacement value weighting coefficient. And estimating the global displacement of the inter-frame image, aligning the area array images of the to-be-registered frames, overlapping the area array images of the to-be-registered frames, and outputting a final TDI image. Experiments prove that the image self-registration TDI imaging method based on kernel density estimation can effectively reduce motion blur caused by relative motion of a camera and a scene in the imaging process, improves the quality of a finally generated image, and can obtain a high-definition TDI image in the remote sensing field, the medical imaging field and the industrial detection field; the method not only improves the imaging quality, but also has a wide application prospect, and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection.

Description

Technical Field

[0001] The invention relates to the technical field of photoelectric imaging, and in particular to an image self-registration TDI imaging method and system based on kernel density estimation. Background Art

[0002] With the continuous development of imaging technology, time delay integration (TDI) imaging technology is increasingly used in various fields, especially in remote sensing, medical imaging and industrial detection. TDI imaging is an efficient imaging method that has attracted widespread attention because it can compensate for the relative motion between the camera and the scene and increase the exposure time equivalently, thereby achieving high sensitivity and high signal-to-noise ratio imaging effects under high-speed motion or low-light conditions.

[0003] Taking the application background in the field of remote sensing as an example, the traditional time delay integration (TDI) imaging system relies on precise motion control and synchronization of image sensors to ensure the effective capture of continuous image frames during the relative motion between the camera and the object. Usually, space cameras use a method that combines mechanical adjustment of the bias flow angle with electronic adjustment of the image motion speed. However, in practical applications, the precision mechanical structures such as bias flow adjustment devices and fast reflectors that traditional TDI imaging systems rely on significantly increase the design and production costs of space cameras. In addition, the increase in the size and mass of the payload has also led to an increase in launch costs. In terms of imaging quality, due to the tremor of the satellite platform, sensor measurement errors, and the uncertainty of dynamic imaging, the traditional TDI imaging method still faces many challenges in eliminating motion blur and improving image clarity.

[0004] With the improvement of onboard computing performance and the advancement of digital image processing technology, the speed and accuracy of onboard image processing have been significantly improved in recent years, and the image processing of remote sensing payloads in the process of on-orbit imaging has a hardware foundation. Therefore, the image registration technology in the field of computer vision is introduced into TDI imaging technology, and a TDI imaging method based on kernel density estimation for image self-registration is developed, which can not only overcome the limitations of traditional TDI imaging technology, but also improve the imaging quality while reducing the cost of use. It has important application value and broad application prospects. Summary of the invention

[0005] The present invention aims at the technical problems of relying on precise motion control and adjustment mechanism in the existing TDI technology and eliminating the high cost of relative motion between scene and image, and provides an image self-registration TDI imaging method and system based on kernel density estimation.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] An image self-registration TDI imaging method based on kernel density estimation comprises the following steps:

[0008] Step 1: Extract feature points from the reference frame array image; use a feature extraction operator to obtain a set of feature points from the reference frame array image, and calculate the feature vector of each feature point;

[0009] Step 2: Extract feature points of the image array frame to be registered; use a feature extraction operator to obtain a set of feature points from the image frame to be registered, and calculate the feature vector of each feature point;

[0010] Step 3: Feature point matching: according to the similarity measurement of each feature vector of the two sets of feature points, the feature points extracted from the reference frame array image and the frame array image to be registered are matched one by one;

[0011] Step 4: Calculate the coordinate difference of each feature point pair between the reference frame array image and the frame array image to be registered;

[0012] Step 5: Calculate the weighted coefficient of the displacement value of the feature point pair; use the kernel density estimation algorithm to calculate a random variable probability density, i.e., the weighted coefficient, for the coordinate difference of each set of matching point pairs;

[0013] Step 6: Estimation of global displacement of images between frames; by analyzing the kernel density estimation results of each coordinate difference point, the weighted average of the number of coordinate difference pixels of the feature point pair obtained according to the weighting coefficient is used as the estimated value of the global displacement between frames;

[0014] Step 7: Aligning the area array images of the frame to be registered; according to the displacement estimation result of the area array images between frames, geometric transformation is performed on each pixel of the area array image of the frame to be registered to obtain the area array image aligned with the reference frame area array image, ensuring that each pixel is aligned one by one with the reference frame area array image in space;

[0015] Step 8: superimposing the image array of the frame to be registered; superimposing the aligned pixel array of the image array of the frame to be registered obtained in step 7 into the digital domain integrator;

[0016] Step 9: Output the final TDI image; repeat steps 2 to 8 until the alignment and superposition of all the frame array images to be registered in the integral series are completed, a complete imaging process is achieved, and the final TDI image is obtained.

[0017] In the above technical solution, in step 2, the type and setting parameters of the feature extraction operator of the frame array image to be registered are the same as those of the feature extraction operator of the reference frame array image in step 1.

[0018] In the above technical solution, step 4 is specifically as follows:

[0019] The coordinate difference in the row direction between the image array of the frame to be registered and the image array of the reference frame is Δx=x 2i -x 1i

[0020] The coordinate difference between the frame to be registered and the reference frame in the column direction Δy=y 2i -y 1i

[0021] Among them, (x 1i ,y 1i ) and (x 2i ,y 2i ) is the set of p feature point pairs between the reference frame array image and the frame array image to be registered Any pair of matching points in .

[0022] In the above technical solution, step 5 is specifically as follows:

[0023] For each coordinate difference point (Δx, Δy), calculate its probability density estimate and obtain the weighting coefficient of the feature point to the displacement value

[0024]

[0025] Among them, n is the number of coordinate difference points in the sample data, h is the smoothing bandwidth, which controls the amount of smoothing. The larger the h value, the smoother the data.

[0026] In the above technical solution, step 6 is specifically as follows:

[0027] Number of pixels of inter-frame area array image displacement in the row direction

[0028] The number of pixels of the inter-frame area array image displacement in the column direction

[0029] in, is the weighting coefficient of the displacement value of the feature point pair, Δx and Δy are the coordinate differences of a pair of matched feature points in the row direction and column direction respectively.

[0030] In the above technical solution, step 7 is specifically as follows:

[0031] The number of pixels of the inter-frame area array image displacement in the row direction Δm and the number of pixels of the inter-frame area array image displacement in the column direction Δn correspond to the integer part Δm′ of the number of pixels of the k-th frame image k , Δn′ k and the fractional part Δm″ k , Δn″ k Respectively expressed as:

[0032] Δm′ k =fix(Δm)

[0033] Δn′ k =fix(Δn)

[0034] Δm″ k =Δm-Δm′ k

[0035] Δn″ k =Δn-Δn′ k

[0036] The pixel array P of the image of the frame to be registered after being aligned with the reference frame image k (i,j) is expressed as:

[0037] P k (i,j)=P(i+Δm′ k ,j+Δn′ k )(1-Δm″ k )+P(i+Δm′ k ,j+Δn′ k +1)(1-Δm″ k )Δn″ k +P(i+Δm′ k +1,j+Δn′ k )Δm″ k (1-Δn″ k )+P(i+Δm′ k +1,j+Δn′ k +1)Δm″ k Δn″ k .

[0038] In the above technical solution, step 8 is specifically as follows:

[0039] After the kth frame is superimposed, the grayscale value of each pixel in the digital domain integrator is expressed as:

[0040] O k (i,j)=O k-1 (i,j)+P k (i,j).

[0041] In the above technical solution, in step 9, when the integration level is M, the grayscale value of each pixel of the final TDI image is expressed as:

[0042]

[0043] A system applicable to the above-mentioned image self-registration TDI imaging method based on kernel density estimation, comprising:

[0044] Feature point pair extraction and matching module, feature point pair coordinate difference calculation module, inter-frame global displacement estimation module, inter-frame image alignment module and image overlay module; wherein,

[0045] The feature point pair extraction and matching module is used to perform steps 1-3;

[0046] The feature point coordinate difference calculation module is used to execute step 4;

[0047] The inter-frame global displacement estimation module is used to perform steps 5 and 6;

[0048] The inter-frame image alignment module is used to execute step 7;

[0049] The image overlay module is used to perform steps 8 and 9.

[0050] The present invention has the following beneficial effects:

[0051] The image self-registration TDI imaging method based on kernel density estimation of the present invention effectively overcomes the dependence of the traditional time delay integration (TDI) imaging system on the precise image motion compensation device by introducing image registration technology, reduces the need for motion rate matching, and thus significantly reduces the design and production costs. Secondly, by utilizing the self-registration technology, it is possible to achieve alignment and superposition of images between frames during the TDI imaging process, eliminate image motion and camera tremor during the imaging process, and is particularly suitable for low-light, fast relative motion application scenarios and high-resolution, high signal-to-noise ratio application requirements. In addition, the implementation of this method relies on the improvement of processor computing performance and the advancement of digital image processing technology, which significantly improves the image processing speed and accuracy, further enhancing the application potential of self-registration TDI imaging.

[0052] The image self-registration TDI imaging method based on kernel density estimation of the present invention can realize accurate estimation of the global displacement of images between frames by extracting and matching feature point pairs between frames; in conjunction with the digital domain TDI architecture, it can realize alignment and superposition of images between frames in the processor, effectively eliminate motion blur, and obtain high-definition TDI images.

[0053] Experiments have shown that the image self-registration TDI imaging method based on kernel density estimation of the present invention can effectively reduce the motion blur caused by the relative motion between the camera and the scene during the imaging process, improve the quality of the final generated image, and obtain high-definition TDI images in the fields of remote sensing, medical imaging and industrial detection.

[0054] The image self-registration TDI imaging method based on kernel density estimation of the present invention not only improves the imaging quality, but also has broad application prospects and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0056] Figure 1 The figure is a schematic flow chart of the steps of the image self-registration TDI imaging method based on kernel density estimation of the present invention.

[0057] Figure 2 This is a schematic diagram of the operation effect of the feature point coordinate difference calculation module during the first experimental verification.

[0058] Figure 3 Schematic diagram of the accuracy analysis of the global displacement estimation module between frames during the first experimental verification.

[0059] Figure 4 This is an imaging effect diagram of the first experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is a satellite remote sensing imaging in the remote sensing field, wherein sub-images (a), (c), (e), and (g) are schematic diagrams of TDI imaging effects without motion compensation, respectively, and (b), (d), (f), and (h) are schematic diagrams of imaging effects of the image self-registration TDI imaging method based on kernel density estimation of the present invention, respectively.

[0060] Figure 5 This is an imaging effect diagram of the second experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is an image taken by an industrial TDI camera in the field of industrial inspection during the PCB production process, wherein sub-figure (a) is a schematic diagram of the TDI imaging effect without motion compensation, and sub-figure (b) is a schematic diagram of the imaging effect of the image self-registration TDI imaging method based on kernel density estimation of the present invention.

[0061] Figure 6 This is an imaging effect diagram of the third experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is a fundus image of a glaucoma case taken by a fundus TDI camera in the field of medical imaging, wherein sub-figure (a) is a schematic diagram of the TDI imaging effect without motion compensation, and sub-figure (b) is a schematic diagram of the imaging effect of the image self-registration TDI imaging method based on kernel density estimation of the present invention.

[0062] The reference numerals in the figures indicate:

[0063] 101-feature point pair extraction and matching module; 102-feature point pair coordinate difference calculation module; 103-inter-frame global displacement estimation module; 104-inter-frame image alignment module; 105-image overlay module. DETAILED DESCRIPTION

[0064] The inventive concept of the present invention is:

[0065] The image self-registration TDI imaging method based on kernel density estimation of the present invention is an imaging method that uses image registration to achieve area array image alignment and superposition during time delay integration imaging. It can adapt to area array image sensors, effectively eliminate motion blur during time delay integration, and obtain high-definition TDI images.

[0066] The image self-registration TDI imaging method based on kernel density estimation of the present invention can use the matched inter-frame feature point pairs to obtain the coordinate differences of all feature point pairs of each frame to be registered and the reference frame, and use the weighted coefficients of the feature point pair coordinate differences obtained by kernel density estimation, and the weighted average of the feature point pair coordinate differences as the estimated value of the global displacement between frames, thereby realizing accurate estimation of the displacement between frames; in conjunction with the digital domain TDI architecture, it can realize the alignment and superposition of inter-frame array images in the digital domain integrator, effectively eliminate motion blur, and obtain a high-definition time-delay integral image.

[0067] The object of the present invention is to provide a time-delayed integral image alignment and superposition method based on image self-registration. The method and the applicable system can realize, in the time-delayed integral push-scan imaging of an area array image sensor, the displacement of an image between frames is obtained by fitting the displacement of a pair of feature points, and then the one-to-one alignment of each pixel of the image between frames is achieved, and the superposition of the aligned images is achieved in a processor, so as to achieve the purpose of eliminating motion mismatch and motion blur without introducing additional motion control and motion compensation mechanisms.

[0068] The present invention is described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the image self-registration TDI imaging method based on kernel density estimation of the present invention comprises the following steps:

[0070] Step 1: Extract feature points of the reference frame array image; use the feature detection operator to extract the reference frame array image to obtain a set of feature points, and calculate the feature vector of each feature point.

[0071] The feature point set extracted from the reference frame array image is: The feature point set includes M feature points. The eigenvector of each point in is expressed as Taking the SIFT (Scale Invariant Feature Transform) operator as an example, the SIFT operator is used to calculate the feature vector corresponding to each feature point of the reference frame array image. The feature vector can describe the appearance information of the area around the feature point.

[0072] Step 2: Extract feature points of the image of the frame to be registered; use the feature detection operator to extract a set of feature points from the image of the frame to be registered, and calculate the feature vector of each feature point.

[0073] The frame array image to be registered is the k-th frame image in an imaging process (2≤k≤M), where M is the integral series. The feature point set extracted from the frame array image to be registered is The feature point set includes N feature points. The eigenvector of each point in is expressed as

[0074] The type and setting parameters of the feature extraction operator of the frame array image to be registered are the same as those of the feature extraction operator of the reference frame array image in step 1.

[0075] Step 3: Feature point matching: According to the similarity measurement of each feature vector of the two sets of feature points, the feature points extracted from the reference frame array image and the frame array image to be registered are matched one by one.

[0076] The two sets of feature points are the feature point sets extracted in step 1 and step 2 and The feature vector is a feature point set and The vectors calculated by the operator for each feature point Feature point matching calculates the similarity between feature points and uses the descriptor matching algorithm to match two sets of feature points. Take the nearest neighbor matching as an example:

[0077]

[0078] During matching, the Euclidean distance between feature vectors is calculated and the feature point pair with the smallest distance is selected as the matching result to achieve the best pairing of feature points extracted from the reference frame array image and the array image to be registered. The set of paired feature point pairs is expressed as

[0079] Step 4: Calculate the coordinate difference of feature point pairs; calculate the coordinate difference of each feature point pair between the reference frame array image and the feature point pair set of the frame array image to be registered.

[0080] The feature point pair set is the p feature point pair sets obtained in step 3 For any pair of matching point coordinates (x 1i ,y 1i ) and (x 2i ,y 2i ), their coordinate difference, i.e. displacement value, can be calculated:

[0081] Δx=x 2i -x 1i

[0082] Δy=y 2i -y1i

[0083] The feature point coordinate difference is obtained by subtracting the matched feature point coordinates.

[0084] Step 5: Calculate the weighted coefficient of the displacement value of the feature point pair; use the kernel density estimation algorithm to calculate a random variable probability density, i.e., the weighted coefficient, for the coordinate difference of each set of matching point pairs, thereby reducing the impact of mismatched points on the inter-frame displacement estimation.

[0085] The displacement value of the feature point pair is the coordinate difference Δx, Δy calculated in step 4. The kernel density estimation algorithm is a method for estimating the probability density function of a random variable that does not rely on a specific distribution assumption of the data. The kernel density estimation algorithm smoothes the data by placing a kernel function on each coordinate difference point (Δx, Δy) to generate a continuous density estimate. The bandwidth of the kernel density estimation is pre-set.

[0086] Taking Gaussian kernel density as an example, for each coordinate difference point (Δx, Δy), its probability density estimate, that is, the weighting coefficient, is calculated:

[0087]

[0088] Among them, n is the number of coordinate difference points in the sample data, h is the smoothing bandwidth, which controls the amount of smoothing. The larger the h value, the smoother the data.

[0089] Step 6: Estimation of the number of pixels of the inter-frame array image displacement; by analyzing the kernel density estimation results of each coordinate difference point, the weighted average of the coordinate differences of the feature points obtained according to the weighting coefficient is used as the estimate of the global displacement.

[0090] The weighting coefficient is calculated in step 5 The weighted average includes: Δm, Δn, which represent the estimated values ​​of the inter-frame image displacement pixel number between the frame to be registered and the reference frame in the row and column directions respectively:

[0091]

[0092] Step 7: Align the area array images of the frame to be registered; according to the estimated number of pixels of the area array image displacement between frames, perform geometric transformation on each pixel of the area array image of the frame to be registered to obtain the area array image aligned with the reference frame area array image, ensuring that each pixel is aligned one by one with the reference frame area array image in space.

[0093] The alignment of the image array of the frame to be registered is calculated by the contribution rate of the integer part pixel unit cell and the decimal part pixel unit cell to the pixel gray value.

[0094] The estimated value of the pixel number of the inter-frame array image displacement is Δm and Δn calculated in step 6, which corresponds to the integer part Δm′ of the pixel number of the k-th frame image. k , Δn′ k and the fractional part Δm″ k , Δn″ k Can be expressed as:

[0095] Δm′ k =fix(Δm)

[0096] Δn′ k =fix(Δn)

[0097] Δm″ k =Δm-Δm′ k

[0098] Δn″ k =Δn-Δn′ k

[0099] Therefore, considering the contribution rate of the integer part pixel cells and the decimal part pixel cells to the pixel gray value, the pixel array P of the frame array image to be registered after being aligned with the reference frame array image is k (i,j) can be expressed as:

[0100] P k (i,j)=P(i+Δm′ k ,j+Δn′ k )(1-Δm″ k )+P(i+Δm′ k ,j+Δn′ k +1)(1-Δm″ k )Δn″ k +P(i+Δm′ k +1,j+Δn′ k )Δm″ k (1-Δn″ k )+P(i+Δm′ k +1,j+Δn′ k +1)Δm″ k Δn″ k

[0101] Step 8: Superimpose the image arrays of the frames to be registered; superimpose the aligned pixel arrays of the image arrays of the frames to be registered obtained in step 7 into the digital domain integrator.

[0102] After the kth frame is superimposed, the grayscale value of each pixel in the digital domain integrator is expressed as:

[0103] O k (i,j)=O k-1 (j,j)+Pk (i,j)

[0104] Step 9: Output the final TDI image; repeat steps 2 to 8 until all the frame array images to be registered in the integral series are aligned and superimposed, that is, a complete imaging process is achieved to obtain the final TDI image.

[0105] Taking the integration level M as an example, the grayscale value of each pixel in the final TDI image can be expressed as:

[0106]

[0107] The image self-registration TDI imaging method based on kernel density estimation of the present invention is applicable to a system comprising: a feature point pair extraction and matching module 101, a feature point pair coordinate difference calculation module 102, an inter-frame global displacement estimation module 103, an inter-frame image alignment module 104 and an image superposition module 105; wherein,

[0108] The feature point pair extraction and matching module 101 is used to execute steps 1-3; the feature point pair coordinate difference calculation module 102 is used to execute step 4; the inter-frame global displacement estimation module 103 is used to execute steps 5 and 6; the inter-frame image alignment module 104 is used to execute step 7; and the image overlay module 105 is used to execute steps 8 and 9.

[0109] Through the feature point pair extraction and matching module 101, the feature point pair coordinate difference calculation module 102, the inter-frame global displacement estimation module 103, the inter-frame image alignment module 104, and the image superposition module 105, it is possible to accurately estimate the global displacement value of the inter-frame sequence image in time-delayed integration push-scan imaging, and then use the integer pixel and sub-pixel global displacement values ​​to eliminate the position deviation of each pixel in each frame of the image, and realize superposition in the digital domain integrator, so as to achieve the purpose of eliminating motion mismatch and motion blur without introducing additional motion control and motion compensation mechanisms.

[0110] The image self-registration TDI imaging method based on kernel density estimation of the present invention effectively overcomes the dependence of the traditional time delay integration (TDI) imaging system on the precise image motion compensation device by introducing image registration technology, reduces the need for motion rate matching, and thus significantly reduces the design and production costs. Secondly, by utilizing the self-registration technology, it is possible to achieve alignment and superposition of images between frames during the TDI imaging process, eliminate image motion and camera tremor during the imaging process, and is particularly suitable for low-light, fast relative motion application scenarios and high-resolution, high signal-to-noise ratio application requirements. In addition, the implementation of this method relies on the improvement of processor computing performance and the advancement of digital image processing technology, which significantly improves the image processing speed and accuracy, further enhancing the application potential of self-registration TDI imaging.

[0111] The image self-registration TDI imaging method based on kernel density estimation of the present invention can realize accurate estimation of the global displacement of images between frames by extracting and matching feature point pairs between frames; in conjunction with the digital domain TDI architecture, it can realize alignment and superposition of images between frames in the processor, effectively eliminate motion blur, and obtain high-definition TDI images.

[0112] Experiments have shown that the image self-registration TDI imaging method based on kernel density estimation of the present invention can effectively reduce the motion blur caused by the relative motion between the camera and the scene during the imaging process, improve the quality of the final generated image, and obtain high-definition TDI images in the fields of remote sensing, medical imaging and industrial detection.

[0113] Figure 2 This is a schematic diagram of the operation effect of the feature point pair coordinate difference calculation module during the first experimental verification. Sub-pictures (a), (b), (c), (d), (e), and (f) are the coordinate differences between the 2nd, 6th, 10th, 15th, 20th, and 24th frames and the reference frame during the integration process. The coordinates of each point in the schematic diagram represent the coordinates of the feature point pair, and the color represents the size of the kernel density estimate of the point. Experiments have shown that there is a kernel density estimate peak area in each frame.

[0114] Figure 3 This is a schematic diagram of the accuracy analysis of the global displacement estimation module between frames during the first experimental verification. The X-axis is the order of the frames in the integration process, the Y-axis is the error between the displacement value estimated by the kernel density and the true displacement value, the blue line is the error in the X direction, and the red line is the error in the Y direction. The experiment shows that the kernel density estimation errors in the X and Y directions during this imaging process are both less than 0.01 pixels.

[0115] Figure 4 This is an imaging effect diagram of the first experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is satellite-to-earth remote sensing imaging in the remote sensing field, wherein sub-images (a), (c), (e), and (g) are schematic diagrams of TDI imaging effects without motion compensation, and (b), (d), (f), and (h) are schematic diagrams of imaging effects of the image self-registration TDI imaging method based on kernel density estimation of the present invention. Experiments show that in satellite-to-earth remote sensing imaging in the remote sensing field, the present invention can significantly reduce motion blur and improve the quality of time delay integral imaging.

[0116] Figure 5This is an imaging effect diagram of the second experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is an image taken by an industrial TDI camera in the field of industrial inspection during the PCB production process, wherein sub-image (a) is a schematic diagram of the TDI imaging effect without motion compensation, and sub-image (b) is a schematic diagram of the imaging effect of the image self-registration TDI imaging method based on kernel density estimation of the present invention. Experiments show that sub-image (b) has better clarity and effectively suppresses the imaging blur caused by the relative movement of the assembly line and the inspected PCB board. In the field of industrial inspection, the present invention can also significantly reduce the motion blur between the camera and the inspected product and improve the quality of time delay integral imaging.

[0117] Figure 6 This is an imaging effect diagram of the third experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the present invention; specifically, it is a fundus image of a glaucoma case taken by a fundus TDI camera in the field of medical imaging, wherein sub-image (a) is a schematic diagram of the TDI imaging effect without motion compensation, and sub-image (b) is a schematic diagram of the imaging effect of the image self-registration TDI imaging method based on kernel density estimation of the present invention. The experiment shows that the present invention is also applicable in the field of medical imaging, and can improve the clarity of medical TDI camera imaging and improve the quality of fundus imaging.

[0118] The image self-registration TDI imaging method based on kernel density estimation of the present invention not only improves the imaging quality, but also has broad application prospects and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection.

[0119] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A TDI imaging method for image self-registration based on kernel density estimation, characterized in that: The following steps are involved: Step 1: Extract feature points of reference frame array image; A set of feature points is obtained from the reference frame array image using a feature extraction operator, and the feature vector of each feature point is calculated; Step 2: Extract feature points of the image array frame to be registered; use a feature extraction operator to obtain a set of feature points from the image frame to be registered, and calculate the feature vector of each feature point; Step 3: Feature point matching: according to the similarity measurement of each feature vector of the two sets of feature points, the feature points extracted from the reference frame array image and the frame array image to be registered are matched one by one; Step 4: Calculate the coordinate difference of feature point pairs; Calculate the coordinate difference of each feature point pair between the reference frame array image and the frame array image to be registered; Step 5: Calculate the weighted coefficient of the displacement value of the feature point pair; use the kernel density estimation algorithm to calculate a random variable probability density, i.e., the weighted coefficient, for the coordinate difference of each set of matching point pairs; Step 6: Estimation of global displacement of images between frames; by analyzing the kernel density estimation results of each coordinate difference point, the weighted average of the number of coordinate difference pixels of the feature point pair obtained according to the weighting coefficient is used as the estimated value of the global displacement between frames; Step 7: Align the image array of the frame to be registered; According to the inter-frame area array image displacement estimation result, each pixel of the area array image of the registration frame is geometrically transformed to obtain the area array image aligned with the reference frame area array image, ensuring that each pixel is spatially aligned with the reference frame area array image. Step 8: superimposing the image array of the frame to be registered; superimposing the aligned pixel array of the image array of the frame to be registered obtained in step 7 into the digital domain integrator; Step 9: Output the final TDI image; repeat steps 2 to 8 until the alignment and superposition of all the frame array images to be registered in the integral series are completed, a complete imaging process is achieved, and the final TDI image is obtained.

2. The image self-registration TDI imaging method based on kernel density estimation according to claim 1, characterized in that: In step 2, the type and setting parameters of the feature extraction operator of the to-be-registered frame area array image are the same as those of the reference frame area array image feature extraction operator in step 1.

3. The image self-registration TDI imaging method based on kernel density estimation according to claim 1, characterized in that: Step 4 is as follows: The coordinate difference in the row direction between the image array of the frame to be registered and the image array of the reference frame is Δx=x 2i -x 1i The coordinate difference between the frame to be registered and the reference frame in the column direction Δy=y 2i -y 1i Among them, (x 1i ,y 1i ) and (x 2i ,y 2i ) is the set of p feature point pairs between the reference frame array image and the frame array image to be registered Any pair of matching points in .

4. The image self-registration TDI imaging method based on kernel density estimation according to claim 3, characterized in that: Step 5 is as follows: For each coordinate difference point (Δx, Δy), calculate its probability density estimate and obtain the weighting coefficient of the feature point to the displacement value Among them, n is the number of coordinate difference points in the sample data, h is the smoothing bandwidth, which controls the amount of smoothing. The larger the h value, the smoother the data.

5. The image self-registration TDI imaging method based on kernel density estimation according to claim 4, characterized in that: Step 6 is as follows: Number of pixels of inter-frame area array image displacement in the row direction The number of pixels of the inter-frame area array image displacement in the column direction in, is the weighting coefficient of the displacement value of the feature point pair, Δx and Δy are the coordinate differences of a pair of matched feature points in the row direction and column direction respectively.

6. The image self-registration TDI imaging method based on kernel density estimation according to claim 5, characterized in that: Step 7 is as follows: The number of pixels of the inter-frame area array image displacement in the row direction Δm and the number of pixels of the inter-frame area array image displacement in the column direction Δn correspond to the integer part Δm′ of the number of pixels of the k-th frame image k , Δn′ k and the fractional part Δm″ k , Δn″ k Respectively expressed as: Δm′ k =fix(Δm) Δn′ k =fix(Δn) Δm″ k =Δm-Δm′ k Δn″ k =Δn-Δn′ k The pixel array P of the image of the frame to be registered after being aligned with the reference frame image k (i,j) is expressed as: P k (i,j)=P(i+Δm′ k ,j+Δn′ k )(1-Δm″ k ) +P(i+Δm′ k ,j+Δn′ k +1)(1-Δm″ k )Δn″ k +P(i+Δm′ k +1,j+Δn′ k )Δm″ k (1-Δn″ k ) +P(i+Δm′ k +1,j+Δn′ k +1)Δm″ k Δn′ k 。 7. The image self-registration TDI imaging method based on kernel density estimation according to claim 6, characterized in that: Step 8 is as follows: After the kth frame is superimposed, the grayscale value of each pixel in the digital domain integrator is expressed as: O k (i,j)=O k-1 (i,j)+P k (i,j)。 8. The image self-registration TDI imaging method based on kernel density estimation according to claim 7, characterized in that: In step 9, when the integration level is M, the grayscale value of each pixel in the final TDI image is expressed as:

9. A system applicable to the image self-registration TDI imaging method based on kernel density estimation according to any one of claims 1 to 8, characterized in that: include: A feature point pair extraction and matching module (101), a feature point pair coordinate difference calculation module (102), an inter-frame global displacement estimation module (103), an inter-frame image alignment module (104) and an image superposition module (105); wherein, The feature point pair extraction and matching module (101) is used to execute steps 1-3; The feature point coordinate difference calculation module (102) is used to execute step 4; The inter-frame global displacement estimation module (103) is used to execute steps 5 and 6; The inter-frame image alignment module (104) is used to execute step 7; The image overlay module (105) is used to execute steps 8 and 9.

Images