An image self-registration TDI imaging method and system based on kernel density estimation

Abstract

The application relates to a kernel density estimation-based image self-registration TDI imaging method and system, which comprises the following steps: feature point extraction of a reference frame plane array image, feature point extraction of a frame plane array image to be registered, feature point matching, feature point pair coordinate difference value calculation, feature point pair displacement value weighting coefficient calculation, inter-frame image global displacement estimation, alignment of the frame plane array image to be registered, superposition of the frame plane array image to be registered, and output of a final TDI image. Experiments prove that the kernel density estimation-based image self-registration TDI imaging method can effectively reduce motion blur caused by camera-scene relative motion in the imaging process, improves the quality of the finally generated image, and can obtain a high-definition TDI image in the fields of remote sensing, medical imaging and industrial detection; not only improves the imaging quality, but also has a wide application prospect and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photoelectric imaging, and in particular to an image self-registration TDI imaging method and system based on kernel density estimation. BACKGROUND

[0002] With the continuous development of imaging technology, time delay integration (TDI) imaging technology is increasingly widely used in various fields, especially in remote sensing, medical imaging and industrial detection. TDI imaging is a highly efficient imaging method, which can compensate for the relative motion between the camera and the scene, and equivalently increase the exposure time, so as to realize high sensitivity and high signal-to-noise ratio imaging effect under high-speed motion or low light conditions, and has received extensive attention.

[0003] Taking the application background in the field of remote sensing as an example, the traditional time delay integration (TDI) imaging system relies on precise motion control and image sensor synchronization to ensure the effective capture of consecutive image frames during the relative motion between the camera and the object. Generally, the spatial camera adopts a method combining mechanical adjustment of the flow angle with electronic adjustment of the image motion speed. However, in actual application, the precise mechanical structures such as flow adjustment device and fast mirror relied on by the traditional TDI imaging system significantly increase the design and production cost of the spatial camera. In addition, the increase in size and mass of the load also leads to the rise of launch cost. In terms of imaging quality, due to the vibration of the satellite platform, sensor measurement error and uncertainty of dynamic imaging, the traditional TDI imaging method still faces many challenges in eliminating motion blur and improving image clarity.

[0004] With the improvement of on-board computing performance and the progress of digital image processing technology, the speed and accuracy of on-board image processing have been significantly improved in recent years, and there is a hardware basis for image processing during on-orbit imaging of remote sensing payloads. Therefore, introducing the image registration technology in the field of computer vision into the TDI imaging technology and developing an image self-registration TDI imaging method based on kernel density estimation not only can overcome the limitations of traditional TDI imaging technology, but also can improve the imaging quality while reducing the use cost, which has important application value and wide application prospect. SUMMARY

[0005] The present application provides an image self-registration TDI imaging method and system based on kernel density estimation to solve the technical problems of relying on precise motion control adjustment mechanism and high cost of eliminating scene-image relative motion in existing TDI technology.

[0006] To solve the above technical problems, the technical scheme of the present application is as follows:

[0007] An image self-registration TDI imaging method based on kernel density estimation, comprising the following steps:

[0008] Step 1: Feature point extraction of the reference frame image; a set of feature points are obtained from the reference frame image by using a feature extraction operator, and the feature vectors of each feature point are calculated;

[0009] Step 2: Feature point extraction of the frame image to be registered; a set of feature points are obtained from the frame image to be registered by using a feature extraction operator, and the feature vectors of each feature point are calculated;

[0010] Step 3: Feature point matching; according to the similarity measure of the feature vectors of the two sets of feature points, the feature points extracted from the reference frame image and the frame image to be registered are paired one by one;

[0011] Step 4: Calculation of coordinate difference values of feature point pairs; the coordinate difference values of each feature point pair between the reference frame image and the frame image to be registered are calculated;

[0012] Step 5: Calculation of displacement value weighting coefficients of feature point pairs; the probability density of a random variable, i.e. the weighting coefficient, is calculated for each set of matched point pairs by using kernel density estimation algorithm;

[0013] Step 6: Estimation of inter-frame global displacement; the weighted average of the number of pixel coordinate difference values of feature point pairs obtained from the weighting coefficients is taken as the estimated value of the inter-frame global displacement by analyzing the kernel density estimation results of each coordinate difference point;

[0014] Step 7: Alignment of the frame image to be registered; according to the displacement estimation results of the inter-frame image, geometric transformation is performed on each pixel of the frame image to be registered to obtain an aligned frame image, ensuring that each pixel of the aligned frame image is aligned with the reference frame image in space;

[0015] Step 8: Superposition of the frame image to be registered; the pixel array of the aligned frame image to be registered obtained in step 7 is superimposed into the digital domain integrator;

[0016] Step 9: Output of the final TDI image; steps 2-8 are repeated until the alignment and superposition of all frame images to be registered in the integration order are completed, a complete imaging process is realized, and the final TDI image is obtained.

[0017] In the above technical solution, in step 2, the type and setting parameters of the feature extraction operator of the frame image to be registered are the same as those of the feature extraction operator of the reference frame image in step 1.

[0018] In the above technical solution, step 4 is specifically:

[0019] The coordinate difference value Δx in the row direction between the frame image to be registered and the reference frame image is x 2i - 1i

[0020] The coordinate difference in the column direction between the frame to be registered and the reference frame is Δy=y 2i -y 1i

[0021] where (x 1i ,y 1i ) and (x 2i ,y 2i ) are any one pair of matching points in the set of p pairs of feature points between the reference frame planar image and the planar image to be registered.

[0022] In the above technical solution, step 5 is specifically:

[0023] For each coordinate difference point (Δx, Δy), the probability density estimation value is calculated to obtain the feature point pair displacement value weighting coefficient

[0024]

[0025] where n is the number of coordinate difference points in the sample data, and h is the smoothing bandwidth, which controls the smoothing amount. The larger the value of h, the smoother the data.

[0026] In the above technical solution, step 6 is specifically:

[0027] The inter-frame planar image displacement pixel number in the row direction

[0028] The inter-frame planar image displacement pixel number in the column direction

[0029] where is the feature point pair displacement value weighting coefficient, and Δx and Δy are the coordinate difference in the row direction and the column direction of the matched pair of feature points, respectively.

[0030] In the above technical solution, step 7 is specifically:

[0031] The inter-frame planar image displacement pixel number Δm in the row direction and the inter-frame planar image displacement pixel number Δn in the column direction, the integer part Δm′ k , Δn′ k of the pixel number of the kth frame image and the decimal part Δm″ k , Δn″ k are respectively represented as:

[0032] Δm′ k = fix(Δm)

[0033] Δn′ k = fix(Δn) ​

[0034] Δm" k = Δm - Δm' k

[0035] Δn" k = Δn - Δn' k

[0036] Pixel array P of the frame face image to be registered after alignment with the reference frame face image k (i,j) is expressed as:

[0037] P k (i,j) = P(i+Δm' k ,j+Δn' k )(1-Δm" k )+P(i+Δm' k ,j+Δn' k +1)(1-Δm" k )Δn" k +P(i+Δm' k +1,j+Δn' k )Δm" k (1-Δn" k )+P(i+Δm' k +1,j+Δn' k +1)Δm" k Δn" k .

[0038] In the above technical solution, step 8 is specifically:

[0039] After the superposition of the kth frame, the gray value of each pixel in the digital domain integrator is expressed as:

[0040] O k (i,j) = O k-1 (i,j) + P k (i,j).

[0041] In the above technical solution, in step 9, when the integration order is M, the gray value of each pixel of the final TDI image is expressed as:

[0042]

[0043] A system suitable for the above-mentioned image self-registration TDI imaging method based on kernel density estimation, comprising:

[0044] The feature point pair extraction and matching module, the feature point pair coordinate difference calculation module, the inter-frame global displacement amount estimation module, the inter-frame image alignment module, and the image superposition module; wherein,

[0045] The feature point pair extraction and matching module is used to perform steps 1-3;

[0046] The feature point pair coordinate difference value calculation module is used to perform step 4;

[0047] The inter-frame global displacement amount estimation module is used to perform steps 5 and 6;

[0048] The inter-frame image alignment module is used to perform step 7;

[0049] The image superposition module is used to perform steps 8 and 9.

[0050] The present application has the following beneficial effects:

[0051] The image self-registration TDI imaging method based on kernel density estimation of the present application effectively overcomes the dependence of the traditional time delay integration (TDI) imaging system on an accurate image motion compensation device, reduces the demand for motion rate matching, and thus significantly reduces the design and production costs. Secondly, by using the self-registration technology, the alignment and superposition of inter-frame images can be realized in the TDI imaging process, the image motion and camera jitter in the imaging process are eliminated, and the method is especially suitable for low-light, fast relative motion application scenarios and high-resolution, high signal-to-noise ratio application requirements. In addition, the implementation of the method depends on the improvement of the processor computing performance and the progress of the digital image processing technology, so that the image processing speed and accuracy are significantly improved, and the application potential of the self-registration TDI imaging is further enhanced.

[0052] The image self-registration TDI imaging method based on kernel density estimation of the present application can realize accurate estimation of the inter-frame image global displacement amount by extracting and matching the inter-frame feature point pairs, and can realize the alignment and superposition of inter-frame images in the processor in cooperation with the digital domain TDI architecture, effectively eliminate motion blur, and obtain high-definition TDI images.

[0053] Experiments prove that the image self-registration TDI imaging method based on kernel density estimation of the present application can effectively reduce the motion blur caused by the camera-landscape relative motion in the imaging process, improve the quality of the finally generated image, and obtain high-definition TDI images in the fields of remote sensing, medical imaging and industrial detection.

[0054] The image self-registration TDI imaging method based on kernel density estimation of the present application not only improves the imaging quality, but also has a wide application prospect and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection. BRIEF DESCRIPTION OF DRAWINGS

[0055] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0056] Figure 1 It is a step flowchart of the image self-registration TDI imaging method based on kernel density estimation of the present application.

[0057] Figure 2 The figure is a schematic diagram for verifying the operation effect of the feature point pair coordinate difference value calculation module in the first experiment.

[0058] Figure 3 The figure is a schematic diagram for analyzing the accuracy of the inter-frame global displacement amount estimation module in the first experiment.

[0059] Figure 4 The figure is an imaging effect diagram of the first experiment verification of the image self-registration TDI imaging method based on kernel density estimation of the present application; specifically, satellite earth remote sensing imaging in the remote sensing field, wherein subgraphs (a), (c), (e), (g) are TDI imaging effect schematic diagrams without motion compensation, and (b), (d), (f), (h) are imaging effect schematic diagrams of the image self-registration TDI imaging method based on kernel density estimation of the present application.

[0060] Figure 5 The figure is an imaging effect diagram of the second experiment verification of the image self-registration TDI imaging method based on kernel density estimation of the present application; specifically, images captured by an industrial TDI camera in the PCB production process in the industrial detection field, wherein subgraph (a) is a TDI imaging effect schematic diagram without motion compensation, and subgraph (b) is an imaging effect schematic diagram of the image self-registration TDI imaging method based on kernel density estimation of the present application.

[0061] Figure 6 The figure is an imaging effect diagram of the third experiment verification of the image self-registration TDI imaging method based on kernel density estimation of the present application; specifically, fundus images of a glaucoma case captured by a fundus TDI camera in the medical imaging field, wherein subgraph (a) is a TDI imaging effect schematic diagram without motion compensation, and subgraph (b) is an imaging effect schematic diagram of the image self-registration TDI imaging method based on kernel density estimation of the present application.

[0062] The reference signs in the figure are as follows:

[0063] 101-feature point pair extraction and matching module; 102-feature point pair coordinate difference value calculation module; 103-inter-frame global displacement amount estimation module; 104-inter-frame image alignment module; 105-image superposition module. DETAILED DESCRIPTION

[0064] The inventive idea of the present application is:

[0065] The kernel density estimation-based image self-registration TDI imaging method of the present invention is an imaging method that uses image registration to achieve area array image alignment and superposition during time delay integration imaging. It can adapt to area array image sensors, effectively eliminate motion blur during time delay integration, and obtain high-definition TDI images.

[0066] The image self-registration TDI imaging method based on kernel density estimation of the present invention can use the matched inter-frame feature point pairs to obtain the coordinate differences of all feature point pairs of each frame to be registered and the reference frame, and use the weighted coefficients of the feature point pair coordinate differences obtained by kernel density estimation, and the weighted average of the feature point pair coordinate differences as the estimated value of the global displacement between frames, thereby realizing accurate estimation of the inter-frame displacement; combined with the digital domain TDI architecture, it can realize the alignment and superposition of inter-frame array images in the digital domain integrator, effectively eliminate motion blur, and obtain a high-definition time-delay integral image.

[0067] The objective of the present invention is to provide a time-delayed integral image alignment and superposition method based on image self-registration. This method and its applicable system can realize, in the time-delayed integral push-scan imaging of an area array image sensor, the displacement of the inter-frame image obtained by fitting the displacement of the feature point pairs, thereby achieving one-to-one alignment of each pixel in the inter-frame image, and realizing the superposition of the aligned images in the processor, thereby achieving the purpose of eliminating motion mismatch and motion blur without introducing additional motion control and motion compensation mechanisms.

[0068] The present invention will be described in detail below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, the image self-registration TDI imaging method based on kernel density estimation of the present invention includes the following steps:

[0070] Step 1: Extract feature points from the reference frame array image; use a feature detection operator to extract the reference frame array image to obtain a set of feature points, and calculate the feature vector of each feature point.

[0071] The feature point set extracted from the reference frame array image is: The feature point set includes M feature points. The eigenvector of each point in is expressed as Taking the SIFT (Scale Invariant Feature Transform) operator as an example, the SIFT operator calculates the feature vector corresponding to each feature point of the reference frame array image, and the feature vector can describe the appearance information of the area around the feature point.

[0072] Step 2: Extract feature points of the image of the frame to be registered; use the feature detection operator to extract a set of feature points from the image of the frame to be registered, and calculate the feature vector of each feature point.

[0073] The frame plane array image to be registered is the kth frame image (2≤k≤M) in the imaging process, and M is the integration order. The feature point set extracted from the frame plane array image to be registered is The feature point set contains N feature points, and the feature point set The feature vector of each point in the feature point set is represented as

[0074] The type and setting parameters of the feature extraction operator of the frame plane array image to be registered are the same as those of the feature extraction operator of the reference frame plane array image in step 1.

[0075] Step 3: Feature point matching; according to the similarity measure of the feature vectors of the two groups of feature points, the feature points extracted from the reference frame plane array image and the frame plane array image to be registered are paired one by one.

[0076] The two groups of feature points are the feature point sets extracted in steps 1 and 2 and The feature vector is the vector calculated by the operator for each feature point in the feature point set and Feature point matching is achieved by calculating the similarity between feature points and using a descriptor matching algorithm to match the two groups of feature points. Taking the nearest neighbor matching as an example:

[0077]

[0078] During matching, the Euclidean distance between the feature vectors is calculated, and the feature point pair with the smallest distance is selected as the matching result, achieving the best pairing of the feature points extracted from the reference frame plane array image and the frame plane array image to be registered. The set of paired feature point pairs is represented as

[0079] Step 4: Feature point pair coordinate difference calculation; calculate the coordinate difference of each feature point pair between the reference frame plane array image and the frame plane array image feature point pair set.

[0080] The feature point pair set is the p feature point pair set obtained in step 3 For any pair of matching points between the reference frame plane array image and the frame plane array image, the coordinate pair (x 1i ,y 1i ) and (x 2i ,y 2i ), the coordinate difference, i.e. the displacement value, can be calculated as:

[0081] Δx=x 2i -x 1i

[0082] Δy=y 2i -y​1i

[0083] The feature point pair coordinate difference value is obtained by subtracting the coordinates of the matched feature point pairs.

[0084] Step 5: Feature point pair displacement value weighting coefficient calculation; a random variable probability density, i.e. a weighting coefficient, is calculated for each group of matched point pair coordinate difference values by using a kernel density estimation algorithm, thereby reducing the influence of the mis-matched point pair inter-frame displacement estimation.

[0085] The feature point pair displacement value is the coordinate difference value Δx, Δy calculated in step 4. The kernel density estimation algorithm is a method for estimating the probability density function of a random variable, which does not rely on the specific distribution assumption of the data. The kernel density estimation algorithm smoothes the data by placing a kernel function at each coordinate difference point (Δx, Δy), thereby generating a continuous density estimation. The bandwidth of the kernel density estimation is pre-set.

[0086] Taking the Gaussian kernel density as an example, for each coordinate difference point (Δx, Δy), the probability density estimation value, i.e. the weighting coefficient, is calculated:

[0087]

[0088] Wherein, n is the number of coordinate difference points in the sample data, h is the smoothing bandwidth, which controls the amount of smoothing, and the larger the h value, the smoother the data.

[0089] Step 6: Inter-frame face array image displacement pixel number estimation; the weighted average of the feature point pair coordinate difference values obtained according to the weighting coefficients is taken as the estimation value of the global displacement by analyzing the kernel density estimation results of each coordinate difference point.

[0090] The weighting coefficient is calculated in step 5 The weighted average includes: Δm, Δn, which respectively represent the estimation value of the inter-frame image displacement pixel number in the row and column directions of the frame to be registered and the reference frame:

[0091]

[0092] Step 7: Registration of the face array image of the frame to be registered; according to the inter-frame face array image displacement pixel number estimation, the pixels of the face array image of the frame to be registered are geometrically transformed to obtain the face array image aligned with the face array image of the reference frame, thereby ensuring that each pixel of the face array image is aligned with each pixel of the face array image of the reference frame in space.

[0093] The registration of the face array image of the frame to be registered is calculated from the contribution rate of the integer part pixel unit cell and the decimal part pixel unit cell to the pixel gray value.

[0094] The inter-frame matrix image pixel displacement pixel number estimation value is Δm, Δn calculated in step 6, which corresponds to the integer part Δm', Δn' of the pixel number of the kth frame image k k and the decimal part Δm", Δn" k k may be respectively expressed as:

[0095] Δm' k = fix(Δm)

[0096] Δn' k = fix(Δn)

[0097] Δm" k = Δm- Δm' k

[0098] Δn" k = Δn- Δn' k

[0099] Therefore, considering the contribution rate of the integer part pixel unit cell and the decimal part pixel unit cell to the pixel gray value, the pixel array P k (i,j) of the to-be-registered frame matrix image aligned with the reference frame matrix image can be expressed as:

[0100] P k (i,j) = P(i+ Δm' k ,j+ Δn' k )(1- Δm" k )+ P(i+ Δm' k ,j+ Δn' k +1)(1- Δm" k )Δn" k + P(i+ Δm' k +1,j+ Δn' k )Δm" k (1- Δn" k )+ P(i+ Δm' k +1,j+ Δn' k +1)Δm" k Δn" k

[0101] Step 8: superimposition of the to-be-registered frame matrix image; superimpose the pixel array of the to-be-registered frame matrix image aligned in step 7 into the digital domain integrator.

[0102] After the kth frame is superimposed, the gray value of each pixel in the digital domain integrator is expressed as:

[0103] O k (i,j) = O k-1 (j,j) + P​​k (i,j)

[0104] Step 9: output the final TDI image; repeat steps 2-8 until the alignment and superposition of all frame plane array images to be registered in the integral order are achieved, that is, a complete imaging process is realized, and the final TDI image is obtained.

[0105] Taking the integral order M as an example, the gray value of each pixel of the final TDI image can be represented as:

[0106]

[0107] The image self-registration TDI imaging method based on kernel density estimation of the application is applicable to a system comprising a feature point pair extraction and matching module 101, a feature point pair coordinate difference value calculation module 102, an inter-frame global displacement value estimation module 103, an inter-frame image alignment module 104, and an image superposition module 105.

[0108] The feature point pair extraction and matching module 101 is used to perform steps 1-3; the feature point pair coordinate difference value calculation module 102 is used to perform step 4; the inter-frame global displacement value estimation module 103 is used to perform steps 5 and 6; the inter-frame image alignment module 104 is used to perform step 7; and the image superposition module 105 is used to perform steps 8 and 9.

[0109] Through the feature point pair extraction and matching module 101, the feature point pair coordinate difference value calculation module 102, the inter-frame global displacement value estimation module 103, the inter-frame image alignment module 104, and the image superposition module 105, the global displacement value of the inter-frame sequence image can be accurately estimated in the time delay integration push-broom imaging, and then the integer pixel and sub-pixel global displacement values are used to eliminate the position deviation of each pixel of each frame image, so that the superposition is realized in the digital integrator, and the purpose of eliminating motion mismatch and motion blur is achieved without introducing additional motion control and motion compensation mechanisms.

[0110] The image self-registration TDI imaging method based on kernel density estimation of the application effectively overcomes the dependence of the traditional time delay integration (TDI) imaging system on the precise image motion compensation device, reduces the demand for motion rate matching, and thus significantly reduces the design and production costs. Secondly, by using the self-registration technology, the alignment and superposition of the inter-frame images can be realized in the TDI imaging process, the image motion and camera jitter in the imaging process are eliminated, and the method is especially suitable for low-light, fast relative motion application scenarios and high-resolution, high signal-to-noise ratio application requirements. In addition, the implementation of the method depends on the improvement of the processor computing performance and the progress of the digital image processing technology, so that the image processing speed and accuracy are significantly improved, and the application potential of the self-registration TDI imaging is further enhanced.

[0111] The kernel density estimation-based image self-registration TDI imaging method of the present invention can achieve accurate estimation of the global displacement of inter-frame images by extracting and matching feature point pairs between frames; combined with the digital domain TDI architecture, it can achieve alignment and superposition of inter-frame images within the processor, effectively eliminating motion blur and obtaining high-definition TDI images.

[0112] Experiments have shown that the image self-registration TDI imaging method based on kernel density estimation of the present invention can effectively reduce the motion blur caused by the relative motion between the camera and the scene during the imaging process, improve the quality of the final generated image, and obtain high-definition TDI images in the fields of remote sensing, medical imaging, and industrial inspection.

[0113] Figure 2 This diagram illustrates the operation of the feature point pair coordinate difference calculation module during the first experimental verification. Sub-figures (a), (b), (c), (d), (e), and (f) show the coordinate differences between each feature point pair and the baseline frame during the integration process for the 2nd, 6th, 10th, 15th, 20th, and 24th frames, respectively. The coordinates of each point in the diagram represent the coordinates of the feature point pair, and the color represents the kernel density estimate for that point. Experiments demonstrate that kernel density estimate peaks exist in each frame.

[0114] Figure 3 This figure shows the accuracy analysis of the inter-frame global displacement estimation module during the first experimental verification. The X-axis represents the order of the frames during the integration process, and the Y-axis represents the error between the kernel density estimated displacement and the true displacement. The blue line represents the error in the X direction, and the red line represents the error in the Y direction. The experiment showed that the kernel density estimation error in both the X and Y directions during this imaging process was less than 0.01 pixel.

[0115] Figure 4 The following diagrams illustrate the imaging results of the first experimental verification of the kernel density estimation-based image self-registration (TDI) imaging method of the present invention; specifically, they illustrate satellite-to-earth remote sensing imaging in the remote sensing field. Sub-images (a), (c), (e), and (g) are schematic diagrams of TDI imaging without motion compensation, while sub-images (b), (d), (f), and (h) are schematic diagrams of imaging using the kernel density estimation-based image self-registration (TDI) imaging method of the present invention. The experiment demonstrates that the present invention can significantly reduce motion blur and improve the quality of time delay integration (TDI) imaging in satellite-to-earth remote sensing imaging.

[0116] Figure 5It is the imaging effect diagram of the second experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the application, specifically, the image shot by the industrial TDI camera in the industrial detection field in the PCB production process, wherein, subgraph (a) is the TDI imaging effect diagram without motion compensation, and subgraph (b) is the imaging effect diagram of the image self-registration TDI imaging method based on kernel density estimation of the application. The experiment shows that the clarity of subgraph (b) is better, and the imaging blur caused by the relative motion of the assembly line and the detected PCB board is effectively inhibited. In the industrial detection field, the application can also significantly reduce the motion blur between the camera and the detected product, and improve the time delay integration imaging quality.

[0117] Figure 6 It is the imaging effect diagram of the third experimental verification of the image self-registration TDI imaging method based on kernel density estimation of the application, specifically, the fundus image of a glaucoma case shot by the fundus TDI camera in the medical imaging field, wherein, subgraph (a) is the TDI imaging effect diagram without motion compensation, and subgraph (b) is the imaging effect diagram of the image self-registration TDI imaging method based on kernel density estimation of the application. The experiment shows that the application is also applicable in the medical imaging field, can improve the clarity of medical TDI camera imaging, and improve the fundus imaging quality.

[0118] The image self-registration TDI imaging method based on kernel density estimation of the application not only improves the imaging quality, but also has a wide application prospect, and is suitable for multiple fields such as remote sensing, medical imaging and industrial detection.

[0119] Obviously, the above embodiments are only examples for clearly illustrating, rather than limiting the implementation modes. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the implementation modes need not and cannot be exhausted. The obvious changes or variations derived therefrom are still within the protection scope of the application.

Claims

1. A method of image self-registration TDI imaging based on kernel density estimation, characterized in that, The method comprises the following steps: Step 1: feature point extraction of the reference frame area array image; A set of feature points are obtained from the reference frame area array image by using a feature extraction operator, and a feature vector of each feature point is calculated; Step 2: feature point extraction of the frame to be registered; a set of feature points are obtained from the frame to be registered by using a feature extraction operator, and a feature vector of each feature point is calculated; Step 3: feature point matching; the feature points extracted from the reference frame area array image and the frame to be registered are matched one by one according to the similarity of the feature vectors of the two sets of feature points; Step 4: coordinate difference calculation of the feature point pairs; The coordinate difference of each feature point pair between the reference frame area array image and the frame to be registered is calculated; Step 5: displacement value weighting coefficient calculation of the feature point pairs; a probability density of a random variable, i.e. a weighting coefficient, is calculated for each set of matched point pairs by using a kernel density estimation algorithm on the coordinate difference values of the matched point pairs; Step 6: inter-frame image global displacement estimation; an estimation value of the inter-frame global displacement is obtained by analyzing the kernel density estimation results of each coordinate difference point, and taking a weighted average of the pixel numbers of the coordinate difference values of the feature point pairs according to the weighting coefficients as the estimation value of the inter-frame global displacement; Step 7: alignment of the frame to be registered; According to the displacement estimation results of the inter-frame area array images, a geometric transformation is performed on each pixel of the frame to be registered to obtain an aligned area array image, so that the frame to be registered is aligned with the reference frame area array image in space. Step 8: superposition of the frame to be registered; the pixel array of the frame to be registered after alignment obtained in step 7 is superposed into a digital domain integrator; Step 9: output of the final TDI image; steps 2-8 are repeated until the alignment and superposition of all the frames to be registered in the integral order are completed, a complete imaging process is realized, and the final TDI image is obtained.

2. The kernel density estimation based image self-registration TDI imaging method of claim 1, wherein, In step 2, the type and setting parameters of the feature extraction operator of the frame to be registered are the same as those of the feature extraction operator of the reference frame area array image in step 1.

3. The kernel density estimation based image self-registration TDI imaging method of claim 1, wherein, Step 4 is specifically: A coordinate difference Δx = x in the line direction between the frame surface array image to be registered and the reference frame surface array image 2i - x 1i coordinate difference in the column direction between the frame to be registered and the reference frame Δy = y 2i - y 1i wherein (x 1i ,y 1i ) and (x 2i ,y 2i ) are any one pair of matching point pairs in the set of p pairs of feature points between the reference frame plane image and the frame plane image to be registered. wherein (x 1i ,y 1i ) and (x 2i ,y 2i ) are any one pair of matching point pairs in the set of p pairs of feature points between the reference frame plane image and the frame plane image to be registered.

4. The kernel density estimation based image self-registration TDI imaging method of claim 3, wherein, Step 5 is specifically: For each coordinate difference point (Δx, Δy), the probability density estimation value is calculated, and the displacement value weighting coefficient of the feature point pair is obtained Wherein, n is the number of coordinate difference points in the sample data, and h is a smoothing bandwidth, which controls the amount of smoothing. The larger the value of h is, the smoother the data is.

5. The kernel density estimation based image self-registration TDI imaging method of claim 4, wherein, Step 6 is specifically: frame interlaced image displacement in the row direction in pixels frame interlace image displacement in column direction wherein, is a feature point pair displacement value weighting coefficient, and Δx and Δy are coordinate difference values in the row direction and the column direction of a matched feature point pair, respectively.

6. The kernel density estimation based image self-registration TDI imaging method of claim 5, wherein, Step 7 is specifically: The number of pixels of the inter-frame area array image displacement in the row direction Δm and the number of pixels of the inter-frame area array image displacement in the column direction Δn, corresponding to the integer part Δm′ of the number of pixels of the k-th frame image k , Δn′ k and the decimal part Δm″ k ,Δn″ k Respectively expressed as: Δm' k = fix(Δm) Δn' k = fix(Δn) Δm" k = Δm - Δm' k Δn" k = Δn - Δn' k Pixel array P of the frame face image to be registered aligned with the reference frame face image k (i,j) is expressed as: P k (i,j) = P(i+Am' k ,j+An' k )(1-Am" k ) +P(i+Δm′ k ,j+Δn′ k +1)(1-Δm″ k )Δn″ k + P(i + Am', j + An') - P(i + Am', j - An') k + P(i - Am', j + An') - P(i - Am', j - An') k + P(i + Am', j + An') - P(i + Am', j - An') k + P(i - Am', j + An') - P(i - Am', j - An') k ​ +P(i+Δm′ k +1,j+Δn′ k +1)Δm″ k Δn′ k 。 7. The kernel density estimation based image self-registration TDI imaging method of claim 6, wherein, Step 8 is specifically: After the k-th superposition, the gray value of each pixel in the digital domain integrator is represented as: O k (i,j) = O k-1 (i,j) + P k (i,j).

8. The kernel density estimation based image self-registration TDI imaging method of claim 7, wherein, In step 9, when the integral order M is M, the gray value of each pixel of the final TDI image is represented as:

9. A system suitable for use in a kernel density estimation based image self- registration TDI imaging method according to any one of claims 1-8, characterized in that, It comprises: a feature point pair extraction and matching module (101), a feature point pair coordinate difference calculation module (102), an inter-frame global displacement estimation module (103), an inter-frame image alignment module (104), and an image superposition module (105); wherein, the feature point pair extraction and matching module (101) is used to execute steps 1-3; the feature point pair coordinate difference calculation module (102) is used to execute step 4; the inter-frame global displacement estimation module (103) is used to execute steps 5 and 6; the inter-frame image alignment module (104) is used to execute step 7; the image superposition module (105) is used to execute steps 8 and 9.

Images