Dynamic target recognition method and system based on image registration

Abstract

The application provides a dynamic target identification method and system based on image registration, comprising the following steps: S1, pre-processing satellite attitude orbit measurement information, corresponding the attitude orbit data of the satellite to the imaging time of the camera; S2, establishing a registration frame forward mapping relationship to obtain the corresponding relationship between the image element in the registration frame image and the latitude and longitude of the ground; S3, establishing a reference frame inverse mapping relationship to obtain the corresponding relationship between the latitude and longitude of the ground and the image element coordinates of the registration frame image; S4, after registering multiple images to the reference frame, identifying the dynamic target through trajectory association.

Description

Technical Field

[0001] This invention relates to the field of satellite remote sensing target recognition technology, and more specifically, to a dynamic target recognition method and system based on image registration. Background Technology

[0002] Target acquisition and identification is a crucial function of remote sensing satellites, especially if target information can be directly output from the satellite, which can significantly enhance the satellite's application capabilities. However, for low-Earth orbit remote sensing satellites, the relative motion between the satellite, the Earth, and the target causes some stationary reference objects to still exhibit dynamic characteristics in remote sensing images, severely affecting the accuracy of target identification.

[0003] A literature review revealed a method combining edge feature extraction and feature point extraction in the paper "A Method for Inter-Frame Image Registration of High-Resolution Remote Sensing Satellites" (Jiangxi Science, 2011, Vol. 29, No. 6). This method addresses the problem of inaccurate edge information extraction caused by cloud occlusion in satellite cloud images. The method extracts edge features and feature points based on an improved Canny algorithm and the Harris detection operator, and uses Hu invariant moments to match image feature points. However, the registration algorithm proposed in this paper is based on image feature point processing, which requires significant image processing computational resources and is difficult to apply to on-board processing.

[0004] The paper "Remote Sensing Image Registration Based on Deep Learning Feature Extraction" (Aerospace Return and Remote Sensing, Vol. 40, No. 6, 2019) discloses an image jitter elimination algorithm. It designs a color histogram matching algorithm based on pre-defined target matching regions (one or more). By analyzing and comparing the similarity of the color histograms of the target region and candidate regions in two images before and after jitter, the optimal matching block is found, and the corresponding block's motion displacement is obtained. This allows for the calculation of the absolute motion displacement of the two images, ultimately eliminating the interference displacement. However, the registration algorithm proposed in this paper is also based on image feature point processing and applies artificial intelligence algorithms, resulting in a high computational load and making it difficult to apply to on-board processing.

[0005] The paper "Analysis of Factors Affecting Image Registration Accuracy of High-Orbit Hyperspectral Remote Sensing Satellites" (Computer Engineering and Applications, 2016, No. 52) discloses an image registration accuracy analysis method. To rationally allocate satellite design specifications, based on research into high-orbit hyperspectral imaging and scanning methods, the paper analyzes influencing factors and proposes possible solutions based on current engineering capabilities, laying the foundation for subsequent satellite development. This article focuses on summarizing the error terms affecting image registration accuracy but does not cover specific registration methods.

[0006] Patent document CN112419380A (application number: 202011340239.5) discloses a high-precision registration method for geostationary satellite image sequences based on cloud masking. The method generates single-frame distortion-free images through virtual re-imaging to eliminate distortion within individual frames of the image sequence. Then, a quadratic adaptive threshold calculated using the Otsu's method is used to quickly detect clouds in the image sequence. Coarse geometric relationships established between the image sequences are used to obtain matching blocks between non-reference frames and reference frames. The SIFT algorithm is then used to match blocks one by one to obtain matching feature points. Afterwards, mismatched points in cloud areas are removed based on the cloud detection results, and random sampling consistency is used to further remove mismatched points, resulting in highly reliable matching point pairs. Finally, a registration model based on global affine transformation is used to register the image sequence.

[0007] In summary, the proposed dynamic target recognition method based on image registration eliminates the need for complex image matching calculations. It utilizes the attitude and orbit measurement information of the satellite platform to obtain the precise registration relationship between two image frames, resulting in low computational complexity and high accuracy, making it suitable for on-board processing. Our research confirms the novelty of this invention. Summary of the Invention

[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide a dynamic target recognition method and system based on image registration.

[0009] A dynamic target recognition method based on image registration according to the present invention includes:

[0010] Step S1: Preprocess the satellite attitude and orbit measurement information to correlate the satellite attitude and orbit data with the camera's imaging time;

[0011] Step S2: Establish the forward mapping relationship of the registration frames to obtain the correspondence between pixels in the registration frame image and the latitude and longitude of the land surface;

[0012] Step S3: Establish the inverse mapping relationship of the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image;

[0013] Step S4: After registering multiple frames of images to the reference frame, identify the dynamic target through trajectory association.

[0014] Preferably, step S1 involves: obtaining the attitude information of the camera at the imaging moment through attitude kinematics recursion; and obtaining the trajectory information of the camera at the imaging moment through orbit kinematics recursion.

[0015] Preferably, step S2 involves obtaining the mapping relationship between pixels in the registration frame image and latitude and longitude of the Earth's surface through attitude transformation, orbit transformation, and Earth rotation transformation in sequence.

[0016] Preferably, step S3 involves: obtaining the mapping relationship between the latitude and longitude of the land surface and the reference frame image by performing a reverse operation on the forward mapping relationship.

[0017] Preferably, step S4 involves: after registering multiple frames of images to a reference frame, identifying dynamic targets by eliminating targets that do not meet preset requirements through trajectory association.

[0018] A dynamic target recognition system based on image registration according to the present invention includes:

[0019] Module M1: Preprocesses satellite attitude and orbit measurement information, and correlates the satellite's attitude and orbit data with the camera's imaging time;

[0020] Module M2: Establishes a forward mapping relationship between registration frames to obtain the correspondence between pixels in the registration frame image and latitude and longitude of the land surface;

[0021] Module M3: Establishes the inverse mapping relationship of the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image;

[0022] Module M4: After registering multiple frames of images to a reference frame, it identifies dynamic targets through trajectory correlation.

[0023] Preferably, module M1 employs: obtaining attitude information of the camera at the imaging moment through attitude kinematics recursion; and obtaining trajectory information of the camera at the imaging moment through orbit kinematics recursion.

[0024] Preferably, module M2 acquires the mapping relationship between pixels in the registration frame image and latitude and longitude of the Earth's surface by sequentially performing attitude transformation, orbit transformation, and Earth rotation transformation.

[0025] Preferably, module M3 employs the following method: obtaining the mapping relationship between the latitude and longitude of the land surface and the reference frame image by performing a reverse operation on the forward mapping relationship.

[0026] Preferably, module M4 employs the following method: after registering multiple frames of images to a reference frame, dynamic targets are identified by eliminating targets that do not meet preset requirements through trajectory association.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. This invention proposes an inter-frame registration method based on the relative motion between stars and the ground, which can effectively identify targets in complex backgrounds;

[0029] 2. The dynamic target recognition method based on image registration proposed in this invention does not require complex image matching calculations. It uses the attitude and orbit measurement information of the satellite platform to obtain the accurate registration relationship between two frames of images. It has low computational load and high accuracy, making it suitable for on-board processing.

[0030] 3. This invention relates to satellite remote sensing target recognition technology. This method can be used to detect dynamic targets in satellite remote sensing images, which is of great significance for improving the application capabilities of remote sensing satellites. Attached Figure Description

[0031] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0032] Figure 1 This is a flowchart of dynamic target recognition based on image registration.

[0033] Figure 2 This is a diagram illustrating the principle of inter-frame image registration.

[0034] Figure 3 It is the target trajectory diagram on the image coordinate system before registration.

[0035] Figure 4 It is the target trajectory diagram on the coordinate system of the registered image. Detailed Implementation

[0036] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0037] The relative motion between the satellite and the Earth causes ground objects to exhibit motion characteristics in the camera's field of view, severely interfering with target recognition. This invention proposes a dynamic target recognition method and system based on image registration. It eliminates the need for complex image matching calculations, utilizing the attitude and orbit measurement information of the satellite platform to obtain the precise registration relationship between two images, thereby improving the accuracy of dynamic target recognition.

[0038] Example 1

[0039] According to the present invention, a dynamic target recognition method based on image registration is provided, such as... Figures 1 to 2 As shown, it includes:

[0040] Step S1: Preprocess the satellite attitude and orbit measurement information to correlate the satellite attitude and orbit data with the camera's imaging time; specifically, the preprocessing of the satellite attitude and orbit measurement information involves: obtaining the attitude information of the camera's imaging time through attitude kinematics recursion; and obtaining the orbit information of the camera's imaging time through orbit kinematics recursion.

[0041] Step S2: Establish a forward mapping relationship for the registration frame to obtain the correspondence between pixels in the registration frame image and latitude and longitude on the Earth's surface; specifically, the establishment of the forward mapping relationship for the registration frame is achieved by sequentially obtaining the mapping relationship between pixels in the registration frame image and latitude and longitude on the Earth's surface through attitude transformation, orbit transformation and Earth rotation transformation.

[0042] Step S3: Establish the inverse mapping relationship of the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image; specifically, the establishment of the inverse mapping relationship of the reference frame is carried out by: performing inverse operation on the forward mapping relationship to obtain the mapping relationship between the latitude and longitude of the land surface and the reference frame image.

[0043] Step S4: After registering multiple frames of images to a reference frame, dynamic targets are identified through trajectory correlation. Specifically, step S4 involves: after registering multiple frames of images to a reference frame, identifying dynamic targets by eliminating various suspected targets through trajectory correlation.

[0044] This invention obtains the precise registration relationship between two frames of images through satellite attitude and orbit measurement information, without the need for complex image processing.

[0045] A dynamic target recognition system based on image registration according to the present invention includes:

[0046] Module M1: Preprocesses the satellite attitude and orbit measurement information, and correlates the satellite attitude and orbit data with the camera's imaging time; specifically, the preprocessing of the satellite attitude and orbit measurement information adopts the following methods: obtaining the attitude information of the camera imaging time through attitude kinematics recursion; obtaining the orbit information of the camera imaging time through orbit kinematics recursion.

[0047] Module M2: Establishes a forward mapping relationship for the registration frame to obtain the correspondence between pixels in the registration frame image and latitude and longitude on the Earth's surface; specifically, the establishment of the forward mapping relationship for the registration frame is achieved by sequentially obtaining the mapping relationship between pixels in the registration frame image and latitude and longitude on the Earth's surface through attitude transformation, orbit transformation and Earth rotation transformation.

[0048] Module M3: Establishes a reverse mapping relationship for the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image; specifically, the establishment of the reverse mapping relationship for the reference frame is achieved by performing a reverse operation on the forward mapping relationship to obtain the mapping relationship between the latitude and longitude of the land surface and the reference frame image.

[0049] Module M4: After registering multiple frames of images to a reference frame, it identifies dynamic targets through trajectory correlation. Specifically, module M4 identifies dynamic targets by: registering multiple frames of images to a reference frame, and then eliminating various suspected targets through trajectory correlation.

[0050] This invention obtains the precise registration relationship between two frames of images through satellite attitude and orbit measurement information, without the need for complex image processing.

[0051] Example 2

[0052] Example 2 is a preferred example of Example 1.

[0053] This invention provides a dynamic target recognition method based on image registration, comprising:

[0054] 1) Preprocessing of satellite attitude and orbit measurement information

[0055] The purpose of satellite attitude and orbit measurement information preprocessing is to obtain satellite attitude and orbit information at the moment of camera imaging.

[0056] First, the attitude information of the camera at the moment of imaging is obtained through attitude kinematics recursion, as shown in the following expression:

[0057]

[0058] θ1=θ0+ω θ t

[0059] ψ1=ψ0+ω ψ t

[0060] In the above formula, It is the roll angle at the moment the camera captures the image. It is the roll angle transmitted by the satellite platform. It is the roll angular velocity. θ1 is the pitch angle at the moment of camera imaging, θ0 is the pitch angle transmitted by the satellite platform, and ω... θ It is the pitch angular velocity. ψ1 is the yaw angle at the moment of camera imaging, ψ0 is the yaw angle transmitted by the satellite platform, and ω ψ It is the yaw rate. t is the difference between the camera imaging time and the platform attitude transmission time.

[0061] Then, the orbital information at the camera imaging moment is obtained through orbital kinematics recursion, as shown in the following expression:

[0062]

[0063] v1=v0+αt

[0064] R1=R0+v1t

[0065] In the above formula, α is the orbital acceleration, R1 is the orbital position at the moment of camera imaging, R0 is the orbital position transmitted by the satellite platform, v1 is the orbital velocity at the moment of camera imaging, v0 is the orbital velocity transmitted by the satellite platform, μ is the Earth's gravitational constant, and t is the difference between the moment of camera imaging and the moment of platform attitude transmission.

[0066] 2) Establish a forward mapping relationship for the registration frames

[0067] The mapping relationship between the registered frame image and the latitude and longitude of the Earth's surface was obtained by sequentially performing attitude transformation, orbit transformation, and Earth rotation transformation. The expression is as follows:

[0068]

[0069] in:

[0070] r c : The projection of the target vector in the camera coordinate system;

[0071] The camera's installation matrix relative to the satellite's coordinate system can be calibrated through ground-based precision surveying;

[0072] The transformation matrix from the satellite's body coordinate system to its orbital coordinate system is calculated based on the attitude information;

[0073] θ, φ, The satellite's three attitude angles (roll angle, pitch angle, and yaw angle);

[0074] The transformation matrix from the satellite orbital coordinate system to the geocentric inertial coordinate system is calculated based on the orbital information;

[0075]

[0076] In the above formula, i is the orbital inclination angle; Ω is the right ascension of the ascending node; and u is the argument of the ascending node.

[0077] The transformation matrix from the geocentric inertial coordinate system to the Earth-fixed coordinate system.

[0078] t1: The time of image registration.

[0079] Based on the above analysis, the projection of the target vector of the registered image onto the Earth-fixed coordinate system can be obtained. Finally, by calculating the intersection point of the target vector and the Earth, the geographical latitude and longitude of the target can be determined.

[0080] 3) Establish the inverse mapping relationship of the reference frame.

[0081] Let the imaging time of the reference image be t0. Based on the orbital information at time t0, the satellite's pointing vector towards the target at that time can be obtained. right After normalization, the components of the satellite pointing at the target in the camera coordinate system at time t0 are obtained by substituting them into the inverse mapping model:

[0082]

[0083] in:

[0084] r c : The component of the satellite pointing at the target in the camera coordinate system at time t0;

[0085] The transformation matrix from the Earth-fixed coordinate system to the geocentric inertial coordinate system at time t0;

[0086] The transformation matrix from the geocentric inertial coordinate system to the satellite orbital coordinate system at time t0;

[0087] The transformation matrix from the satellite orbit coordinate system to the body coordinate system at time t0.

[0088] Finally, by combining the camera's imaging model, the pixel coordinates of the target in the reference image can be obtained, thus achieving on-board inter-frame registration.

[0089] 4) Dynamic target recognition method

[0090] By registering suspected targets from multiple consecutive frames of images to a reference frame, and then eliminating various suspected targets through trajectory correlation, dynamic targets are identified. Figure 3 It can be seen that before registration, all types of targets exhibit motion characteristics on the detector, making it impossible to identify truly dynamic targets. From Figure 4 As can be seen, after inter-frame registration, only dynamic targets will exhibit motion characteristics in the detector. By calculating the motion velocity corresponding to each trajectory, if the motion velocity exceeds a certain threshold, it can be identified as a dynamic target.

[0091] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0092] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A dynamic target recognition method based on image registration, characterized in that, include: Step S1: Preprocess the satellite attitude and orbit measurement information to correlate the satellite attitude and orbit data with the camera's imaging time; Step S1 involves: obtaining the attitude information of the camera at the imaging moment through attitude kinematics recursion; and obtaining the trajectory information of the camera at the imaging moment through orbit kinematics recursion. Step S2: Establish the forward mapping relationship of the registration frames to obtain the correspondence between pixels in the registration frame image and the latitude and longitude of the land surface; Step S2 involves sequentially obtaining the mapping relationship between pixels in the registration frame image and latitude and longitude of the Earth's surface through attitude transformation, orbit transformation, and Earth rotation transformation. Step S3: Establish the inverse mapping relationship of the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image; Step S3 involves: performing a reverse operation on the forward mapping relationship to obtain the mapping relationship between the latitude and longitude of the land surface and the reference frame image; Step S4: After registering multiple frames of images to the reference frame, identify the dynamic target through trajectory association; Step S4 involves: after registering multiple frames of images to a reference frame, identifying dynamic targets by removing targets that do not meet preset requirements through trajectory association.

2. A dynamic target recognition system based on image registration, characterized in that, include: Module M1: Preprocesses satellite attitude and orbit measurement information, and correlates the satellite's attitude and orbit data with the camera's imaging time; In module M1: attitude information at the camera imaging moment is obtained by attitude kinematics recursion; trajectory information at the camera imaging moment is obtained by orbit kinematics recursion. Module M2: Establishes a forward mapping relationship between registration frames to obtain the correspondence between pixels in the registration frame image and latitude and longitude of the land surface; In module M2: the mapping relationship between pixels in the registration frame image and latitude and longitude of the Earth's surface is obtained sequentially through attitude transformation, orbit transformation and Earth rotation transformation; Module M3: Establishes the inverse mapping relationship of the reference frame to obtain the correspondence between the latitude and longitude of the land surface and the pixel coordinates of the registered frame image; In module M3: the mapping relationship between the latitude and longitude of the land surface and the reference frame image is obtained by performing a reverse operation on the forward mapping relationship; Module M4: After registering multiple frames of images to a reference frame, it identifies dynamic targets through trajectory correlation; In module M4: after multiple frames of images are registered to a reference frame, dynamic targets are identified by eliminating targets that do not meet the preset requirements through trajectory association.

Images