Telescope electronic image stabilization method and image stabilization system based on theoretical guidance trajectory

Abstract

The present invention relates to the field of telescope image processing technology, specifically providing a telescope electronic image stabilization method and system based on theoretical guidance trajectories. This method calculates the telescope pointing error based on encoder pointing data and the telescope's guidance direction, and then calculates the jitter offset of the target image. A cropping region is calculated based on this jitter offset, and the target image is cropped based on the cropping region. The cropped target image is then output to achieve image stabilization. This method is independent of hardware equipment and does not require target feature extraction. It is technically low in difficulty, simple to implement, and provides excellent image stabilization results.

Description

Technical Field

[0001] The present invention relates to the technical field of telescope image processing, and in particular provides a telescope electronic image stabilization method and an image stabilization system based on a theoretical guidance trajectory. Background Art

[0002] Ground-based electro-optical telescopes have a variety of guidance methods, including theoretical trajectory guidance, external guidance, fixed-point guidance, constant velocity guidance, and closed-loop guidance based on miss distance. Whether the telescope image is jittery generally depends on the smoothness and accuracy of the telescope's guidance trajectory. Theoretical trajectory guidance is a common method used for telescopes. For example, observations of stars, satellites, and space stations are all based on observation trajectories calculated from theoretical orbital models. Fixed-point guidance and constant velocity guidance are commonly used for ground and sea surface searches and island observations.

[0003] In the theoretical trajectory guidance method, the guided trajectory is smooth and accurate, and usually does not cause image jitter. However, during the observation process, strong winds often occur, and it is inevitable that the telescope will be disturbed by strong winds and cause image jitter.

[0004] In a strong wind, a telescope was used to conduct an observation experiment. The telescope used in the experiment had an aperture of 500mm and was equipped with a short-wave infrared camera. The observation field was , the image resolution is , the horizontal and vertical angular resolutions are .

[0005] In Experiment 1, we used the fixed-point mode to observe the azimuth 206.6° and pitch -0.5°. The error curves of the azimuth and pitch axes are shown in the figure below. Figure 1 As shown. The absolute values ​​of the maximum errors of the azimuth and pitch axes exceed , where the azimuth axis error RMS (Root Mean Square) is , the RMS error of the pitch axis is The horizontal RMS of the image jitter amplitude converted by angular resolution is: , the vertical RMS is .

[0006] Experiment 2 uses the theoretical trajectory to guide the telescope to track and observe the International Space Station (ISS). Figure 2 The weighted centroid algorithm is used to calculate the pixel centroid of the International Space Station. The difference between the centroids of the frames is used as the criterion for judging the image jitter amplitude. The error curves of the azimuth and pitch axes are shown in Figure 3 As shown, Figure 3 The maximum absolute value of the horizontal and vertical jitter amplitude of the International Space Station exceeds , where the horizontal jitter amplitude RMS is , the vertical jitter amplitude RMS is .

[0007] Image jitter caused by strong winds not only affects the user experience but also negatively impacts tasks such as target search, capture, and extraction. Therefore, research on electronic image stabilization (EIS) is crucial to address this issue. Electronic image stabilization (EIS) uses image processing algorithms to eliminate video jitter, primarily for dynamic scene capture. Traditional image processing-based stabilization methods include block matching, grayscale projection, and feature point matching.

[0008] The Chinese patent application, CN117687346A, published on March 12, 2024, and titled "Spatial Image Stabilization Control System and Control Method for Shipborne Electro-Optical Theodolite," proposes a spatial image stabilization control system for a shipborne electro-optical theodolite. The system includes a main control module, an inertial navigation module, a servo control module, an encoder module, an image enhancement module, and a control console, representing a multi-system joint control image stabilization solution. However, some ground-based electro-optical telescopes lack one or more of these modules, making them inoperable for already installed and deployed telescopes. Furthermore, these telescopes are often difficult to modify, and even if they can be modified, the cost is prohibitive.

[0009] The Chinese patent publication number is CN118334063A, and the publication date is July 21, 2024. The patent name is "A method for stabilizing a ground-based telescope based on kernel correlation filtering". The invention patent application uses kernel correlation filtering to extract target feature areas, and calculates jitter compensation by the centroid offset of the target feature area of ​​two adjacent frames of images to achieve electronic image stabilization. The problem with this method is that when the target in the image is a dim target or there is strong signal background noise interference in the image, the kernel correlation filter is prone to incorrectly extract the target, resulting in image stabilization failure. In this case, the kernel correlation filter cannot be used for feature extraction, and when the telescope observes the ground, sea surface, and islands at a certain rotation speed, the background itself is in motion, and the image stabilization method based on target feature extraction should not be used. Summary of the Invention

[0010] To address these issues, the present invention provides a telescope electronic image stabilization method and system based on a theoretical guidance trajectory. This method calculates the target's jitter offset based on encoder pointing data and the telescope's guidance direction, then determines a cropping region and crops the target image to achieve image stabilization. This method is decoupled from mechanical equipment, is easy to implement, and provides excellent image stabilization results.

[0011] The present invention provides a telescope electronic image stabilization method based on a theoretical guidance trajectory, comprising:

[0012] S1: Get the exposure center time corresponding to the target image ;

[0013] S2: Get the encoder pointing data of the last two frames;

[0014] S3: Obtain the telescope guidance direction corresponding to the two most recent frames of encoder direction data in step S2;

[0015] S4: Calculate telescope pointing error :

[0016] ;

[0017] in,

[0018] ;

[0019] ;

[0020] Indicates the receiving time corresponding to the encoder pointing data, A sequence number indicating the time of receipt, Display and reception time The corresponding encoder points to the data, Display and reception time The corresponding encoder points to the data, Display and reception time The corresponding telescope guide pointing;

[0021] S5: Calculate the jitter offset of the target image :

[0022] :

[0023] ;

[0024] ;

[0025] in, Indicates the horizontal jitter offset, Indicates the vertical jitter offset, Indicates encoder pointing data The encoder pitch value of

[0026] S6: Calculate a cropping area according to the jitter offset, crop the target image according to the cropping area, and output the cropped target image.

[0027] Preferably, in step S2, the encoder pointing data of the last two frames are and ,in, Indicates the receiving time corresponding to the data pointed to by the receiving encoder The serial number, .

[0028] Preferably, in step S3, the telescope guidance points are: Corresponding telescope guidance pointing , and receiving time Corresponding telescope guidance pointing .

[0029] Preferably, the clipping area is calculated as follows:

[0030] ;

[0031] in, Indicates the coordinates of the upper left corner pixel of the target image, , , Indicates the horizontal cropping ratio, Indicates the vertical cropping ratio, Indicates the width of the target image, Indicates the height of the target image.

[0032] Preferably, the coordinates of the upper left corner pixel of the target image are The calculation method is:

[0033] ;

[0034] ;

[0035] in, , .

[0036] Preferably, the coordinates of the upper left corner pixel of the target image are The value range satisfies 、 .

[0037] A telescope electronic image stabilization system based on a theoretical guidance trajectory is used to implement a telescope electronic image stabilization method based on a theoretical guidance trajectory, comprising an image processing module, a telescope control module, a camera, and a servo control system;

[0038] The telescope control module obtains guidance information and sends it to the servo control system; the servo control system controls the rotation of the telescope based on the guidance information and sends the telescope's encoder pointing data; the camera captures the target image and sends it to the image processing module; after receiving the target image sent by the camera, the image processing module calculates the target's jitter offset and the corresponding cropping area, and crops the target image; the image processing module outputs the cropped image to the display area.

[0039] Preferably, it also includes a data sharing area, which stores guidance information and encoder pointing data.

[0040] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0041] This method calculates the target image's cropping region based on various data from the telescope's guidance process, and then crops any jittery images. This solves the image jitter problem caused by windy weather interference when tracking a target using a known theoretical guidance trajectory. This method does not require feature extraction or high definition, and has a wide range of applications, including search missions in background environments such as land, sea, and islands.

[0042] At the same time, the method of the present invention does not rely on external conditions such as mechanical structure and optical system, and can be applied to existing equipment without the need to modify old equipment. It has low cost, low technical difficulty, is simple and easy to implement, and has a good image stabilization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a graph of azimuth and elevation jitter when observing in fixed-point mode of the telescope mentioned in the background art;

[0044] Figure 2 This is the schematic diagram of the International Space Station imaging mentioned in the background technology;

[0045] Figure 3 This is a graph of the azimuth and pitch jitter when observing the International Space Station using the theoretical trajectory guidance method mentioned in the background technology;

[0046] Figure 4 is a flow chart of a telescope electronic image stabilization method based on a theoretical guidance trajectory according to an embodiment of the present invention;

[0047] Figure 5 is a layout diagram of a telescope electronic image stabilization system based on a theoretical guidance trajectory according to an embodiment of the present invention;

[0048] Figure 6 3 is a graph showing azimuth and pitch jitter curves after image stabilization using the method of the present invention according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention. Similar elements in different embodiments use associated similar element numbers. In the following embodiments, many detailed descriptions are intended to enable the present invention to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present invention are not shown or described in the specification. This is to avoid the core part of the present invention being overwhelmed by too much description. For those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0050] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other to form various implementation methods. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various orders in the description and the drawings are only for the purpose of clearly describing a certain embodiment and are not intended to be a required order, unless otherwise specified that a certain order must be followed.

[0051] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0052] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0053] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0054] The telescope contains a camera. When observing a target through the telescope, the camera captures images of the observed target, resulting in multiple frames of continuous target images. To address the issue of target jitter in the images, these multiple frames need to be processed and output for image stabilization. The target images mentioned in the embodiments of the present invention are not limited to images of individual targets such as aircraft and satellites. They also include images of targets such as land, islands, and sea surfaces. Any object that can be observed and tracked by a telescope guided according to a theoretical guidance trajectory can serve as a target.

[0055] like Figure 4 As shown, the present invention provides a telescope electronic image stabilization method based on a theoretical guidance trajectory, the specific contents of which are as follows:

[0056] In this embodiment of the present invention, after the camera transmits a captured target image, the corresponding image processing module receives it, triggering the image stabilization process. Each triggered process is a single-frame process. Each received target image undergoes the following image stabilization process to obtain a cropped image. During continuous observation, multiple triggered single-frame processes are combined to produce the final image stabilization result.

[0057] S1: Get the exposure center time corresponding to the target image ;

[0058] After the camera captures the target image, the image processing module receives the target image sent by the camera and receives the exposure center time corresponding to the target image. When a frame of target image needs to be stabilized, the exposure center time corresponding to the target image needs to be obtained. .

[0059] S2: Get the encoder pointing data of the last two frames.

[0060] When observing a target, a telescope is guided according to a guidance trajectory. This trajectory is a sequence of guidance information, a smoothly varying curve, calculated using a theoretical orbital model of the observed target. Each piece of guidance information includes a guidance time, a guidance azimuth, and a guidance pitch. When guided according to the guidance trajectory, each rotation of the telescope based on this guidance information is accompanied by encoder pointing data, which reflects the actual telescope's current pointing direction. This series of encoder pointing data is combined to form the telescope's actual motion trajectory. The actual motion trajectory should be a smooth curve in calm or light winds, but will become non-smooth in strong winds, resulting in jitter.

[0061] The encoder pointing data generated during telescope observation is stored in the data sharing area. The receiving time corresponding to a frame of encoder pointing data received by the data sharing area is set to , A serial number representing the reception time. It should be noted that each time a guidance message is applied to guide the telescope, corresponding encoder pointing data is generated. Therefore, the guidance time contained in the guidance message is the same as the reception time here.

[0062] The encoder pointing data includes the encoder azimuth value and the encoder pitch value. Indicates that any encoder points to data, then , where the encoder points to the data in Indicates that the encoder points to data that is received at time The corresponding encoder points to the data, Indicates encoder pointing data The encoder position value, Indicates encoder pointing data The encoder pitch value.

[0063] Get the encoder pointing data of two adjacent frames from the data sharing area and ,in, Indicates encoder pointing data Is the receiving time The corresponding encoder points to the data, and the encoder points to the data of the two adjacent frames obtained and The corresponding receiving time satisfies .

[0064] S3: Obtain the telescope guidance direction corresponding to the two most recent frames of encoder direction data in step S2.

[0065] The guidance trajectory used to guide the telescope is composed of a series of guidance information, each of which includes a guidance time, a guidance azimuth value, and a guidance pitch value. The coordinates composed of the guidance azimuth value and the guidance pitch value of the guidance information are the guidance direction of the telescope. Indicates that the telescope guidance direction is also determined by the guidance azimuth value and guidance pitch value Composition, that is .

[0066] Obtain two adjacent frames of guidance information from the data sharing area, and then obtain two frames of telescope guidance pointing. The obtained two frames of telescope guidance pointing are respectively consistent with the pointing data in step S2. and Correspondingly, the obtained two-frame telescope guidance pointing is and .

[0067] S4: Calculate the telescope pointing error.

[0068] The pointing error of the telescope is calculated based on the obtained encoder pointing data and the telescope guidance pointing. Indicates exposure center time The corresponding pointing error, the encoder pointing data and the telescope guidance pointing both contain azimuth and pitch values, and the corresponding pointing error also contains azimuth error and pitch error, that is, , then the pointing error The calculation method is:

[0069] ;

[0070] in, and Calculated by the following formula:

[0071] ,

[0072] .

[0073] S5: Calculate the jitter offset of the target image.

[0074] To complete image stabilization, it is necessary to calculate the equivalent offset value of the target in the target image. In the embodiment of the present invention, the equivalent offset is called the jitter offset. Indicates, correspondingly, the jitter offset It also includes a horizontal component and a vertical component. In the embodiment of the present invention, the azimuth value and the pitch value are respectively the horizontal jitter offset and vertical jitter offset , then .

[0075] Jitter offset The calculation method is as follows:

[0076] Assume that the angular resolution of the target image is , , the angular resolution is are the parameters of the telescope optical system, which are the known parameters of the telescope. middle, Indicates the angular resolution of the target image in the horizontal direction (corresponding to the orientation of the horizontal coordinate system). Indicates the angular resolution of the target image in the vertical direction (corresponding to the pitch of the horizontal coordinate system), and The units are Jitter offset The calculation formula is:

[0077] ,

[0078] .

[0079] S6: Calculate the cropping area based on the jitter offset , based on the cropping area Crop the target image and output the cropped target image.

[0080] The target image is cropped according to the jitter offset calculated in step S5. The cropping method is as follows:

[0081] Set the width of the target image captured by the camera to , the height is , define the cropping region as ,in, Indicates the coordinates of the upper left corner pixel of the target image, , , Indicates the horizontal cropping ratio, Indicates the vertical cropping ratio, The value range meets 、 .

[0082] The coordinates of the upper left pixel of the target image The calculation method is:

[0083] ,

[0084] ,

[0085] in, , .

[0086] Get the clipping area based on calculation The target image is cropped to obtain the cropped image and output it to the display device. The calculation method has been used for coordinates The value of is limited to ensure that there is no clipping out of bounds.

[0087] After the cropped image completes the above cropping process frame by frame and is output to the display device frame by frame, a continuous, stabilized target observation result can be obtained.

[0088] In order to implement the above-mentioned telescope electronic image stabilization method based on the theoretical guidance trajectory, the embodiment of the present invention further provides a method as follows: Figure 5 The image stabilization system shown includes: a control and data processing terminal, a telescope, the control and data processing terminal includes an image processing module, a data sharing area and a telescope control module, and the telescope includes a camera and a servo control system.

[0089] The data sharing area stores the telescope's guidance trajectory required for a specific observation plan. This trajectory is a sequence of guidance information, each of which includes the guidance time, guidance azimuth, and guidance elevation. The data sharing area also stores the encoder azimuth and elevation values ​​received from the servo control system at each guidance time point, known as encoder pointing data. Both the telescope control module and the image processing module can access the data sharing area and retrieve and apply the data.

[0090] The telescope control module accesses the data sharing area to obtain the guidance trajectory, and then obtains the guidance information. According to the guidance information sequence, the module sends the guidance azimuth value and the guidance pitch value to the servo control system at each guidance time point.

[0091] At each guidance time point, the servo control system receives the guidance azimuth and guidance pitch values ​​sent by the telescope control module, and controls the telescope's rotation to point to the guidance azimuth and guidance pitch values. Simultaneously, the servo control module transmits the telescope's encoder azimuth and encoder pitch values ​​corresponding to that guidance time point to the data sharing area. Each time the telescope is guided, the servo control system guides the telescope according to its internal logic. However, in strong winds, the telescope may not be able to rotate to the specified direction as controlled by the servo control system, resulting in encoder azimuth and encoder pitch errors, which in turn cause target jitter.

[0092] The camera is connected to the image processing module. In the process of guiding the telescope to rotate, the camera captures images of the target at a specified frequency, and sends the captured target images and the corresponding exposure center time of the target images to the image processing module frame by frame, triggering the image stabilization process.

[0093] The image processing module receives the target image from the camera, along with the corresponding exposure center time. Based on this information, it calculates the target's jitter offset and the corresponding cropping area (this calculation avoids over-cropping), completes the cropping process, and ultimately outputs the cropped image to the display area.

[0094] To verify the method of the present invention, the following experiments were performed:

[0095] The present invention uses the jitter amplitude of the target mass center with obvious characteristics to evaluate the image stabilization effect, and sets the relevant parameter values ​​as: , , , , the camera frame rate is 10Hz. Figure 6 As shown in the stabilization effect diagram, the maximum absolute value of the horizontal and vertical jitter amplitude of the target in the image is less than , where the horizontal jitter amplitude RMS is , the vertical jitter amplitude RMS is , the image stabilization effect is significant.

[0096] Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0097] The above specific embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A telescope electronic image stabilization method based on a theoretical guided trajectory, wherein the telescope includes a camera that acquires a target image of an observed target, characterized in that: include: S1: Obtain the exposure center time corresponding to the target image ; S2: Get the encoder pointing data of the last two frames; S3: Obtain the telescope guidance direction corresponding to the two most recent frames of encoder direction data in step S2; S4: Calculate telescope pointing error : ； in, ； ； Indicates the receiving time corresponding to the encoder pointing data, A sequence number indicating the time of receipt, Display and reception time The corresponding encoder points to the data, Display and reception time The corresponding encoder points to the data, Display and reception time The corresponding telescope guide pointing; S5: Calculate the jitter offset of the target image : ： ； ； in, Indicates the horizontal jitter offset, Indicates the vertical jitter offset, Indicates encoder pointing data The encoder pitch value, Indicates the horizontal angular resolution of the target image, Indicates the angular resolution of the target image in the vertical direction, pointing error Including bearing error and pitch error ; S6: Calculating a cropping area according to the jitter offset, cropping the target image according to the cropping area, and outputting the cropped target image.

2. The telescope electronic image stabilization method based on theoretical guidance trajectory according to claim 1, characterized in that: In step S2, the encoder pointing data of the last two frames are and ,in, Indicates the receiving time corresponding to the data pointed to by the receiving encoder The serial number, .

3. The telescope electronic image stabilization method based on theoretical guidance trajectory according to claim 1, characterized in that: In step S3, the telescope guidance points are: Corresponding telescope guidance pointing , and receiving time Corresponding telescope guidance pointing .

4. The telescope electronic image stabilization method based on theoretical guidance trajectory according to claim 1, characterized in that: The calculation method of the clipping area is: ； in, Indicates the coordinates of the upper left corner pixel of the target image, , , Indicates the horizontal cropping ratio, Indicates the vertical cropping ratio, Indicates the width of the target image, Indicates the height of the target image.

5. The telescope electronic image stabilization method based on theoretical guidance trajectory according to claim 4, characterized in that: The coordinates of the upper left corner pixel of the target image The calculation method is: ； ； in, , .

6. The telescope electronic image stabilization method based on theoretical guidance trajectory according to claim 4, characterized in that: The coordinates of the upper left corner pixel of the target image The value range satisfies 、 .

7. A telescope electronic image stabilization system based on a theoretical guidance trajectory, for implementing the telescope electronic image stabilization method based on a theoretical guidance trajectory according to any one of claims 1 to 6, characterized in that: Includes image processing module, telescope control module, camera and servo control system; The telescope control module obtains guidance information and sends the guidance information to the servo control system; the servo control system controls the rotation of the telescope based on the guidance information and obtains the pointing data of the telescope encoder; the camera captures the target image and sends it to the image processing module; after receiving the target image sent by the camera, the image processing module calculates the jitter offset of the target and the corresponding cropping area, and crops the target image; the image processing module outputs the cropped image to the display area.

8. The telescope electronic image stabilization system based on theoretical guidance trajectory according to claim 7, characterized in that: It also includes a data sharing area, which stores the guidance information and encoder pointing data.

Images