An aerial full-link cross-domain cooperative image motion compensation system and method

Abstract

The present application relates to aerospace photoelectric imaging technology field, and specifically provides a kind of aviation full-link cross-domain collaborative image motion compensation system, comprising: cross-domain collaborative perception unit, for real-time acquisition of the multi-source heterogeneous motion information of carrier aircraft and the charge state information of imaging detector;Image motion solving and decision unit is connected with the cross-domain collaborative perception unit, for receiving the multi-source heterogeneous motion information and the charge state information, based on pre-constructed full-link multi-physical field coupling model real-time solving dynamic image motion vector field, and accordingly generate decision parameter;Chip compensation execution unit is connected with the image motion solving and decision unit, for receiving the decision parameter, accordingly generate driving signal and output to image motion composite control mechanism;The image motion composite control mechanism includes imaging detector, optical compensation mechanism and mechanical compensation mechanism.The present application solves the problem that prior art is not comprehensive to complex image motion perception, and compensation is not accurate.

Description

Technical Field

[0001] This invention belongs to the field of aerospace optoelectronic imaging technology, and particularly relates to an image shift compensation system and method for cross-domain collaborative aerospace end-to-end. Background Technology

[0002] High-altitude, long-range aerial cameras are crucial equipment for acquiring strategic intelligence. To enhance survivability, modern reconnaissance aircraft often employ high-speed, low-altitude, and highly maneuverable flight modes, coupled with complex imaging attitudes such as slant-looking and sweep-scanning. This results in severe and complex high-dynamic image shifts on the imaging target surface. These image shifts not only have a forward component but also complex multidimensional components due to attitude changes. Furthermore, the image shift velocity and direction vary at different pixels within the same image plane (i.e., allometric image shift), leading to image blurring and reduced resolution, severely limiting imaging quality and information extraction capabilities.

[0003] Traditional image shift compensation often employs discrete solutions: using an inertial stabilization platform for overall mechanical compensation, or post-processing digital image processing for software correction. Mechanical compensation systems are slow to respond, have limited bandwidth, struggle to handle high-frequency jitter and complex multidimensional image shifts, and are also bulky and heavy. Digital processing methods, on the other hand, are post-exposure remedies and cannot fundamentally eliminate image shift blur during exposure. Furthermore, existing technologies often focus on a single image shift source (such as forward velocity), lacking a unified modeling and perception of the entire coupling effect of "aircraft motion - platform vibration - detector charge transfer," resulting in inaccurate compensation models and large compensation residuals in high-speed, high-dynamic scenarios. Summary of the Invention

[0004] In view of this, the present invention aims to provide an image motion compensation system and method for cross-domain collaboration across the entire aviation link, so as to solve the problems of incomplete perception and inaccurate compensation of complex image motion in the prior art.

[0005] To achieve the above objectives, the technical solution created by this invention is implemented as follows: In a first aspect, the present invention provides an image shift compensation system for cross-domain collaboration across the entire aviation link, comprising: The cross-domain collaborative sensing unit is used to acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information of the imaging detector in real time. Image shift calculation and decision unit, connected to the cross-domain collaborative sensing unit, is used to receive the multi-source heterogeneous motion information and the charge state information, calculate the dynamic image shift vector field in real time based on the pre-built full-link multi-physics coupling model, and generate decision parameters accordingly; An on-chip compensation execution unit, connected to the image shift calculation and decision unit, is used to receive the decision parameters, generate a drive signal accordingly, and output it to the image shift composite control mechanism; the image shift composite control mechanism includes an imaging detector, an optical compensation mechanism, and a mechanical compensation mechanism.

[0006] Furthermore, the cross-domain collaborative sensing unit includes an inertial measurement module, a satellite positioning module, and a charge state monitoring module; wherein: the charge state monitoring module is connected to the imaging detector and is used to monitor the position, transfer efficiency, and potential well state information of the charge packet of the imaging detector in real time.

[0007] Furthermore, the image shift calculation and decision unit includes: The multi-source information fusion module, connected to the inertial measurement module and the satellite positioning module, is used to filter and fuse multi-source motion information to obtain the motion state of the carrier. The end-to-end image shift modeling module is connected to the multi-source information fusion module and the charge state monitoring module. It is used to calculate the dynamically changing image shift vector field on the imaging detector by using the end-to-end dynamic tensor coupling model, combined with the pre-stored optical system parameters, the motion state of the carrier, and the charge state information. The intelligent decision-making module, connected to the end-to-end image shift modeling module, is used to calculate the optimal compensation strategy based on the image shift vector field through a reverse derivation engine, and generate the decision parameters accordingly.

[0008] Furthermore, the driving signals include detector driving signals, FSM driving signals, and platform servo driving signals; the on-chip compensation execution unit includes: The timing reverse derivation and control module is connected to the intelligent decision module and is used to convert the charge transfer rate, line transfer period and electronic shutter parameters in the decision parameters into corresponding driving clock phase control words and timing control logic, respectively. A programmable drive timing generator, connected to the timing inverse derivation and control module, is used to generate corresponding drive timing pulses according to the drive clock phase control word and timing control logic; A multi-dimensional collaborative driving module, connected to the programmable driving timing generator, is used to receive the driving timing pulses, amplify and synchronize the power, and generate the detector driving signal to drive the imaging detector. The multi-dimensional collaborative drive module is also connected to the intelligent decision module and is used to generate corresponding FSM drive signals and platform servo drive signals based on the optical compensation component and mechanical compensation component in the decision parameters, respectively, to drive the optical compensation mechanism and the mechanical compensation mechanism.

[0009] Furthermore, it also includes a closed-loop feedback adjustment loop, which is connected to the cross-domain collaborative sensing unit and the on-chip compensation execution unit respectively; the closed-loop feedback adjustment loop is used to receive the residual error feedback signal from the cross-domain collaborative sensing unit, and to fine-tune the driving amount allocated by the multi-dimensional collaborative driving module to the image shift composite control mechanism according to the residual error feedback signal.

[0010] Furthermore, it also includes an image quality assessment module for evaluating the modulation transfer function (MTF) value of the image, and the residual error feedback signal also includes the modulation transfer function (MTF) value of the image.

[0011] Furthermore, the photosensitive array of the imaging detector is divided into multiple independent control groups in the vertical direction, and each group is equipped with an independent charge transfer rate control electrode and a corresponding readout circuit.

[0012] Secondly, this invention provides an image shift compensation method for cross-domain collaboration across the entire aviation link, comprising the following steps: S10. Acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information of the imaging detector; S20. Filter and fuse the multi-source heterogeneous motion information to obtain the carrier motion state; S30. Based on the pre-constructed full-link dynamic tensor coupling model, combined with the carrier motion state, optical system parameters and charge state information, the image shift vector field on the imaging detector is calculated, and decision parameters are generated accordingly. S40. Generate driving signals according to the decision parameters. The driving signals include detector driving signals output to the imaging detector, FSM driving signals output to the optical compensation mechanism, and platform servo driving signals output to the mechanical compensation mechanism.

[0013] Furthermore, the multi-source heterogeneous motion information includes: aircraft angular velocity and linear acceleration information, aircraft position and velocity information; the charge state information includes: the position, transfer efficiency and potential well depth state information of the imaging detector charge packet.

[0014] Furthermore, step S40 includes: S41. Convert the charge transfer rate, line transfer period and electronic shutter parameters in the decision parameters into the corresponding driving clock phase control word and timing control logic, respectively. S42. Generate corresponding driving timing pulses according to the driving clock phase control word and timing control logic; S43. Amplify and synchronize the driving timing pulse to generate the detector driving signal; convert the optical compensation component in the decision parameters into the FSM driving signal; convert the mechanical compensation component in the decision parameters into the platform servo driving signal.

[0015] Furthermore, a residual error feedback signal is received through a closed-loop feedback adjustment circuit, and the drive signal is fine-tuned based on the residual error feedback signal.

[0016] Compared with the prior art, the present invention can achieve the following beneficial effects: (1) The present invention realizes accurate, real-time, on-chip compensation for complex image shifts in high dynamic range of aviation, which effectively improves the imaging quality.

[0017] (2) The present invention divides the photosensitive array of the imaging detector into multiple independent groups in the vertical direction. Each group is equipped with an independent charge transfer speed control electrode. The on-chip compensation execution unit applies a vertical transfer clock with independently adjustable frequency and phase to each group, so that the charge packets of each group are transferred longitudinally at different speeds to match the image movement speed of their respective fields of view. This achieves precise matching of the image movement field at different speeds, and the compensation accuracy can reach the sub-pixel level. It effectively eliminates the local blurring and geometric distortion of the image caused by the difference in image movement speed.

[0018] (3) The present invention creates a unified scheduling of the two-axis stabilization platform, the fast reflector and the imaging detector through the on-chip compensation execution unit, so that the three can perform mechanical coarse stabilization, optical fine adjustment and electronic fine matching respectively. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 A schematic diagram of the structure of the image shift compensation system for cross-domain collaboration across the entire aviation link as described in the embodiment of the present invention; Figure 2 A schematic diagram of the cross-domain collaborative sensing unit described in the embodiment of the present invention; Figure 3 A schematic diagram of the image shifting calculation and decision-making unit as described in the embodiment of the present invention; Figure 4 A schematic diagram of the structure of the compensation execution unit described in the embodiment of the present invention; Figure 5 A schematic diagram of the electrode structure of the imaging detector described in an embodiment of the present invention; Figure 6A schematic diagram of the multiphase electrode configuration of the imaging detector described in an embodiment of the present invention; Figure 7 A schematic diagram of the high dynamic multidimensional image shift described in the embodiment of the present invention; Figure 8 This is a flowchart illustrating the image shift compensation method for cross-domain collaboration across the entire aviation link as described in an embodiment of the present invention.

[0020] Explanation of reference numerals in the attached figures: 100. Cross-domain collaborative sensing unit; 101. Inertial measurement module; 102. Satellite positioning module; 103. Charge state monitoring module; 200. Image motion calculation and decision-making unit; 201. Multi-source information fusion module; 202. End-to-end image motion modeling module; 203. Intelligent decision-making module; 300. On-chip compensation execution unit; 301. Timing inverse derivation and control module; 302. Programmable drive timing generator; 303. Multi-dimensional collaborative drive module; 500. Imaging detector; 501. Photosensitive array; 502. CCD register. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and do not constitute a limitation thereof. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the invention are not shown or described in the specification. This is to avoid overwhelming the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined to form various implementations. Furthermore, the order of the steps or actions in the method description can be changed or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various orders in the specification and drawings are merely for the clear description of a particular embodiment and do not imply a mandatory order, unless otherwise stated that a particular order must be followed.

[0023] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. The term "based on" should be understood as "at least partially based on." Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more, and the term "including" means "including but not limited to." Various embodiments of the present invention may exist in the form of a range; it should be understood that the description in the form of a range is merely for convenience and brevity and should not be construed as a rigid limitation on the scope of the invention; therefore, it should be considered that the range description has specifically disclosed all possible sub-ranges and single numerical values ​​within that range; for example, it should be considered that the range description from 1 to 6 has specifically disclosed sub-ranges, such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and single numbers within the range, such as 1, 2, 3, 4, 5, and 6, regardless of the range. Furthermore, whenever a numerical range is referred to herein, it means including any referenced number (fraction or integer) within the range referred to.

[0024] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0025] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] Example 1 like Figure 1As shown, this embodiment of the invention provides an image shift compensation system for cross-domain collaboration across the entire aviation link (hereinafter referred to as the system), including: a cross-domain collaborative sensing unit 100, an image shift calculation and decision unit 200, and an on-chip compensation execution unit 300.

[0027] The cross-domain collaborative sensing unit 100 is responsible for acquiring information from different physical domains in real time. Specifically, the cross-domain collaborative sensing unit 100 is used to acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information inside the imaging detector in real time. Please also refer to... Figure 2 The cross-domain collaborative sensing unit 100 includes an inertial measurement module 101, a satellite positioning module 102, and a charge state monitoring module 103. The inertial measurement module 101 acquires the angular velocity and linear acceleration information of the carrier aircraft; the satellite positioning module 102 acquires the position and velocity information of the carrier aircraft; the charge state monitoring module 103 is connected to the imaging detector or directly integrated inside the imaging detector, and is used to monitor the position, transfer efficiency, and potential well state information of the charge packet of the imaging detector in real time. By monitoring the reference charge packet or electrode signals of a special structure, it can deduce the microscopic states such as charge transfer velocity, direction, and potential well depth.

[0028] The cross-domain collaborative sensing unit 100 provides a full-link data foundation for accurate modeling through cross-domain (motion domain and electronic domain) collaborative sensing.

[0029] Image shift calculation and decision unit 200 is connected to cross-domain collaborative sensing unit 100. Image shift calculation and decision unit 200 is used to receive the aforementioned multi-source heterogeneous motion information and charge state information, calculate the dynamic image shift vector field in real time based on a pre-built end-to-end multiphysics coupling model, and generate decision parameters based on the calculation results. In specific implementations, image shift calculation and decision unit 200 is typically implemented using a high-performance system-on-a-chip (SoC) or FPGA. In a specific embodiment, image shift calculation and decision unit 200 can be implemented using the Xilinx Zynq UltraScale+ MPSoC platform, where the ARM processor core runs the information fusion and intelligent decision algorithm, and the FPGA logic implements the real-time model calculation.

[0030] Please refer to the following at the same time Figure 3 The image motion calculation and decision-making unit 200 includes a multi-source information fusion module 201, a full-link image motion modeling module 202, and an intelligent decision-making module 203. Among them: The multi-source information fusion module 201 is connected to the inertial measurement module 101 and the satellite positioning module 102. The multi-source information fusion module 201 is used to filter and fuse multi-source motion information to obtain a high-precision carrier motion state.

[0031] The end-to-end image motion modeling module 202 is connected to the multi-source information fusion module 201 and the charge state monitoring module 103. The end-to-end image motion modeling module 202 has a built-in pre-built end-to-end dynamic tensor coupling model, which is used to calculate the dynamically changing image motion vector field with spatial differences on the imaging detector by combining the pre-stored optical system parameters, carrier motion state, and charge state information.

[0032] The pre-constructed full-link dynamic tensor coupling model in this invention breaks through the limitations of the traditional geometric optics model. It couples the mechanical vibration spectrum, the attenuation of the optical modulation transfer function (MTF), and the charge diffusion effect for calculation, thereby solving a dynamic image shift vector field that is more in line with physical reality.

[0033] Specifically, this end-to-end dynamic tensor coupling model describes the entire physical process from aircraft motion to focal plane image shift, encompassing physical processes at three scales: macroscopic aircraft motion, mesoscopic mechanical vibration, and microscopic detector charge transfer and diffusion. Macroscopic aircraft motion includes the aircraft's three-axis angular and three-axis motions, measured in real-time by the airborne inertial navigation system and projected onto the detector's focal plane after coordinate transformation, forming low-frequency, large-amplitude image shift components in each field of view. Mesoscopic mechanical vibration includes broadband vibrations generated by the aircraft skin and optical system structure under aerodynamic loads and engine excitation, picked up in real-time by strapdown-mounted accelerometers or angular velocity sensors on the optical reference. After modal decomposition, the contribution of each resonant mode to the focal plane image shift is extracted, while also considering the attenuation of the optical system's modulation transfer function with increasing vibration frequency and amplitude. Microscopic detector charge transfer and diffusion includes the stepwise longitudinal transfer of photogenerated charge packets under the time-sequential drive of each zone control electrode, as well as the accompanying charge transfer losses during the transfer process.

[0034] Significant timescale differences exist among the physical processes at the three scales mentioned above: the rate of change of macroscopic aircraft motion ranges from 10Hz to 50Hz, the response rate of mesoscopic mechanical vibration ranges from 100Hz to 1kHz, and the response rate of charge transfer in the imaging detector depends on the driving clock frequency (1MHz to 100MHz). To address this cross-scale correlation, a relaxation factor is introduced, defined as the ratio of the response time constant at each scale to the system's reference time constant. This relaxation factor is used to normalize the timescale of the physical processes at each scale. When a physical process at a certain scale changes, the relaxation factor controls the rate of change of the compensation amount at that scale, ensuring it smoothly follows the target value, thereby correcting the superposition error of the compensation amount caused by the differences in response time across scales. The relaxation factors at each scale are obtained through offline calibration or online adaptive identification and are dynamically adjusted according to the system's operating state.

[0035] The physical processes at the three scales mentioned above, after being corrected by relaxation factors, are uniformly mapped to the detector focal plane coordinate system. Image velocity vectors are calculated at each spatial location on the focal plane, thus forming an instantaneous image velocity vector field with spatial distribution characteristics. This instantaneous image velocity vector field is a real-time output snapshot of the dynamic image velocity vector field within the current control cycle; that is, the value of the dynamic image velocity vector field at a specific moment. Together, they constitute a complete link of "full-time-series modeling - real-time snapshot driving." This dynamic image velocity vector field is the output result of the full-link dynamic tensor coupling model.

[0036] The intelligent decision-making module 203 is connected to the full-link image shift modeling module 202 and is used to calculate the optimal compensation strategy based on the image shift vector field through the built-in reverse derivation engine. The optimal compensation strategy includes the precise transfer rate, line scan frequency, electronic shutter parameters, etc. required by each part of the charge transfer unit. Decision parameters can be formed based on the optimal compensation strategy.

[0037] The workflow of the image motion calculation and decision-making unit 200 is as follows: The multi-source information fusion module 201 uses algorithms such as federated Kalman filtering to fuse the data output by the inertial measurement module 101 and the satellite positioning module 102, and outputs high-precision, high-update-rate carrier six-degree-of-freedom motion state data. The end-to-end image motion modeling module 202 receives this high-precision, high-update-rate carrier motion state, and combines it with the attitude angle from the flight control system, the focal length and distortion parameters from the optical system, and the information fed back from the imaging detector charge state monitoring circuit 103, and runs its internally pre-built end-to-end dynamic tensor coupling model.

[0038] The intelligent decision-making module 203 receives the dynamic image shift vector field. At its core is a "reverse derivation engine." This engine accesses a pre-established database (or fitted polynomial model) of "image shift velocity - charge transfer velocity - driving phase" mapping established through simulation and experimentation. For each region in the current image shift vector field (corresponding to a group of imaging detectors), using the image shift velocity of that region as the target, it quickly looks up the table or calculates to deduce the optimal target charge transfer velocity V_charge, along with associated parameters such as line frequency, electronic shutter slit speed and width, forming decision parameters. These decision parameters include charge transfer rate, line transfer period, electronic shutter parameters, optical compensation components, and mechanical compensation components.

[0039] The on-chip compensation execution unit 300 is connected to the image shift calculation and decision unit 200. It receives decision parameters, generates driving signals based on these parameters, and outputs them to the image shift composite control mechanism. The image shift composite control mechanism includes an imaging detector, an optical compensation mechanism (e.g., a fast reflector), and a mechanical compensation mechanism (e.g., a two-axis stabilization platform). The on-chip compensation execution unit 300 drives the imaging detector to match the charge transfer rate with the image shift velocity, and simultaneously coordinates the control of the optical and mechanical compensation mechanisms to achieve coordinated image shift compensation across the optical, mechanical, and electronic domains.

[0040] Please refer to the following at the same time Figure 4 The on-chip compensation execution unit 300 includes a timing inverse derivation and control module 301, a programmable drive timing generator 302, and a multi-dimensional collaborative drive module 303. Among them: The timing inverse derivation and control module 301 is used to convert the target charge transfer rate V_charge in the decision parameters into a specific driving clock phase control word Φ. Specifically, based on parameters such as the pixel size, electrode pitch, and TDI level of the imaging detector, the timing inverse derivation and control module 301 inversely derives the operating frequency and phase configuration required for the driving clock, and its output is a set of digitized frequency / phase control parameters. It should be noted that the timing inverse derivation and control module 301 only handles the charge transfer requirements of the imaging detector and does not involve the processing of optical or mechanical compensation parameters.

[0041] The programmable drive timing generator 302 is used to generate row and field drive timing pulses with a gradual rate of change, low jitter, and a fixed phase difference between rows and columns, based on the drive clock phase control word Φ. The programmable drive timing generator 302 only generates the timing pulses required by the imaging detector and does not generate optical or mechanical drive signals. In specific implementations, the programmable drive timing generator 302 can be implemented using a dedicated timing chip with a high-resolution digital delay chain (such as Texas Instruments' LMK series) or the fine timing resources of a high-end FPGA (such as Xilinx Kintex UltraScale).

[0042] The input of the multi-dimensional collaborative drive module 303 is connected to the programmable drive timing generator 302 and the intelligent decision-making module 203 in the image shift calculation and decision-making unit 200. The input of the programmable drive timing generator 302 includes three sources: one is the detector drive timing pulse output by the programmable drive timing generator 302; the second is the optical compensation component in the decision parameters, that is, the desired deflection angle of the fast reflector; and the third is the mechanical compensation component in the decision parameters, that is, the desired frame angle or angular velocity of the two-axis stabilizing platform.

[0043] The multi-dimensional collaborative drive module 303 further includes three parallel sub-drive sections: a detector drive circuit, an optical compensation driver, and a mechanical compensation driver. Each of the three sub-drive sections receives different input signals, and after format conversion and power adaptation, generates a detector drive signal, an FSM drive signal, and a platform servo drive signal. These signals drive the imaging detector 500, the fast reflector, and the two-axis stabilized platform, respectively, achieving three-level collaborative compensation: electronic fine-tuning, optical fine-adjustment, and mechanical coarse stabilization.

[0044] The detector drive circuit amplifies and synchronously distributes the drive timing pulses generated by the programmable drive timing generator 302 to generate a detector drive signal. This signal is then applied to the respective independent groups (G1, G2, …, Gn) of the imaging detector, controlling the charge transfer rate of each group to precisely track the image migration velocity of its respective region, thus achieving electronic fine compensation in the charge domain. The optical compensation driver receives the optical compensation component from the decision parameters, converts it into an analog voltage signal or a PWM duty cycle signal (FSM drive signal), and outputs it to the piezoelectric ceramic or voice coil motor driver of the Fast Steering Mirror (FSM). This controls the mirror to generate corresponding angular displacements in the pitch and yaw degrees of freedom, achieving optical coarse compensation for the high-frequency component of the forward image migration. The mechanical compensation driver receives the mechanical compensation component from the decision parameters, converts it into a PWM power drive signal or a speed / position command (platform servo drive signal), and outputs it to the frame motor of the two-axis stabilized platform. This drives the two-axis stabilized platform to perform servo adjustment, achieving mechanical compensation for low-frequency swaying of the carrier attitude.

[0045] Preferably, the system further includes a closed-loop feedback adjustment loop, which is connected to the cross-domain collaborative sensing unit 100 and the on-chip compensation execution unit 300, respectively. More specifically, it is connected to the multi-dimensional collaborative driving module 303 in the on-chip compensation execution unit 300. The closed-loop feedback adjustment loop is used to receive residual error feedback signals, which include direct error signals from the charge state monitoring module in the cross-domain collaborative sensing unit. Based on the residual error feedback signals, the loop finely adjusts the driving quantities allocated by the multi-dimensional collaborative driving module to each independent control group of the two-axis stabilization platform, the fast reflector, and the imaging detector frame by frame until the residual error converges to within a preset threshold, thereby achieving optimal coordination of optical, mechanical, and electronic three-layer compensation.

[0046] In practical implementation, the system also includes an image quality assessment module, which evaluates the modulation transfer function (MTF) value of the image. This module performs spatial frequency response analysis on the acquired current frame image, calculates the MTF value, and uses it as a quantitative indicator to evaluate the image shift compensation effect of the current frame. At this time, the residual error feedback signal also includes the image's MTF value.

[0047] The direct error signal provided by the charge state monitoring module is a process feedback, which can correct charge transfer deviations in real time during exposure integration. The MTF value provided by the image quality assessment module is a result feedback, used to evaluate the compensation effect of the previous frame and guide the optimization of decision parameters for the next frame. The two feedback mechanisms complement each other, forming a two-layer closed-loop control of intra-frame real-time correction + inter-frame iterative optimization.

[0048] This invention achieves closed-loop coordination of optical coarse compensation, mechanical intermediate compensation, and electronic fine compensation by synchronously controlling the optical compensation mechanism (fast reflector), the mechanical compensation mechanism (two-axis stabilization platform), and the imaging detector. This enables the cross-domain collaborative sensing unit 100, the image shift calculation and decision-making unit 200, and the on-chip compensation execution unit 300 to form an on-chip intelligent closed loop of "perception-decision-execution".

[0049] In this invention, data and control commands are exchanged between units through high-speed serial interfaces (such as LVDS, JESD204B) and real-time networks (such as Ethernet AVB) to ensure low latency and high synchronization accuracy.

[0050] In one specific embodiment, the imaging detector employs a time-delay integration (TDI) CCD array or a CMOS image sensor with equivalent functionality, and supports independent control of charge transfer rates for each group, thus providing a hardware foundation for on-chip electronic precision compensation for high-dynamic-range image shifts in aviation. In one specific embodiment, such as... Figure 5 As shown, the photosensitive array 501 of the imaging detector 500 is divided into n independent groups (G1, G2, …, Gn) in the vertical direction. Each group contains a vertical CCD register 502 composed of multiple MOS capacitors. Each register 502 has an independent charge transfer rate control electrode (or circuit) above it and a readout circuit connected below it. Driven by multiphase clock pulses applied to each charge transfer rate control electrode, the charge packets are transferred step-by-step along the vertical direction through periodic changes in the potential well depth. The groups are isolated from each other by isolation channels or barrier regions to ensure that the charge packets within each zone do not interfere with each other during the vertical transfer process.

[0051] Figure 5The diagram also illustrates the correspondence between the detector target pixel arrangement and the charge transfer direction. Each rectangular square represents a pixel, and each pixel contains a fine grid to characterize the photosensitive region division and charge potential well distribution within the pixel. A double arrow indicating the "row transfer direction" is marked above the pixel array, illustrating the path of the charge packet's horizontal (row direction) transfer row by row, representing the horizontal transfer process of the incident photogenerated charge within the vertical CCD register. A double arrow indicating the "column transfer direction" is marked on the side of the pixel array, illustrating the path of the charge packet's vertical (column direction) movement to the readout circuit after each row transfer. The row and column transfer directions are orthogonal to each other, together forming the complete transfer path of the detector charge packet from the photosensitive region to the readout region.

[0052] In the CCD register 502, which is divided into n independent groups in the vertical direction, each group (G1 to Gn) corresponds to an independent vertical transfer channel. The independent charge transfer speed control electrode above each channel receives different timing pulses from the on-chip compensation execution unit 300. Figure 5 Based on the grouping structure shown, each group can further include multiple horizontal transfer registers in the vertical direction to realize row-by-row transfer of charge packets in the horizontal direction (row direction). The coordinated operation of the row transfer direction and the column transfer direction ensures that the image velocity vector on the focal plane can be accurately tracked during the integration and transfer of the charge packet in the vertical CCD register, thus completing the synchronous tracking of photogenerated charge and target image point.

[0053] When the on-chip compensation execution unit 300 applies different driving timing pulses, the charge packets in each group will be transferred downward at different average speeds (V1≠V2≠…≠Vn), thereby compensating for the image shift in the corresponding field of view of that group and achieving accurate matching of the entire allometric image shift field.

[0054] Based on this, please also refer to Figure 6 , Figure 6 This is a schematic diagram of the multiphase transfer electrode configuration of the imaging detector of the present invention. The vertical transfer electrodes in each group adopt a multiphase structure (a three-phase or six-phase structure in this embodiment). Voltage is applied sequentially by timing pulses (CI1, CI2, CI3, and extended multiphase beats T1~T6) on each phase electrode, causing the photogenerated charge packets to be transferred step by step from one electrode to the next electrode along the vertical direction, thereby realizing the directional delivery from the photosensitive area to the readout circuit 502 below. The photosensitive area is divided into multiple independent groups in the vertical direction, and the charge transfer speed of each group can be independently controlled, thereby directly compensating for the corresponding allometric image shift in that area.

[0055] The charge packets in each vertical CCD register are transferred step-by-step along the vertical direction under the drive of clock pulses. Specifically, the driving timing pulses applied to each phase electrode are as follows: Figure 7As shown, the driving pulse timing includes the vertical transfer clock sequences CI1, CI2, and CI3, a multi-phase clock consisting of multi-phase clocks T1 to T6, horizontal readout clocks H1 and H2, and a reset clock CRX. The frequency and phase of the vertical transfer clock determine the transfer speed of the charge packet in the vertical CCD register. The horizontal readout clock drives the charge packet into the horizontal shift register and completes the row readout. CRX is used to reset the output node. The period jitter of each timing pulse is better than 10 ps RMS to ensure precise controllability of the charge packet transfer time between registers.

[0056] After the on-chip compensation execution unit 300 receives the image motion calculation results from the front end, it adjusts the vertical transfer clock frequency and phase configuration of the corresponding group (G1~Gn) according to the image motion velocity distribution of each field of view, so that the longitudinal transfer velocity of each group of charge packets is matched with their respective image motion velocity, and finally achieves accurate compensation for the entire allometric image motion field.

[0057] After completing the vertical transfer of a full frame, each group of charge packets is read out line by line and pixel by pixel via the horizontal shift register (driven by H1 / H2) and the output node (controlled by CRX for reset), forming complete image data containing allometric image shift compensation information.

[0058] Example 2 like Figure 8 As shown, this invention provides an image shift compensation method for cross-domain collaboration across the entire aviation link, comprising the following steps: S10. Acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information of the imaging detector; The aforementioned multi-source heterogeneous motion information includes: real-time acquisition of the aircraft's angular velocity and linear acceleration information via an inertial measurement module, and real-time acquisition of the aircraft's position and velocity information via a satellite positioning module. Charge state information includes: real-time monitoring of the real-time position, transfer efficiency, and potential well state information of the charge packets within the imaging detector via a charge state monitoring module. The aforementioned aircraft motion information and charge state information are synchronously acquired using timestamp alignment and transmitted in real-time to the image motion calculation and decision-making unit.

[0059] S20. Filter and fuse the multi-source heterogeneous motion information to obtain the carrier motion state; The motion calculation and decision-making unit performs time synchronization and coordinate alignment on the received angular velocity, linear acceleration, position and velocity information, and uses the federated Kalman filter algorithm to perform fusion filtering on the above information to output the carrier motion state with high precision and high update rate.

[0060] S30. Based on the pre-constructed full-link dynamic tensor coupling model, combined with the carrier motion state, optical system parameters and charge state information, the image shift vector field on the imaging detector is calculated, and decision parameters are generated accordingly. The image shift calculation and decision unit receives the carrier's motion state and simultaneously acquires current flight attitude angle information, optical system parameters (focal length and distortion parameters), and charge state information (imaging detector charge state feedback information). All this information is input into a pre-constructed end-to-end multiphysics coupling model to calculate the spatially differentiated image shift vector field that dynamically changes over time at each pixel position on the imaging detector. This pre-constructed end-to-end coupling model is the same as in Example 1 and will not be described again here.

[0061] The image shift calculation and decision unit further derives the decision parameters required to synchronize the charge transfer rate with the image shift velocity based on the dynamic image shift vector field using an inverse optimization algorithm. The dynamic image shift vector field is region-matched according to the independent group division of the imaging detector's photosensitive array. For each independent group's corresponding image shift vector field region, the image shift velocity vector for that region is extracted. The image shift calculation and decision unit integrates an inverse derivation engine. This engine uses a pre-established mapping database or fitting model of image shift velocity, charge transfer velocity, and driving phase, with the image shift velocity corresponding to each region in the dynamic image shift vector field as the target, to inversely calculate the target charge transfer rate, line frequency, and electronic shutter parameters for each independent group, forming the decision parameters. The decision parameters include charge transfer rate, line transfer period, electronic shutter parameters, optical compensation components, and mechanical compensation components.

[0062] S40. Generate driving signals according to the decision parameters. The driving signals include detector driving signals output to the imaging detector, FSM driving signals output to the optical compensation mechanism, and platform servo driving signals output to the mechanical compensation mechanism.

[0063] In this embodiment, the structure of the imaging detector is as follows: Figure 5 , Figure 6 As shown, it will not be described in detail here. The optical compensation mechanism is a fast-reflecting mirror, and the mechanical compensation mechanism is a two-axis stabilizing platform.

[0064] In specific implementation, step S40 includes: S41. Convert the charge transfer rate, line transfer period and electronic shutter parameters in the decision parameters into the corresponding driving clock phase control word and timing control logic, respectively. Specifically, based on parameters such as the pixel size, electrode pitch, and TDI level of the imaging detector, the required operating frequency and phase configuration of the driving clock are derived in reverse, and the output is a set of digitized frequency / phase control parameters. The structure of the imaging detector is as follows: Figure 5 , Figure 6 As shown, it will not be elaborated further here.

[0065] S42. Generate corresponding driving timing pulses according to the driving clock phase control word and timing control logic; The driving timing pulses, output from the detector driving circuit, can be applied to the charge transfer rate control circuits of each independent group of the imaging detector. This controls the charge transfer of each group at its corresponding target transfer rate, achieving sub-pixel-level precise matching between the charge transfer rate and the image migration velocity of the corresponding area, thus completing electronic precision compensation. More specifically, the driving timing pulses are as follows: Figure 7 As shown, it will not be elaborated further here.

[0066] S43. Amplify and synchronize the driving timing pulse to generate the detector driving signal; convert the optical compensation component in the decision parameters into the FSM driving signal; convert the mechanical compensation component in the decision parameters into the platform servo driving signal.

[0067] More specifically, the detector driving signal is applied to each independent group (G1, G2, …, Gn) corresponding to the imaging detector, and the charge of each group is controlled to transfer at different rates, so that it can accurately track the image movement velocity of its respective region, thereby achieving electronic precision compensation of the charge domain.

[0068] The FSM drive signal can be an analog voltage drive signal or a PWM drive signal. The FSM drive signal is output to the piezoelectric ceramic driver or voice coil motor driver of the fast reflector to control the fast reflector to generate corresponding angular displacements in the pitch and yaw degrees of freedom, thereby achieving optical coarse compensation for the high-frequency components of the forward image shift.

[0069] The platform servo drive signal can be a PWM power drive signal or a speed / position command. The platform servo drive signal is output to the frame motor of the two-axis stabilized platform to drive the two-axis stabilized platform to perform servo adjustment and realize mechanical compensation for low-frequency swaying of the carrier attitude.

[0070] Preferably, after step S40, the method further includes: receiving a residual error feedback signal through a closed-loop feedback adjustment circuit. This residual error feedback signal includes a direct error signal from the charge state monitoring module in the cross-domain collaborative sensing unit. Based on this residual error feedback signal, the driving quantities allocated by the multi-dimensional collaborative driving module to each independent control group of the two-axis stabilization platform, the fast reflector, and the imaging detector are fine-tuned frame-by-frame until the residual error converges to within a preset threshold, thereby achieving optimal coordination of optical, mechanical, and electronic three-layer compensation. The residual error feedback signal also includes the modulation transfer function (MTF) value of the image. The MTF value of the image is calculated by performing spatial frequency response analysis on the acquired current frame image.

[0071] The above steps S10 to S40 are executed sequentially according to the closed-loop logic of "perception-fusion-decision-execution". Among them, S10 is the cross-domain collaborative perception step, which is started after receiving the frame synchronization trigger signal from the imaging detector, performs a round of data acquisition, and provides multi-source heterogeneous motion information and charge state information for S20; S20 is the image shift calculation and intelligent decision-making step, and its output decision parameters serve as the input of S30; S30 is the on-chip compensation execution step, and its output drive signals drive the various control mechanisms of the image shift composite control mechanism (i.e., the imaging detector, the optical compensation mechanism, and the mechanical compensation mechanism); during the execution process, the drive signals are fine-tuned according to the residual error feedback signal to form closed-loop control until the current exposure window ends; after the exposure is completed, the imaging detector enters an idle state, waits for the next frame trigger signal, and begins the next round of the process cycle from S10 to S40.

[0072] This invention acquires multi-source heterogeneous motion information of the carrier aircraft and detector charge state information in real time through a cross-domain collaborative sensing unit. The image shift calculation and decision unit calculates the dynamic image shift vector field in real time based on the full-link multi-physics coupling model and generates the optimal decision parameters. Finally, the on-chip compensation execution unit drives the optical compensation mechanism (fast reflector), the mechanical compensation mechanism (two-axis stabilization platform), and the imaging detector (variable-speed charge transfer in each independent control component region) to achieve optical, mechanical, and electronic hierarchical collaborative compensation. This can effectively eliminate multidimensional image shift in highly dynamic and complex environments and significantly improve the imaging quality of airborne optoelectronic imaging payloads.

[0073] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0074] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An image shift compensation system for cross-domain collaboration across the entire aviation link, characterized in that, include: The cross-domain collaborative sensing unit is used to acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information of the imaging detector in real time. Image shift calculation and decision unit, connected to the cross-domain collaborative sensing unit, is used to receive the multi-source heterogeneous motion information and the charge state information, calculate the dynamic image shift vector field in real time based on the pre-built full-link multi-physics coupling model, and generate decision parameters accordingly; An on-chip compensation execution unit, connected to the image shift calculation and decision unit, is used to receive the decision parameters, generate a drive signal accordingly, and output it to the image shift composite control mechanism; the image shift composite control mechanism includes an imaging detector, an optical compensation mechanism, and a mechanical compensation mechanism.

2. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 1, characterized in that, The cross-domain collaborative sensing unit includes an inertial measurement module, a satellite positioning module, and a charge state monitoring module; wherein: the charge state monitoring module is connected to the imaging detector and is used to monitor the position, transfer efficiency, and potential well state information of the charge packet of the imaging detector in real time.

3. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 2, characterized in that, The image shift calculation and decision unit includes: The multi-source information fusion module, connected to the inertial measurement module and the satellite positioning module, is used to filter and fuse multi-source motion information to obtain the motion state of the carrier. The end-to-end image shift modeling module is connected to the multi-source information fusion module and the charge state monitoring module. It is used to calculate the image shift vector field on the imaging detector by using the end-to-end dynamic tensor coupling model, combined with the pre-stored optical system parameters, the motion state of the carrier, and the charge state information. The intelligent decision-making module, connected to the end-to-end image shift modeling module, is used to calculate the optimal compensation strategy based on the image shift vector field through a reverse derivation engine, and generate the decision parameters accordingly.

4. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 3, characterized in that, The driving signals include detector driving signals, FSM driving signals, and platform servo driving signals; The on-chip compensation execution unit includes: The timing reverse derivation and control module is connected to the intelligent decision module and is used to convert the charge transfer rate, line transfer period and electronic shutter parameters in the decision parameters into corresponding driving clock phase control words and timing control logic, respectively. A programmable drive timing generator, connected to the timing inverse derivation and control module, is used to generate corresponding drive timing pulses according to the drive clock phase control word and timing control logic; A multi-dimensional collaborative driving module, connected to the programmable driving timing generator, is used to receive the driving timing pulses, amplify and synchronize the power, and generate the detector driving signal to drive the imaging detector. The multi-dimensional collaborative drive module is also connected to the intelligent decision module and is used to generate corresponding FSM drive signals and platform servo drive signals based on the optical compensation component and mechanical compensation component in the decision parameters, respectively, to drive the optical compensation mechanism and the mechanical compensation mechanism.

5. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 1, characterized in that, It also includes a closed-loop feedback adjustment loop, which is connected to the cross-domain collaborative sensing unit and the on-chip compensation execution unit respectively; the closed-loop feedback adjustment loop is used to receive the residual error feedback signal from the cross-domain collaborative sensing unit, and to fine-tune the driving amount allocated by the multi-dimensional collaborative driving module to the image shift composite control mechanism according to the residual error feedback signal.

6. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 5, characterized in that, It also includes an image quality assessment module for evaluating the modulation transfer function (MTF) value of the image, and the residual error feedback signal also includes the modulation transfer function (MTF) value of the image.

7. The image shift compensation system for cross-domain collaboration across the entire aviation link according to claim 1, characterized in that, The photosensitive array of the imaging detector is divided into multiple independent control groups in the vertical direction, and each group is equipped with an independent charge transfer rate control electrode and a corresponding readout circuit.

8. A cross-domain collaborative image shift compensation method for the entire aviation link, characterized in that, Including the following steps: S10. Acquire multi-source heterogeneous motion information of the carrier aircraft and charge state information of the imaging detector; S20. Filter and fuse the multi-source heterogeneous motion information to obtain the carrier motion state; S30. Based on the pre-constructed full-link dynamic tensor coupling model, combined with the carrier motion state, optical system parameters and charge state information, the image shift vector field on the imaging detector is calculated, and decision parameters are generated accordingly. S40. Generate driving signals according to the decision parameters. The driving signals include detector driving signals output to the imaging detector, FSM driving signals output to the optical compensation mechanism, and platform servo driving signals output to the mechanical compensation mechanism.

9. The image shift compensation method for cross-domain collaboration across the entire aviation link according to claim 8, characterized in that, Step S40 includes: S41. Convert the charge transfer rate, line transfer period and electronic shutter parameters in the decision parameters into the corresponding driving clock phase control word and timing control logic, respectively. S42. Generate corresponding driving timing pulses according to the driving clock phase control word and timing control logic; S43. Amplify and synchronize the driving timing pulse to generate the detector driving signal; convert the optical compensation component in the decision parameters into the FSM driving signal; convert the mechanical compensation component in the decision parameters into the platform servo driving signal.

10. The image shift compensation method for cross-domain collaboration across the entire aviation link according to claim 8, characterized in that: The residual error feedback signal is received through a closed-loop feedback adjustment circuit, and the drive signal is fine-tuned according to the residual error feedback signal.

Images