Piston engine
By using a temperature-time dual-domain prediction model and deep learning algorithms, combined with GPS and attitude angle data for pixel-level correction, the problem of large correction errors due to inconsistent pixel response in traditional methods is solved, and high-precision pixel response correction and image quality improvement are achieved in an on-orbit environment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- STAR LIGHT YAO (BEIJING) TECHNOLOGY CO LTD
- Filing Date
- 2025-09-09
- Publication Date
- 2026-05-05
AI Technical Summary
Traditional pixel response inconsistency correction methods rely on laboratory calibration and cannot dynamically track changes in the on-orbit environment, resulting in pixel response characteristic drift. In particular, they cannot effectively eliminate pixel response deviations when temperatures change abruptly, affecting the quality of infrared images.
A temperature-time dual-domain prediction model combined with deep learning algorithms is adopted. The dynamic offset of pixels is calculated through real-time temperature data, and pixel-level correction is performed by integrating GPS and attitude angle data. The image quality is optimized through a bad pixel self-repair module, and gain compensation is performed using a deep reinforcement learning model that combines convolutional neural networks and long short-term memory networks.
It achieves high-precision pixel response correction under changing on-orbit environment, reduces pixel response inconsistency, improves image quality and imaging stability, and meets the requirements of high-resolution remote sensing.
Smart Images

Figure CN121259113B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of pixel response correction technology, and more particularly to a method and system for correcting pixel response inconsistency. Background Technology
[0002] In the field of remote sensing camera imaging, response characteristic parameters of each pixel are obtained through laboratory calibration to establish a pixel response inconsistency model. Linear or nonlinear fitting methods are typically used to describe the relationship between pixel response and radiance. Secondly, based on multi-frame image pixel-by-pixel non-uniformity correction technology, deep learning methods such as 3D residual networks are used to dynamically compensate for pixel response differences. This method can effectively handle non-uniform noise in remote sensing images and improve image quality. Next, geometric correction is performed using ground control points. By selecting ground feature points with known geographic coordinates, a coordinate transformation relationship is established between the original and corrected images, eliminating geometric distortions caused by sensor attitude, Earth curvature, and other factors. Then, the image is resampled using the control point correction method to ensure the accuracy of the corrected image in geographic coordinates, while unifying the coordinate system and projection parameters. Finally, radiometric calibration converts the digital quantization values of pixels into radiance values, further eliminating the impact of pixel response inconsistency on quantitative remote sensing analysis. This entire process combines laboratory calibration, deep learning algorithms, and geometric correction techniques, significantly improving the radiometric consistency and geometric accuracy of remote sensing images.
[0003] For example, the invention patent announcement CN115567651B discloses a method, system, electronic device, and medium for correcting pixel response non-uniformity, including: acquiring low-light observation image data output by a spaceborne low-light imager; the low-light observation image data includes latitude, longitude, and radiation values; preprocessing the low-light observation image data; sequentially performing relative deviation calculation and histogram processing on the preprocessed low-light observation image data to determine the bright line data of the observation image; and sequentially performing bilinear interpolation processing and mapping correction processing on the bright line data of the observation image to obtain the corrected low-light observation image data.
[0004] For example, the invention patent announcement CN103118235B discloses a device and method for correcting the nonlinear response of pixel output in an image sensor, comprising: a photosensitive pixel array, dark pixels, row selection switches for photosensitive pixels, dark pixel switches, bias current sources for photosensitive pixels, bias current sources for dark pixels, a linearization circuit, an analog column readout circuit, and a row selection and exposure control decoder. Compared with traditional image sensors, this invention adds a nonlinear correction circuit composed of dark pixels and a linearization circuit, which occupies a smaller chip area.
[0005] The above-mentioned technology has at least the following technical problems:
[0006] Traditional pixel response inconsistency correction heavily relies on the baseline plan field coefficients calibrated in the laboratory using equipment such as integrating spheres. These coefficients only reflect the pixel response characteristics of the camera under standard ground conditions. However, after the camera enters on-orbit operation, it is exposed to complex environments such as space radiation, temperature cycling, and detector aging, leading to continuous drift in pixel response characteristics. To compensate for the limitations of laboratory calibration, some techniques employ on-orbit scene statistical methods (such as updating the plan field using uniform ground features, cloud tops, etc.) for correction. However, these methods rely on statistical averaging calculations of global or local areas of the image. During the calculation process, high-frequency details in the image (such as ground feature edges and fine textures) are inevitably smoothed, resulting in a decrease in the camera's modulation transfer function.
[0007] For infrared cameras, pixel response is significantly affected by temperature, especially in scenarios like the Tibetan Plateau with its large diurnal temperature range and sudden temperature changes in on-orbit cameras. Traditional two-point correction methods (which determine correction coefficients using dark and bright frames) cannot dynamically track the nonlinear effects of temperature changes on pixel response. When temperatures change abruptly, the model assumptions of the two-point correction method deviate greatly from the actual pixel response patterns, causing the correction residual to exceed a preset threshold. Furthermore, it cannot effectively eliminate pixel response deviations caused by temperature drift, resulting in obvious striped noise or uneven brightness in infrared images. This makes it difficult to meet the needs of thermal infrared monitoring of the plateau ecological environment (such as lake heat distribution and vegetation transpiration analysis). In existing correction techniques, obtaining dark levels (dark currents) requires acquiring dark frames by blocking the optical path. This process consumes the camera's effective observation time, reducing the effective observation duration and leading to large correction errors due to pixel response inconsistencies. Summary of the Invention
[0008] This application provides a pixel response inconsistency correction method and system, which solves the problem of large pixel response inconsistency correction error in the prior art and improves the pixel response inconsistency correction accuracy.
[0009] On the one hand, a pixel response inconsistency correction method is provided, including the following steps: acquiring dark frame images, combining real-time camera temperature data acquired by a temperature sensor, and calculating pixel dynamic offset based on a temperature-time dual-domain prediction model; obtaining the flat field coefficient, which is generated by a fusion algorithm from laboratory calibration baseline values and on-orbit uniform scene observation update values; the laboratory calibration baseline values are obtained through laboratory micro-integrating sphere calibration, covering the entire camera spectrum; the scene observation update values are generated by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal domain joint analysis of bad pixel detection, inconsistency difference detection, and neighborhood interpolation processing; and performing pixel-level correction on the original remote sensing camera image based on the pixel dynamic offset and flat field coefficient. During the process, satellite GPS position data, attitude angle data, and real-time temperature data are integrated simultaneously to assist in optimizing the correction accuracy. Attitude angle data is used to compensate for pixel response deviations caused by camera sway, and GPS position data is used to match imaging parameters at different orbital altitudes. After correction, the image is further optimized through a dead pixel self-repair module, which is based on the pixel gain compensation matrix output by a deep reinforcement learning model. The inputs of the deep reinforcement learning model include real-time temperature, satellite attitude angle, and historical pixel response curves. The network structure combines convolutional neural networks with temporal units of long short-term memory networks. The output pixel gain compensation matrix is used to compensate for response deviations of individual pixels caused by long-term on-orbit aging and radiation damage.
[0010] On the other hand, a pixel response inconsistency correction system is provided, including a pixel dynamic offset calculation module, a flat field coefficient acquisition module, a pixel-level comprehensive correction module, and a bad pixel self-repair secondary optimization module: The pixel dynamic offset calculation module is used to acquire dark frame images and, combined with real-time camera temperature data acquired by a temperature sensor, calculates the pixel dynamic offset based on a temperature-time dual-domain prediction model; the flat field coefficient acquisition module is used to acquire the flat field coefficient, which is generated by a fusion algorithm from laboratory calibration baseline values and on-orbit uniform scene observation update values; the laboratory calibration baseline values are obtained through laboratory micro-integrating sphere calibration, covering the full spectrum of the camera; the scene observation update values are generated by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal domain joint analysis of bad pixel detection, inconsistency difference detection, and neighborhood interpolation processing; the pixel-level comprehensive correction module is used to... Pixel dynamic offset and flat field coefficient are used to perform pixel-level correction on the original images of the remote sensing camera. During the correction process, satellite GPS position data, attitude angle data, and real-time temperature data are integrated simultaneously to assist in optimizing the correction accuracy. Attitude angle data is used to compensate for pixel response deviations caused by camera sway, and GPS position data is used to match imaging parameters at different orbital altitudes. The bad pixel self-repair secondary optimization module is used to perform secondary optimization on the image after correction. The bad pixel self-repair module is based on the pixel gain compensation matrix output by the deep reinforcement learning model. The input of the deep reinforcement learning model includes real-time temperature, satellite attitude angle, and historical pixel response curves. The network structure adopts a combination of convolutional neural network and long short-term memory network temporal units. The output pixel gain compensation matrix is used to compensate for the response deviations of individual pixels caused by long-term on-orbit aging and radiation damage.
[0011] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:
[0012] 1. In the pixel dynamic offset calculation stage, a combined approach of "non-interrupted dark frame acquisition, temperature-time dual-domain prediction model, and cosmic ray filtering" was adopted to achieve high accuracy and low interference in offset calculation. On the one hand, dark frames were acquired during the satellite orbit shadow period and at the moment the mechanical shutter closed, avoiding the reduction in effective observation time due to light path obstruction, thus ensuring that the imaging capability of the camera per revolution was not affected. At the same time, the temperature of the focal plane component was obtained by a temperature sensor integrated into the detector readout circuit, ensuring the accuracy of the temperature input data. Cosmic ray contamination was filtered through the inter-frame standard deviation detection algorithm to reduce the single-frame contamination rate. Combined with the focal plane circuit clamping correction function, the quantization value after effective pixel clamping was made less than the quantization reference value, further eliminating dark current interference and providing a stable reference offset for subsequent pixel-level correction. The accuracy of dynamic offset calculation was significantly improved, effectively eliminating dark current and environmental interference, laying the foundation for high-precision correction.
[0013] 2. By employing "full-spectrum laboratory calibration, on-orbit uniform scene update, and dynamic fusion algorithm," the pain points of traditional flat field correction, which relies on laboratory calibration and suffers from residual errors caused by on-orbit environmental drift, have been resolved. During the laboratory calibration phase, full-spectrum calibration was performed using a miniature integrating sphere in an environment with controlled temperature, humidity, and cleanliness (Class 100,000), covering the entire color spectrum and eight multispectral bands. Simultaneously, quantization errors and system nonlinearity errors were corrected to ensure that the baseline laboratory calibration values met the requirements for absolute and relative calibration accuracy. During the on-orbit update phase, deep-space and cloud-top uniform scenes were selected by combining satellite attitude data and GPS position data. Defect detection (inter-frame standard deviation detection) and inconsistency detection (comparing pixel responses at different times and spaces) were performed through spatiotemporal joint analysis. Wavelet transform combined with median filtering and neighborhood interpolation were used to repair the data, ensuring that the relative calibration accuracy of the on-orbit observation update values was less than or equal to the calibration accuracy reference value. This ensured that the flattening coefficients retained the baseline accuracy of the laboratory calibration while adapting to environmental drift caused by on-orbit temperature changes and radiation effects, ultimately reducing the non-uniformity residuals of the flattening coefficients. This guaranteed image radiation uniformity and detail integrity, significantly enhancing the dynamic adaptability of the flattening coefficients, balancing laboratory calibration accuracy with on-orbit environmental changes, and effectively reducing pixel response inconsistencies.
[0014] 3. In the pixel-level comprehensive correction and bad pixel self-repair stages, a comprehensive improvement in image quality was achieved through a two-layer optimization of "multi-auxiliary data fusion correction and deep reinforcement learning repair". In the comprehensive correction stage, the dynamic offset of pixels and the flat field coefficient are the core, and satellite attitude angle and GPS data are integrated to reduce the gradient difference between the repaired pixels and the surrounding area, thereby improving the bad pixel repair rate. Moreover, the model supports on-orbit parameter uploading and periodic fine-tuning, and can continuously adapt to changes such as long-term detector aging and cumulative space radiation dose. This ensures that the corrected image can meet the requirements of high-resolution remote sensing and adapt to the reliability requirements of long-term on-orbit operation. At the same time, the synergistic effect of pixel-level comprehensive correction and bad pixel self-repair comprehensively compensates for multi-source errors and repairs long-term damage, achieving a second leap in image quality. Attached Figure Description
[0015] Figure 1 A flowchart of the pixel response inconsistency correction method provided in the embodiments of this application;
[0016] Figure 2 This is a schematic diagram of the pixel response inconsistency correction system provided in an embodiment of this application. Detailed Implementation
[0017] This application provides a method and system for correcting pixel response inconsistency, which solves the problem of large correction errors in existing technologies. It collects dark frame images and real-time temperature data, and uses a temperature-time dual-domain prediction model to calculate the dynamic offset of pixels. Then, it integrates the baseline values calibrated by the full spectrum of a laboratory micro-integrating sphere, and updated values from in-orbit observations of deep space and cloud tops, processed by bad pixel detection, to obtain the flat field coefficient. Next, it combines GPS, attitude angles, and real-time temperature to perform pixel-level correction using the offset and flat field coefficient. Finally, it uses a deep reinforcement learning model containing convolutional neural networks and long short-term memory networks to output a gain compensation matrix, and a bad pixel self-repair module further optimizes the image, improving the accuracy of pixel response inconsistency correction.
[0018] The technical solution in this application embodiment aims to solve the problem of large correction errors due to pixel response inconsistency, and the overall approach is as follows:
[0019] By acquiring dark frame images and combining them with real-time camera temperature data collected by a temperature sensor, the dynamic offset of pixels is calculated based on a temperature-time dual-domain prediction model. The flat field coefficient is generated by a fusion algorithm using laboratory-calibrated baseline values and on-orbit uniform scene observation update values. Pixel-level correction is performed on the original remote sensing camera images based on the dynamic offset and flat field coefficient. During the correction process, satellite GPS position data, attitude angle data, and real-time temperature data are simultaneously integrated to assist in optimizing the correction accuracy. Attitude angle data is used to compensate for pixel response deviations caused by camera sway, and GPS position data is used to match imaging parameters at different orbital altitudes. After correction, the image is further optimized through a bad pixel self-repair module, which is based on the pixel gain compensation matrix output by a deep reinforcement learning model. The inputs of the deep reinforcement learning model include real-time temperature, satellite attitude angle, and historical pixel response curves. The network structure combines convolutional neural networks with long short-term memory network temporal units. The output pixel gain compensation matrix is used to compensate for response deviations of individual pixels caused by long-term on-orbit aging and radiation damage, improving the accuracy of pixel response inconsistency correction.
[0020] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0021] like Figure 1The diagram shows a flowchart of a pixel response inconsistency correction method provided in this application embodiment. The method includes the following steps: acquiring dark frame images, combining real-time camera temperature data acquired by a temperature sensor, and calculating pixel dynamic offset based on a temperature-time dual-domain prediction model; obtaining the flat field coefficient, which is generated by a fusion algorithm from laboratory calibration baseline values and on-orbit uniform scene observation update values; the laboratory calibration baseline values are obtained through laboratory micro-integrating sphere calibration, covering the entire camera spectrum; the scene observation update values are generated by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal domain joint analysis for bad pixel detection, inconsistency difference detection, and neighborhood interpolation; and performing pixel-wise recalculation of the original remote sensing camera image based on the pixel dynamic offset and flat field coefficient. Level correction involves simultaneously integrating satellite GPS position data, attitude angle data, and real-time temperature data to assist in optimizing correction accuracy. Attitude angle data is used to compensate for pixel response deviations caused by camera sway, and GPS position data is used to match imaging parameters at different orbital altitudes. After correction, the image is further optimized through a dead pixel self-repair module, which is based on the pixel gain compensation matrix output by a deep reinforcement learning model. The inputs to the deep reinforcement learning model include real-time temperature, satellite attitude angle, and historical pixel response curves. The network structure combines convolutional neural networks with temporal units of long short-term memory networks. The output pixel gain compensation matrix is used to compensate for response deviations of individual pixels caused by long-term on-orbit aging and radiation damage.
[0022] In this embodiment, a four-step progressive process is used to achieve high-precision optimization of the imaging quality of the remote sensing camera. The technical details and effects of each step are as follows: In the pixel dynamic offset calculation step, the dark frame image is acquired without interruption during the satellite orbit shadow period or at the moment the mechanical shutter closes, avoiding the occupation of the effective time for Earth observation. At the same time, the real-time temperature of the focal plane component is obtained by the temperature sensor integrated into the detector readout circuit. The offset is calculated based on the temperature-time dual-domain prediction model. The detector aging attenuation coefficient is updated monthly based on dark frame data to adapt to the design life. The second-order temperature fitting parameters are combined with laboratory temperature chamber tests and on-orbit data for joint fitting to ensure adaptation to the high temperature sensitivity of the infrared band. At the same time, cosmic rays are filtered by inter-frame standard deviation detection to reduce the offset residual and improve the accuracy of traditional fixed dark level correction. In addition, the effective pixel quantization value is reduced by combining focal plane circuit clamping correction, eliminating dark current interference. In the flat field coefficient acquisition stage, the laboratory stage uses a miniature integrating sphere for calibration in a standard environment with controlled temperature, humidity, and cleanliness. This process simultaneously corrects quantization errors and system nonlinearity errors to obtain the baseline values for laboratory calibration. In the on-orbit stage, satellite three-axis attitude data and GPS position data are combined to screen for deep space and cloud-top uniform scenes. Through spatiotemporal joint analysis, bad pixel detection (inter-frame standard deviation determination) and inconsistency difference detection (comparing responses of different spatiotemporal pixels) are completed. Wavelet transform combined with median filtering and neighborhood interpolation are used to repair the data and generate updated on-orbit observation values. Finally, a fusion algorithm is used to generate the flat field coefficient, reducing non-uniform residuals and minimizing high-frequency information loss. This method is far superior to traditional scene statistical methods, balancing laboratory calibration accuracy and on-orbit environmental adaptability. In the pixel-level correction and secondary optimization stage, the correction is first optimized by integrating satellite-aided data based on the pixel dynamic offset and flat field coefficient: the camera side-swing angle is extracted to calculate the compensation coefficient, which compensates for the pixel response deviation caused by the side-swing, thus reducing the deviation under the side-swing state; the matching coefficient is calculated according to the orbital altitude, and the local weight of the flat field coefficient is adjusted to ensure that the correction accuracy fluctuation is reduced when the orbital altitude changes. Dark current subtraction, flat field correction, attitude compensation, and orbit matching are completed through radiation-resistant FPGA pipeline processing; then, the bad pixel self-repair module performs secondary optimization. The module relies on a 3-layer CNN (3×3 / 32, 3×3 / 64, 2×2 / 128 convolutional kernels, ReLU activation) to extract the spatial features of real-time temperature and attitude angle, combined with LSTM temporal units (256 hidden units, Adam) The optimizer mines the temporal patterns of the historical response curves of pixels and generates a pixel gain compensation matrix through a fully connected layer. It repairs aging and radiation-damaged bad pixels with deviation rates, reducing the gradient difference between the repaired pixel and its surroundings. Ultimately, it improves the bad pixel repair rate, reduces image MTF loss, improves the high-end signal-to-noise ratio of panchromatic and multispectral resolutions, and meets the imaging requirements of panchromatic and multispectral resolutions. The model also supports on-orbit parameter injection and periodic fine-tuning.
[0023] Furthermore, the specific process for acquiring dark frame images is as follows: During the satellite orbit shadow period, exposure is triggered momentarily by closing the mechanical shutter to achieve uninterrupted acquisition of dark frame images, thus avoiding occupying the effective time for camera Earth observation; Temperature data of the camera focal plane component is acquired in real time through a temperature sensor integrated into the camera detector readout circuit to ensure the accuracy of the temperature data input to the temperature-time dual-domain prediction model; Based on the acquired temperature data and dark frame images, the dynamic dark level offset is calculated using the temperature-time dual-domain prediction model.
[0024] In this embodiment, the dark frame image acquisition process focuses on "non-interrupted acquisition, precise temperature measurement, and dynamic modeling." Through the collaboration of multiple technologies, high-quality acquisition of dark frame data and accurate calculation of dynamic offset are achieved. The specific process and technical effects are as follows: The dark frame acquisition timing strictly matches the satellite orbit shadow period. Data acquisition is completed by triggering exposure at the moment the mechanical shutter closes. The time error of this shutter triggering mechanism is controlled to be less than the error reference value. Dark frame capture can be completed without obstructing the camera's optical path for Earth observation. This completely avoids the problem of reduced effective observation time caused by optical path obstruction in traditional dark frame acquisition. It ensures that the camera's on-orbit imaging capability of more than a preset threshold per orbit is not affected, and fully adapts to the needs of long-term continuous observation tasks such as large-scale ecological monitoring and border control in Tibet. Temperature data acquisition relies on a dedicated temperature sensor integrated into the camera detector readout circuit. This sensor is in direct contact with the focal plane assembly (the core working area of the detector). The temperature measurement range covers the temperature of electronic equipment inside the cabin from -20℃ to 55℃ and the temperature outside the payload cabin from -100℃ to 100℃. It can capture in real time the minute temperature fluctuations of the focal plane assembly caused by changes in its on-orbit working status (such as integration time adjustment and imaging mode switching), providing high-fidelity temperature input for subsequent model calculations and avoiding offset calculation errors caused by temperature data deviations. The dynamic dark level offset calculation employs a temperature-time dual-domain prediction model, with model parameters optimized using both laboratory and on-orbit data. The detector aging attenuation coefficient is updated monthly using dark frame data, accurately adapting to the slow attenuation trend of the detector response within the camera's 5-year design life. The second-order temperature fitting parameters are obtained through joint fitting of laboratory chamber tests (temperature control accuracy ±0.1℃, simulating extreme temperature environments of -40℃ to 60℃) and on-orbit measured temperature-dark current data, particularly suitable for the high temperature sensitivity of the infrared band (such as the camera's B8 multispectral band 857±20nm-1039±20nm). The final calculated dynamic dark level offset residual is more accurate than the traditional fixed dark level correction method. Furthermore, combined with the clamping correction function of the camera's focal plane circuit, it can clamp all effective pixels of the linear array output to the dark level reference corresponding to the dynamic offset, reducing the quantization value after clamping and effectively eliminating dark current interference, thus laying a stable reference for subsequent pixel-level correction.
[0025] Furthermore, the detector aging attenuation coefficient in the temperature-time dual-domain prediction model is updated periodically using monthly collected dark frame data. By collecting a preset number of dark frame data during the satellite orbit shadow period, the detector response attenuation trend is fitted based on multiple sets of data to calculate the detector aging attenuation coefficient for the month, thus achieving dynamic adaptation of the model to the detector aging state. The second-order temperature fitting parameter in the temperature-time dual-domain prediction model is determined by fitting data from both the laboratory phase and the on-orbit phase. In the laboratory phase, temperature-dark current data are collected by simulating different temperature environments using a miniature integrating sphere. In the on-orbit phase, temperature sensor data and dark frame data are synchronized. The two sets of data are jointly fitted to obtain the second-order temperature fitting parameter to ensure the model's adaptability to nonlinear responses to temperature changes, especially suitable for scenarios with high temperature sensitivity in the infrared band.
[0026] In this embodiment, the detector aging attenuation coefficient in the temperature-time dual-domain prediction model is updated periodically using monthly collected dark frame data. Specifically, a preset number of dark frame data sets are collected during the satellite's orbital shadow period. Based on the fit of multiple data sets to the detector response attenuation trend, the monthly detector aging attenuation coefficient is calculated, enabling the model to dynamically adapt to the detector's aging state. This mechanism accurately reflects the attenuation of the detector's response performance due to radiation damage, material aging, and other factors during its on-orbit operation, giving the model long-term self-learning capabilities and continuously maintaining high-precision compensation for pixel response non-uniformity. Simultaneously, the second-order temperature fitting parameters in the temperature-time dual-domain prediction model are determined by joint fitting of laboratory and on-orbit data: in the laboratory phase, temperature-dark current data are collected by simulating different temperature environments using a miniature integrating sphere; in the on-orbit phase, temperature sensor data and dark frame data are synchronized. Both sets of data participate in parameter fitting, ensuring the model has a high degree of adaptability to nonlinear responses to temperature changes, especially suitable for temperature-sensitive scenarios such as the infrared band. This two-stage joint fitting strategy not only improves the model's accuracy in compensating for temperature drift but also significantly enhances its robustness and generalization ability in complex on-orbit environments. This dynamic update and joint fitting mechanism achieves the following key breakthroughs: First, by fitting and updating the aging attenuation coefficient using monthly dark frame data, the model can track the detector's performance degradation trend in real time, ensuring that the correction accuracy attenuation rate is controlled within the expected range at the end of the on-orbit lifespan (e.g., after 3 years), thus significantly extending the camera's effective lifespan. Second, the joint fitting of laboratory and on-orbit data reduces the adaptation error of the second-order temperature fitting parameters for scenarios with daily temperature differences ≥20℃, particularly addressing the failure of traditional correction methods due to the high temperature sensitivity of the infrared band. Third, through the dynamic parameter update mechanism, the model's misjudgment rate for abnormal interference such as cosmic ray contamination and sudden temperature changes is reduced, while simultaneously lowering the pixel response standard deviation of the corrected image compared to the original data, ultimately achieving sub-DN level residual control. This provides reliable technical support for the accurate imaging of the 0.25m high-resolution remote sensing camera in complex environments such as Tibet.
[0027] Furthermore, the process involves acquiring dark frame images, followed by: filtering cosmic ray contamination using an inter-frame standard deviation detection algorithm; constructing a pixel stacking region for the same pixel in the dark frame image from multiple frames; calculating the standard deviation and median standard deviation of the pixel stacking region constructed from multiple frames; determining whether the standard deviation is greater than three times the median standard deviation; if so, replacing the contaminated pixel with the median value within the 3×3 neighborhood of that pixel; otherwise, leaving it unchanged; calculating the pixel dynamic offset based on a temperature-time dual-domain prediction model; and combining the clamping correction function of the camera's focal plane circuit to clamp the pixel output signal to the dark level reference corresponding to the dynamic offset, so that the quantization value of all effective pixels output by the linear array after clamping is less than the preset quantization reference value.
[0028] In this embodiment, an inter-frame standard deviation detection algorithm is used to efficiently filter cosmic ray contamination in dark frame images. The specific process is as follows: First, a pixel stacking region is constructed for the same pixel in multiple consecutive dark frame data. The standard deviation and median standard deviation of this region are calculated. If the standard deviation exceeds three times the median standard deviation, the pixel is determined to be contaminated by cosmic rays, and the median value within its 3×3 neighborhood is used to replace the contaminated pixel; otherwise, the original value remains unchanged. This method can accurately identify and remove sudden noise points caused by high-energy cosmic ray particle impacts, avoiding interference with dark level calculations. Simultaneously, after calculating the pixel dynamic offset based on a temperature-time dual-domain prediction model, the clamping correction function of the camera's focal plane circuit is combined to clamp the pixel output signal in real time to the dark level reference corresponding to the dynamic offset. This ensures that the quantized values of all effective pixels output by the linear array are less than a preset quantization reference value (e.g., 80DN), thereby achieving dynamic and stable control of the dark level.
[0029] Furthermore, the steps for obtaining the baseline values for laboratory calibration are as follows: A full-spectrum coverage calibration of the calibration scene is performed using a laboratory micro integrating sphere. The radiance calibration error of the micro integrating sphere is less than or equal to the calibration error reference value, the radiance stability is less than or equal to the radiance reference value, the surface uniformity peak-valley value is less than or equal to the peak-valley value reference value, and the angular uniformity is less than or equal to the angular uniformity reference value. During the full-spectrum coverage calibration process, the laboratory ambient temperature is controlled within the ambient temperature reference range to obtain the baseline values for laboratory calibration covering the entire spectrum of the camera. The full spectrum of the camera includes the full chromatographic range and the multispectral range. The steps for obtaining the updated values of the on-orbit uniform scene observation are as follows: In the phase... During the on-orbit operation, deep space and cloud tops were selected as uniform observation scenarios. Defect detection and inconsistency detection were performed through spatiotemporal joint analysis, and neighborhood interpolation was used to generate scene observation update values. Defect detection was achieved by inter-frame standard deviation detection. If the standard deviation of pixel stacking was greater than three times the median standard deviation, it was judged as a defect and replaced with median. Inconsistency detection was based on the spatiotemporal characteristics of camera imaging data. The response differences of pixels at different times and locations were compared to identify pixel regions with inconsistent responses. Neighborhood interpolation used wavelet transform combined with median filtering to repair the detected defective pixels and regions with inconsistent responses.
[0030] In this embodiment, a full-spectrum calibration of the calibration scene is performed using a laboratory miniature integrating sphere. The radiance calibration error of the miniature integrating sphere is ≤5%, radiance stability is ≤0.5%, surface uniformity (peak-valley value) is ≤1%, and angular uniformity is ≤0.5%. During the calibration process, the laboratory ambient temperature is strictly controlled within the range of 18-22℃, with temperature stability variation not exceeding ±0.2℃ over 24 hours. The temperature difference between any two points within each room does not exceed ±0.1℃, humidity is controlled at 30%-50%, and the cleanliness level is Class 100,000 to eliminate the interference of environmental factors on the calibration accuracy. This yields laboratory calibration baseline values covering the entire spectrum of the camera. The full spectrum of the instrument includes the full chromatographic range (450±20nm-800±20nm) and 8 multispectral ranges (B1: 450±20nm-520±20nm, B2: 520±20nm-590±20nm, B3: 630±20nm-690±20nm, B5: 626±20nm-696±20nm, B6: 698±20nm-749±20nm, B7: 765±20nm-899±20nm, B8: 770±20nm-890±20nm, and another B8: 857±20nm-1039±20nm). The steps for obtaining the on-orbit uniform scene observation update values are as follows: During the camera's on-orbit operation, deep space or cloud top is selected as the uniform observation scene. Defect detection and inconsistency detection are performed through spatiotemporal joint analysis, and neighborhood interpolation is used to generate scene observation update values. Defect detection is achieved through inter-frame standard deviation detection; if the standard deviation of pixel stacking is greater than three times the median standard deviation, it is determined to be a defect and replaced with the median. Inconsistency detection is based on the spatiotemporal characteristics of the camera imaging data, comparing the response differences of pixels at different times and locations to identify pixel regions with inconsistent responses. Neighborhood interpolation uses wavelet transform combined with median filtering to repair the detected defective pixels and regions with inconsistent responses, ensuring that the uniformity of the generated scene observation update values meets the requirement of relative calibration accuracy ≤3%. This provides core technical support for the accurate imaging of a 0.25m high-resolution remote sensing camera in complex environments such as Tibet.
[0031] Furthermore, the specific steps for obtaining the flat field coefficient are as follows: the flat field coefficient is generated by the fusion algorithm, the fusion weight decays with the detector aging index, the detector aging index is obtained by monitoring monthly dark frame data, and the value of the fusion weight is dynamically adjusted according to the aging degree of the detector. In the initial state, the fusion weight is within the fusion weight reference range. As the aging degree of the detector increases, the fusion weight value gradually decreases, and the minimum value is not lower than the lower limit of the fusion weight reference, so as to ensure that the flat field coefficient can reflect the basic accuracy calibrated in the laboratory.
[0032] In this embodiment, the acquisition of the flat field coefficient is based on the intelligent decay mechanism of the fusion weight with the detector aging index. It is dynamically generated through multi-stage collaboration. The specific process and technical effects are as follows: First, the detector aging index is acquired through monthly dark frame data monitoring. Every month, 5-10 sets of dark frame data are collected during the satellite orbit shadow period (without occupying the effective time of Earth observation). The temperature of the focal plane component is recorded simultaneously (based on the temperature sensor integrated in the detector readout circuit) to eliminate temperature interference. Based on the pixel response values in multiple sets of dark frame data, the decay trend of the detector with the time on orbit is fitted, and the detector aging decay coefficient for the month is calculated. This accurately quantifies the degree of detector aging. This monthly update mechanism shortens the response delay of the model to long-term drift such as radiation damage and material aging to within 30 days, significantly improving the real-time adaptability to changes in the on-orbit environment. Secondly, a fusion algorithm is used to complete the full-spectrum calibration of the micro-integrating sphere in an environment with controllable temperature and humidity and a cleanliness level of 100,000. This provides a reliable guarantee for the long-term stable imaging of the 0.25m high-resolution remote sensing camera in the complex environment of the Tibetan Plateau (such as large temperature difference between day and night and high radiation intensity), and meets the stringent requirements of image radiation uniformity for tasks such as glacier monitoring and border control.
[0033] Furthermore, the specific process of pixel-level correction is as follows: acquire satellite three-axis stable attitude data, extract camera yaw angle, calculate pixel response deviation compensation coefficient based on yaw angle, incorporate yaw angle into pixel-level correction formula, and compensate for pixel response non-uniformity caused by yaw to reduce pixel response deviation under yaw state; acquire satellite GPS position data, calculate imaging parameter matching coefficient based on orbital altitude, and based on ground pixel resolution formula, if orbital altitude deviates from orbital altitude reference value, adjust the local weight of flat field coefficient through imaging parameter matching coefficient to match imaging resolution requirements under different orbital altitudes; perform pixel-level correction on the original image using correction formula, and the correction process is implemented in a pipelined manner through radiation resistance, sequentially completing dark current subtraction, flat field correction, attitude deviation compensation, and orbital parameter matching.
[0034] In this embodiment, attitude data-driven lateral tilt compensation reduces the pixel response deviation of the camera under ±45° lateral tilt maneuvering by 70% compared to traditional methods, especially solving the problem of non-uniformity of edge field of view response during large-angle lateral tilts. Secondly, dynamic matching of GPS position data and orbital parameters improves the consistency of imaging resolution at different orbital altitudes (480-530km) to 98%, avoiding image quality fluctuations caused by orbital attenuation. Thirdly, radiation-resistant FPGA pipeline processing enables real-time collaborative four-level correction with a correction delay of ≤15ms, supporting continuous pushbroom imaging (≥10 minutes per revolution). Simultaneously, through cascaded optimization of dark current subtraction and flat field correction, the standard deviation of pixel response in the corrected image is reduced by more than 60% compared to the original data, ultimately ensuring the imaging stability and accuracy of the 0.25m high-resolution remote sensing camera under complex terrain and variable orbital conditions such as Tibet.
[0035] Furthermore, the specific steps for secondary image optimization using the dead pixel self-repair module are as follows: Real-time temperature data, satellite attitude angle data, and pixel historical response curves are collected during camera operation in orbit, with the update frequency consistent with the integral time. A structure combining convolutional neural networks and long short-term memory network temporal units is adopted. The convolutional neural network is used to extract spatial features from the real-time temperature and attitude angle data, while the long short-term memory network temporal units are used to mine the temporal variation patterns of the pixel historical response curves. The output features of both are fused through a fully connected layer to generate a pixel gain compensation matrix. The objective function is to minimize the non-uniformity residual of the corrected pixel response. A pixel response dataset simulating on-orbit aging and radiation damage in the laboratory is used for further processing. Pre-training is performed, and fine-tuning is carried out using newly acquired pixel response data during on-orbit operation to reduce the prediction error of individual pixel response deviations. The corrected image is matched pixel by pixel with the pixel gain compensation matrix output by the deep reinforcement learning model. The deviation rate between the actual response value and the theoretical response value of the pixel is calculated. It is determined whether the deviation rate is greater than the deviation rate reference value. If it is, the pixel is determined to be a bad pixel with abnormal response caused by long-term on-orbit aging and radiation damage. If not, it is not determined. For the determined bad pixels with abnormal response, the gain compensation value of the corresponding pixel in the pixel gain compensation matrix is used for repair to reduce the gradient difference between the repaired pixel value and the surrounding normal pixel value. Local MTF detection is performed on the image after secondary optimization.
[0036] In this embodiment, firstly, the deep mining of multimodal data through a CNN-LSTM hybrid architecture improves the accuracy of the pixel gain compensation matrix in predicting response anomalies caused by aging / radiation damage, achieving a 40% improvement over traditional threshold detection methods. Secondly, the collaborative mechanism of laboratory pre-training and on-orbit fine-tuning reduces the model's adaptation error to sudden temperature changes (e.g., daily temperature difference ≥20℃) and radiation accumulation effects in complex environments such as Tibet, particularly addressing the problem of misjudgment of bad pixels due to the high temperature sensitivity of the infrared band. Thirdly, pixel-by-pixel gain compensation and gradient difference control ensure that the residual bad pixel rate of the repaired image is ≤1%, and the pixel response standard deviation is lower than that of the original data. At the same time, local MTF detection ensures that the image details are not lost during the repair process. Fourthly, the on-orbit update mechanism of model parameters (implemented via CAN bus, with fine-tuning time ≤10 seconds) enables the system to maintain a bad pixel repair rate above the preset bad pixel repair rate reference value at the end of its 3-year lifespan. Ultimately, this ensures the imaging stability and accuracy of the 0.25m high-resolution remote sensing camera during long-term on-orbit operation, providing core technical support for accurate monitoring in complex terrains such as Tibet.
[0037] Furthermore, the image is further optimized through a dead pixel self-repair module, which includes: on-orbit uploading of model parameters via the satellite CAN bus, during which the focal plane circuit operates in a low-power mode and the uploaded data is frame-checked; a circular storage mechanism is adopted, which automatically overwrites the oldest historical data when the amount of stored data reaches the upper limit, while periodically filtering the historical data to remove invalid data caused by cosmic ray interference and sudden temperature changes; after each track imaging is completed, a uniform region in the corrected image is extracted, the standard deviation of the pixel response in that region is calculated, and it is determined whether the standard deviation of the pixel response is greater than the standard deviation reference value. If so, the model parameter fine-tuning process is triggered. The fine-tuning process only updates the hidden layer parameters of the temporal units of the long short-term memory network, and the fine-tuning time is less than the fine-tuning time reference value to ensure that the pixel gain compensation matrix output by the model continuously adapts to changes in the on-orbit environment.
[0038] In this embodiment, the success rate of model parameter updates is improved and the power consumption is reduced compared to the conventional mode by using low-power uploading and frame verification mechanisms via the CAN bus, thus avoiding the burden on the satellite's energy system. Secondly, the combination of cyclic storage and intelligent screening improves the effective utilization rate of historical data and reduces the interference of invalid data on model training, especially addressing the problem of cosmic ray contamination in high-radiation environments such as Tibet. Thirdly, adaptive fine-tuning based on the standard deviation of uniform regions ensures that the model's response delay to changes in the on-orbit environment is within a preset time reference value. After fine-tuning, the standard deviation of pixel response is reduced, and the local update strategy of LSTM hidden layer parameters (updating only 10% of the parameters) avoids model overfitting, ensuring long-term stability. Fourthly, the fully automated design (from data screening to fine-tuning triggering without manual intervention) enables the system to maintain a bad pixel repair rate above the preset repair rate threshold at the end of its 3-year lifespan, ultimately ensuring continuous high-precision imaging of the 0.25m high-resolution remote sensing camera in complex on-orbit environments, providing reliable technical support for remote sensing monitoring under extreme conditions such as Tibet.
[0039] like Figure 2 The diagram shown is a structural schematic of the pixel response inconsistency correction system provided in this application embodiment. The pixel response inconsistency correction system includes: a pixel dynamic offset calculation module, a flat field coefficient acquisition module, a pixel-level comprehensive correction module, and a bad pixel self-repair secondary optimization module. The pixel dynamic offset calculation module is used to acquire dark frame images and, combined with real-time camera temperature data acquired by a temperature sensor, calculate the pixel dynamic offset based on a temperature-time dual-domain prediction model. The flat field coefficient acquisition module is used to acquire the flat field coefficient, which is generated by a fusion algorithm from laboratory calibration baseline values and on-orbit uniform scene observation update values. The laboratory calibration baseline values are obtained through laboratory micro-integrating sphere calibration, covering the entire camera spectrum. The scene observation update values are obtained by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal domain joint analysis of bad pixel detection, inconsistency difference detection, and neighborhood interpolation. The system includes: a pixel-level integrated correction module for pixel-level correction of the original remote sensing camera image based on pixel dynamic offset and flat field coefficient; a second bad pixel self-repair and secondary optimization module for optimizing the image after correction, which integrates satellite GPS position data, attitude angle data, and real-time temperature data to assist in optimizing the correction accuracy; attitude angle data to compensate for pixel response deviation caused by camera sway; and GPS position data to match imaging parameters at different orbital altitudes. The third bad pixel self-repair and secondary optimization module is used to perform secondary optimization of the image after correction. This module is based on the pixel gain compensation matrix output by a deep reinforcement learning model. The inputs to the deep reinforcement learning model include real-time temperature, satellite attitude angle, and historical pixel response curves. The network structure combines convolutional neural networks with long short-term memory network temporal units. The output pixel gain compensation matrix is used to compensate for response deviations caused by long-term on-orbit aging and radiation damage in individual pixels.
[0040] In this embodiment, through the synergistic effect of the pixel dynamic offset calculation module, the flat field coefficient acquisition module, the pixel-level comprehensive correction module, and the bad pixel self-repair secondary optimization module, high-precision correction and adaptive optimization of pixel response inconsistency of the remote sensing camera are achieved across the entire link, with significant technical effects: First, the pixel dynamic offset calculation module calculates the dark level offset in real time based on a temperature-time dual-domain prediction model, and combines temperature sensor data to achieve sub-DN level residual control, effectively suppressing the impact of temperature drift on imaging; Second, the flat field coefficient acquisition module dynamically integrates the laboratory calibration baseline value and the on-orbit uniform scene observation update value through a fusion algorithm, and the fusion weight intelligently decays with the detector aging index, ensuring the accuracy and stability of the flat field coefficient under long-term on-orbit environment; Third, The pixel-level integrated correction module synchronously integrates satellite GPS position data, attitude angle data, and real-time temperature data. Through attitude compensation and orbit parameter matching, it eliminates pixel response deviations caused by lateral sway and orbital altitude changes, achieving high-precision correction through multi-source data collaboration. Finally, the dead pixel self-repair secondary optimization module generates a pixel gain compensation matrix based on a deep reinforcement learning model combining convolutional neural networks and long short-term memory networks. It intelligently repairs dead pixels with abnormal responses caused by long-term on-orbit aging and radiation damage, reducing the pixel response standard deviation of the corrected image compared to the original data and lowering the dead pixel residual rate. Ultimately, it provides a full-process, high-precision, and adaptive technical guarantee for the long-term stable imaging of the 0.25m high-resolution remote sensing camera in complex environments such as Tibet.
[0041] It should be added that the detector driver function completes the power supply, timing drive, communication, data reception, and imaging parameter control (level, gain, clamping, etc.) of the specified detector, and the detector must not be damaged under any circumstances. CAN bus communication / remote control / telemetry function: communicates with an external CAN bus to control imaging parameters and imaging process, collects the board's operating status information, and feeds it back to the CAN bus; through the second pulse synchronization function, it receives external primary and backup second pulses and inputs the hardware second pulse time and the CAN bus software second pulse time into the image auxiliary data; the hardware second pulse recording time resolution is 1µs, and the time accuracy is better than 20µs. It receives an external +6V power input and converts it into the power required for the detector and the board's operation; it arranges the detector's image data format according to the specified format. After circuit initialization, it must be able to output correct images using default parameters and preset parameters stored after power failure, and the image output should not be affected by the communication status of external interfaces such as the CAN bus and second pulse. It should at least implement pixel response inconsistency correction, dark level correction, clamping correction, digital filtering, and bad pixel correction for image data. The satellite auxiliary data and the board's operating status information are encoded into image auxiliary data according to a specified format. This is used for synchronization between different detectors. One possible method: a jumper is used to set whether the board is actively or passively synchronized. If it is actively synchronized, the corresponding line synchronization is generated according to the CAN bus integration time, and the line synchronization signal is output externally; if it is passively synchronized, the board synchronizes itself according to the input line synchronization signal. If no line synchronization is received within 5 seconds after initialization, it operates according to the CAN bus integration time. The uploading function allows for uploading and updating of correction coefficients and FPGA programs; during uploading, the detector operates with low power consumption. The forward / reverse scan function controls the TDI detector's forward / reverse push-broom imaging function according to CAN bus control commands. The test mode is used for ground testing. Area array output mode: In this mode, all levels of the detector are output to form an area array image; 14-bit output mode: In this mode, all quantization bits of the detector are output.
[0042] It's important to understand that in the sunlit area, push-broom imaging of the imaging region is achieved according to instructions, simultaneously outputting panchromatic and four-band multispectral data. The camera subsystem only images in the sunlit area, with a maximum single-loop imaging time of 10 minutes. The camera automatically shuts down for protection after 15 minutes of imaging. The video processing is in standby mode, with no image data output. The camera is initiated or deactivated from standby at specified times, with command execution accuracy less than 10ms. The video processing is in a powered-off state, with no image data output and no power consumption. The video processing is in a powered-on but non-imaging state, receiving uplink data packets from the integrated electronic relay, containing parameters and programs.
[0043] The imaging process is as follows: Under the control of the integrated satellite electronic switch, the secondary power supply first performs main backup settings, then powers on, and outputs +6V power to the focal plane circuit. After power-on initialization, the focal plane circuit is ready for imaging, completing image output and various data arrangement and processing requirements. The focal plane circuit executes various commands and broadcast data sent by the integrated satellite electronic system and returns telemetry data. Upon receiving the "Start Standby" command forwarded by the integrated satellite electronic system, the focal plane circuit puts the corresponding circuits of the focal plane electronic components into standby mode to save power, and simultaneously stops outputting images to the data transmission subsystem; upon receiving the "Stop Standby" command forwarded by the integrated satellite electronic system, it resumes imaging. During a single imaging process, the "Start Standby" and "Stop Standby" commands can appear multiple times in pairs. During the "Start Standby" process, the focal plane circuit is required to have the ability to receive and save the parameters set by the integrated satellite electronic system. After "Stop Standby," imaging is performed according to the last set parameters. The uploading process is as follows: Uploading is divided into two types: one is FPGA software configuration bitstream uploading; the other is correction coefficient uploading. The focal plane circuit uploading process must be initiated when the focal plane circuit is powered on. Upon receiving the uploading preparation command from the integrated electronics, the focal plane circuit stops imaging, minimizes its own system, and awaits the uploading command. The focal plane circuit receives uploading data packet by packet and performs frame-by-frame verification. After uploading is complete and the circuit restarts, it executes with the new program / new correction coefficients.
[0044] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0045] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0048] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for correcting pixel response inconsistency, characterized in that, Includes the following steps: Dark frame images are acquired, and combined with camera temperature data collected in real time by a temperature sensor, the dynamic offset of pixels is calculated based on a temperature-time dual-domain prediction model. The flat field coefficients are obtained by a fusion algorithm from laboratory calibration baseline values and on-orbit uniform scene observation update values. The laboratory calibration baseline values are obtained through laboratory micro-integrating sphere calibration, covering the full spectrum of the camera. The scene observation update values are generated by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal joint analysis of bad spot detection, inconsistency difference detection and neighborhood interpolation. Pixel-level correction is performed on the original images of the remote sensing camera based on pixel dynamic offset and flat field coefficient. During the correction process, satellite GPS position data, attitude angle data and real-time temperature data are integrated simultaneously to help optimize the correction accuracy. Attitude angle data is used to compensate for pixel response deviation caused by camera side sway, and GPS position data is used to match imaging parameters at different orbital altitudes. After correction, the image is further optimized through a bad pixel self-repair module, which is based on the pixel gain compensation matrix output by the deep reinforcement learning model. The input of the deep reinforcement learning model includes real-time temperature, satellite attitude angle and pixel historical response curve. The network structure adopts a combination of convolutional neural network and long short-term memory network temporal unit. The output pixel gain compensation matrix is used to compensate for the response deviation of individual pixels caused by long-term on-orbit aging and radiation damage.
2. The pixel response inconsistency correction method as described in claim 1, characterized in that, The specific process for acquiring the dark frame image is as follows: During the satellite orbit shadow period, exposure is triggered momentarily by closing the mechanical shutter to achieve uninterrupted acquisition of dark frame images, thus avoiding occupying the effective time for camera observation of the Earth. By using a temperature sensor integrated into the camera detector readout circuit, the temperature data of the camera focal plane component is collected in real time to ensure the accuracy of the temperature data input to the temperature-time dual-domain prediction model. Based on the collected temperature data and dark frame images, a temperature-time dual-domain prediction model is used to calculate the dynamic dark level offset.
3. The pixel response inconsistency correction method as described in claim 1, characterized in that, The detector aging attenuation coefficient in the temperature-time dual-domain prediction model is updated periodically using monthly collected dark frame data. By collecting a preset number of dark frame data during the satellite orbit shadow period, the detector response attenuation trend is fitted based on multiple sets of data, and the detector aging attenuation coefficient for the month is calculated, thus realizing the dynamic adaptation of the model to the detector aging state. The second-order temperature fitting parameters in the temperature-time dual-domain prediction model are determined by fitting data from both the laboratory phase and the on-orbit phase. In the laboratory phase, temperature-dark current data are collected by simulating different temperature environments using a micro-integrating sphere. In the on-orbit phase, temperature sensor data and dark frame data are synchronized. The two sets of data are jointly fitted to obtain the second-order temperature fitting parameters to ensure the model's adaptability to nonlinear responses to temperature changes and to adapt to scenarios with high temperature sensitivity in the infrared band.
4. The pixel response inconsistency correction method as described in claim 1, characterized in that, The process of acquiring the dark frame image then includes: Cosmic ray contamination is filtered out by the inter-frame standard deviation detection algorithm, and pixel stacking regions are constructed for the same pixel in dark frame images from multiple frames of data. Calculate the standard deviation and median standard deviation of the pixel stacking region constructed by the multi-frame data, and determine whether the standard deviation is greater than three times the median standard deviation. If so, replace the contaminated pixel with the median value in the 3×3 neighborhood of the pixel; otherwise, leave it unchanged. After calculating the dynamic offset of pixels based on the temperature-time dual-domain prediction model, the clamping correction function of the camera focal plane circuit is combined to clamp the pixel output signal to the dark level reference corresponding to the dynamic offset, so that the quantization value of all effective pixels output by the linear array is less than the preset quantization reference value after clamping.
5. The pixel response inconsistency correction method as described in claim 1, characterized in that, The steps for obtaining the baseline values for laboratory calibration are as follows: A full-spectrum calibration of the calibration scene is performed using a laboratory micro integrating sphere. The micro integrating sphere has a radiance calibration error less than or equal to the calibration error reference value, radiance stability less than or equal to the radiance reference value, surface uniformity peak-valley value less than or equal to the peak-valley value reference value, and angular uniformity less than or equal to the angular uniformity reference value. During the full-spectrum calibration process, the laboratory ambient temperature is controlled within the ambient temperature reference range to obtain laboratory calibration baseline values covering the entire spectrum of the camera. The full spectrum of the camera includes the full chromatographic range and the multispectral range. The steps for obtaining the on-orbit uniform scene observation update values are as follows: During the camera's on-orbit operation, deep space and cloud tops were selected as uniform observation scenarios. Defect detection and inconsistency detection were performed through spatiotemporal joint analysis, and neighborhood interpolation was used to generate scene observation update values. The defective pixel detection is achieved through inter-frame standard deviation detection. If the standard deviation of pixel stacking is greater than three times the median standard deviation, it is determined to be a defective pixel and the median is used for replacement. The inconsistency difference detection is based on the spatiotemporal domain features of camera imaging data, comparing the response differences of pixels at different times and locations to identify pixel regions with inconsistent responses; The neighborhood interpolation process uses wavelet transform combined with median filtering to repair the detected bad pixels and regions with inconsistent responses.
6. The pixel response inconsistency correction method as described in claim 1, characterized in that, The specific steps for obtaining the flat field coefficient are as follows: The flat field coefficient is generated by a fusion algorithm. The fusion weight decays with the detector aging index, which is obtained by monitoring monthly dark frame data. The value of the fusion weight is dynamically adjusted according to the aging degree of the detector. In the initial state, the fusion weight is within the fusion weight reference range. As the aging degree of the detector increases, the fusion weight value gradually decreases, and the minimum value is not lower than the lower limit of the fusion weight reference, so as to ensure that the flat field coefficient can reflect the basic accuracy calibrated in the laboratory.
7. The pixel response inconsistency correction method as described in claim 1, characterized in that, The specific process for performing pixel-level correction is as follows: Acquire satellite three-axis stable attitude data, extract camera side-swing angle, calculate pixel response deviation compensation coefficient based on side-swing angle, incorporate the side-swing angle into pixel-level correction formula, and compensate for pixel response non-uniformity caused by side-swing to reduce pixel response deviation under side-swing state. The system acquires satellite GPS position data, calculates imaging parameter matching coefficients based on orbital altitude, and adjusts the local weights of the flat field coefficients to match the imaging resolution requirements at different orbital altitudes if the orbital altitude deviates from the orbital altitude reference value, based on the ground pixel resolution formula. The original image is corrected at the pixel level using a correction formula. The correction process is implemented in a pipelined manner through radiation resistance, and dark current subtraction, flat field correction, attitude deviation compensation and orbit parameter matching are completed sequentially.
8. The pixel response inconsistency correction method as described in claim 1, characterized in that, The specific steps for secondary image optimization using the bad pixel self-repair module are as follows: The real-time temperature data, satellite attitude angle data, and historical pixel response curves collected during the camera's on-orbit operation are updated at the same frequency as the integration time. The structure combines a convolutional neural network and a long short-term memory network temporal unit. The convolutional neural network is used to extract spatial features from real-time temperature and attitude angle data, while the long short-term memory network temporal unit is used to mine the temporal variation pattern of the historical response curve of the pixel. The output features of the two are fused by a fully connected layer to generate a pixel gain compensation matrix. With the objective function of minimizing the non-uniformity residual of the corrected pixel response, a pre-training was performed using a pixel response dataset that simulates on-orbit aging and radiation damage in the laboratory. During on-orbit operation, fine-tuning was performed using newly acquired pixel response data to reduce the prediction error of individual pixel response deviation. The corrected image is matched pixel by pixel with the pixel gain compensation matrix output by the deep reinforcement learning model. The deviation rate between the actual response value and the theoretical response value of the pixel is calculated. It is determined whether the deviation rate is greater than the deviation rate reference value. If so, the pixel is determined to be a bad pixel with abnormal response caused by long-term on-orbit aging and radiation damage. If not, no determination is made. For the identified abnormal pixels, the gain compensation value of the corresponding pixel in the pixel gain compensation matrix is used for repair to reduce the gradient difference between the repaired pixel value and the surrounding normal pixel value. Local MTF detection is then performed on the image after secondary optimization.
9. The pixel response inconsistency correction method as described in claim 8, characterized in that, The secondary optimization of the image through the bad pixel self-repair module also includes: The model parameters are injected into orbit via satellite CAN bus. During the injection process, the focal plane circuit is in a low-power operating mode, and the injected data is frame-checked. A circular storage mechanism is adopted. When the amount of stored data reaches the upper limit, the oldest historical data is automatically overwritten. At the same time, the historical data is periodically filtered to remove invalid data caused by cosmic ray interference and sudden temperature changes. After each track imaging is completed, a uniform region in the corrected image is extracted, and the standard deviation of the pixel response in that region is calculated. It is then determined whether the standard deviation of the pixel response is greater than the standard deviation reference value. If so, the model parameter fine-tuning process is triggered. The fine-tuning process only updates the hidden layer parameters of the temporal units of the Long Short-Term Memory network, and the fine-tuning time is less than the fine-tuning time reference value to ensure that the pixel gain compensation matrix output by the model continuously adapts to changes in the on-orbit environment.
10. A pixel response inconsistency correction system, wherein the pixel response inconsistency correction system applies a pixel response inconsistency correction method as described in any one of claims 1-9, characterized in that, It includes a pixel dynamic offset calculation module, a flat field coefficient acquisition module, a pixel-level comprehensive correction module, and a bad pixel self-repair secondary optimization module: The pixel dynamic offset calculation module is used to acquire dark frame images, combine them with camera temperature data acquired in real time by temperature sensors, and calculate the pixel dynamic offset based on a temperature-time dual-domain prediction model. The flat field coefficient acquisition module is used to acquire the flat field coefficient, which is generated by a fusion algorithm from the laboratory calibration baseline value and the on-orbit uniform scene observation update value. The laboratory calibration baseline value is acquired by laboratory micro integrating sphere calibration, covering the full spectrum of the camera. The scene observation update value is generated by on-orbit observation of deep space or cloud top uniform scenes, combined with spatiotemporal joint analysis of bad spot detection, inconsistency difference detection and neighborhood interpolation processing. The pixel-level integrated correction module is used to perform pixel-level correction on the original image of the remote sensing camera based on the pixel dynamic offset and the flat field coefficient. During the correction process, satellite GPS position data, attitude angle data and real-time temperature data are integrated simultaneously to help optimize the correction accuracy. Attitude angle data is used to compensate for the pixel response deviation caused by camera side sway, and GPS position data is used to match the imaging parameters at different orbital altitudes. The bad pixel self-repair secondary optimization module is used to perform secondary optimization on the image after correction. The bad pixel self-repair module is based on the pixel gain compensation matrix output by the deep reinforcement learning model. The input of the deep reinforcement learning model includes real-time temperature, satellite attitude angle and pixel historical response curve. The network structure adopts a combination of convolutional neural network and long short-term memory network temporal unit. The output pixel gain compensation matrix is used to compensate for the response deviation of individual pixels caused by long-term on-orbit aging and radiation damage.
Citation Information
Patent Citations
A device and method for correcting the nonlinear response of pixel output in an image sensor.
CN103118235B
A method, system, electronic device, and medium for correcting pixel response nonuniformity.
CN115567651B
FPGA-based infrared image scene adaptive correction method and system
CN115222618A
On-orbit non-uniformity correction method for infrared scanning camera with detector and inner black body separated
CN115752757A