A kind of multispectral TDI CCD mosaic camera inner orientation element automatic calibration system and method

Abstract

An automatic calibration system and method for the interior orientation elements of a multispectral TDI CCD mosaic camera is disclosed. A CAN bus communication card module is used for communication between a high-performance server module and the camera; a program-controlled power module is used for powering on and off the camera; a high-precision turntable control module is used for controlling the turntable; a high-speed image acquisition card module is used for receiving high-speed image data output by the camera; a NI TestStand test management software module is used for editing, managing and running an automatic calibration sequence; a high-precision centroid extraction algorithm module is used for extracting image data spot centroids; a camera interior orientation element solving algorithm module is used for calculating the principal point, principal distance, distortion fitting curve and residual error of the distortion fitting curve of a test spectral band; an abnormal data automatic review algorithm module is used for reviewing and calculating the centroid, distortion fitting data and residual error; a MATLAB 2016a data processing software module is designed and developed; and a high-performance server module is used for adjusting the position of the turntable and monitoring the state of the turntable.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic calibration of camera internal orientation elements, and in particular to a system and method for automatic calibration of internal orientation elements of a multi-spectral TDI CCD splicing camera. Background Art

[0002] Accurate calibration of the camera's internal orientation elements is a prerequisite for high-precision mapping. The calibration of the camera's internal orientation elements in the laboratory is usually completed using the precision goniometer method. The calibration system consists of a high-precision turntable, a turntable control system, a collimator, an integrating sphere, a star point target, a camera control system, a camera to be tested, and a high-stability optical vibration isolation platform. The precision goniometer method uses the star point emitted by the light tube as the target target, rotates the turntable to enable the camera to image the infinitely distant target at different field angles, and uses the mathematical relationship between the target point imaging coordinates and the field angle to solve the camera's internal orientation elements. The basic principle diagram is shown below. Figure 1 As shown in the figure, O is the center of the image, point A is the theoretical image point of the image plane, A' is the actual image point, and N is the intersection of the optical axis and the image plane. Figure 1 The distortion formula is:

[0003] Δ i =(x i +x0)-f x tan(α i +θ) (1)

[0004] Among them, x0 is the coordinate of the principal point in the x direction, f x is the principal distance in the x direction, α i is the angle value of the turntable during calibration, x i In order to calibrate the x-direction coordinate value of the image point during the calibration process, the centroid of the spot image collected at different rotation angles of the turntable is extracted.

[0005] Since tanθ=x0 / f x , and the deviation of the principal point is generally small, so the value of θ is small, thus:

[0006]

[0007] Substituting formula (2) into formula (1), the distortion expression can be organized as:

[0008] Δ i =x i -f x tanα i -x0·tan 2 α i (3)

[0009] The interior orientation elements are solved based on the distortion square sum minimum algorithm. According to the least squares principle, the expression of the principal point and the principal distance can be obtained as follows:

[0010]

[0011]

[0012] Based on the basic principles of the precision goniometric method described above, the calibration of camera orientation elements requires obtaining image point positions at multiple angles. To ensure calculation accuracy, a camera typically requires obtaining image point positions at at least 60 different angles. For mass-produced cameras, each camera must be precisely calibrated before use. Manual image storage, turntable control, and position recording are cumbersome, requiring the collaborative work of multiple testers to complete the experiment. This results in low experimental efficiency and high labor costs. Traditional laboratory orientation element calibration data processing typically involves specialized data processing personnel analyzing and processing the data after the calibration test is completed. This results in slow processing and poor real-time performance. If there are any issues with the data collected during the test, feedback to the tester for retesting cannot be provided promptly, further increasing the time and labor costs of camera testing.

[0013] For multi-TDI CCD stitching cameras, the centroid data of multiple CCDs must be obtained before calculating intrinsic position elements. This centroid data is obtained by extracting the centroid of spot images captured at different turntable rotation angles. The accuracy of centroid extraction is affected by various factors, including image sensor noise, background noise, aberrations, and imaging angle. Directly extracting the centroid from the original image results in low centroid accuracy, further affecting the accuracy of the camera's intrinsic position element solution. Furthermore, the accuracy of centroid extraction at the spliced ​​CCD slices also affects the calculation of the number of CCD stitching pixels. Large errors in the measurement of the number of stitching pixels between CCDs will directly affect the camera's intrinsic position element solution, resulting in low reliability of the obtained camera distortion calibration data.

[0014] In summary, the existing camera internal orientation element calibration test method has a complicated and tedious testing process, low test and data processing efficiency, and the centroid data extracted from different spectral bands of the multispectral camera may contain large errors, affecting the calibration accuracy of the internal orientation elements. Summary of the Invention

[0015] The present invention solves the problems of the existing camera internal orientation element calibration test method, such as the complicated testing process, low test and data processing efficiency, and the possible large error in the centroid data extracted from different spectral bands of the multispectral camera, which affects the calibration accuracy of the internal orientation elements.

[0016] The present invention discloses an automatic calibration system for internal orientation elements of a multispectral TDI CCD splicing camera, comprising a CAN bus communication card module, a programmable power supply module, a high-precision turntable control module, a high-speed image acquisition card module, an NI TestStand test management software module, a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, an abnormal data automatic review algorithm module, a MATLAB 2016a data processing software module, and a high-performance server module.

[0017] The CAN bus communication card module is used to realize the communication between the high-performance server module and the camera;

[0018] The programmable power module controls the power on and off of the camera through the serial port. During the calibration process, the camera current is monitored in real time. If the camera current is abnormal, the programmable power module will power off and end the current task.

[0019] The high-precision turntable control module is used to control the turntable;

[0020] The high-speed image acquisition card module is used to receive high-speed image data output by the camera;

[0021] The NI TestStand test management software module is used to edit, manage, and run automated calibration sequences. The NI TestStand test management software module interacts with the MATLAB 2016a data processing software module to complete real-time processing and analysis of calibration data and generate test reports. The NI TestStand test management software module interacts with the high-speed image acquisition card module to complete image acquisition and storage requirements during the automated calibration process.

[0022] The high-precision centroid extraction algorithm module is used to extract the centroid of the image data spot;

[0023] The camera internal orientation element solution algorithm module calculates the principal point, principal distance, distortion fitting curve and residual of the distortion fitting curve of the current test spectrum segment according to the obtained centroid and turntable angle data and the minimum square distortion algorithm;

[0024] The abnormal data automatic review algorithm module is used to review and calculate the centroid, distortion fitting data and residuals of all spectral segments after the calibration of all spectral segments of the multispectral camera is completed;

[0025] The MATLAB 2016a data processing software module is used to implement the design and development of a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, and an abnormal data automatic review algorithm module, and communicates with the NI TestStand test management software module, accepting automatic calls from the NI TestStand test management software module to realize automatic processing of calibration data;

[0026] The high-performance server module communicates through the high-precision turntable control module to automatically adjust the turntable position and monitor the turntable status. The high-performance server module communicates with the high-speed image acquisition card module to acquire and store image data and complete image analysis according to the image data protocol. The high-performance server module provides an operating environment for the MATLAB 2016a data processing software module and the NI TestStand test management software module.

[0027] The present invention provides a method for automatically calibrating the internal orientation elements of a multispectral TDI CCD stitching camera, which is implemented using the automatic calibration system for the internal orientation elements of a multispectral TDI CCD stitching camera described in the above method, and includes the following steps:

[0028] Step S1: Set up the calibration environment and level the camera;

[0029] Step S2, preparing tables required for camera automated testing, the tables including a power parameter configuration table, an imaging parameter configuration table, and a turntable parameter configuration table;

[0030] Step S3: The camera automated test software reads the table in step S2 and completes the communication port settings between the programmable power module and the high-precision turntable control module according to the power parameter configuration table and the turntable parameter configuration table;

[0031] Step S4: After the camera is powered on, the imaging parameters of the camera are set according to the imaging parameter configuration table in step S2 through the CAN bus communication card module and the camera engineering parameter frame is read to confirm that the camera imaging parameter configuration is complete. The programmable power supply module continuously monitors the power-on status of the camera;

[0032] Step S5: According to the turntable parameter configuration table, the high-precision turntable control module sets the turntable to the first position and continuously reads the turntable's current position. If |current position - set value| ≤ 1", the turntable rotation is complete, and the high-speed image acquisition card module acquires image data at the current position. After image acquisition is complete, the high-precision turntable control module controls the turntable to rotate to the next position for image acquisition. This process continues until all positions in the turntable parameter configuration table have been traversed, completing the test task.

[0033] In step S6, the local background noise automatic extraction and subtraction method and the camera tap-by-tap fine calibration coefficient are used, and the calibration data of the current test task are processed separately in combination with the high-precision centroid extraction algorithm module and the camera internal orientation element solution algorithm module. The abnormal data automatic review algorithm module then verifies the processed data.

[0034] Furthermore, in one embodiment of the present invention, the setting up of the calibration environment and leveling of the camera are specifically as follows:

[0035] Level the turntable and collimator, fix the camera on the turntable, level the camera, and rotate the turntable to ensure that the star image emitted by the collimator is always formed on the camera target surface when the camera scans from one end of the field of view to the other end.

[0036] Furthermore, in one embodiment of the present invention, the local background noise automatic extraction and subtraction method is specifically:

[0037] The background noise region of the tap where the current light spot is located is selected, the background noise distribution characteristics are analyzed in the background noise region, and after the background noise is modeled, the established background noise model is deducted from the current image.

[0038] Furthermore, in one embodiment of the present invention, the high-precision centroid extraction algorithm module uses a Gaussian surface fitting method to extract the centroid.

[0039] Furthermore, in one embodiment of the present invention, the Gaussian surface fitting method establishes a model of the relationship between the grayscale value, position and center position of each pixel in the image area based on the Gaussian characteristics of the image grayscale distribution and the correlation between the grayscale value and height of each pixel in the image.

[0040] An electronic device according to the present invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0041] Memory for storing computer programs;

[0042] The processor is used to implement the method steps described in the above method when executing the program stored in the memory.

[0043] The computer-readable storage medium of the present invention stores a computer program, and when the computer program is executed by a processor, the method steps described in the above method are implemented.

[0044] This invention solves the problems of existing camera internal orientation element calibration test methods, such as the complex and cumbersome testing process, low test and data processing efficiency, and the potential for large errors in the centroid data extracted from different spectral bands of a multispectral camera, which affects the calibration accuracy of the internal orientation elements. Specific beneficial effects include:

[0045] The automated calibration system for intrinsic elements of a multispectral TDI CCD stitching camera, described in the present invention, utilizes a combined hardware and software design to achieve joint automated control of the camera and turntable. The system also features a one-click test software operation mode and automated, real-time calibration data processing, significantly improving the efficiency of camera intrinsic element calibration. Improvements to the centroid extraction and intrinsic element solution algorithms enhance the accuracy and data reliability of camera intrinsic element solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0047] Figure 1 This is the principle diagram of the internal orientation element calibration described in the background technology;

[0048] Figure 2 This is a diagram of an automatic calibration system for internal orientation elements of a multispectral TDI CCD splicing camera described in a specific embodiment;

[0049] Figure 3 This is a flow chart of an automatic calibration method for internal orientation elements of a multispectral TDI CCD splicing camera described in a specific implementation manner. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe various embodiments of the present invention in conjunction with the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0051] This embodiment provides an automated calibration system for internal position elements of a multispectral TDI CCD stitching camera, comprising a CAN bus communication card module, a programmable power supply module, a high-precision turntable control module, a high-speed image acquisition card module, an NI TestStand test management software module, a high-precision centroid extraction algorithm module, a camera internal position element solution algorithm module, an abnormal data automated review algorithm module, a MATLAB 2016a data processing software module, and a high-performance server module.

[0052] The CAN bus communication card module is used to realize the communication between the high-performance server module and the camera;

[0053] The programmable power module controls the power on and off of the camera through the serial port. During the calibration process, the camera current is monitored in real time. If the camera current is abnormal, the programmable power module will power off and end the current task.

[0054] The high-precision turntable control module is used to control the turntable;

[0055] The high-speed image acquisition card module is used to receive high-speed image data output by the camera;

[0056] The NI TestStand test management software module is used to edit, manage, and run automated calibration sequences. The NI TestStand test management software module interacts with the MATLAB 2016a data processing software module to complete real-time processing and analysis of calibration data and generate test reports. The NI TestStand test management software module interacts with the high-speed image acquisition card module to complete image acquisition and storage requirements during the automated calibration process.

[0057] The high-precision centroid extraction algorithm module is used to extract the centroid of the image data spot;

[0058] The camera internal orientation element solution algorithm module calculates the principal point, principal distance, distortion fitting curve and residual of the distortion fitting curve of the current test spectrum segment according to the obtained centroid and turntable angle data and the minimum square distortion algorithm;

[0059] The abnormal data automatic review algorithm module is used to review and calculate the centroid, distortion fitting data and residuals of all spectral segments after the calibration of all spectral segments of the multispectral camera is completed;

[0060] The MATLAB 2016a data processing software module is used to implement the design and development of a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, and an abnormal data automatic review algorithm module, and communicates with the NI TestStand test management software module, accepting automatic calls from the NI TestStand test management software module to realize automatic processing of calibration data;

[0061] The high-performance server module communicates through the high-precision turntable control module to automatically adjust the turntable position and monitor the turntable status. The high-performance server module communicates with the high-speed image acquisition card module to acquire and store image data and complete image analysis according to the image data protocol. The high-performance server module provides an operating environment for the MATLAB 2016a data processing software module and the NI TestStand test management software module.

[0062] This embodiment is based on the multi-spectral TDI CCD splicing camera internal orientation element automatic calibration system described in the present invention, combined with Figure 2 To better understand this implementation, a practical implementation is provided:

[0063] The system includes a CAN bus communication card module, a serial port card module, a programmable power supply module, a high-precision turntable control module, a high-speed image acquisition card module, an NI TestStand test management software module, a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, an abnormal data automatic review algorithm module, a MATLAB 2016a data processing software module, and a high-performance server module;

[0064] The programmable power supply module can be controlled through the serial port to realize the power on and off of the camera and the real-time monitoring of the camera current during the calibration process. If an abnormal current is detected, the power is immediately turned off to end the current task;

[0065] The CAN bus communication card module is used to realize the communication between the high-performance server module and the camera. The high-performance server module sends commands to the camera through the CAN bus communication card module and performs parameter verification based on the telemetry commands returned by the camera.

[0066] The high-precision turntable control module and serial port card module are used to control the turntable. The high-performance server module communicates with the high-precision turntable control module via RS485 to automatically adjust the turntable's position and monitor its status. Specifically, to capture image data corresponding to a specific field of view angle, the turntable receives position adjustment instructions from the server, which then continuously queries the turntable's position. When the difference between the turntable's current position and the set position value is less than the turntable's accuracy index, the high-speed image acquisition card module is controlled to complete image acquisition.

[0067] The high-speed image acquisition card module is used to receive high-speed image data output by the camera. The high-performance server communicates with the image acquisition card via the PCIE bus to acquire and store image data, and completes image analysis according to the image data protocol. Due to the large amount of data required by high-resolution cameras, high requirements are placed on the real-time reception and analysis of images. To meet real-time requirements, an image acquisition card is designed using an FPGA + high-speed serial interface protocol solution, which can achieve real-time reception of 600MB / s data rate and perform spectral image analysis based on the image data format.

[0068] The NI TestStand test management software module is the core for camera internal orientation element calibration and is used to edit, manage, and run automated calibration sequences. The NI TestStand development environment enables code module development, code writing, and interface converter driver invocation. It also interacts with the MATLAB 2016a data processing software module via an ActiveX interface to perform real-time processing, analysis, and test report generation of calibration data. Specifically, the NI TestStand test management software module interacts with the image acquisition software via TCP communication to meet the image acquisition and storage requirements of the automated calibration process.

[0069] The high-precision centroid extraction algorithm module is used to realize the centroid extraction of the image data spot, and the Gaussian surface fitting method is used to extract the centroid position. Specifically, laboratory centroid extraction usually adopts the traditional centroid method or the square weighted centroid method. Both algorithms are based on the grayscale values ​​of the feature point pixels for correlation operations, which are simple to implement and have a small amount of calculation. However, when the imaging angle changes and causes the spot distribution to change, it will cause a certain error in its centroid extraction. The Gaussian surface fitting method is based on the Gaussian characteristics of the image grayscale distribution and the correlation between the grayscale value and height of each pixel in the image. It establishes a model of the relationship between the grayscale value, position and center position of each pixel in the image area, and uses the Gaussian surface fitting formula to determine the coordinates of the centroid position of the spot in the image. The determination of its centroid position does not only rely on the grayscale value of the spot pixel, but also reduces the influence of the change in spot distribution caused by the change in imaging angle on the accuracy of centroid extraction;

[0070] Aiming at the image dark field non-uniformity and dark current noise caused by temperature during the imaging process, an automatic extraction and subtraction method of local background noise based on the CCD tap where the current light spot is located is proposed. The background noise area of ​​the tap where the current light spot is located is automatically selected, the background noise distribution characteristics are analyzed within the background noise area, and the background noise is modeled and subtracted from the current image to reduce the influence of image dark field non-uniformity, dark current noise and circuit noise on centroid extraction.

[0071] Aiming at the influence of CCD fixed pattern noise and bright field non-uniformity caused by inconsistent pixel light response on centroid extraction, a tap-by-tap fine calibration method is proposed to obtain relative calibration coefficients. The relative calibration coefficients are used for calibration data preprocessing, which can more effectively eliminate the influence of CCD fixed pattern noise and camera bright field non-uniformity on the accuracy of centroid extraction.

[0072] For the image data after background noise subtraction and non-uniformity correction, the Gaussian surface fitting method is used to obtain the calibration data centroid, eliminating the influence of different camera imaging angles on centroid extraction and further improving the accuracy of centroid extraction.

[0073] The camera internal orientation element solution algorithm module calculates the principal point, principal distance, distortion fitting curve and residual of the distortion fitting curve of the current test spectrum segment according to the obtained centroid and turntable angle data and the minimum square distortion algorithm;

[0074] The abnormal data automatic review algorithm module is used to calculate the centroid, distortion fitting data and residuals of all spectral segments after the calibration of all spectral segments of the multispectral camera is completed. When the distortion fitting order or fitting residual exceeds the set value, it is considered that there is an abnormal value in the distortion data of the current spectral segment. The location of the abnormal value is determined again. If this data is located near the overlap area between CCD slices, it can be determined that the abnormality is caused by inaccurate overlap data. Compare the test results of all spectral segments, take the overlap data of spectral segment 1 with the smallest residual value as the overlap data of spectral segment 2 whose residual exceeds the limit, and re-solve the distortion data of spectral segment 2 to obtain relatively accurate distortion data results. If the abnormal value is not located near the overlap area between CCD slices, the abnormal data is eliminated, and similarly, the distortion data of spectral segment 2 is re-solved. In this way, the accuracy of the distortion data solution result can be further improved;

[0075] The MATLAB 2016a data processing software module is key to analyzing camera internal orientation element data. It is used to implement the design and development of the high-precision centroid extraction algorithm module, the camera internal orientation element solution algorithm module, and the abnormal data automatic review algorithm module. It also communicates with the NI TestStand test management software module through an ActiveX interface and accepts automatic calls from the NI TestStand test management software module to realize automatic processing of calibration data.

[0076] The high-performance server module is an operating platform for the automated calibration and analysis system, providing a high-performance operating environment for automated calibration of internal orientation elements and algorithm design and implementation.

[0077] The method for automatically calibrating the internal position elements of a multispectral TDI CCD stitching camera described in this embodiment is implemented using the automatic calibration system for the internal position elements of a multispectral TDI CCD stitching camera described in the above embodiment, and includes the following steps:

[0078] Step S1: Set up the calibration environment and level the camera;

[0079] Step S2, preparing tables required for camera automated testing, the tables including a power parameter configuration table, an imaging parameter configuration table, and a turntable parameter configuration table;

[0080] Step S3: The camera automated test software reads the table in step S2 and completes the communication port settings between the programmable power module and the high-precision turntable control module according to the power parameter configuration table and the turntable parameter configuration table;

[0081] Step S4: After the camera is powered on, the imaging parameters of the camera are set according to the imaging parameter configuration table in step S2 through the CAN bus communication card module and the camera engineering parameter frame is read to confirm that the camera imaging parameter configuration is complete. The programmable power supply module continuously monitors the power-on status of the camera;

[0082] Step S5: According to the turntable parameter configuration table, the high-precision turntable control module sets the turntable to the first position and continuously reads the turntable's current position. If |current position - set value| ≤ 1", the turntable rotation is complete, and the high-speed image acquisition card module acquires image data at the current position. After image acquisition is complete, the high-precision turntable control module controls the turntable to rotate to the next position for image acquisition. This process continues until all positions in the turntable parameter configuration table have been traversed, completing the test task.

[0083] In step S6, the local background noise automatic extraction and subtraction method and the camera tap-by-tap fine calibration coefficient are used, and the calibration data of the current test task are processed separately in combination with the high-precision centroid extraction algorithm module and the camera internal orientation element solution algorithm module. The abnormal data automatic review algorithm module then verifies the processed data.

[0084] In this embodiment, the steps of setting up the calibration environment and leveling the camera are as follows:

[0085] Level the turntable and collimator, fix the camera on the turntable, level the camera, and rotate the turntable to ensure that the star image emitted by the collimator is always formed on the camera target surface when the camera scans from one end of the field of view to the other end.

[0086] In this embodiment, the local background noise automatic extraction and subtraction method is specifically as follows:

[0087] The background noise region of the tap where the current light spot is located is selected, the background noise distribution characteristics are analyzed in the background noise region, and after the background noise is modeled, the established background noise model is deducted from the current image.

[0088] In this embodiment, the high-precision centroid extraction algorithm module uses Gaussian surface fitting method to extract the centroid.

[0089] In this embodiment, the Gaussian surface fitting method is based on the Gaussian characteristics of the image grayscale distribution and the correlation between the grayscale value and height of each pixel in the image to establish a model of the relationship between the grayscale value and position of each pixel in the image area and the center position of the pixel.

[0090] This embodiment is based on the automatic calibration method of the internal orientation elements of a multi-spectral TDI CCD splicing camera described in the present invention, combined with Figure 3 To better understand this implementation, a practical implementation is provided:

[0091] Step 1. Set up the calibration environment and level the camera.

[0092] Before calibration begins, first level the turntable and collimator. Then, secure the camera to be tested on a high-precision turntable and level the camera. Rotate the turntable to ensure that the star image emitted by the collimator remains on the camera target as the camera scans from one end of the field of view to the other.

[0093] Specifically, the multispectral TDI CCD stitching camera used for calibration in this example includes five spectral bands: P, B1, B2, B3, and B4. It is composed of three TDI CCD optical stitching elements and has an X-direction field of view of (-0.95°, 0.95°). The high-precision turntable used has a control accuracy of ±1″.

[0094] In order to improve the signal-to-noise ratio of the light spot in the image and reduce the influence of background noise, dark current noise, etc. on the accuracy of light spot centroid extraction, the brightness of the integrating sphere is adjusted to ensure that the grayscale value of the light spot in the tested spectral band is within 50% to 80% of its saturation grayscale value.

[0095] Step 2. Prepare for camera automated testing.

[0096] Prepare the forms required for camera automation testing, including:

[0097] (1) Power parameter configuration table: The automated test sequence sets the COM port for communication with the programmable power supply, the programmable power supply port, the camera's supply voltage, supply current, voltage limit, and current limit according to this table;

[0098] (2) Imaging parameter configuration table: The automated test sequence sets the TDI integration level and gain of different spectral bands of the camera according to this table;

[0099] (3) Turntable Parameter Configuration Table: The automated test sequence uses this table to set the COM port for communication with the high-precision turntable and the turntable position sampling points used for calibration. In this example, the turntable angle range is set to (-0.9°, 0.9°), with sampling points spaced 0.03° apart, for a total of 61 sampling points.

[0100] Step 3. Camera automated testing.

[0101] The automated testing process was developed based on NI TestStand test management software. After the test system was built, the automated testing software was run.

[0102] (1) The automated test software first reads the test table described in step 2 and completes the communication port settings between the programmable power module and the high-precision turntable module according to the power parameter configuration table and the turntable parameter configuration table;

[0103] (2) Power on the camera and configure the imaging parameters. Read the camera engineering parameter frame through the CAN bus communication card module interface to confirm that the camera imaging parameter configuration is complete. During this process, the programmable power supply module continuously monitors the camera power-on status, including the power supply voltage and current. If the limits are exceeded, the camera power-on is determined to be abnormal, and the programmable power supply module is controlled to power off, ending the current task.

[0104] (3) According to the turntable parameter configuration table, set the first position of the turntable and continuously read the current position of the turntable through RS485. If |current position-set value|≤1″, the turntable rotation is completed and the high-speed image acquisition card module starts to collect image data at the current position. After the image acquisition is completed, the high-precision turntable control module controls the turntable to rotate to the next position for image acquisition, and so on, until all positions in the turntable parameter configuration table are traversed, and the current test task is completed.

[0105] (4) Start the data processing software module based on MATLAB 2016a to process the calibration data of the current test task.

[0106] First, a local automatic extraction and subtraction method for background noise based on the current image is proposed. This algorithm is then used to process the calibration raw data to reduce the impact of image dark field non-uniformity and temperature-induced dark current noise on centroid extraction. Specifically, the column col containing the maximum grayscale value of the current image is obtained. Considering that the grayscale values ​​of the CCD dark field image vary significantly at different taps and locations due to CCD reset noise and image dark field non-uniformity, the number of taps in col is determined and the background noise window size (L0, W0) is designed. Pixels other than the light spot location are not illuminated and can be considered background noise in the current image. Within the tap range of the image's maximum grayscale value, the image outside the light spot range is taken as the background noise image range in the current tap. Within the background noise image region, a background noise window region (L0, W0) is taken near the light spot location. This background noise includes image dark field non-uniformity, dark current noise, and circuit noise. The background noise distribution characteristics are analyzed within the background noise window, and a background noise model is constructed. The established background noise model is then subtracted from the current image f(x, y). This can reduce the impact of image dark field non-uniformity and dark current noise on centroid extraction.

[0107] Furthermore, the camera relative calibration coefficient is used to process the data after deducting the background noise, so as to eliminate the influence of the camera bright field non-uniformity on the centroid extraction accuracy. The camera relative calibration coefficient is obtained by analyzing the camera radiation calibration data. Specifically, a laboratory 1.2m integrating sphere is used as a reference radiation source. The light output aperture of the integrating sphere can cover the entire field of view of the camera, the spectral range is 400nm-2500nm, the surface uniformity is better than 98%, and the radiant brightness instability during the calibration work is less than 1% / 1h. The camera is used to collect image data of the integrating sphere at different brightness levels. Usually, the relative calibration coefficient uses the response column mean at each brightness level as the horizontal coordinate and the image mean as the vertical coordinate. The least squares method is used for linear fitting to obtain the slope and intercept of the fitting as the relative calibration coefficient of each column of pixels under the current working conditions. A calibration algorithm for fine calibration of different CCD taps is proposed. The mean of each tap is calculated according to the calibration data, and the column mean and the tap mean are linearly fitted according to the tap where the current pixel column is located to obtain the fitting coefficient of the current column, thereby improving the accuracy of the relative calibration coefficient.

[0108] Based on the relative calibration coefficients obtained through tap-by-tap fine calibration, the calibration data is preprocessed using Equation 6. Where DN′ is the preprocessed pixel value, k and b are the slope and intercept of the relative calibration coefficient, respectively, and DN0 is the pixel value after subtracting dark field noise. This preprocessing eliminates the effects of bright field non-uniformity on centroid extraction accuracy.

[0109] DN′=k×DN0+b (6)

[0110] Furthermore, the centroid data of the relatively corrected image is extracted using a Gaussian surface fitting method. Specifically, the image column mean is first calculated, and the column with the maximum column mean is obtained, colmax. The image grayscale column mean is obtained within the range (colmax±50). A grayscale threshold, thresh, is set. When the image grayscale column mean in the region is greater than thresh, it is considered the location of the bright spot. Coordinate values ​​pix1 and pix2 are obtained on either side of colmax. Pixels in the column with a mean greater than pix2 or less than pix1 do not meet the set grayscale threshold. In this case, the bright spot is located between (pix1, pix2). The region where the bright spot is located is intercepted as colarea = g(1:row, pix1:pix2), where g is the relatively corrected image and row is the total number of rows in the original image. The column mean of colarea is calculated, and the centroid data is extracted using a Gaussian surface fitting method.

[0111] Furthermore, based on the basic principles of the goniometric method and the distortion calculation formula, the principal point and principal distance of the camera under test are calculated using the distortion sum-of-squares minimum algorithm, and the camera distortion and distortion curve fitting residuals are given. Specifically, the principal point and principal distance are calculated according to formulas (4) and (5), and the distortion is calculated according to formula (3) using the least squares principle under the condition of minimizing the residual error sum-of-squares. The distortion curve fitting residual is the difference between the function value at each point of the fitting curve and the measured distortion data.

[0112] Step 4. Verify the internal orientation element solution data.

[0113] After completing the calibration of the internal orientation elements of all spectral segments of the camera under test, the abnormal data automatic review algorithm module verifies the internal orientation element solution data.

[0114] Specifically, first read the centroid and distortion fitting data and residuals of all spectral segments obtained in step 3. In this example, when the distortion fitting order reaches 10 and the fitting residual is still greater than 1.5 pixels, it is considered that there is an abnormal value in the distortion data of the current spectral segment. Read the location of the centroid data of the abnormal residual. If the centroid position is located in the overlap area between CCD slices, it can be determined that this abnormality is caused by inaccurate overlap data. Compare the test results of all spectral segments, take the overlap data of the spectral segment with the smallest residual value as the overlap data of spectral segment 2 whose residual exceeds the limit, re-solve the distortion data of spectral segment 2, and obtain relatively accurate distortion data results. If the abnormal value is not located near the overlap area between CCD slices, it is considered that this data is unusable and needs to be eliminated, and then the distortion data of spectral segment 2 is re-solved. After completing the data verification, all data results are integrated, and the programming interface technology of Word is used to call its COM component to generate the camera internal orientation element calibration report.

[0115] In summary, by improving the test process and data processing algorithm, the proposed camera intrinsic orientation element automatic calibration algorithm, device and system effectively improve the camera intrinsic orientation element calibration efficiency, improve the data solution accuracy, reduce the testing cost, and achieve high-accuracy and real-time analysis of the intrinsic orientation element calibration data.

[0116] An electronic device described in this embodiment includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0117] Memory for storing computer programs;

[0118] The processor is configured to implement the method steps described in the above embodiment when executing the program stored in the memory.

[0119] This embodiment describes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method steps described in the above embodiment are implemented.

[0120] The memory in the embodiments of the present application can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to comprise, but not be limited to, these and any other suitable types of memory.

[0121] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired, such as coaxial cable, optical fiber, digital subscriber line (DSL) or wireless, such as infrared, wireless, microwave, etc. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media, such as floppy disks, hard disks, magnetic tapes, optical media, such as digital video discs (DVD), or semiconductor media, such as solid state discs (SSD), etc.

[0122] In the implementation process, each step of the above method can be completed by integrated logic circuit of hardware in the processor or instruction in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution or combined with hardware and software modules in the processor to complete the execution. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0123] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-described method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-described processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-described method.

[0124] The above describes in detail the system and method for automatic calibration of internal orientation elements of a multispectral TDI CCD stitching camera proposed in the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the method and core concept of the present invention. At the same time, for those skilled in the art, according to the concept of the present invention, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. An automatic calibration system for internal position elements of a multi-spectral TDI CCD splicing camera, characterized by: The system includes a CAN bus communication card module, a programmable power supply module, a high-precision turntable control module, a high-speed image acquisition card module, an NI TestStand test management software module, a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, an abnormal data automatic review algorithm module, a MATLAB 2016a data processing software module, and a high-performance server module. The CAN bus communication card module is used to realize the communication between the high-performance server module and the camera; The programmable power module controls the power on and off of the camera through the serial port. During the calibration process, the camera current is monitored in real time. If the camera current is abnormal, the programmable power module will power off and end the current task. The high-precision turntable control module is used to control the turntable; The high-speed image acquisition card module is used to receive high-speed image data output by the camera; The NI TestStand test management software module is used to edit, manage, and run automated calibration sequences. The NI TestStand test management software module interacts with the MATLAB 2016a data processing software module to complete real-time processing and analysis of calibration data and generate test reports. The NI TestStand test management software module interacts with the high-speed image acquisition card module to complete image acquisition and storage requirements during the automated calibration process. The high-precision centroid extraction algorithm module is used to extract the centroid of the image data spot; The camera internal orientation element solution algorithm module calculates the principal point, principal distance, distortion fitting curve and residual of the distortion fitting curve of the current test spectrum segment according to the obtained centroid and turntable angle data and the minimum square distortion algorithm; The abnormal data automatic review algorithm module is used to review and calculate the centroid, distortion fitting data and residuals of all spectral segments after the calibration of all spectral segments of the multispectral camera is completed; The MATLAB 2016a data processing software module is used to implement the design and development of a high-precision centroid extraction algorithm module, a camera internal orientation element solution algorithm module, and an abnormal data automatic review algorithm module, and communicates with the NI TestStand test management software module, accepting automatic calls from the NI TestStand test management software module to realize automatic processing of calibration data; The high-performance server module communicates with the high-precision turntable control module to automatically adjust the turntable position and monitor its status. The high-performance server module communicates with the high-speed image acquisition card module to collect and store image data and perform image analysis according to the image data protocol. The high-performance server module provides an operating environment for the MATLAB 2016a data processing software module and the NI TestStand test management software module. First, obtain the column col where the maximum grayscale value of the current image is located, determine the number of taps where col is located, and design the background noise window size (L0, W0). Within the tap range where the maximum grayscale value of the image is located, take the image outside the light spot range as the background noise image range. In the background noise image area, take the background noise window area (L0, W0) near the light spot position. This background noise includes image dark field non-uniformity, dark current noise, and circuit noise. Analyze the background noise distribution characteristics in the background noise window, and model the background noise. Use the current image f(x, y) to subtract the established background noise model.

2. A method for automatically calibrating internal position elements of a multispectral TDI CCD splicing camera, the method being implemented using the automatic calibration system for internal position elements of a multispectral TDI CCD splicing camera according to claim 1, characterized in that: The following steps are involved: Step S1: Set up the calibration environment and level the camera; Step S2, preparing tables required for camera automated testing, the tables including a power parameter configuration table, an imaging parameter configuration table, and a turntable parameter configuration table; Step S3: The camera automated test software reads the table in step S2 and completes the communication port settings between the programmable power module and the high-precision turntable control module according to the power parameter configuration table and the turntable parameter configuration table; Step S4: After the camera is powered on, the imaging parameters of the camera are set according to the imaging parameter configuration table in step S2 through the CAN bus communication card module and the camera engineering parameter frame is read to confirm that the camera imaging parameter configuration is complete. The programmable power supply module continuously monitors the power-on status of the camera; Step S5: According to the turntable parameter configuration table, the high-precision turntable control module sets the turntable to the first position and continuously reads the turntable's current position. If |current position - set value| ≤ 1", the turntable rotation is complete, and the high-speed image acquisition card module acquires image data at the current position. After image acquisition is complete, the high-precision turntable control module controls the turntable to rotate to the next position for image acquisition. This process continues until all positions in the turntable parameter configuration table have been traversed, completing the test task. In step S6, the local background noise automatic extraction and subtraction method and the camera tap-by-tap fine calibration coefficient are used, and the calibration data of the current test task are processed separately in combination with the high-precision centroid extraction algorithm module and the camera internal orientation element solution algorithm module. The abnormal data automatic review algorithm module then verifies the processed data.

3. The method for automatic calibration of internal orientation elements of a multispectral TDI CCD splicing camera according to claim 2, characterized in that: The calibration environment is built and the camera is leveled as follows: Level the turntable and collimator, fix the camera on the turntable, level the camera, and rotate the turntable to ensure that the star image emitted by the collimator is always formed on the camera target surface when the camera scans from one end of the field of view to the other end.

4. The method for automatic calibration of internal orientation elements of a multispectral TDI CCD splicing camera according to claim 2, characterized in that: The local background noise automatic extraction and subtraction method is specifically as follows: The background noise region of the tap where the current light spot is located is selected, the background noise distribution characteristics are analyzed in the background noise region, and after the background noise is modeled, the established background noise model is deducted from the current image.

5. The method for automatic calibration of internal orientation elements of a multispectral TDI CCD splicing camera according to claim 2, characterized in that: The high-precision centroid extraction algorithm module uses a Gaussian surface fitting method to extract the centroid.

6. The method for automatic calibration of internal orientation elements of a multispectral TDI CCD splicing camera according to claim 5, characterized in that: The Gaussian surface fitting method is based on the Gaussian characteristics of the image grayscale distribution and the correlation between the grayscale value and height of each pixel in the image, to establish a model of the relationship between the grayscale value and position of each pixel in the image area and the center position of the pixel.

7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 2 to 6 when executing a program stored in a memory.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 2 to 6 is implemented.

Images