A high-precision image registration precision fusion device and testing method

Abstract

The application discloses a high-precision fused image registration precision testing device and a testing method, relates to the field of optical metrology, and adopts the technical scheme that the testing device comprises a target imaging unit, an image acquisition unit and a control unit, the testing method comprises the steps of debugging the testing device, determining a plurality of test points, positioning test point coordinates and target image center coordinates in different imaging modes, sequentially acquiring turntable offset coordinates of the plurality of test points, and calculating offset distances of each test point and total offset absolute distances to evaluate registration precision. The application can perform standard testing on the registration precision of real machine fused images of different types and models of measured equipment, does not need to export imaging data of the measured equipment, is simple and convenient in the testing process, the testing precision can reach 1 pixel level, and the selection of multiple test points can comprehensively evaluate the registration precision.

Description

Technical Field

[0001] This invention relates to the field of optical metrology, and in particular to a high-precision fusion image registration accuracy testing device and method. Background Technology

[0002] Image fusion technology plays a crucial role in military, security, and surveillance fields. Fusion of images from different modalities can effectively improve image clarity and discernibility. For example, in night vision devices, images obtained by fusing enhanced weak light sources and infrared signals are fused, enabling users to obtain clear visual information in low-light environments. However, in practical applications, imaging devices such as night vision devices and infrared detectors often need to operate in different spectral modes, such as low-light and infrared imaging modes. Due to differences in optical characteristics and environmental conditions, the registration accuracy of the fused image becomes a decisive factor.

[0003] Traditional registration accuracy testing involves static testing, which involves exporting images obtained from different modalities of the device under test to professional testing software for analysis. In reality, this tests the product's registration algorithm rather than conducting real-time testing on the actual device. Moreover, there is currently no method for testing the registration accuracy of a physical device. Summary of the Invention

[0004] To address the problem that existing technologies lack methods for testing the registration accuracy of actual devices under test, this invention provides a high-precision fusion image registration accuracy testing device and method.

[0005] This invention provides the following technical solution: a high-precision fusion image registration accuracy testing device, comprising:

[0006] The target imaging unit includes a target, a test target, a reflector for reflecting the target image onto the test target, and a light source, wherein the light source includes a visible light source and an infrared radiation source.

[0007] The image acquisition unit includes a dual-axis turntable, the center of which is located on a vertical plane where the center of the target image on the test target is located, and the vertical plane is perpendicular to the test target; the dual-axis turntable is provided with a first clamp and a second clamp, the first clamp is disposed between the second clamp and the test target, and the second clamp is provided with an image acquisition device;

[0008] The control unit includes a central processing module, a motion control module and a data processing module electrically connected to the central processing module. The motion control module is electrically connected to the dual-axis turntable, and the central processing module is electrically connected to the image acquisition device.

[0009] Preferably, the target is a cross-shaped target.

[0010] Preferably, both the first clamp and the second clamp include a three-axis robotic arm and a fixing device disposed on the three-axis robotic arm.

[0011] A method for testing the registration accuracy of high-precision fused images includes the following steps:

[0012] Step 1: Fix the device under test to the first fixture and perform debugging. After debugging, the target image is located in the center of the imaging screen of the device under test, and the image acquisition device acquires the entire imaging screen of the device under test.

[0013] Step 2: Acquire the image from the image acquisition device and determine multiple test points within the boundary of the image from the device under test;

[0014] Step 3: In the single-modal imaging mode of the device under test, locate the coordinates of a test point and the center coordinates of the target image. Calculate the turntable offset angle and rotate the dual-axis turntable at least once until the test point coincides with the center of the target image. Switch to the fusion modal imaging mode, relocate the coordinates of the test point and the center coordinates of the target image, calculate the turntable offset angle, and rotate the dual-axis turntable at least once again until the test point coincides with the center of the target image. Record the turntable offset coordinate P from the center of the target image in the single-modal imaging mode to the center of the target image in the fusion modal imaging mode. n (x n ,y n ); Sequentially obtain the turntable offset coordinates of multiple test points, and calculate the offset distance l of each test point. n The formulas for calculating the total offset absolute distance L are as follows:

[0015]

[0016]

[0017] In the formula, x n y n These represent the horizontal and pitch offset distances of the dual-axis turntable as it moves from the center of the target image in the single-modality imaging mode to the center of the target image in the fusion-modality imaging mode.

[0018] Step 4: Evaluate the registration accuracy based on the total absolute offset distance. The smaller the total absolute offset distance, the higher the registration accuracy.

[0019] Preferably, in step 1, the debugging specifically includes:

[0020] Step 11: Use a reflector to reflect the target image onto the test target, and adjust the brightness and temperature of the target image to ensure that the image of the device under test is clear;

[0021] Step 12: Adjust the first fixture and set the device under test at the center of the dual-axis rotary table;

[0022] Step 13: Adjust the offset angle of the dual-axis turntable in the horizontal and vertical directions to ensure that the target image is located in the center of the imaging screen of the device under test;

[0023] Step 14: Adjust the second fixture to ensure that the image acquisition device captures the entire image of the device under test.

[0024] Preferably, in step 2, the test points include an initial center point, an upper center point, a lower center point, a left center point, and a right center point. The method for determining the test points is as follows:

[0025] Step 21: Acquire the image from the image acquisition device;

[0026] Step 22: Draw a horizontal rectangle and a vertical rectangle within the image of the device under test. One intersection of the horizontal rectangle and the vertical rectangle coincides with the center of the image of the device under test, and the intersection is taken as the initial center point.

[0027] Step 22: Draw a positioning rectangle within the image frame of the device under test. The length and width of the frame border of the device under test's image frame are proportionally reduced to the length and width of the positioning rectangle, and the center of the positioning rectangle coincides with the center of the image frame of the device under test. Among the multiple intersection points of the horizontal rectangle and the positioning rectangle, the two intersection points that are collinear with the initial center point are the left center point and the right center point, respectively. Among the multiple intersection points of the vertical rectangle and the positioning rectangle, the two intersection points that are collinear with the initial center point are the upper center point and the lower center point, respectively.

[0028] Preferably, the positioning rectangle is a 3 / 4 rectangle of the frame of the image of the device under test.

[0029] Preferably, in step 3, the specific method for locating the center coordinates of the target image is as follows: the image acquisition device sequentially performs image binarization processing, target detection, and contour detection to obtain the target image contour and generate an outer rectangle, completes the extraction of the center point of the outer rectangle, and outputs the center coordinates of the target image.

[0030] Preferably, the method for extracting the center point specifically involves: after generating the circumscribed rectangle of the target image contour, identifying the coordinates E(x) of the lower left vertex of the circumscribed rectangle. e ,y e The target image center coordinates N(x), width w, and height h are calculated. N ,y N The calculation formula is:

[0031]

[0032] Preferably, in step 3, the method of aligning a test point with the center of the target image by rotating the dual-axis turntable specifically involves: based on the coordinates N(x) of the target image center... N ,y N ) and test point coordinates M(x) M ,y M ), calculate the horizontal offset angle and pitch offset angle that the dual-axis turntable needs to move, and control the dual-axis turntable to complete the overlap;

[0033] The formulas for calculating the horizontal offset angle and the pitch offset angle are as follows:

[0034]

[0035]

[0036] In the formula, This is the horizontal offset angle. α is the pitch offset angle, β is the horizontal scaling factor, L0 is the pitch scaling factor, and L0 is the straight-line distance from the device under test to the test target.

[0037] The beneficial effects of this invention are: this method can be used to perform standard tests on the registration accuracy of fused images of different types and models of devices under test, without the need to export the imaging data of the device under test. The testing process is simple and convenient, the testing accuracy can reach the level of 1 pixel, and multiple test points can be selected to comprehensively evaluate the registration accuracy. Attached Figure Description

[0038] Figure 1 This is an overall schematic diagram of one embodiment of the testing apparatus.

[0039] Figure 2 This is a schematic diagram of the image acquisition device's imaging screen in one embodiment of the test method.

[0040] Figure 3 This is a flowchart illustrating the test point offset coordinate test of one embodiment of the test method.

[0041] Figure 4 This is a flowchart illustrating the target image localization process of one embodiment of the testing method.

[0042] Reference numerals: 11-Light source, 12-Target, 13-Reflector, 14-Test target, 21-Device under test, 22-Image acquisition device, 23-Dual-axis turntable, 24-First fixture, 25-Second fixture, 30-Control unit. Detailed Implementation

[0043] The embodiments of the present invention will be described in more detail below with reference to the accompanying drawings and reference numerals, so that those skilled in the art can implement them after reading this specification. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0044] Example 1

[0045] Please refer to Figure 1 This embodiment provides a high-precision fusion image registration accuracy testing device, including a target imaging unit, an image acquisition unit, and a control unit.

[0046] The target imaging unit includes a light source 11, a target 12, a reflector 13, and a test target 14. Various types of targets 12 are available, with crosshairs providing a clearer view of the center point—the intersection of the horizontal and vertical lines—facilitating identification by the device under test. The light source 11 illuminates the target 12 to form an image. Specifically, the light source 11 integrates a visible light source and an infrared radiation source, used to adjust the brightness and temperature of the target image for easier identification by the device under test. The side of the test target 14 facing the image acquisition unit is a plane mirror. The image formed by the light source 11 illuminating the target 12 is reflected by the reflector 13 and the test target 14 to the device under test.

[0047] The image acquisition unit includes a dual-axis turntable 23 for driving the device under test 21 and the image acquisition device 22 to perform horizontal and pitch offsets. The center of the dual-axis turntable 23 is located on the vertical plane where the center of the target image on the test target 14 is located, and the vertical plane is perpendicular to the test target, which simplifies the subsequent calculation process. The center of the dual-axis turntable is the intersection of the horizontal rotation axis and the pitch rotation axis of the dual-axis turntable. The dual-axis turntable 23 is provided with a first clamp 24 and a second clamp 25. The first clamp 24 is disposed between the second clamp 25 and the test target 14 for fixing the device under test 21, and the second clamp 25 is provided with the image acquisition device 22. Specifically, the first clamp 24 and the second clamp 25 both include a three-axis robotic arm and a fixing device disposed on the three-axis robotic arm. The three-axis robotic arm can accurately adjust the planar position and height of the device under test and the image acquisition device on the dual-axis turntable. The fixing device is used to position the device under test and the image acquisition device, and can be a clamp or a restraint strap. The device under test 21 is a night vision device, infrared detector, or other device with multiple imaging modes such as single-modal imaging and fused modal imaging; the image acquisition device 22 can be a high-definition camera used to monitor the imaging process of the device under test on the target.

[0048] The control unit 30 includes a central processing module and a motion control module and a data processing module electrically connected to the central processing module. The central processing module and motion control module can be microcontrollers. The central processing module is used for comprehensive processing of various types of data and communication. The motion control module is electrically connected to the dual-axis turntable and is used to control the movement of the dual-axis turntable and feed back offset data to the data processing module through the central processing module. The data processing module uses a computer or server with data input / output, calculation, and storage capabilities. Data acquired by the image acquisition device and the motion control module is transmitted to the data processing module for storage and calculation through the central processing module.

[0049] Example 2

[0050] This embodiment provides a testing method for the high-precision fusion image registration accuracy testing device as described in Embodiment 1, including the following steps.

[0051] Step 1: Fix the device under test (DUT) to the first fixture. The DUT is a night vision device with both single-modal imaging and fusion modal imaging modes. Perform the following adjustments to enable the DUT to observe the target at the center of its imaging field. Single-modal imaging is either visible light imaging or infrared imaging; fusion modal imaging combines the imaging results of visible light and infrared light using a fusion algorithm to generate a single image.

[0052] Step 11: Turn on the light source and use the reflector to reflect the target image onto the test target. Control the intensity of the light source to adjust the brightness and temperature of the target image, ensuring that the target image is clear in the imaging screen of the device under test in multiple imaging modes.

[0053] Step 12: Adjust the first fixture to position the device under test at the center of the dual-axis turntable. The center of the dual-axis turntable is the intersection of the horizontal rotation axis and the pitch rotation axis of the dual-axis turntable. Positioning the device under test at the center of the dual-axis turntable generally means that the three-dimensional assembly center of the device under test coincides with the center of the dual-axis turntable.

[0054] Step 13: Adjust the offset angle of the dual-axis turntable in the horizontal and vertical directions to ensure that the target image on the test target is located in the center of the imaging screen of the device under test, and adjust the imaging parameters of the device under test to ensure that the target image is clear in the imaging screen of multiple imaging modes.

[0055] Step 14: Adjust the focal length, aperture, and other imaging parameters of the second fixture and the image acquisition device to ensure that the image acquisition device captures a complete and clear image of the device under test. Preferably, the image of the device under test occupies the center of the image acquisition device's image, and the centers of the two coincide.

[0056] After debugging, the device under test is located at the center of the dual-axis turntable. The target image in its imaging screen is clear and located in the center of the device under test's imaging screen. The imaging screen of the image acquisition device is also clear, and the imaging screen of the device under test occupies the center of the imaging screen of the image acquisition device. The centers of the two coincide.

[0057] Step 2: The data processing module acquires the image from the image acquisition device and determines multiple test points within the boundary of the image from the device under test. In this embodiment, the test points include an initial center point, an upper center point, a lower center point, a left center point, and a right center point. Please refer to... Figure 2 The method for determining the test points is as follows.

[0058] Step 21: The data processing module acquires the image from the image acquisition device. Figure 2 The area displayed by the middle border 41 is the image captured by the image acquisition device, and the area displayed by the border 42 is the image captured by the device under test.

[0059] Step 22: The data processing module draws a horizontal rectangle 43 and a vertical rectangle 44 in the image frame of the image acquisition device within the image frame of the device under test. One intersection of the horizontal rectangle 43 and the vertical rectangle 44 coincides with the center of the image frame of the device under test, and the intersection is used as the initial center point for registration test.

[0060] Step 22: The data processing module draws a positioning rectangle 45 within the image frame of the device under test in the image acquisition device's image frame. The length and width of the frame of the device under test's image frame are proportionally reduced to the length and width of the positioning rectangle 45, and the center of the positioning rectangle 45 coincides with the center of the image frame of the device under test. In this embodiment, the positioning rectangle 45 is a 3 / 4 rectangle of the frame of the image frame of the device under test, and the length and width of the positioning rectangle 45 are 3 / 4 of the length and width of the frame of the image frame of the device under test, respectively.

[0061] Among the multiple intersections of the horizontal rectangle 43 and the positioning rectangle 45, the two intersections that are collinear with the initial center point are the left center point and the right center point, respectively; among the multiple intersections of the vertical rectangle 44 and the positioning rectangle 45, the two intersections that are collinear with the initial center point are the upper center point and the lower center point, respectively.

[0062] Step 3: Establish a planar coordinate system based on the image captured by the image acquisition device. The coordinates of the test points mentioned above are all known quantities. The testing device sequentially tests multiple test points according to a specified or random order, obtains the offset distance of the test points, and then combines the offset distances of multiple test points to obtain the total absolute offset distance to evaluate the registration accuracy of the fused image in the fused modal imaging mode of the device under test.

[0063] Specifically, please refer to Figure 3The method for detecting the offset distance of a single test point is as follows.

[0064] Step 31: In the single-modal imaging mode of the device under test, locate the coordinates of a test point and the center coordinates of the target image, calculate the turntable offset angle and rotate the dual-axis turntable at least once until the test point coincides with the center of the target image.

[0065] First, taking the initial center point as an example, its coordinates are M(x M ,y M ), and is a known quantity, while the target image center coordinates N(x) N ,y N ) through such Figure 4 The target image localization process shown is as follows: A frame of image acquisition equipment containing a target image is input to the data processing module. The image acquisition equipment's image is then subjected to binarization, target detection, and contour detection to obtain the target image contour. After generating the circumscribed rectangle of the target image contour, the coordinates E(x) of the lower left vertex of the circumscribed rectangle are obtained. e ,y e The center point of the circumscribed rectangle is extracted based on the following formula, given the width w and height h. The coordinates of the center point are then output, which are the center coordinates N(x) of the target image. N ,y N The calculation formula is:

[0066]

[0067] The binarization, target detection, contour detection, and bounding box generation can all refer to relevant technologies; in this embodiment, the target detection model is a YOLO v2 network trained on a self-made dataset. In target detection, if the target image is not identified, a new frame of the image captured by the image acquisition device is acquired.

[0068] Secondly, based on the target image center coordinates N(x) N ,y N ) and test point coordinates M(x) M ,y M ), calculate the required rotation angle of the dual-axis turntable, whereby the rotation angle includes horizontal and pitch angles, and the calculation formulas are as follows:

[0069]

[0070]

[0071] In the formula, This is the horizontal offset angle. α is the pitch offset angle, β is the horizontal scaling factor, L0 is the pitch scaling factor, and L0 is the straight-line distance from the center of the dual-axis turntable to the center of the test target.

[0072] Then, the horizontal offset angle and pitch offset angle are output to the motion control module to control the movement of the dual-axis turntable. Due to the existence of errors, it is necessary to reposition the target image center coordinates to check whether they coincide with the test point coordinates. If they do not coincide, this step is repeated multiple times until they coincide completely. The turntable coordinates at this point are then set as the origin.

[0073] The horizontal and pitch scaling factors can be obtained through multiple iterations using a binary search method. Specifically, taking the horizontal scaling factor α as an example, an initial value α0 is preset, and the horizontal offset angle is obtained through α0. After the dual-axis rotary table moves according to this horizontal angle, the relative position Δx between the test point and the center of the target image is recalculated. There are three possible scenarios:

[0074] (1) Δx = 0 indicates that the two coincide, and α0 does not need to be iterated and optimized again;

[0075] (2) Δx>0 indicates that the distance moved is too large, let Calculate the new horizontal offset angle and move again;

[0076] (3) Δx < 0 indicates that the distance moved is too small, let Calculate the new horizontal offset angle and move again.

[0077] Step 32: Switch the device under test to fusion modal imaging mode. Reposition the coordinates of the test point and the center coordinates of the target image according to the method in step 31. Calculate the turntable offset angle and rotate the dual-axis turntable. After at least one rotation, align the test point with the center of the target image again. Record the turntable coordinates at this point, which are the turntable offset coordinates P. n (x n ,y n ), where x n y n These represent the horizontal offset distance and the pitch offset distance, respectively, when the dual-axis turntable moves from the center of the target image in the single-modality imaging mode to the center of the target image in the fusion-modality imaging mode.

[0078] The turntable offset coordinates of multiple test points are obtained sequentially using the method described above, and the offset distance l of each test point is calculated. n The formulas for calculating the total offset absolute distance L are as follows:

[0079]

[0080]

[0081] Step 4: Evaluate the registration accuracy based on the total absolute offset distance. The smaller the total absolute offset distance, the higher the registration accuracy.

[0082] In other embodiments, step 3 may also involve manually locating the coordinates of the test point and the center coordinates of the target image using visual positioning, and manually controlling the movement of the dual-axis turntable to align the test point and the center of the target image, thereby obtaining the turntable offset coordinates P. n (x n ,y n ), thus obtaining the total absolute offset distance.

[0083] The above describes one or more embodiments of the present invention in a relatively specific and detailed manner, but it should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A high-precision fused image registration accuracy testing device, characterized in that, include: The target imaging unit includes a target, a test target, a reflector for reflecting the target image onto the test target, and a light source, wherein the light source includes a visible light source and an infrared radiation source. The image acquisition unit includes a dual-axis turntable, the center of which is located on a vertical plane where the center of the target image on the test target is located, and the vertical plane is perpendicular to the test target; the dual-axis turntable is provided with a first clamp and a second clamp, the first clamp is disposed between the second clamp and the test target, and the second clamp is provided with an image acquisition device; The control unit includes a central processing module, a motion control module and a data processing module electrically connected to the central processing module. The motion control module is electrically connected to the dual-axis turntable, and the central processing module is electrically connected to the image acquisition device.

2. The high-precision fusion image registration accuracy testing device according to claim 1, characterized in that: The target is a cross-shaped target.

3. The high-precision fusion image registration accuracy testing device according to claim 1, characterized in that: Both the first clamp and the second clamp include a three-axis robotic arm and a fixing device mounted on the three-axis robotic arm.

4. The testing device and testing method according to any one of claims 1 to 3, characterized in that, Includes the following steps: Step 1: Fix the device under test to the first fixture and perform debugging. After debugging, the target image is located in the center of the imaging screen of the device under test, and the image acquisition device acquires the entire imaging screen of the device under test. Step 2: Acquire the image from the image acquisition device and determine multiple test points within the boundary of the image from the device under test; Step 3: In the single-modal imaging mode of the device under test, locate the coordinates of a test point and the center coordinates of the target image. Calculate the turntable offset angle and rotate the dual-axis turntable at least once until the test point coincides with the center of the target image. Switch to the fusion modal imaging mode, relocate the coordinates of the test point and the center coordinates of the target image, calculate the turntable offset angle, and rotate the dual-axis turntable at least once again until the test point coincides with the center of the target image. Record the turntable offset coordinate P from the center of the target image in the single-modal imaging mode to the center of the target image in the fusion modal imaging mode. n (x n ,y n ); Sequentially obtain the turntable offset coordinates of multiple test points, and calculate the offset distance l of each test point. n The formulas for calculating the total offset absolute distance L are as follows: In the formula, x n y n These represent the horizontal and pitch offset distances of the dual-axis turntable as it moves from the center of the target image in the single-modality imaging mode to the center of the target image in the fusion-modality imaging mode. Step 4: Evaluate the registration accuracy based on the total absolute offset distance. The smaller the total absolute offset distance, the higher the registration accuracy.

5. The test method according to claim 4, characterized in that, In step 1, the debugging specifically involves: Step 11: Use a reflector to reflect the target image onto the test target, and adjust the brightness and temperature of the target image to ensure that the image of the device under test is clear; Step 12: Adjust the first fixture and set the device under test at the center of the dual-axis rotary table; Step 13: Adjust the offset angle of the dual-axis turntable in the horizontal and vertical directions to ensure that the target image is located in the center of the imaging screen of the device under test; Step 14: Adjust the second fixture to ensure that the image acquisition device captures the entire image of the device under test.

6. The test method according to claim 4, characterized in that, In step 2, the test points include an initial center point, an upper center point, a lower center point, a left center point, and a right center point. The method for determining the test points is as follows: Step 21: Acquire the image captured by the image acquisition device; Step 22: Draw a horizontal rectangle and a vertical rectangle within the image of the device under test. One intersection of the horizontal rectangle and the vertical rectangle coincides with the center of the image of the device under test, and the intersection is taken as the initial center point. Step 22: Draw a positioning rectangle within the image frame of the device under test. The length and width of the frame border of the device under test's image frame are proportionally reduced to the length and width of the positioning rectangle, and the center of the positioning rectangle coincides with the center of the image frame of the device under test. Among the multiple intersection points of the horizontal rectangle and the positioning rectangle, the two intersection points that are collinear with the initial center point are the left center point and the right center point, respectively. Among the multiple intersection points of the vertical rectangle and the positioning rectangle, the two intersection points that are collinear with the initial center point are the upper center point and the lower center point, respectively.

7. The test method according to claim 6, characterized in that, The positioning rectangle is a 3 / 4 rectangle of the frame of the image of the device under test.

8. The test method according to claim 4, characterized in that, In step 3, the specific method for locating the center coordinates of the target image is as follows: the image acquisition device sequentially performs image binarization processing, target detection, and contour detection to obtain the target image contour and generate an outer rectangle. The center point of the outer rectangle is extracted, and the center coordinates of the target image are output.

9. The test method according to claim 8, characterized in that, The method for extracting the center point specifically involves: after generating the circumscribed rectangle of the target image contour, identifying the coordinates E(x) of the lower left vertex of the circumscribed rectangle. e ,y e The target image center coordinates N(x), width w, and height h are calculated. N ,y N The calculation formula is:

10. The test method according to claim 8, characterized in that, In step 3, the method of aligning a test point with the center of the target image by rotating the dual-axis turntable is specifically as follows: based on the coordinates N(x) of the target image center... N ,y N ) and test point coordinates M(x) M ,y M ), calculate the horizontal offset angle and pitch offset angle that the dual-axis turntable needs to move, and control the dual-axis turntable to complete the overlap; The formulas for calculating the horizontal offset angle and the pitch offset angle are as follows: In the formula, The horizontal offset angle. Where α is the pitch offset angle, β is the horizontal scaling factor, and L0 is the straight-line distance from the device under test to the test target.

Images