A heterogeneous video fusion method and system based on external synchronous exposure

By adopting external synchronous exposure and time code alignment methods in drone reconnaissance technology, the problem of poor drone video fusion effect is solved, efficient heterologous image alignment and fusion are achieved, and the real-time performance of image processing is improved and Accuracy.

CN115086573BActive Publication Date: 2024-11-05BEIJING INST OF AEROSPACE CONTROL DEVICES
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210556490.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-19
Publication Date
2024-11-05
Estimated Expiration
2042-05-19

AI Technical Summary

Technical Problem

In the field of drone reconnaissance technology, it is difficult for the prior art to achieve efficient video fusion, especially when the scene characteristics are not rich or there is motion jitter on the photoelectric reconnaissance platform, the image fusion effect is poor, and real-time and alignment accuracy are difficult to meet the requirements.

Method used

Using a heterologous video fusion method based on external synchronous exposure, the visible light and infrared camera exposure are triggered regularly through a timer, and time code is added to realize the image time code alignment, and spatial alignment is performed through mechanical and software compensation, and finally image fusion is performed through wavelet transformation. .

Benefits of technology

It ensures simultaneous and spatial alignment of heterologous images, avoids image catching up, improves fusion effect, reduces the computing burden of the image processor, and enhances the accuracy of the center alignment of the field of view.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115086573B_ABST
    Figure CN115086573B_ABST
Patent Text Reader

Abstract

The present invention relates to a heterogeneous video fusion method based on external synchronous exposure, and belongs to the field of unmanned aerial vehicle optoelectronic reconnaissance technology. It includes: timing trigger imaging; image acquisition; time alignment: by caching infrared images, using visible light time code to find the corresponding infrared image, to achieve time alignment; space alignment: using mechanical alignment and dynamic compensation to determine the rotation angle between visible light and infrared cameras and the offset under different fields of view, and perform space alignment; image fusion: calculating the wavelet transform of visible light images and infrared images for multi-scale decomposition, extracting their respective low-frequency and high-frequency components for fusion, and then performing inverse wavelet transform to obtain a time domain fused image. The present invention realizes image fusion, enables visible light and infrared cameras to image simultaneously, and ensures the simultaneity of heterogeneous images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a heterogeneous video fusion method based on external synchronous exposure, belonging to the technical field of unmanned aerial vehicle optoelectronic reconnaissance. Background Art

[0002] In order to achieve day and night reconnaissance of target scenes, optoelectronic reconnaissance platforms are generally equipped with visible light imaging systems and infrared imaging systems. Due to the differences in the physical properties and imaging mechanisms of the sensors themselves, the image information collected by different sensors can be fully utilized for reconnaissance processing, thereby achieving multi-dimensional description and expression of the same scene. Therefore, video fusion technology has developed rapidly.

[0003] On an ordinary PC, the video fusion effect and performance can achieve good results. However, in the field of UAV reconnaissance technology, it is difficult to achieve better image fusion effects due to limitations such as the small size, low power consumption, and fast computing of the airborne terminal.

[0004] It is difficult for ordinary embedded platforms to meet the real-time requirements of image feature extraction and matching algorithms; when the scene features are not rich enough, the fusion effect becomes poor and does not meet the requirements; when the optoelectronic reconnaissance platform and the target move relative to each other or it shakes itself, due to delays in image acquisition and transmission, if there is no better image processing algorithm to achieve scene alignment, the ghosting phenomenon will be serious and the fusion effect will be very poor. Summary of the invention

[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art and propose a heterogeneous video fusion method based on external synchronous exposure, in which a visible light camera and an infrared camera are regularly triggered to expose by a timer, and the time code corresponding to the exposure moment is superimposed on the image data. The image data is collected and transmitted to a video fusion module, and the visible light and infrared image frames are time-aligned according to the image time code. By pre-calibration or feature extraction and matching, the transformation relationship between the two images is obtained, so that the two are spatially aligned to achieve image fusion.

[0006] The technical solution of the present invention:

[0007] A heterogeneous video fusion method based on external synchronous exposure comprises the following steps:

[0008] Step 1: Timed trigger imaging: Use an external timer to send imaging instructions to the visible light camera and infrared camera at the same time at a certain frequency, output YUV420 format image frames, and fill the time code at this moment into the last 32 pixels of the first line of the image;

[0009] Step 2: Image acquisition: The image acquisition module is used to acquire the YUV420 format image frames output by the visible light camera and the infrared camera, and the time code in the upper right corner of the image is read to make the time code correspond to the image frame one by one;

[0010] Step 3: Time alignment: By caching infrared images, using visible light time codes to find the corresponding infrared images, time alignment is achieved;

[0011] Step 4: Spatial alignment: Use mechanical alignment and dynamic compensation to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment;

[0012] Step 5: Image fusion: Calculate the wavelet transform of the visible light image and the infrared image for multi-scale decomposition, extract their respective low-frequency and high-frequency components for fusion, and then perform inverse wavelet transform to obtain a time-domain fused image.

[0013] Furthermore, the timing triggering process of step 1 includes the following steps:

[0014] Step 1-1: The timer sends an imaging instruction containing a time code;

[0015] Step 1-2: The camera starts imaging and generates image frames in YUV420 format;

[0016] Step 1-3: Write the time code into the 32 pixels in the upper right corner of the Y component of the image, where each pixel represents one bit of the time code;

[0017] Step 1-4: Output image frames through MIPI or SDI interface;

[0018] Step 1-5: Wait for the imaging instruction issued by the timer, and then execute the above four steps in a loop.

[0019] Furthermore, the image acquisition process of step 2 includes the following steps:

[0020] Step 2-1: Collect image signals;

[0021] Step 2-2: restore the time code according to the time code filling logic, and use the number of 1s in the binary representation of the pixel as the criterion. If it is greater than or equal to 4, the bit of the time code is 1, otherwise it is 0;

[0022] Step 2-3: Pack the time code and data together and send them to the video fusion module for time alignment.

[0023] Furthermore, the time alignment process of step 3 includes the following steps:

[0024] Step 3-1: When the video fusion module receives the infrared image, it uses the time code as the key and the data as the value to form a key-value pair, uses the time code as the index to search for the corresponding image data, and records the current time as the time when the infrared image last arrived;

[0025] Step 3-2: If the video fusion module receives a visible light image, it reads the visible light image frame time code and uses the time code to search for the infrared image data. If the infrared image data can be found, it is fused and the infrared image cache with a smaller time code than the current time code is recovered. If the infrared image data cannot be found, the last arrival time of the infrared image is checked. If there is no data for more than 100ms, it indicates that the infrared image is abnormal and no fusion is performed. Otherwise, the latest infrared image is used for fusion and other cached infrared image data is finally recovered.

[0026] Furthermore, the spatial alignment process of step 4 includes the following steps:

[0027] Step 4-1: The video fusion module turns on the fusion mode and adjusts the visible light to the minimum field of view. At this time, the field of view linkage is automatically turned on, and the infrared field of view automatically changes to ensure that the vertical field of view angles of visible light and infrared are consistent;

[0028] Step 4-2: Aim the visible light at the collimator, the target used by the collimator is a cross target, adjust the rotation angle of the infrared image to ensure that the cross target in the image is parallel, and record the angle;

[0029] Step 4-3: Modify the position of the cross target in the infrared image in the visible light image, ensure that the center of the cross target is completely aligned, and record the current focal length value, X-direction offset, and Y-direction offset;

[0030] Step 4-4: Adjust the focal length of visible light, increase the field of view in sections, and record the current focal length value, X-direction offset, and Y-direction offset until the maximum field of view is reached;

[0031] Step 4-5: In the data recorded in step 4-3, search for the XY offset according to the current focal length value f. If the search fails, perform linear interpolation to calculate the offset corresponding to the current focal length, thereby compensating for the visible light and infrared image offsets to achieve spatial alignment.

[0032] A heterogeneous video fusion system based on external synchronous exposure, comprising:

[0033] Timing trigger imaging unit: send imaging instructions to the visible light camera and infrared camera at the same time through an external timer at a certain frequency, output YUV420 format image frame, and fill the time code at this moment into the last 32 pixels of the first line of the image;

[0034] Image acquisition unit: The image acquisition module is used to acquire the YUV420 format image frames output by the visible light camera and the infrared camera, and the time code in the upper right corner of the image is read to make the time code correspond to the image frame one by one;

[0035] Time alignment unit: by caching infrared images and using visible light time codes to find corresponding infrared images, time alignment is achieved;

[0036] Spatial alignment unit: uses mechanical alignment and dynamic compensation to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment;

[0037] Image fusion unit: Calculate the wavelet transform of visible light image and infrared image for multi-scale decomposition, extract their respective low-frequency and high-frequency components for fusion, and then perform inverse wavelet transform to obtain the time domain fusion image.

[0038] The beneficial effects of the present invention compared with the prior art are:

[0039] (1) The present invention uses an external synchronous trigger exposure technology to trigger the visible light camera and the infrared camera to expose simultaneously through a timer, and superimposes the time code corresponding to the exposure moment onto the image data. The image data is collected and transmitted to the video fusion module, and the visible light and infrared image frames are time-aligned according to the image time code. The transformation relationship between the two images is obtained by pre-calibration or feature extraction and matching, so that the two images are aligned in space, image fusion is achieved, and the visible light and infrared cameras are imaged simultaneously, thereby ensuring the simultaneity of heterogeneous images.

[0040] (2) The present invention uses time code superposition to ensure that the visible light and infrared images used for fusion are exposed at the same time, thereby achieving time alignment of different images and effectively avoiding the image "catch-up phenomenon";

[0041] (3) The present invention ensures the spatial alignment of visible light and infrared images through mechanical and software calibration, and the center alignment accuracy of the field of view reaches 2 pixels, which effectively reduces the computational burden of the image processor and avoids the situation where the image registration effect drops sharply or even cannot be fused when the image scene has no good features. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The present invention is a video fusion flow chart for realizing external synchronous exposure. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the embodiments.

[0044] like Figure 1 As shown, a heterogeneous video fusion method based on external synchronous exposure includes the following steps:

[0045] Step 1: Timed trigger imaging: Use an external timer to send imaging instructions to the visible light camera and infrared camera at the same time every 40ms, output the YUV420 format image frame, and fill the time code at this time into the last 32 pixels of the first line of the image;

[0046] Step 2: Image acquisition: The image acquisition module is used to acquire the YUV420 format image frames output by the visible light camera and the infrared camera, and the time code in the upper right corner of the image is read to make the time code correspond to the image frame one by one;

[0047] Step 3: Time alignment: By caching infrared images, using visible light time codes to find the corresponding infrared images, time alignment is achieved;

[0048] Step 4: Spatial alignment: Use mechanical alignment and dynamic compensation to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment;

[0049] Step 5: Image fusion: Calculate the wavelet transform of the visible light image and the infrared image for multi-scale decomposition, extract their respective low-frequency and high-frequency components for fusion, and then perform inverse wavelet transform to obtain a time-domain fused image.

[0050] The timing triggering process of step 1 includes the following steps:

[0051] Step 1-1: The timer sends an imaging instruction containing a time code;

[0052] Step 1-2: The camera starts imaging and generates image frames in YUV420 format;

[0053] Step 1-3: Write the time code into the 32 pixels in the upper right corner of the Y component of the image, where each pixel represents one bit of the time code;

[0054] Step 1-4: Output image frames through MIPI or SDI interface;

[0055] Step 1-5: Wait for the imaging instruction issued by the timer, and then execute the above steps in a loop.

[0056] The image acquisition process of step 2 includes the following steps:

[0057] Step 2-1: Collect image signals;

[0058] Step 2-2: Restore the time code according to the time code filling logic of step 1-3, and use the number of 1s in the binary representation of the pixel as the criterion. If it is greater than or equal to 4, the bit of the time code is 1, otherwise it is 0.

[0059] Step 2-3: Pack the time code and data together and send them to the video fusion module for time alignment.

[0060] The time alignment of step 3 includes the following steps:

[0061] Step 3-1: When the video fusion module receives the infrared image, it uses the time code as the key and the data as the value to form a key-value pair, uses the time code as the index to search for the corresponding image data, and records the current time as the time when the infrared image last arrived;

[0062] Step 3-2: If the video fusion module receives a visible light image, it reads the visible light image frame time code and uses the time code to search for the infrared image data. If the infrared image data can be found, it is fused and the infrared image cache with a smaller time code than the current time code is recovered. If the infrared image data cannot be found, the last arrival time of the infrared image is checked. If there is no data for more than 100ms, it indicates that the infrared image is abnormal and no fusion is performed. Otherwise, the latest infrared image is used for fusion and other cached infrared image data is finally recovered.

[0063] The spatial alignment of step 4 includes the following steps:

[0064] Step 4-1: The video fusion module turns on the fusion mode and adjusts the visible light to the minimum field of view. At this time, the field of view linkage is automatically turned on, and the infrared field of view automatically changes to ensure that the vertical field of view angles of visible light and infrared are consistent;

[0065] Step 4-2: Aim the visible light at the collimator. The target used by the collimator is a high-precision cross target. Adjust the rotation angle of the infrared image to ensure that the cross target in the image is parallel and record the angle.

[0066] Step 4-3: Modify the position of the cross target in the infrared image in the visible light image, ensure that the center of the cross target is completely aligned, and record the current focal length value, X-direction offset, and Y-direction offset;

[0067] Step 4-4: Adjust the focal length of visible light, increase the field of view in sections, and record the current focal length value, X-direction offset, and Y-direction offset until the maximum field of view is reached;

[0068] Step 4-5: Search the XY offset in the data recorded in step 4-3 according to the current focal length value f. If the search fails, perform linear interpolation to calculate the offset corresponding to the current focal length, thereby compensating for the visible light and infrared image offsets to achieve spatial alignment.

[0069] A heterogeneous video fusion system based on external synchronous exposure, comprising:

[0070] Timing trigger imaging unit: send imaging instructions to the visible light camera and infrared camera at the same time through an external timer at a certain frequency, output YUV420 format image frame, and fill the time code at this moment into the last 32 pixels of the first line of the image;

[0071] Image acquisition unit: The image acquisition module is used to acquire the YUV420 format image frames output by the visible light camera and the infrared camera, and the time code in the upper right corner of the image is read to make the time code correspond to the image frame one by one;

[0072] Time alignment unit: by caching infrared images and using visible light time codes to find corresponding infrared images, time alignment is achieved;

[0073] Spatial alignment unit: uses mechanical alignment and dynamic compensation to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment;

[0074] Image fusion unit: Calculate the wavelet transform of visible light image and infrared image for multi-scale decomposition, extract their respective low-frequency and high-frequency components for fusion, and then perform inverse wavelet transform to obtain the time domain fusion image.

[0075] Example

[0076] A heterogeneous video fusion method based on external synchronous exposure, the steps are as follows:

[0077] Step 1: Timed trigger imaging: The trigger frequency is 25Hz, and the instruction is 0x01, which means exposure. Assuming the current time is 11:22:33:444, which means 11 hours, 22 minutes, 33 seconds and 444 milliseconds, the time code is 40953444, and the hexadecimal (little endian) is 64E6 70 02. Finally, add a byte check 0x38, and the instruction (hexadecimal) is: EB 90 01 64 E6 70 0238. After receiving the instruction, the camera takes an image and superimposes the time code bit by bit.

[0078] Step 2: Image acquisition: read the video output by the camera and restore the time code in the upper right corner;

[0079] Step 3: Time alignment: The video fusion module caches the infrared image. When the visible light data arrives, it queries the infrared image time code based on the visible light image time code to obtain the corresponding infrared image data.

[0080] Step 4: Spatial alignment: Use the high-precision cross target of the collimator to obtain the rotation angle of infrared relative to visible light and the offset under different fields of view. After the calibration is completed, interpolate the calibrated data according to the focal length value of the current camera to obtain the current image offset and achieve spatial alignment;

[0081] Step 5: Image fusion: Use the traditional and mature wavelet fusion method for fusion.

[0082] The present invention uses external synchronous trigger exposure technology to enable visible light and infrared cameras to simultaneously image, thereby ensuring the simultaneity of heterogeneous images;

[0083] The present invention uses time code superposition to ensure that the visible light and infrared images used for fusion are exposed at the same time, so that different images are aligned in time and the image "catch-up phenomenon" is effectively avoided.

[0084] The present invention ensures the spatial alignment of visible light and infrared images through mechanical and software calibration, and the field of view center alignment accuracy reaches 2 pixels, which effectively reduces the computing burden of the image processor and avoids the situation where the image registration effect drops sharply or even cannot be fused when the image scene has no good features.

[0085] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A heterogeneous video fusion method based on external synchronous exposure, characterized in that: include: Timing trigger imaging: send imaging instructions to the visible light camera and infrared camera at the same time through an external timer at a certain frequency, output visible light image and infrared image, and fill the time code at this moment into the last 32 pixels of the first line of the image; Image acquisition: collect visible light images and infrared images output by visible light cameras and infrared cameras, and read the time code in the upper right corner of the image to make the time code correspond to the image frame one by one; Time alignment: By caching infrared images, using visible light time codes to find the corresponding infrared images, time alignment is achieved; Spatial alignment: Mechanical alignment and dynamic compensation are used to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment. Image fusion: Perform wavelet transform and multi-scale decomposition on visible light images and infrared images, extract the low-frequency and high-frequency components of the visible light image and infrared image respectively and fuse them, then perform inverse wavelet transform to obtain the time-domain fused image; The image acquisition process includes the following steps: Step 2-1: Collect image signals; Step 2-2: restore the time code according to the time code filling logic, and use the number of 1s in the binary representation of the pixel as the criterion. If it is greater than or equal to 4, the bit of the time code is 1, otherwise it is 0; Step 2-3: Pack the time code and data together and send them to the video fusion module for time alignment.

2. The heterogeneous video fusion method based on external synchronous exposure according to claim 1, characterized in that: The timing trigger process includes the following steps: Step 1-1: The timer sends an imaging instruction containing a time code; Step 1-2: The camera starts imaging and generates image frames in YUV420 format; Step 1-3: Write the time code into the 32 pixels in the upper right corner of the Y component of the image, where each pixel represents one bit of the time code; Step 1-4: Output image frames through MIPI or SDI interface; Step 1-5: Wait for the imaging instruction issued by the timer, and then execute the above four steps in a loop.

3. The heterogeneous video fusion method based on external synchronous exposure according to claim 1, characterized in that: The time alignment process includes the following steps: Step 3-1: When the video fusion module receives the infrared image, it uses the time code as the key and the data as the value to form a key-value pair, uses the time code as the index to search for the corresponding image data, and records the current time as the time when the infrared image last arrived; Step 3-2: If the video fusion module receives a visible light image, it reads the visible light image frame time code and uses the time code to search for the infrared image data. If the infrared image data can be found, it is fused and the infrared image cache with a smaller time code than the current time code is recovered. If the infrared image data cannot be found, the last arrival time of the infrared image is checked. If there is no data for more than 100ms, it indicates that the infrared image is abnormal and no fusion is performed. Otherwise, the latest infrared image is used for fusion and other cached infrared image data is finally recovered.

4. The heterogeneous video fusion method based on external synchronous exposure according to claim 1, characterized in that: The spatial alignment process includes the following steps: Step 4-1: The video fusion module turns on the fusion mode and adjusts the visible light to the minimum field of view. At this time, the field of view linkage is automatically turned on, and the infrared field of view automatically changes to ensure that the vertical field of view angles of visible light and infrared are consistent; Step 4-2: Aim the visible light at the collimator, the target used by the collimator is a cross target, adjust the rotation angle of the infrared image to ensure that the cross target in the image is parallel, and record the angle; Step 4-3: Modify the position of the cross target in the infrared image in the visible light image, ensure that the center of the cross target is completely aligned, and record the current focal length value, X-direction offset, and Y-direction offset; Step 4-4: Adjust the focal length of visible light, increase the field of view in sections, and record the current focal length value, X-direction offset, and Y-direction offset until the maximum field of view is reached; Step 4-5: In the data recorded in step 4-3, search for the XY offset according to the current focal length value f. If the search fails, perform linear interpolation to calculate the offset corresponding to the current focal length, thereby compensating for the visible light and infrared image offsets to achieve spatial alignment.

5. A heterogeneous video fusion system based on external synchronous exposure, characterized in that: include: Timing trigger imaging unit: send imaging instructions to the visible light camera and infrared camera at the same time through an external timer at a certain frequency, output visible light image and infrared image, and fill the time code at this moment into the last 32 pixels of the first line of the image; Image acquisition unit: collects visible light images and infrared images output by visible light cameras and infrared cameras, and reads the time code in the upper right corner of the image to make the time code correspond to the image frame one by one; Time alignment unit: by caching infrared images and using visible light time codes to find corresponding infrared images, time alignment is achieved; Spatial alignment unit: uses mechanical alignment and dynamic compensation to determine the rotation angle between the visible light and infrared cameras and the offset under different fields of view for spatial alignment; Image fusion unit: Calculate the wavelet transform of visible light image and infrared image for multi-scale decomposition, extract their respective low-frequency and high-frequency components for fusion, and then perform inverse wavelet transform to obtain the time-domain fusion image; The image acquisition process of the image acquisition unit includes the following steps: Step 2-1: Collect image signals; Step 2-2: restore the time code according to the time code filling logic, and use the number of 1s in the binary representation of the pixel as the criterion. If it is greater than or equal to 4, the bit of the time code is 1, otherwise it is 0; Step 2-3: Pack the time code and data together and send them to the video fusion module for time alignment.

6. The heterogeneous video fusion system based on external synchronous exposure according to claim 5, characterized in that: The timing trigger process includes the following steps: Step 1-1: The timer sends an imaging instruction containing a time code; Step 1-2: The camera starts imaging and generates image frames in YUV420 format; Step 1-3: Write the time code into the 32 pixels in the upper right corner of the Y component of the image, where each pixel represents one bit of the time code; Step 1-4: Output image frames through MIPI or SDI interface; Step 1-5: Wait for the imaging instruction issued by the timer, and then execute the above four steps in a loop.

7. The heterogeneous video fusion system based on external synchronous exposure according to claim 5, characterized in that: The time alignment process includes the following steps: Step 3-1: When the video fusion module receives the infrared image, it uses the time code as the key and the data as the value to form a key-value pair, uses the time code as the index to search for the corresponding image data, and records the current time as the time when the infrared image last arrived; Step 3-2: If the video fusion module receives a visible light image, it reads the visible light image frame time code and uses the time code to search for the infrared image data. If the infrared image data can be found, it is fused and the infrared image cache with a smaller time code than the current time code is recovered. If the infrared image data cannot be found, the last arrival time of the infrared image is checked. If there is no data for more than 100ms, it indicates that the infrared image is abnormal and no fusion is performed. Otherwise, the latest infrared image is used for fusion and other cached infrared image data is finally recovered.

8. The heterogeneous video fusion system based on external synchronous exposure according to claim 5, characterized in that: The spatial alignment process includes the following steps: Step 4-1: The video fusion module turns on the fusion mode and adjusts the visible light to the minimum field of view. At this time, the field of view linkage is automatically turned on, and the infrared field of view automatically changes to ensure that the vertical field of view angles of visible light and infrared are consistent; Step 4-2: Aim the visible light at the collimator, the target used by the collimator is a cross target, adjust the rotation angle of the infrared image to ensure that the cross target in the image is parallel, and record the angle; Step 4-3: Modify the position of the cross target in the infrared image in the visible light image, ensure that the center of the cross target is completely aligned, and record the current focal length value, X-direction offset, and Y-direction offset; Step 4-4: Adjust the focal length of visible light, increase the field of view in sections, and record the current focal length value, X-direction offset, and Y-direction offset until the maximum field of view is reached; Step 4-5: In the data recorded in step 4-3, search for the XY offset according to the current focal length value f. If the search fails, perform linear interpolation to calculate the offset corresponding to the current focal length, thereby compensating for the visible light and infrared image offsets to achieve spatial alignment.