Air moving small target tracking device and method based on double-waveband common-aperture and double-turntable linkage

Abstract

The device comprises a large rotary table, a short-focus SWITR-MWIR camera, a small rotary table, a long-focus SWITR-RGB camera, a damping device and a distance measuring device, the large rotary table is responsible for large-range search, the small rotary table is responsible for high-precision tracking, and the long-focus SWITR-RGB camera is responsible for high-precision tracking. The two dual-band common-aperture cameras adapt to complex environments such as haze, sand and dust and night through multi-band combination, the damping device inhibits mechanical vibration, and the distance measuring device provides target distance and speed information; according to the method, preprocessing and block division are carried out on an image, moving target detection is carried out, linkage tracking is realized by combining coarse adjustment of a large turntable and fine adjustment of a small turntable, moving targets are sequentially locked to carry out target identification, a differential tracking strategy is adopted according to a target area position, and corresponding processing is carried out aiming at a target loss condition. According to the invention, long-distance, high-precision and stable tracking of a small target moving in the air in complex environments such as haze, dust and night can be realized.

Description

A device and method for tracking small aerial moving targets based on dual-band common aperture and dual-turntable linkage Technical Field

[0001] This invention belongs to the field of photoelectric tracking technology, specifically relating to a tracking device and method for small moving targets in the air based on dual-band common aperture and dual turntable linkage. Background Technology

[0002] With the rapid development of low-altitude security and airspace management, the demand for long-range detection and high-precision tracking of small aerial targets such as drones is becoming increasingly urgent. These targets are characterized by their small size, high speed, and low flight altitude, and often exist in complex environments such as fog, dust, and nighttime, resulting in low target imaging contrast, strong background interference, and easy escape from the field of view, posing a huge challenge to stable tracking.

[0003] Most existing mainstream photoelectric tracking systems employ a single-band sensor and a single turntable structure. Single-band systems have inherent drawbacks: using the visible light band is susceptible to illumination and atmospheric scattering, leading to a sharp drop in performance under adverse weather conditions; while using the infrared band offers some fog penetration and night vision capabilities, its detail resolution is limited, making accurate target identification difficult. The single turntable structure presents a performance contradiction: turntables focused on wide-range, rapid searching struggle with high-precision angle adjustment; conversely, turntables prioritizing high-precision tracking often have limited search ranges, making them ill-suited for high-speed maneuvering targets.

[0004] Therefore, existing technologies cannot simultaneously achieve target detection, wide-range search, and high-precision tracking capabilities in complex environments, and an innovative technical solution is urgently needed to address this issue. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an aerial moving small target tracking device and method based on dual-band common aperture and dual turntable linkage, so as to achieve long-distance, high-precision and stable tracking of aerial moving small targets in complex environments such as fog, dust, and night.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an aerial moving small target tracking device based on dual-band common aperture and dual turntable linkage, comprising:

[0007] The large turntable serves as the system base, supporting the short-focal-length SWIR-MWIR camera, the small turntable and its load, and integrates a motion feedback device to provide the overall horizontal and pitch rotation capabilities of the device.

[0008] The short-focal-length SWIR-MWIR camera is mounted on a large turntable and adopts a common aperture design. It uses a wavelength selective beam splitter to separate the incident light from a single lens into two coaxial imaging optical paths: SWIR (Short-Wave Infrared) and MWIR (Mid-Wave Infrared), outputting a wide-field-of-view SWIR image and MWIR image.

[0009] A small turntable, mounted on a large turntable, supports a long-focus SWIR-RGB camera and rangefinder through a shock-absorbing device. It integrates a motion feedback device and a high-bandwidth servo system, providing high-precision and fast horizontal and pitch rotation capabilities independent of the large turntable.

[0010] The long-zoom SWIR-RGB camera, mounted on a vibration damping device, adopts a common aperture design. It uses a wavelength selective beam splitter to separate the incident light from a single lens into two coaxial imaging optical paths: the SWIR band and the RGB (Red Green Blue) band, outputting high-resolution SWIR and RGB images.

[0011] A vibration damping device, mounted on a small turntable, is used to suppress mechanical vibrations transmitted to the telephoto SWIR-RGB camera and rangefinder.

[0012] The rangefinder is mounted parallel to the optical axis of the telephoto SWIR-RGB camera on the vibration damping device to obtain distance and speed information of the target.

[0013] Furthermore, the motion feedback device integrated in the large turntable and the motion feedback device integrated in the small turntable are used to collect the horizontal angle, pitch angle and rotation speed of the corresponding turntable in real time, and to adjust the angle and speed of the corresponding turntable.

[0014] Furthermore, the high-bandwidth servo system compensates for optical axis deviation and image shift caused by the movement of the small turntable in real time based on the motion information of the small turntable collected by the motion feedback device.

[0015] This invention also provides a method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage, implemented using the above-mentioned device, and including the following steps:

[0016] S1. Image preprocessing: Preprocess the four image data output from the short-focus SWIR-MWIR camera and the long-focus SWIR-RGB camera respectively;

[0017] S2. Block division: Divide the two short-focus images output by the short-focus SWIR-MWIR camera into N×N uniform blocks, and further divide them into central region, edge region and transition region;

[0018] S3. Moving target detection: The frame difference method is used to detect moving targets in two short-focus images, and the block number where the moving target is located is recorded. The centroid coordinates of the target are calculated and assigned to the block where the centroid is located.

[0019] S4: Target Locking: Determine the reference block based on the number of moving targets: For a single target, use the block containing that target as the reference block; for multiple targets, calculate the average center point of all moving targets and use the block containing the average center point as the reference block; if the average center point of a single or multiple targets is within the reference block, the large turntable remains stationary; if the average center point of a single or multiple targets enters a new block, drive the large turntable to pull it back to the reference block; calculate the threat level of the moving targets and lock them in order of threat level; obtain the block number of the telephoto field of view center through image matching; combine the target block number with the interval mapping table to obtain the rotation angle of the small turntable; drive the small turntable to rotate to the corresponding angle; and make fine adjustments based on the real-time pixel deviation of the target within the block.

[0020] S5: Target Recognition: Recognize moving targets in the two telephoto images output by the telephoto SWIR-RGB camera, retain the target type, and eliminate interference types. If the recognition results of the two telephoto images conflict, the RGB image is analyzed. If the average brightness of the RGB image is lower than the threshold T1 and the contrast is lower than the threshold T2, it is determined to be a low light or bad weather environment, and the recognition result of the SWIR image shall prevail; otherwise, the recognition result of the RGB image shall prevail.

[0021] S6: Target Tracking: Prioritize tracking the target with the highest threat level. If the target is located in the central area, the large turntable remains stationary, and the small turntable adjusts its angle based on the interval mapping table and the positional deviation within the area. If the target is located in the transition area, the large turntable rotates at the first speed to pull the target back to the central area, and the small turntable immediately compensates by the same angle in the opposite direction. If the target is located in the edge area, the large turntable rotates at a second speed greater than the first speed to pull the target back to the central area, and the small turntable remains stationary. If the target is not detected within a preset time T, the threat level of the target within the short focal length field of view is recalculated, and the target with the highest threat level continues to be tracked. If there are no other targets within the short focal length field of view, return to step S3.

[0022] Further, in step S1, the preprocessing includes: performing bad pixel correction, lens distortion correction and grayscale enhancement, and performing dehazing enhancement on the RGB image; and performing image registration on the short-focus SWIR image and MWIR image, and the long-focus SWIR image and RGB image, respectively.

[0023] Further, in step S2, the two short-focus images are divided into N×N uniform blocks, where N is a positive integer, and the blocks are sequentially labeled with numbers 0 to N. 2-1, set the central block of size [N / 3]×[N / 3] as the central region, the outermost block of size [N / 9] as the edge region, and the remaining blocks as the transition region, [ ] indicates rounding up.

[0024] Furthermore, in step S4, the threat level of the moving target is calculated based on a weighted average of the target's speed, distance, and size characteristics, using the following formula:

[0025]

[0026] in, The threat level of a moving target. For the target radial velocity, This represents the maximum speed. For the target distance, The minimum distance, The pixel area occupied by the target in the short focal length image. The maximum pixel area. , , All are weighting coefficients.

[0027] Furthermore, in step S4, the rotation angle of the large turntable is calculated based on the field of view, image resolution, and pixel deviation between the target block and the center block of the short-focal-length SWIR-MWIR camera.

[0028] Furthermore, in step S4, the interval mapping table is established through a pre-calibration process, which is used to determine the correspondence between each block on the short focal length image and the deflection angle of the small turntable.

[0029] Furthermore, in step S4, the rotation angle of the small turntable is a closed-loop control process, and its control quantity is calculated based on the real-time pixel deviation between the centroid of the target and the center of the field of view in the telephoto image.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This invention adopts a dual-band common aperture imaging design, which takes into account the detail resolution capability of visible light and the fog penetration and night vision characteristics of infrared band. It can stably detect small moving targets in the air in complex environments such as fog, dust, and night, and solves the problems of traditional single-band imaging systems being easily affected by environmental interference and easily losing targets.

[0032] 2. This invention utilizes a control strategy of coarse adjustment on a large turntable and fine adjustment on a small turntable to fully leverage the large turntable's wide range of rotation capabilities and the small turntable's high-precision adjustment capabilities, forming an efficient division of labor and collaboration mode. This approach leverages the large field-of-view coverage advantage of the large turntable while utilizing the high-speed and precise characteristics of the small turntable to achieve stable locking, thus solving the problems of the traditional single turntable where range and precision cannot be simultaneously achieved, and the poor collaboration between dual turntables.

[0033] 3. This invention adopts a differentiated tracking strategy for targets in different regions of the field of view. It accurately locks onto single targets and prioritizes high-threat targets when there are multiple targets. Furthermore, it avoids invalid actions of large turntables through the "block boundary triggering" mechanism, thus solving the problems of rigidity, poor adaptability to multiple targets, and easy lag in cross-regional movement of traditional tracking strategies. Attached Figure Description

[0034] Figure 1 is a schematic diagram of the structure of the aerial moving small target tracking device based on dual-band common aperture and dual turntable linkage provided in an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of the structure of the beam-splitting device in an embodiment of the present invention;

[0036] Figure 3 is a flowchart of the aerial moving small target tracking method based on dual-band common aperture and dual turntable linkage provided in an embodiment of the present invention;

[0037] Figure 4 is a block diagram of telephoto field of view control in an embodiment of the present invention;

[0038] Figure 5 is a block diagram of short focal length field of view control in an embodiment of the present invention;

[0039] Figure 6 is a schematic diagram of block division in an embodiment of the present invention;

[0040] Figure 7 is a flowchart of the frame difference method in an embodiment of the present invention. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0043] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0044] As shown in Figure 1, this embodiment provides an aerial moving small target tracking device based on dual-band common aperture and dual turntable linkage, including: a large turntable 1, a short-focus SWIR-MWIR camera 2, a small turntable 3, a long-focus SWIR-RGB camera 4, a vibration damping device 5, and a ranging device 6.

[0045] The large turntable 1 serves as the system base, supporting the short-focal-length SWIR-MWIR camera, the small turntable and its load, and integrating a motion feedback device to provide the overall horizontal and pitch rotation capabilities of the device.

[0046] The short-focal-length SWIR-MWIR camera 2 is mounted on a large turntable and adopts a common aperture design. It uses a wavelength selective beam splitter to separate the incident light from a single lens into two coaxial imaging optical paths: the SWIR (Short-Wave Infrared) band and the MWIR (Mid-Wave Infrared) band, and outputs a wide-field-of-view SWIR image and MWIR image.

[0047] The small turntable 3 is mounted on the large turntable and carries the long-focus SWIR-RGB camera and rangefinder through a shock absorption device. It integrates a motion feedback device and a high-bandwidth servo system, providing high-precision and fast horizontal and pitch rotation capabilities independent of the large turntable.

[0048] Among them, the motion feedback device integrated in the large turntable 1 and the motion feedback device integrated in the small turntable 3 are used to collect the horizontal angle, pitch angle and rotation speed of the corresponding turntable in real time, and are used to adjust the angle and speed of the corresponding turntable.

[0049] The high-bandwidth servo system uses motion information such as horizontal / pitch angle deviation and rotation speed fluctuation of a small turntable collected by a motion feedback device to compensate for optical axis deviation and image shift caused by turntable movement in real time, thus maintaining imaging stability.

[0050] The long-focus SWIR-RGB camera 4 is mounted on a shock-absorbing device and adopts a common aperture design. It uses a wavelength selective beam splitter to separate the incident light from a single lens into two coaxial imaging optical paths: the SWIR band and the RGB (Red Green Blue) band, and outputs high-resolution SWIR and RGB images.

[0051] The vibration damping device 5 is mounted on a small turntable to suppress mechanical vibrations transmitted to the telephoto SWIR-RGB camera and the rangefinder.

[0052] The ranging device 6 is mounted parallel to the optical axis of the telephoto SWIR-RGB camera on the vibration damping device, and is used to obtain the distance and speed information of the target.

[0053] As shown in Figure 3, this embodiment provides a method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage. Based on the above device, it is implemented according to the following steps.

[0054] S1. Image preprocessing: Preprocess the four image data output from the short-focus SWIR-MWIR camera and the long-focus SWIR-RGB camera respectively.

[0055] The preprocessing specifically includes: performing bad pixel correction, lens distortion correction and grayscale enhancement; performing dehazing enhancement on RGB images; and performing image registration between short-focus SWIR images and MWIR images, and between long-focus SWIR images and RGB images.

[0056] S2. Block division: Divide the two short-focus images output by the short-focus SWIR-MWIR camera into N×N uniform blocks, and further divide them into central region, edge region and transition region.

[0057] Specifically, the two short-focus images are divided into N×N uniform blocks, where N is a positive integer, and the blocks are numbered sequentially from 0 to N. 2 -1, set the central block of size [N / 3]×[N / 3] as the central region, the outermost block of size [N / 9] as the edge region, and the remaining blocks as the transition region, [ ] indicates rounding up.

[0058] S3. Moving target detection: The frame difference method is used to detect moving targets in two short-focus images, and the block number where the moving target is located is recorded. The centroid coordinates of the target are calculated and assigned to the block where the centroid is located.

[0059] S4: Target Locking: Determine the reference block based on the number of moving targets: For a single target, use the block containing that target as the reference block; for multiple targets, calculate the average center point of all moving targets and use the block containing the average center point as the reference block; if the average center point of a single or multiple targets is within the reference block, the large turntable remains stationary; if the average center point of a single or multiple targets enters a new block, drive the large turntable to pull it back to the reference block; calculate the threat level of the moving targets and lock them in order of threat level; obtain the block number of the telephoto field of view center through image matching; combine the target block number with the interval mapping table to obtain the rotation angle of the small turntable; drive the small turntable to rotate to the corresponding angle; and make fine adjustments based on the real-time pixel deviation of the target within the block.

[0060] The threat level of the moving target is calculated based on a weighted average of the target's speed, distance, and size characteristics, using the following formula:

[0061]

[0062] in, The threat level of a moving target. For the target radial velocity, This represents the maximum speed. For the target distance, The minimum distance, The pixel area occupied by the target in the short focal length image. The maximum pixel area. , , All are weighting coefficients.

[0063] The rotation angle driving the large turntable is calculated based on the field of view, image resolution, and pixel deviation between the target block and the center block of the short-focal-length SWIR-MWIR camera.

[0064] The interval mapping table is established through a pre-calibration process, which is used to determine the correspondence between each block on the short focal length image and the deflection angle of the small turntable.

[0065] The rotation angle of the small turntable is a closed-loop control process, and its control quantity is calculated based on the real-time pixel deviation between the centroid of the target and the center of the field of view in the telephoto image.

[0066] S5: Target Recognition: Recognize moving targets in the two telephoto images output by the telephoto SWIR-RGB camera, retain the target type, and eliminate interference types. If the recognition results of the two telephoto images conflict, the RGB image is analyzed. If the average brightness of the RGB image is lower than the threshold T1 and the contrast is lower than the threshold T2, it is determined to be a low light or bad weather environment, and the recognition result of the SWIR image shall prevail. Otherwise, the recognition result of the RGB image shall prevail.

[0067] S6: Target Tracking: Prioritize tracking the target with the highest threat level. If the target is located in the central area, the large turntable remains stationary, and the small turntable adjusts its angle based on the interval mapping table and the positional deviation within the area. If the target is located in the transition area, the large turntable rotates at the first speed to pull the target back to the central area, and the small turntable immediately compensates by the same angle in the opposite direction. If the target is located in the edge area, the large turntable rotates at a second speed greater than the first speed to pull the target back to the central area, and the small turntable remains stationary. If the target is not detected within a preset time T, the threat level of the target within the short focal length field of view is recalculated, and the target with the highest threat level continues to be tracked. If there are no other targets within the short focal length field of view, return to step S3.

[0068] The following is a further detailed description of this embodiment.

[0069] In this embodiment, the overall device is shown in Figure 1, and the specific configuration of the device is as follows:

[0070] Large turntable: driven by servo motor, horizontal rotation range ±180°, pitch rotation range -20° to +90°, repeatability ±0.01°, absolute accuracy ±0.015°, maximum rotation speed 60° / s, integrated high-precision photoelectric encoder as motion feedback device.

[0071] Short-focal-length SWIR-MWIR camera: lens focal length 12mm, field of view 50°×38°, built-in dichroic beam splitter separates incident light into SWIR and MWIR bands, SWIR band range 0.9-1.7μm, MWIR band range 3.0-5.0μm, image resolution 640×512, and a schematic diagram of the beam splitting device is shown in Figure 2.

[0072] Small turntable: driven by servo motor, horizontal rotation range ±135°, pitch rotation range ±90°, repeatability ±0.005°, absolute accuracy ±0.008°, maximum rotation speed 120° / s, integrated high-precision photoelectric encoder as motion feedback device.

[0073] Long-zoom SWIR-RGB camera: lens focal length 135mm, field of view 5.5°×4°, built-in dichroic beam splitter to separate incident light into SWIR and RGB bands, SWIR band range 0.9-1.7μm, RGB band range 400-760nm, image resolution 1920x1080.

[0074] Vibration damping device: It adopts an air spring passive vibration isolator, which is installed on a small turntable. It consists of a sealed airbag, a pressure regulating valve and a height control valve. Its natural frequency is 2-5Hz, and its mechanical vibration attenuation rate in the 10-200Hz frequency band is ≥90%.

[0075] Ranging device: Employs millimeter-wave radar with its optical axis installed parallel to the optical axis of the telephoto SWIR-RGB camera, with a parallelism error of less than 0.5 mrad. The radar operates at a frequency of 76-81 GHz, has a ranging range of 0.1-5 km, and a ranging accuracy of ±0.5 m. It can simultaneously provide target distance and velocity information.

[0076] In this embodiment, the method implementation flowchart is shown in Figure 3, the telephoto field-of-view control block diagram is shown in Figure 4, and the short-focal-length field-of-view control block diagram is shown in Figure 5. The specific process of the method is as follows:

[0077] S1: Image preprocessing: In this embodiment, the following preprocessing is performed sequentially on the four images:

[0078] Defect correction: The 3x3 neighborhood mean replacement method is used for the two short-focus images, and the 3x3 neighborhood median replacement method is used for the two long-focus images. The 3σ principle is used for defect detection to ensure that normal pixels are not incorrectly corrected.

[0079] Lens distortion correction: The Brown-Conrady distortion model is adopted. During the system calibration stage, the camera's intrinsic parameter matrix and distortion coefficients are solved by capturing images of the checkerboard calibration board in different poses. During the real-time processing stage, the four images are corrected by mapping transformation based on the intrinsic parameter matrix and distortion coefficients.

[0080] Gray-scale enhancement: The CLAHE algorithm is used for gray-scale enhancement. The two short-focus images are divided into 16x16 blocks and the two long-focus images are divided into 8x8 blocks. All blocks are eliminated by bilinear interpolation. Histogram equalization is performed on each block. At the same time, the contrast enhancement is limited to avoid excessive amplification of noise.

[0081] Dehazing Enhancement: For RGB images, a dark channel prior algorithm is used for dehazing enhancement to improve the image quality of RGB images under severe weather conditions such as fog and haze.

[0082] Image registration: The short-focus SWIR and MWIR images and the long-focus SWIR and RGB images were registered separately. Feature points were extracted based on the SIFT algorithm, and the KNN-matching algorithm was used to match the feature points. The RANSAC algorithm was then applied to filter the matching feature points. Finally, the pixel-level registration between images of different bands was achieved through perspective transformation and bilinear interpolation.

[0083] S2: Block division: In this embodiment, N=9 is taken, and the two short-focus images are divided into 9×9=81 uniform blocks. The field of view of a single block is about 5.6°×4.2°. The blocks are numbered from 0 to 80 from left to right and from top to bottom. The central 3×3=9 blocks are defined as the central region, the outermost 2 blocks are the edge region, and the remaining blocks are the transition region. The block division diagram is shown in Figure 6.

[0084] S3: Moving Target Detection: In this embodiment, the two-frame difference method is used to detect moving targets in two short-focus images. The two consecutive short-focus images are converted into grayscale images and preprocessed with Gaussian filtering. The absolute difference between the two images is calculated to obtain a difference image. The difference image is thresholded to obtain a binary image containing candidate motion regions. Morphological opening and closing operations are performed on the binary image to eliminate noise and fill holes. Connected components are extracted and the centroid coordinates of the moving target are calculated to determine the block to which the moving target belongs. The flowchart of the frame difference method is shown in Figure 7. The centroid coordinate calculation method is as follows: For each detected moving target connected component, the horizontal / vertical coordinates of all pixels inside it are summed and divided by the total number of pixels to obtain the horizontal / vertical coordinates of the centroid.

[0085] S4: Target Locking: In this embodiment, when there is only a single moving target in the short focal field of view, the block where the moving target is located is taken as the reference block. When there are multiple moving targets, the block where the average center point of all moving targets is located is taken as the reference block. The average center point is the average value of the centroid coordinates of all moving targets.

[0086] When the target / center moves from the reference block to the new block, the large turntable pulls the target / center back to the reference block. The formula for calculating the rotation angle of the large turntable between blocks is as follows:

[0087]

[0088] in, The horizontal rotation angle, For the horizontal field of view of a short-focal-length camera, For the horizontal resolution of short-focal-length images, As the base block center Coordinates and the center of the target block The difference in coordinates, pitch rotation angle The calculation is similar.

[0089] Calculate the threat level of moving targets and lock onto them sequentially according to their threat level. The formula for calculating the threat level is as follows:

[0090]

[0091] in, For target threat level, For the target radial velocity, D is the target distance. , The pixel area occupied by the target in the short focal length image. The weighting coefficient is taken as: , , .

[0092] The rotation angle of the small turntable between blocks is obtained by querying a pre-calibrated interval mapping table. The calibration method is as follows: In an indoor collimator environment, a light-emitting target is placed at infinity. The large turntable is kept stationary, and the small turntable is driven in sequence so that the center of the telephoto camera's field of view is precisely aligned with the center point of each block of the short-focus camera, a total of 81 points. The horizontal and pitch angles of the small turntable are recorded, thereby establishing an interval mapping table of "block number - small turntable angle".

[0093] Fine-tuning of a small turntable is a closed-loop control process, and its control deviation is... Pixel deviation between the target centroid and the center of the field of view in a telephoto field of view The calculation is as follows:

[0094]

[0095] in, This is the proportionality coefficient. The horizontal pixel deviation is determined based on the camera's focal length and image resolution, and the pitch angle is finely adjusted. The calculation is similar.

[0096] S5: Target Recognition: In this embodiment, a deep learning-based target recognition algorithm is used to establish a dataset containing a large number of drone, bird, and kite images under different lighting conditions, backgrounds, and target sizes. The dataset is used to train the YOLOv5 model. The trained model is used to perform target recognition on telephoto SWIR and RGB images, retaining drone targets and excluding bird and kite targets. If the recognition results of the SWIR image and the RGB image conflict, the average brightness and contrast of the RGB image are calculated. When the average brightness is less than the threshold T1=40 and the contrast is less than the threshold T2=30, the recognition result of the SWIR image is taken as the standard; otherwise, the recognition result of the RGB image is taken as the standard.

[0097] S6: Target tracking: In this embodiment, the first speed is set to 20° / s, the second speed is set to 50° / s, and the preset time is T=2s.

[0098] Based on the calculated threat levels, prioritize tracking targets with the highest threat level.

[0099] If the target is located in the central area, the large turntable remains stationary, while the small turntable performs precise tracking based on the interval mapping table and real-time pixel deviation.

[0100] If the target enters the transition zone, the large turntable rotates at the first speed to pull the target back to the center area. At the same time, the small turntable immediately compensates for the same angle of rotation of the large turntable in the opposite direction to maintain continuous lock on the target.

[0101] If the target enters the edge area, the large turntable rotates at a second speed to quickly pull the target back to the center area. The small turntable remains stationary during this stage to avoid the risk of overshoot or oscillation.

[0102] If the target is not detected in the short-focus and long-focus fields of view within the preset time T, the threat level of all targets in the short-focus field of view is recalculated, and the target with the highest threat level is switched to be tracked. If there are no other targets in the short-focus field of view, the process returns to the moving target detection step S3 to perform a large-scale search again.

[0103] Experiments show that the device and method described in this embodiment can stably track UAV targets at a distance of 1 km and a flight speed of less than 20 m / s in foggy weather with visibility of less than 3 km. The tracking accuracy is better than 0.1 mrad, which is significantly better than the traditional single-band single turntable system.

[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A tracking device for small moving targets in the air based on dual-band common aperture and dual turntable linkage, characterized in that, include: A large turntable, serving as the system base, supports the short-focal-length SWIR-MWIR camera, a small turntable, and their loads. It integrates motion feedback and provides the overall horizontal and vertical rotation capabilities of the system. The short-focal-length SWIR-MWIR camera, mounted on the large turntable, employs a common-aperture design. A wavelength-selective beam splitter separates the incident light from the single lens into two coaxial imaging paths—one for the SWIR band and one for the MWIR band—outputting wide-field SWIR and MWIR images. The small turntable, mounted on the large turntable, supports the long-focal-length SWIR-RGB camera and a rangefinder via a vibration damping device. It integrates motion feedback and a high-bandwidth servo motor. The system provides high-precision, rapid horizontal and vertical rotation capabilities independent of the large turntable; the long-focus SWIR-RGB camera, mounted on the vibration damping device, adopts a common aperture design and uses a wavelength selective beam splitter to separate the incident light from a single lens into two coaxial imaging optical paths, one in the SWIR band and the other in the RGB band, outputting high-resolution SWIR and RGB images; the vibration damping device, mounted on a small turntable, is used to suppress mechanical vibrations transmitted to the long-focus SWIR-RGB camera and the rangefinder; the rangefinder, mounted parallel to the optical axis of the long-focus SWIR-RGB camera on the vibration damping device, is used to acquire distance and velocity information of the target.

2. The aerial moving small target tracking device based on dual-band common aperture and dual turntable linkage according to claim 1, characterized in that, The motion feedback device integrated in the large turntable and the motion feedback device integrated in the small turntable are used to collect the horizontal angle, pitch angle and rotation speed of the corresponding turntable in real time, and to adjust the angle and speed of the corresponding turntable.

3. The aerial moving small target tracking device based on dual-band common aperture and dual turntable linkage according to claim 1, characterized in that, The high-bandwidth servo system compensates for optical axis deviation and image shift caused by the movement of the small turntable in real time based on the motion information collected by the motion feedback device.

4. A method for tracking small moving targets in the air based on dual-band common aperture and dual-turntable linkage, characterized in that, Based on the apparatus described in any one of claims 1-3, the method includes the following steps: S1, image preprocessing: preprocessing the four-channel image data output from the short-focus SWIR-MWIR camera and the long-focus SWIR-RGB camera respectively; S2, block division: dividing the two short-focus images output from the short-focus SWIR-MWIR camera into N×N uniform blocks, further dividing them into central regions, edge regions, and transition regions; S3, moving target detection: detecting moving targets in the two short-focus images using the frame difference method, recording the block number where the moving target is located, calculating the centroid coordinates of the target, and assigning it to the block where the centroid is located; S4 Target Locking: The baseline block is determined based on the number of moving targets: For a single target, the block containing that target is used as the baseline block; for multiple targets, the average center point of all moving targets is calculated, and the block containing the average center point is used as the baseline block. If the average center point of a single or multiple targets is within the baseline block, the large turntable remains stationary; if the average center point of a single or multiple targets enters a new block, the large turntable is driven to pull it back to the baseline block. The threat level of the moving targets is calculated, and moving targets are locked sequentially according to their threat level. The block number of the telephoto field of view center is obtained through image matching, and the interval mapping table is consulted in conjunction with the target block number to obtain the small turntable's... S5: Target Recognition: Target recognition is performed on moving targets in the two telephoto images output by the telephoto SWIR-RGB camera. The target type is retained, and interference types are eliminated. If the recognition results of the two telephoto images conflict, the RGB image is analyzed. If the average brightness of the RGB image is lower than threshold T1 and the contrast is lower than threshold T2, it is determined to be a low-light or severe weather environment, and the recognition result of the SWIR image shall prevail; otherwise, the recognition result of the RGB image shall prevail. For the target with the highest threat level, if the target is located in the central area, the large turntable remains stationary, and the small turntable adjusts its angle based on the interval mapping table and the positional deviation within the area. If the target is located in the transition area, the large turntable rotates at a first speed to pull the target back to the central area, and the small turntable immediately compensates by the same angle in the opposite direction. If the target is located in the edge area, the large turntable rotates at a second speed greater than the first speed to pull the target back to the central area, and the small turntable remains stationary. If the target is not detected within a preset time T, the threat level of the target within the short focal length field of view is recalculated, and the target with the highest threat level continues to be tracked. If there are no other targets within the short focal length field of view, the process returns to step S3.

5. The method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S1, the preprocessing includes: performing bad pixel correction, lens distortion correction and grayscale enhancement, and performing dehazing enhancement on the RGB image; and performing image registration on the short-focus SWIR image and MWIR image, and the long-focus SWIR image and RGB image, respectively.

6. The method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S2, the two short-focus images are divided into N×N uniform blocks, where N is a positive integer, and the blocks are numbered sequentially from 0 to N. 2 -1, set the central block of size [N / 3]×[N / 3] as the central region, the outermost block of size [N / 9] as the edge region, and the remaining blocks as the transition region, [ ] indicates rounding up.

7. The method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S4, the threat level of the moving target is calculated based on a weighted average of the target's speed, distance, and size characteristics, using the following formula: in, The threat level of a moving target. For the target radial velocity, This represents the maximum speed. For the target distance, The minimum distance, The pixel area occupied by the target in the short focal length image. The maximum pixel area. 、 、 All are weighting coefficients.

8. The method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S4, the rotation angle of the large turntable is calculated based on the field of view, image resolution, and pixel deviation between the target block and the center block of the short-focal-length SWIR-MWIR camera.

9. The method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S4, the interval mapping table is established through a pre-calibration process, which is used to determine the correspondence between each block on the short focal length image and the deflection angle of the small turntable.

10. A method for tracking small moving targets in the air based on dual-band common aperture and dual turntable linkage according to claim 4, characterized in that, In step S4, the rotation angle of the small turntable is a closed-loop control process, and its control quantity is calculated based on the real-time pixel deviation between the centroid of the target and the center of the field of view in the telephoto image.

Images