Unmanned aerial vehicle adaptive target identification method and system based on optical detection

Abstract

The invention relates to the technical field of optical recognition, and further relates to an unmanned aerial vehicle self-adaptive target recognition method and system based on optical detection, and the method comprises the following steps: 1, collecting a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and carrying out the alignment preprocessing of the image frame pair to obtain an aligned frame pair; step 2, based on the aligned frame pair, respectively generating a neutral comparison graph corresponding to the visible light image frame and a neutral heat graph corresponding to the infrared image frame, and generating a topological manifold graph based on the neutral comparison graph and the neutral heat graph; 3, threshold segmentation is executed in the neutral comparison graph and the neutral heat graph, screening is carried out in combination with geometric and spatial constraints, and a candidate set is obtained; and if the judgment is passed, outputting an identification result according to the shape trunk characteristics. According to the method, the recognition accuracy, robustness and practical value are remarkably improved in a complex background and a dynamic environment.

Description

Technical Field

[0001] This invention belongs to the field of optical detection technology, specifically relating to an adaptive target recognition method and system for unmanned aerial vehicles based on optical detection. Background Technology

[0002] In the development of UAV target detection and recognition technology, with the widespread application of UAVs in both civilian and military fields, the need for reliable detection and classification in complex environments is becoming increasingly urgent. Existing UAV detection and recognition methods are mainly divided into single-sensor imaging methods and multi-sensor information fusion methods. Single-sensor based detection methods include visible light image analysis and infrared imaging analysis. Visible light imaging relies on natural lighting conditions, extracting and analyzing edge, texture, and color features from video streams or static images to identify UAV targets. These methods have high recognition accuracy in well-lit environments with simple backgrounds, but in low light, backlight, shadow occlusion, or complex backgrounds, image contrast and edge sharpness decrease significantly, leading to a sharp decline in detection performance. Infrared imaging methods rely on the thermal radiation characteristics generated by the UAV's body, power system, and electronic equipment, and can still work stably at night or in low-light environments. However, infrared images typically have low resolution, insufficient detail, and are easily affected by changes in ambient temperature, background heat sources, and meteorological conditions (such as fog and rain), resulting in unstable thermal characteristics. In addition, infrared imaging has limited ability to distinguish the fine shapes of small targets at long distances, and it is difficult to accurately distinguish drone types by relying solely on thermal distribution.

[0003] To overcome the limitations of single sensors, existing technologies have developed fusion methods for visible light and infrared imaging, aiming to obtain complementary information under different lighting and background conditions. A common approach in these methods is to spatially register and align visible light and infrared images, and then fuse them at the pixel or feature level, for example using weighted superposition, feature stitching, or deep learning multimodal fusion models. However, existing multi-sensor fusion methods still face several prominent problems in practical engineering: First, insufficient registration accuracy leads to cross-channel information misalignment. Sensors of different bands have inherent parallax and distortion due to differences in optical systems, installation positions, and imaging characteristics. Especially when imaging medium- to long-range targets, even slight registration errors can cause inconsistencies in boundaries, textures, and thermal areas, thus affecting the fusion effect and subsequent recognition. Second, insufficient suppression of environmental interference during the fusion process. Visible light images are prone to false detections under conditions of strong reflection, shadows, and complex background textures, while infrared images may produce false targets when background heat sources are present. Existing methods often simply superimpose or average the two types of information, lacking targeted suppression strategies, resulting in a still relatively high false detection rate. Third, existing fusion detection methods rely heavily on the saliency of a single channel or simple geometric constraints based on the bounding rectangle for candidate region selection and target identification. They lack comprehensive utilization of multi-dimensional features such as target geometry, boundary morphology, and heat distribution patterns. Especially in distinguishing UAV types, existing technologies struggle to accurately differentiate between multi-rotor and fixed-wing aircraft with significant structural differences. Fourth, existing methods lack robustness in complex backgrounds or dynamic scenes. When there are moving objects (such as birds, wind-blown leaves, or cloud changes) or dynamic heat sources in the background, the detection system often struggles to stably track and accurately identify UAV targets. This is particularly true when the background and target are similar in size and speed, leading to missed detections and misclassifications. Summary of the Invention

[0004] The main objective of this invention is to provide an adaptive target recognition method and system for unmanned aerial vehicles (UAVs) based on optical detection. By simultaneously acquiring visible light and infrared image frames and performing high-precision alignment preprocessing, accurate temporal and spatial matching of cross-channel data is achieved. Combined with the construction of neutral contrast maps and neutral heatmaps, interference from illumination, reflection, and false heat sources is effectively suppressed. Furthermore, multi-dimensional structured feature fusion is achieved using topological manifold maps. In the candidate region screening and consistency determination process, multi-channel joint constraints based on geometric, textural, and thermal features are employed, significantly reducing the false detection rate. It can also automatically distinguish between multi-rotor and fixed-wing UAVs based on their core shape features, thereby significantly improving the accuracy, robustness, and practical value of recognition in complex backgrounds and dynamic environments.

[0005] To solve the above problems, the technical solution of the present invention is implemented as follows: An adaptive target recognition method for unmanned aerial vehicles (UAVs) based on optical detection, the method comprising the following steps: Step 1: Acquire a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and then perform alignment preprocessing on the image frame pair to obtain an aligned frame pair; Step 2: Based on the aligned frame pairs, generate neutral contrast maps corresponding to visible light image frames and neutral heat maps corresponding to infrared image frames, and generate topological manifold maps based on the neutral contrast maps and neutral heat maps. Step 3: Perform threshold segmentation on the neutral contrast map and the neutral heatmap respectively, and filter them by combining geometric and spatial constraints to obtain a candidate set; extract multi-dimensional features for each candidate region in the candidate set, and make consistency judgment according to preset rules. If the judgment passes, output the recognition result based on its shape backbone features.

[0006] Furthermore, in step 1, the alignment preprocessing includes: performing timestamp alignment on the visible light image frame and the infrared image frame, and then performing a 3x3 median filter on the visible light image frame and the infrared image frame respectively.

[0007] Furthermore, the alignment preprocessing process specifically includes: on the visible light image frame, edge detection is used to obtain a binary edge map; on the binary edge map, corner point detection is used to obtain a set of visible light corner points, and line segment detection is used to obtain a set of visible light line segments; on the infrared image frame, threshold segmentation is used to obtain a binary map of the thermal target; morphological opening and closing operations are performed on the binary map of the thermal target to extract an infrared contour set; and contour corner points are uniformly sampled on the infrared contour to obtain an infrared corner point set; matching pairs are generated based on the parallelism, approximate collinearity, and proximity of endpoints between the visible light line segments and the infrared contour; random sampling consistency screening is used to remove outliers from the matching pairs, and geometric transformations between the two image frames are fitted, prioritizing perspective transformations; if a stable fit is not possible, it degenerates into an affine transformation; the fitted perspective or affine transformation is used to perform geometric correction on the infrared image frame, aligning the infrared image frame and the visible light image frame in the same pixel coordinate system, generating aligned frame pairs, and generating a bidirectional reversible mapping table for coordinate mapping.

[0008] Furthermore, the process of generating a neutral contrast map in step 2 includes: performing the following process on the visible light image frames of the aligned frame pairs: using top-hat and bottom-hat operations to suppress isolated strong reflections and deep shadows; performing contrast-limited histogram equalization on the processed image to enhance the contrast of medium brightness areas; and using guided filtering to perform edge-preserving smoothing in strong reflection candidate areas to weaken reflection peaks while preserving edge positions, thereby obtaining a neutral contrast map.

[0009] Furthermore, the process of generating a neutral thermal image in step 2 includes: performing the following process on the infrared image frames of the aligned frame pairs: obtaining a binary mask of hot and cold areas by threshold segmentation; performing morphological reconstruction on the hot area mask to remove pseudo heat sources with too small an area; performing edge-preserving smoothing on the reconstructed hot areas and performing local contrast stretching on the cold areas to obtain a neutral thermal image.

[0010] Furthermore, the process of generating the topological manifold map in step 2 includes: ensuring that the neutral contrast map and the neutral heatmap are pixel-aligned based on a bidirectional reversible mapping table; marking the connected components of the neutral contrast map and the neutral heatmap respectively; extracting the shape backbone for each connected component and calculating its hole count, connectivity level, boundary chain code histogram, boundary inflection point sequence and binary texture pattern histogram; and establishing an entry-level correspondence between the two spatially overlapping connected components through the bidirectional reversible mapping table to form the topological manifold map.

[0011] Furthermore, in step 3, the process of obtaining the candidate set includes: on the neutral contrast map, segmenting it using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a contrast candidate mask, and then removing burrs and discarding areas with an area smaller than a first set area threshold through an opening operation; on the neutral heatmap, segmenting it using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a heat candidate mask, and then removing burrs and discarding areas with an area smaller than a second set area threshold through an opening operation; projecting the heat candidate mask and the contrast candidate mask onto the same coordinate system, calculating their overlap ratio, and retaining only areas that meet the following conditions as candidate areas: the overlap ratio is not lower than a set lower overlap limit, and the aspect ratio of its bounding rectangle falls within a set aspect ratio range; all candidate areas together constitute the candidate set.

[0012] Furthermore, in step 3, the consistency determination process includes: for each candidate region, obtaining the corresponding entry from the topological manifold graph and generating a configuration signature, which includes: the aspect ratio of the circumscribed rectangle, the proportion of the shape backbone length, the number of holes, the connectivity level, the concentration of the main direction of the chain code histogram, the boundary transition density, the high-frequency proportion of the binary texture pattern, the proportion of hot areas, the continuity of hot boundaries, and the density of hot-cold junctions; when a candidate region meets at least one of the following criteria, it is determined to pass: geometric determination passes: the aspect ratio of the candidate region is within the set aspect ratio range, is greater than the set first aspect ratio threshold, the proportion of the shape backbone length exceeds the set length threshold, and the number of holes is not greater than one; boundary texture determination passes: the main direction of the chain code of the candidate region is concentrated, the boundary transition density is within the set density range, and the high-frequency proportion of the binary texture pattern is within the set proportion range; thermal feature determination passes: the proportion of hot areas of the candidate region is within the set hot area range, the hot boundary is continuous, and the density of hot-cold junctions is within the set hot-cold junction density range.

[0013] Furthermore, the process of outputting the recognition result includes: if a candidate region passes the consistency judgment, then its shape backbone features are further judged: if the shape backbone is cross or star-shaped, and the aspect ratio of its circumscribed rectangle is close to 1, then the output recognition result is a multi-rotor drone; if the shape backbone is long and thin and sharp at both ends, then the output recognition result is a fixed-wing drone.

[0014] An adaptive target recognition system for unmanned aerial vehicles (UAVs) based on optical detection includes: an acquisition unit for acquiring a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and then performing alignment preprocessing on the image frame pair to obtain an aligned frame pair; an image processing unit for generating a neutral contrast map corresponding to the visible light image frame and a neutral heat map corresponding to the infrared image frame based on the aligned frame pair, and generating a topological manifold map based on the neutral contrast map and the neutral heat map; and a recognition unit for performing threshold segmentation on the neutral contrast map and the neutral heat map respectively, and filtering by combining geometric and spatial constraints to obtain a candidate set; extracting multi-dimensional features for each candidate region in the candidate set, and performing consistency judgment according to preset rules; if the judgment passes, the recognition result is output based on its shape backbone features.

[0015] The UAV adaptive target recognition method and system based on optical detection of the present invention has the following beneficial effects: By acquiring visible light image frames and infrared image frames at the same time point and performing high-precision alignment preprocessing, strict synchronization of multispectral data in time and space is ensured, effectively eliminating the impact of cross-channel information misalignment on subsequent recognition. Based on this, the present invention constructs a neutral contrast map for the visible light channel to suppress isolated strong reflections and deep shadow interference, enhancing the contrast of medium-brightness areas; and constructs a neutral heat map for the infrared channel to remove false heat sources with excessively small areas and enhance the detail representation of effective heat areas, thereby improving the effective target saliency of both channels in complex environments. Furthermore, the present invention utilizes a bidirectional reversible mapping table to achieve precise pixel alignment between the neutral contrast map and the neutral heat map, and extracts multi-dimensional topological and texture features to construct a topological manifold map, providing a unified structured description for cross-channel target association and consistency determination. In the candidate region screening stage, the present invention effectively eliminates non-target interference areas and significantly reduces the false detection rate by combining histogram peak-valley threshold segmentation, morphological operations, area constraints, overlap ratio calculation, and aspect ratio range screening. In the consistency determination stage, this invention achieves refined target screening through multi-channel joint constraints of geometric features, boundary texture features, and thermal features. Based on this, it outputs the UAV type according to the main shape features, realizing automatic differentiation between multi-rotor and fixed-wing aircraft. This design not only maintains high recognition accuracy in static complex backgrounds but also possesses stability and robustness in dynamic scenes. It can effectively adapt to complex conditions such as changes in lighting, background interference, and multi-target scenarios, thereby significantly improving the reliability and practical value of the UAV target recognition system in combat applications. Attached Figure Description

[0016] Figure 1 A schematic diagram of the method flow for the UAV adaptive target recognition method based on optical detection provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of histogram bimodal distribution threshold segmentation analysis in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the principle of shape backbone feature determination and analysis in an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] refer to Figure 1 An adaptive target recognition method for unmanned aerial vehicles (UAVs) based on optical detection, the method comprising the following steps: Step 1: Acquire a pair of visible light and infrared image frames at the same time point to form an image frame pair. Timestamp alignment is used to ensure that the image frame pair reflects scene information at the same observation instant. Timestamp alignment is used to correct micro-time-sequential deviations caused by sampling and transmission delays from different sensors, and to remove abnormal frames or remap nearby frames, ensuring that moving targets within the two channels are consistent in time. After completing time-domain synchronization, a 3x3 median filter is applied to both the visible light and infrared image frames to suppress transient noise, fine speckle, and salt-and-pepper interference, while minimizing excessive smoothing of edges and textures, thus creating conditions for stable extraction of geometric elements.

[0019] Building upon this foundation, to achieve spatial domain alignment, edge detection is performed on visible light image frames to generate binary edge maps. Corner detection is then performed on these binary edge maps to obtain a set of visible light corner points, and line segment detection is used to obtain a set of visible light line segments. For infrared image frames, threshold segmentation is used to obtain a binary map of the thermal target. Morphological opening and closing operations are then performed on the binary map of the thermal target to remove isolated pseudo-heat sources and correct regular boundaries. Contour corner points are then uniformly sampled from the corrected infrared contour set to obtain an infrared corner point set. Utilizing the geometric relationship between visible light line segments and infrared contours, matching pairs are generated based on the aforementioned parallelism, approximate collinearity, and endpoint proximity as matching conditions. Outlier removal is performed on the matching pairs through random sampling consistency screening to improve the robustness of the estimated geometric relationship.

[0020] With the support of effective matching pairs, a geometric transformation between two image frames is fitted. Perspective transformation is fitted first to accommodate projection differences between imaging planes. When the scene structure or feature distribution is insufficient to stably support the perspective model, it degenerates into an affine transformation to maintain alignment stability and feasibility. Subsequently, the fitted perspective or affine transformation is used to perform geometric correction on the infrared image frame, mapping it to the pixel coordinate system of the visible light image frame. This aligns the two-channel data in the same pixel coordinate system, generating aligned frame pairs. During the geometric correction process, a bidirectional reversible mapping table for coordinate mapping is simultaneously generated. This table records the forward and reverse correspondences between pixels in the visible light and infrared image frames and can identify interpolated, occluded, and invalid regions, facilitating subsequent precise pixel-level association and region projection between the neutral contrast map and the neutral heatmap.

[0021] Step 2: Based on the aligned frame pairs and bidirectional reversible mapping table obtained in Step 1, ensure that the visible light image frame and the infrared image frame can be bidirectionally mapped and accessed consistently at every pixel position. In the visible light image frame channel, to construct a neutral contrast map, top-hat and bottom-hat operations are first performed on the original brightness to suppress the intensity bias caused by isolated strong reflections and deep shadows. Then, after the residual uneven illumination is suppressed, contrast-limited histogram equalization is performed on the overall image to highlight the weak contrast in the medium brightness area, so that the slender fuselage, propeller outline, arms and other structures have a more balanced local contrast performance in the background.

[0022] For candidate regions with potentially strong reflections, guided filtering is used for edge-preserving smoothing. By leveraging the structural priors of the original image, the contour sharpness is maintained at the edges, while the reflection peaks within the regions are weakened. This reduces the interference of high-light saturation on boundary detection and connected component labeling, ultimately resulting in a neutral contrast map that exhibits robust boundary performance even against backgrounds of strong light, shadow, and complex textures. In the infrared image frame channel, to construct a neutral thermal map, binary masks for hot and cold regions are first obtained by threshold segmentation. Morphological reconstruction is used to remove pseudo-heat sources that are too small, irregularly shaped, and lack spatial continuity with the real thermal target. Edge-preserving smoothing is then applied to the reconstructed hot regions to reduce broken boundaries caused by thermal noise and local temperature fluctuations, while maintaining the continuity of thermal boundaries for possible parts such as the fuselage, wings, rotor motors, or tail nozzles. For cold regions, local contrast stretching is performed to flatten the slow gradient of the background temperature field, highlighting subtle intensity transitions at the hot-cold interface. This results in a neutral thermal map that is more stable and significant for the real thermal target in complex thermal scenes.

[0023] After neutralization enhancement of both channels, a bidirectional reversible mapping table is used to ensure strict alignment between the neutral contrast map and the neutral heatmap at the pixel level. Connectivity labeling is performed on each channel to obtain several candidate connected regions with closed boundaries and internal consistency. For each connected region, the shape backbone is extracted to characterize its elongation and main axis connectivity. At the same time, the number of holes and the connectivity level are calculated to reflect the topological features brought about by possible blade gaps, arm intersections, or installation structures inside the target. At the boundary level, the boundary chain code histogram and boundary inflection point sequence are calculated to record the main direction distribution and inflection density of the contour, thereby forming a relatively stable description of the cross structure of the multirotor, the slender edge of the fixed wing, and the connection relationship between the fuselage and auxiliary components. At the texture level, the binary texture pattern histogram is statistically analyzed to express the proportion of binary texture patterns inside the connected region and the boundary neighborhood, helping to distinguish between false contours caused by background textures or ground feature details and real structures generated by UAV entities.

[0024] Subsequently, a bidirectional reversible mapping table is used to establish entry-level correspondences between the spatially overlapping or nearest-neighbor connected components of the two channels, forming a topological manifold map containing cross-channel paired entries, geometric attributes, texture statistics, and thermal features. In this topological manifold map, each entry can be regarded as a unified description of the same spatial entity under both visible and infrared observations. It includes information such as contour sharpness, shape backbone length ratio, and boundary transition density from the neutral contrast map, as well as thermal information such as hot zone ratio, thermal boundary continuity, and hot-cold interface density from the neutral thermal map. The adjacency, inclusion, and contact relationships between entries constitute a connectivity hierarchy and spatial relationship network, which reflects the structural organization between the target and the background, and between target components, in complex scenes.

[0025] The topological manifold map generated through the above process, on the one hand, complements the advantages of visible light image frames and infrared image frames into a unified data structure, reducing ambiguity caused by strong light reflection, shadow occlusion, uneven heat source distribution, or complex background texture in a single channel; on the other hand, it provides a direct basis for subsequent steps to perform threshold segmentation result projection, geometric and spatial constraint screening, and configuration signature generation under the same reference coordinates and the same entry semantics, so that UAV targets can still be robustly identified through entry-level consistency under cross-spectral condition changes and scene disturbances.

[0026] Step 3: Based on the neutral contrast map and neutral heatmap generated from the aligned frame pairs, the target items are screened, measured, and judged under a unified pixel coordinate system. After passing the consistency judgment, the recognition result is output based on the main shape features. First, threshold segmentation is performed in the neutral contrast map, using the lowest point between the second peak and the first peak of its histogram as the threshold to obtain the contrast candidate mask. An opening operation is used to remove burrs and eliminate regions with an area smaller than the first set area threshold, thus constraining false responses caused by reflections, shadows, or fine textures. Subsequently, the same threshold segmentation strategy is performed in the neutral heatmap, using the lowest point between the second peak and the first peak of its histogram as the threshold to obtain the heat candidate mask. Similarly, an opening operation is used to remove burrs and eliminate regions with an area smaller than the second set area threshold to reduce interference from isolated false heat sources or noisy hotspots.

[0027] The thermal candidate mask and the contrast candidate mask are projected onto the same coordinate system, and their spatial overlap ratio is calculated. Only regions that meet the set lower overlap limit and whose circumscribed rectangle aspect ratio falls within the set range are retained as candidate regions. All candidate regions together constitute the candidate set. To maintain semantic consistency across channel information, when extracting multi-dimensional features for each candidate region in the candidate set, the entry-level correspondence established by the topological manifold map is directly used to retrieve the two-channel connected domain entries that spatially overlap with the candidate region, comprehensively obtaining geometric, boundary, and thermal elements. During feature construction, a configuration signature for consistency analysis is generated. The configuration signature includes elements such as the circumscribed rectangle aspect ratio, the proportion of the shape backbone length, the number of holes, the connectivity level, the concentration of the main direction of the chain code histogram, the boundary transition density, the high-frequency proportion of the binary texture pattern, the proportion of hot areas, the continuity of hot boundaries, and the density of hot-cold interfaces. This allows the boundary and shape information from the neutral contrast map and the thermal distribution information from the neutral heat map to be associated and expressed under the same entry.

[0028] When performing consistency determination based on preset rules, the criteria are based on three categories: geometric determination, boundary texture determination, and thermal feature determination. A candidate region is considered to have passed if, in geometric determination, its aspect ratio is within a set range and greater than a set first aspect ratio threshold, the length of its main shape exceeds a set length threshold, and the number of holes is no greater than one. Similarly, a candidate region is considered to have passed if, in boundary texture determination, its chain code main direction is concentrated, its boundary transition density is within a set density range, and the high-frequency ratio of its binary texture mode is within a set ratio range. Likewise, a candidate region is considered to have passed if, in thermal feature determination, its hot zone ratio is within a set hot zone range, its hot boundary is continuous, and its hot-cold interface density is within a set hot-cold interface density range. Meeting any of these preset rules constitutes a successful consistency determination. For candidate regions that pass the consistency determination, the recognition result is further output based on their shape backbone features: when the shape backbone is cross or star and the aspect ratio of its circumscribed rectangle is close to 1, the output recognition result is a multi-rotor drone; when the shape backbone is long and thin and sharp at both ends, the output recognition result is a fixed-wing drone.

[0029] Furthermore, the goal of alignment preprocessing is to ensure the consistency of visible light image frames and infrared image frames in both the temporal and spatial domains. This allows the data from both channels to be jointly analyzed in the same pixel coordinate system and form aligned frame pairs, providing reliable input for subsequent generation of neutral contrast maps, neutral heatmaps, and topological manifold maps. First, timestamp alignment is performed to eliminate triggering errors and latency differences caused by different imaging links during acquisition, buffering, and transmission. Timestamp alignment uses the timestamps of each pair of image frames as a basis to match neighboring frames across channels. If necessary, abnormal frames that do not meet the same time point requirement are removed, or a resampling strategy for adjacent times is adopted to ensure that the temporal position of the same moving target remains consistent in both channels, avoiding cross-channel time shifts caused by the relative motion of the aircraft and the background. After completing temporal domain synchronization, a 3x3 median filter is applied to both the visible light and infrared image frames. The median operator is used to suppress impulse noise, speckle, and random brightness abrupt changes, while minimizing excessive smoothing of edges and details, providing conditions for stable extraction of subsequent geometric elements. Subsequently, the spatial domain enters the stage of structured feature construction and cross-channel constraint matching. On the visible light image frame, edge detection is first used to obtain a binary edge map. The binarized edge map can highlight the structural outline without relying on the absolute value of illumination. On the binary edge map, a set of visible light corner points is obtained through corner point detection to capture the location of local curvature or strong directional changes, thereby enhancing the repeatability and distinguishability of features. A set of visible light segments is obtained through line segment detection to extract straight line elements formed by the arms, fuselage edges, blade projections, or ground structures, providing a basis for subsequent geometric relationship constraints.

[0030] On the infrared image frame, a binary image of the thermal target is obtained by threshold segmentation to highlight hot areas that may be caused by power systems, electronic loads, or frictional heating. Morphological opening and closing operations are performed on the binary image of the thermal target to extract an infrared contour set. Opening operations help remove pseudo-heat sources with excessively small areas, while closing operations help repair fracture boundaries caused by thermal noise, making the infrared contour more consistent with the shape continuity of a real object. Contour corner points are uniformly sampled on the infrared contour to obtain an infrared corner point set, thus forming discrete structural anchor points in the thermal representation corresponding to the visible light corner point set. Based on the above two-channel element sets, an initial cross-channel association is established: matching pairs are generated using the parallelism, approximate collinearity, and endpoint proximity between the visible light segment and the infrared contour as matching conditions. This matching condition utilizes the consistency of the visible light edge and the infrared contour in macroscopic geometric direction, while suppressing mismatches caused by accidental parallelism through endpoint proximity constraints. To improve the robustness of the matching, random sampling consistency screening is used to remove outliers from the matching pairs. The model interior point set that best explains the vast majority of matching pairs is selected in several random subset evaluations, reducing erroneous associations caused by noise, local occlusion, and non-rigid backgrounds. Based on the valid matching pairs after consistency screening, the geometric transformation between the two image frames is fitted. Perspective transformation is fitted first to adapt to the projection differences caused by imaging geometry, viewpoint changes and the assumption of scene planarization. When the scene structure, feature distribution or disparity conditions are insufficient to stably support the perspective model, it degenerates into affine transformation to ensure that a stable spatial mapping can still be obtained under the condition of sparse features or minimal disparity.

[0031] Geometric correction is performed on the infrared image frame using the fitted perspective or affine transformation, mapping the infrared image frame to the pixel coordinate system of the visible light image frame. This achieves pixel-by-pixel alignment of the two channels in the same pixel coordinate system, generating aligned frame pairs. During the geometric correction process, to ensure the interpretability and efficiency of subsequent cross-channel data interaction, a bidirectional reversible mapping table is generated for coordinate mapping. This bidirectional reversible mapping table records the forward mapping from the visible light image frame to the infrared image frame and the inverse mapping from the infrared image frame to the visible light image frame. It includes the correspondence of pixel coordinates, interpolation method identifiers, invalid region markers due to boundary extrapolation or parallax occlusion, and local registration quality indicators. This allows for reliable region projection, connected component correspondence, and feature convergence between the two channels in subsequent processing.

[0032] Furthermore, the purpose of generating a neutral contrast map on the visible light image frames of the aligned frame pairs is to suppress the intensity bias caused by illumination and materials and improve the separability of structural boundaries, so as to provide a stable contrast basis for subsequent connected component labeling, configuration signature construction and entry-level correspondence in a unified pixel coordinate system. In the specific implementation process, firstly, top-hat operation and bottom-hat operation are applied sequentially on the visible light image frames of the aligned frame pairs. The top-hat operation is used to weaken the local bright spikes caused by isolated strong reflections, so that the abnormal highlights generated by specular reflection, wet surfaces or metal parts no longer dominate the local intensity distribution; the bottom-hat operation is used to fill the dark pits and boundary collapses caused by deep shadows, reduce the low brightness trail caused by backlighting, occlusion or ground undulations, so as to achieve a more balanced appearance of background brightness without changing the real geometry. After the above morphological shaping, the global and local brightness imbalance of the image is significantly alleviated. However, there may still be insufficient contrast in the medium brightness area. Therefore, the processed image is subjected to contrast-limited histogram equalization. Through block histogram redistribution and over-amplification suppression mechanism, the gradient visibility of the medium brightness area is significantly enhanced while keeping the bright and dark areas from oversaturating. This allows the slender arms, blade roots, fuselage outlines and supports to obtain clearer grayscale transitions in complex background textures.

[0033] Considering that unavoidable strong reflections in the environment may still remain locally and interfere with boundary detection and subsequent connected component labeling, guided filtering is further employed for edge-preserving smoothing within strong reflection candidate regions. Strong reflection candidate regions can be obtained jointly based on the brightness peak distribution and neighborhood gradient stability, forming a mask to limit the processing range. Guided filtering uses visible light image frames as a guide, performing edge-preserving smoothing within the candidate regions to uniformly weaken the reflection peaks of highlight areas without crossing real edges, while maintaining the sharpness of grayscale transitions at edges, thus avoiding boundary expansion and virtual connectivity caused by highlight tailing. This strategy ensures that high-contrast regions do not experience edge drift due to simple smoothing and also avoids over-enhancing low-contrast regions, resulting in a globally neutral contrast representation of the image, i.e., a neutral contrast map. The generation of neutral contrast images emphasizes two points: first, after suppressing isolated strong reflections and deep shadows, the background intensity distribution of the scene is brought back to a scale comparable to the real target structure; second, while enhancing the medium brightness areas, an edge-preserving strategy is adopted to ensure that the slender structures, cross connections, and local gaps related to the drone are not blurred or misconnected.

[0034] Since the aligned frame pairs are strictly aligned with the infrared channel through a bidirectional reversible mapping table, the neutral contrast map can not only be directly used for edge extraction and connected component labeling in this channel, but also provide reliable geometric contour information when performing entry-level registration with the neutral heatmap in the subsequent process. This allows the saliency from thermal distribution and the shape information from visible light boundaries to complement each other in the same pixel coordinate system. To ensure the stability of the downstream decision-making process, the neutral contrast map can be directly used for threshold segmentation after generation. Its histogram structure, after contrast-limited histogram equalization, is more bimodal and separable, which is beneficial for using the lowest point between the second peak and the first peak as the threshold for segmentation and obtaining a contrast candidate mask with less noise. At the same time, edge-preserving smoothing suppresses the reflection peaks in the strong reflection candidate regions, so that these regions no longer form unnecessary false large connected components during the segmentation and morphological processing stages.

[0035] Furthermore, the purpose of generating a neutral thermal map on the infrared image frames of the aligned frame pairs is to establish an intensity basis that stably expresses thermal saliency and accurately corresponds to the visible light channels, enabling subsequent connected component labeling, entry-level correspondence, and configuration signature construction to be reliably performed in the same pixel coordinate system. Specifically, firstly, relying on the aligned frame pairs and a bidirectional reversible mapping table, it is ensured that each pixel position in the infrared image frame has a definite spatial correspondence during cross-channel processing. Subsequently, binary masks for hot and cold regions are obtained by threshold segmentation. Potential heat sources caused by power systems, electronic components, or aerodynamic friction are concentrated and marked within the hot region mask, while environmental background, sky areas, or ground shadows are distinguished within the cold region mask, thus initially completing the foreground and background division of thermal distribution. Considering that infrared imaging is susceptible to random thermal noise, isolated hotspots, and sensor drift, directly using the initial thermal mask can easily generate a large number of pseudo-heat sources that are too small, irregular in shape, and lack spatial continuity during subsequent connected component analysis. Therefore, morphological reconstruction of the thermal mask is performed. By using the original mask as a constraint and restoring the connectivity consistency of the reliable structure, pseudo-heat sources with too small an area are deleted, and minor breaks caused by noise are repaired. This makes the reconstructed thermal area more consistent with the overall physical shape and boundary integrity of the target. After obtaining the reconstructed thermal area, to balance the continuity of the thermal boundary and the stability of the internal texture, edge-preserving smoothing is performed on the reconstructed thermal area. The edge-preserving strategy is used to suppress high-frequency noise and micro-brightness fluctuations caused by internal temperature fluctuations without crossing the boundary. This ensures that areas related to the UAV, such as the fuselage, wings, rotor motors, or heat-generating accessories, exhibit a continuous, stable, and easily labeled intensity distribution in terms of connectivity.

[0036] Meanwhile, to improve the visibility of subtle gradients at the hot-cold interface in the background area, local contrast stretching is performed on the cold area. This inter-regional grayscale stretching enhances the expression of the hot-cold interface density, making subtle intensity changes in the boundary transition zone easier to separate in subsequent boundary detection and thresholding, reducing boundary adhesion and false mergers caused by slow temperature gradients. The neutral thermal map obtained through these operations exhibits stronger discriminative power of hot area proportions, higher thermal boundary continuity, and more stable hot-cold interface density representation in complex backgrounds compared to the original infrared image frame. When performing entry-level registration with the neutral contrast map, this neutral thermal map can achieve precise pixel-to-pixel projection through a bidirectional reversible mapping table, thereby providing structural evidence from a thermal dimension for each entry in the topological manifold map, including information such as the spatial distribution of hot areas, boundary orientation consistency, and local intensity gradients at the interface. In downstream processing, the neutral heatmap directly participates in threshold segmentation and connected component labeling. Its connectivity after morphological reconstruction and boundary stability after edge preservation and smoothing significantly reduce the interference of pseudo heat sources, hot spot fragmentation and boundary jaggedness on candidate region generation. Local contrast stretching makes the cold area background present a more balanced intensity field, which makes it easier to obtain a clearer bimodal structure when using the lowest point between the second peak and the first peak as the threshold for segmentation, thereby improving the accuracy of candidate region selection.

[0037] Furthermore, the process of generating the topological manifold map first relies on the strict registration of the neutral contrast map and the neutral heatmap within the same pixel coordinate system. Therefore, a bidirectional reversible mapping table is used as the sole basis for pixel-level association. Pixel alignment is checked on the two-channel images, and uncertain pixels with occlusion, extrapolation, or interpolation are marked to avoid contamination in subsequent statistics. After confirming pixel alignment, connected component labeling is performed on the neutral contrast map and the neutral heatmap respectively. Regions with closed boundaries and internal consistency are divided into several connected components, and the bounding rectangle, area, perimeter, and adjacency relationships of each connected component are recorded. At the same time, the original pixel index is maintained for direct interaction with the bidirectional reversible mapping table. Subsequently, shape backbones are extracted for each connected component. These shape backbones describe the connected skeleton of slender and branched structures, stably representing slender or intersecting features such as the UAV arms, fuselage, and propeller projections. Based on this, the number of holes is calculated to reflect internal cavities, propeller gaps, or internal voids caused by component occlusion. The connectivity hierarchy is calculated to characterize the inclusion and containment relationships, contact relationships, and adjacency order between connected components, forming a topological sequence composed of parent-child and sibling relationships. Boundary chain code histograms are further calculated for each connected component based on boundary and texture features to statistically analyze the distribution and stability of the main boundary directions. Boundary inflection point sequences are calculated to record the location and order of directional abrupt changes on the boundary, characterizing the degree of angularity and the rhythm of the broken lines. Binary texture pattern histograms are calculated to statistically analyze the frequency of binary texture patterns within the connected component and its boundary neighborhood, reflecting the differences in binary structure caused by material, propeller modulation, or background texture.

[0038] After representing the connected components of each of the two channels, an entry-level correspondence is established between the spatially overlapping connected components of the neutral contrast map and the neutral heatmap using a bidirectional reversible mapping table: based on pixel mapping and spatial overlap, the connected components in the neutral contrast map are used as the main body of the candidate entries, and the connected components in the neutral heatmap that have effective overlap or adjacent boundaries are paired to generate entries containing the attributes of both channels; when there is a one-to-many or many-to-one situation, the entries are split or merged according to the overlap area, boundary consistency and hot-cold boundary distribution, and the correspondence and conflict resolution strategy are explicitly recorded in the entries to ensure the traceability and interpretability of the entries. The set of entries constitutes the topological manifold graph, which includes the geometric elements of the entry itself (shape backbone, number of holes, connectivity level), boundary elements (boundary chain code histogram, boundary inflection point sequence), and texture elements (binary texture pattern histogram), as well as the overlap ratio, boundary distribution, and pixel alignment quality indicators of the two channels. The relationships between entries are stored in a graph structure, recording the edge information of adjacency, containment, contact, and cross-channel mapping, thus providing a unified geometric-thermal joint representation in the same pixel coordinate system. This topological manifold graph can be directly retrieved in the subsequent candidate region generation and consistency determination stages: for any candidate region, entries and related entries with consistent spatial coverage can be found in the graph, and the calculated features such as shape backbone, number of holes, connectivity level, boundary chain code histogram, boundary inflection point sequence, and binary texture pattern histogram can be quickly retrieved, and a comprehensive evaluation can be performed by combining the hot zone distribution and hot-cold boundary information inherited from the neutral heatmap.

[0039] Furthermore, the brightness distribution on the neutral contrast map is analyzed and a histogram is constructed. The lowest point between the second and first peaks of the histogram is used as a threshold for segmentation to obtain contrast candidate masks. This threshold selection minimizes aliasing between the background and structural boundaries in a statistical sense. Since the neutral contrast map has suppressed excessive reflections and deep shadows through top-hat and bottom-hat operations, and enhanced medium-brightness areas through contrast-limited histogram equalization, the contrast candidate masks generally exhibit good bimodal separability. However, they may still contain small-scale burrs and isolated patches caused by fine textures, residual reflections, or weak shadows. Therefore, an opening operation is applied to the mask to remove burrs, and areas with excessively small areas are eliminated based on a first set area threshold, thereby preserving responses with geometric integrity and boundary continuity. Simultaneously, the same threshold determination mechanism is performed on the neutral heatmap, using the lowest point between the second and first peaks of its histogram as a threshold for segmentation to obtain thermal candidate masks. The neutral heatmap has been morphologically reconstructed to remove spurious heat sources with excessively small areas, and edge-preserving smoothing has been implemented within the hot zones. Therefore, the candidate heatmap exhibits high connectivity and boundary stability for the real hot zones. To further suppress noise-induced connected regions caused by background temperature fluctuations, opening operations are applied to the candidate heatmap to remove glitch-related artifacts, and regions with excessively small areas are eliminated based on a second set area threshold, preventing random hot spots or jagged edges from being mistakenly identified as valid targets. After constructing independent candidate masks for both channels, the candidate heatmap and the contrast candidate mask are projected onto the same coordinate system for spatial consistency constraints. This projection relies on the previously established aligned frame pairs and bidirectional reversible mapping table to achieve pixel-by-pixel mapping, ensuring that the two masks are geometrically strictly comparable.

[0040] For each connected component present in the comparison candidate mask, its spatial overlap ratio with the thermal candidate mask is calculated. To avoid occasional mismatches caused by boundary jitter or interpolation, the mapping validity and occlusion markers are filtered during the overlap calculation process, and only pixels with valid mappings are counted. Subsequently, connected components are filtered according to a preset overlap lower limit. Only when the overlap ratio is not lower than the set overlap lower limit is the region considered to have both geometric contrast and thermal significance. To constrain false detection regions with abnormal shapes, the aspect ratio of the bounding rectangle of the regions that pass the overlap filtering is calculated, and it is checked whether it falls within the set aspect ratio range. This step is used to exclude extremely elongated or extremely flat connected components formed by ground textures, linear reflections, or temperature bands, while preserving the geometric shape consistent with the projection of the UAV arm, fuselage, or propellers. For adjacent connected regions with one-to-many or many-to-one relationships, priority is given to merging or splitting them based on overlap ratio, boundary consistency, and compatibility of the circumscribed rectangle. The aspect ratio and overlap ratio of the circumscribed rectangle are then recalculated for the merged or split regions to ensure that each retained region satisfies the condition that the overlap ratio is not lower than the lower overlap limit and the aspect ratio of the circumscribed rectangle falls within the aspect ratio range.

[0041] All candidate regions obtained through the above steps constitute a candidate set. This candidate set, through the combined effects of statistical thresholds, morphological constraints, cross-channel overlap, and geometric constraints, significantly reduces false positives and false negatives caused by single-channel noise, local strong reflections, random hotspots, and background textures. This provides a moderately sized and reliable input for subsequent multi-dimensional feature extraction and consistency determination. Since each candidate region in the candidate set can be directly correlated with entries in the topological manifold graph with the support of a bidirectional reversible mapping table, information such as the shape backbone, number of holes, connectivity level, boundary chain code histogram, boundary inflection point sequence, and binary texture pattern histogram of the candidate region can be directly read. This information is then jointly evaluated with the proportion of hot zones, the continuity of hot boundaries, and the density of hot-cold interfaces. This improves the robustness and generalizability of the recognition while maintaining computational efficiency, laying a reliable foundation for the final recognition result output based on shape backbone features.

[0042] Furthermore, for any candidate region, the corresponding entry is first obtained in the topological manifold map through its spatial coverage relationship, and the joint elements from the neutral contrast map and the neutral heat map are aggregated within the entry to generate a configuration signature. The configuration signature consists of components such as the aspect ratio of the circumscribed rectangle, the proportion of the shape backbone length, the number of holes, the connectivity level, the concentration of the main direction of the chain code histogram, the boundary transition density, the high frequency ratio of the binary texture pattern, the proportion of hot areas, the continuity of hot boundaries, and the density of hot-cold junctions. Among them, the aspect ratio of the circumscribed rectangle is used to measure the elongation and directional stability of the overall shape of the candidate region; the proportion of the shape backbone length is used to reflect the dominance of slender and branched structures in the region; the number of holes and the connectivity level are used to express the internal topological organization caused by blade gaps, component occlusion, or support structures; the concentration of the main direction of the chain code histogram and the boundary transition density are used to describe the consistency of the boundary orientation and the angular complexity; the high frequency ratio of the binary texture pattern is used to distinguish the structural texture near the real target boundary from the background noise texture; and the proportion of hot areas, the continuity of hot boundaries, and the density of hot-cold junctions characterize the spatial coverage, boundary coherence, and gradient stability of the junction zone of the target's heat-generating parts from a thermal perspective. When generating configuration signatures, the mapping references from pixels to entries and from entries to candidate regions are preserved, and any local uncertainties caused by interpolation, occlusion, or parallax are marked to ensure that the contribution of uncertainty is limited in subsequent decisions.

[0043] When determining consistency based on preset rules, the satisfaction of three criteria is independently calculated for each candidate region. A strategy of passing is adopted if at least one criterion is met to enhance robustness under complex scenes and cross-channel disturbances. The geometric criteria for passing are that the aspect ratio of the candidate region is within the set aspect ratio range and greater than the set first aspect ratio threshold, the length ratio of the main shape exceeds the set length threshold, and the number of holes is no more than 1. This combination of conditions is used to filter out thick blocky regions and irregular fragmented regions caused by ground textures or shadow boundaries, so that the remaining candidate regions are more consistent with the geometric priors of UAV arms, fuselages, or airfoils. The boundary texture criteria for passing are that the main direction of the chain code of the candidate region is concentrated, the boundary turning density is within the set density range, and the high frequency ratio of the binary texture mode is within the set ratio range. This set of conditions is used to suppress boundary jitter and high frequency pseudo-textures caused by noise, reflection, or background stripes, and prioritizes the selection of entity contours with stable main directions and moderate angles, which conform to the boundary statistical characteristics of multi-rotor cross structures or fixed-wing slender edges. The conditions for passing the thermal feature determination are that the proportion of hot areas in the candidate region is within the set hot area range, the hot boundary is continuous, and the density of hot and cold junctions is within the set hot and cold junction density range. This set of conditions is used to ensure that the candidate region exhibits continuous rather than sporadic thermal significance in the infrared channel and maintains a continuous and separable intensity gradient on the boundary and junction zone, so as to exclude isolated hot spots, drifting thermal noise, or false responses caused by gradual changes in background temperature.

[0044] In actual decision-making, the three criteria are evaluated simultaneously. A consistency judgment is passed if any one criterion is met. If multiple criteria are met simultaneously, their priority levels are marked in the entry record for subsequent conflict resolution and type output reference. If a region is satisfied only at the boundary texture or thermal feature level but not at the geometric level, the candidate region is retained, and temporal consistency verification is sought in adjacent frames or multiple viewpoints to reduce the risk of missed judgments due to instantaneous occlusion or local parallax. For candidate regions that pass the consistency judgment, the process of outputting the recognition result begins, using the shape backbone features as the direct basis for type discrimination. Under the premise of the same pixel coordinate system and entry-level mapping, the overall topological shape of the shape backbone and its consistency with the aspect ratio of its circumscribed rectangle are checked first: if the shape backbone is cross-shaped or star-shaped, and the aspect ratio of its circumscribed rectangle is close to 1, the output recognition result is a multi-rotor UAV. This indicates that the candidate region contains multiple approximately orthogonal or radial main branches, consistent with the cross-support pattern formed by a multi-rotor arm under top-down or oblique-view conditions. In the neutral contrast image, this appears as a multi-directional converging contour, often accompanied by multiple point or ring-shaped hot zones and continuous thermal boundaries in the neutral heatmap. If the main branches are elongated and sharp at both ends, the output recognition result is a fixed-wing UAV.

[0045] This scenario corresponds to the elongated projection of a fixed-wing fuselage or wing along the line of sight. The main boundary directions are concentrated, and the boundary transition density is low. The aspect ratio of the circumscribed rectangle significantly deviates from 1. Simultaneously, in the neutral heatmap, it may exhibit linear or strip-shaped hot zones distributed along the main axis, forming continuous and uniformly oriented gradient bands at the hot-cold interface. To further reduce misjudgments, a consistency cross-check is performed on entries adjacent to this candidate region before type output. If there are significantly overlapping entries with different main shape types, the primary entry is selected for output based on the connectivity level and overlap degree of the entry in the topological manifold graph, and secondary entries are marked as subordinate structures or background interference. For cases where local main structures are discontinuous within the same entry, consistency correction is performed by referring to the concentration of the main direction and the continuity of the hot boundary in the chain code histogram. If necessary, the weight of components affected by occlusion in the configuration signature is reduced. After outputting the recognition result, the configuration signature components used in this decision and the threshold satisfaction status are registered in the entry history for use as a stability measure in the temporal association of subsequent frames. If multiple consecutive frames maintain the same shape backbone type and similar aspect ratios of the circumscribed rectangle, the recognition confidence of the target is increased. If a transition from a cross or star shape to a slender strip shape occurs in consecutive frames, accompanied by a monotonous change in the aspect ratio of the circumscribed rectangle, the recognition category is updated or the output is temporarily suspended according to the temporal consistency rule until the configuration signature stabilizes. Through the above-mentioned consistency judgment and output recognition result coordination mechanism, geometric, boundary texture, and thermal features are uniformly expressed at the entry level, and the shape backbone features serve as the key evidence for the final type distinction, thereby maintaining stable distinction and reliable output for UAV multi-rotor and fixed-wing UAVs under complex backgrounds, lighting changes, and thermal field disturbances.

[0046] Figure 2This paper details the core algorithm principle of threshold selection based on histogram analysis. The histogram uses a standard two-dimensional coordinate system, with the horizontal axis representing grayscale values ​​ranging from 0 to 255, and the vertical axis representing the number of pixels ranging from 0 to 4500. The histogram curve exhibits a typical bimodal distribution, a key characteristic shared by neutral contrast maps and neutral heatmaps. The first peak is located at grayscale value 56, corresponding to 2200 pixels, marked in red and indicated by a vertical dashed line; the second peak is located at grayscale value 180, corresponding to 3800 pixels, also marked in red. A distinct valley exists between the two peaks, located at grayscale value 118, corresponding to 600 pixels, highlighted in blue and marked with a solid blue line. According to the threshold segmentation algorithm, the system automatically identifies the positions of the first and second peaks of the histogram and then finds the lowest point between the two peaks as the segmentation threshold. The specific calculation formula is: T = argmin(H(x)) for x ∈ [first peak position, second peak position], where H(x) is the number of pixels corresponding to the gray value x. This threshold divides the image into two regions: a background region with a gray value less than 118 and a contrast candidate region or hot candidate region with a gray value greater than or equal to 118. The technical advantage of this bimodal distribution threshold segmentation method lies in its ability to adaptively determine the optimal segmentation point, effectively separating the target region from the background region, thus laying a solid foundation for subsequent candidate region extraction and geometric constraint screening. Experimental verification shows that this threshold selection algorithm maintains stable segmentation results under various lighting conditions and thermal environments.

[0047] Figure 3This paper demonstrates the technical principle of UAV type recognition based on shape backbone features. The analysis graph uses aspect ratio as the horizontal axis, ranging from 0.1 to 2.1, and the proportion of shape backbone length as the vertical axis, ranging from 0% to 90%, establishing a two-dimensional feature determination space. The graph shows the distribution characteristics of two types of UAV targets: multi-rotor UAV samples are marked in red, mainly distributed in the aspect ratio range of 0.8 to 1.4, with the corresponding shape backbone length proportion concentrated in the range of 65% to 85%; fixed-wing UAV samples are marked in blue, mainly distributed in the aspect ratio range of 1.5 to 2.0, with the corresponding shape backbone length proportion concentrated in the range of 45% to 70%. Shape backbone feature determination includes several key thresholds: the first aspect ratio threshold is set to 1.5, indicated by an orange dashed line; the length threshold is set to 60%, indicated by a green dashed line. For the identification of multi-rotor drones, the criteria are: the aspect ratio of the candidate region is within the set range and close to 1.0±0.2; the length of the main shape segment accounts for more than 65%; and the main shape segment is cross-shaped or star-shaped. For the identification of fixed-wing drones, the criteria are: the aspect ratio is greater than 1.5; the length of the main shape segment accounts for more than 45%; and the main shape segment is long and thin with sharp ends. Experimental results show that for the 7 multi-rotor drone samples, the average aspect ratio is 0.98±0.15, and the average length segment accounts for 72.3%±8.7%; for the 7 fixed-wing drone samples, the average aspect ratio is 1.67±0.18, and the average length segment accounts for 58.6%±11.2%. This feature-based identification method effectively distinguishes between the two types of drones and provides a reliable basis for the main shape segment features for subsequent identification results.

[0048] This example uses a dual-channel imaging system, with a visible light camera and an infrared camera mounted on the same pan-tilt unit. The visible light camera has a resolution of 1280×720 and a frame rate of 30, while the infrared camera has a resolution of 640×512 and a frame rate of 30. The system is triggered by a unified time source with a timestamp accuracy of 1 millisecond.

[0049] First, acquire a pair of visible light image frames and infrared image frames at the same time point, denoted as follows: and .in Represents a visible light image frame. Represents an infrared image frame. Timestamp alignment uses time difference. Criterion, if Then it is considered aligned, where The difference between the timestamps of the two frames. This is the time threshold; in this example, we take... Milliseconds. This data was collected in this instance. Milliseconds, satisfying the alignment condition. Then, respectively... and Perform a 3×3 median filter once to obtain the denoised image. and The median filter kernel size is denoted as... .

[0050] In spatial domain alignment, first in Perform Canny edge detection, and take low and high thresholds respectively. (Grayscale range 0 to 255) to obtain a binary edge map .exist The Shi-Tomasi method was used to extract the corner set. Maximum number of corner points Minimum mass fraction minimum distance Pixels; and extract the set of line segments using probabilistic Hough transform. Minimum line length Pixels, maximum gap Pixels. In Based on the bimodal shape of the grayscale histogram, the lowest point between the two peaks is taken as the threshold. In this example, the actual measurement yielded... Threshold segmentation yields a binary image of the thermal target. .right Perform morphological opening and closing operations, with the structuring element being the radius. A circular shape yields a set of infrared contours with better connectivity. Sampling of corner points at equal arc length intervals along each infrared profile, with a step size of [missing information]. Pixels, to obtain the infrared corner point set .

[0051] Construct cross-channel matching pairs based on geometric relationships. and Pair elements that are approximately parallel, approximately collinear, and whose endpoints are close together. Set an endpoint proximity threshold. Pixels, collinear angle threshold Degree. Obtain the initial set of matching pairs. In this example .right Random Sampling Consensus (RANSAC) is used, with an in-point criterion pixel geometric error threshold. Pixels, maximum number of iterations Obtain the inner point subset In this example .exist Supports prioritizing the fitting of perspective transformation matrices (Hotopy matrix), if ill-conditioned, it degenerates into an affine transformation. This example has a stable fit, yielding... .

[0052] in Used to transfer infrared coordinates Mapped to visible light coordinates The mapping relationship is as follows: , For homogeneity metric. right Geometric correction was performed to obtain And generate a bidirectional invertible mapping table for coordinate mapping. ,in At the same time, invalid and interpolation flags are recorded.

[0053] Construct a neutral contrast map in the visible light channel. Apply top-hat and bottom-hat operations sequentially, structuring element radius Pixels, to obtain the image after brightness shaping Then, contrast-limited histogram equalization (CLAHE) is performed, with the block grid size set. pruning threshold , to obtain the enhanced image For candidate regions with strong reflections, guided filtering is used to preserve edges and smooth the edges. Candidate regions are then selected based on a threshold value. Combined with local gradient instability determination, guiding the filter radius Regular terms Output neutral contrast chart .

[0054] Construct a neutral thermal map in the infrared channel. By threshold The hot and cold regions are segmented to obtain a binary mask. and .right Perform morphological reconstruction and mark the minimum effective area threshold. Pixels, removing false heat sources, to obtain the reconstructed heat area. .exist The inner execution edge is preserved and smoothed using bilateral filtering and spatial standard deviation. Strength standard deviation Output .exist Perform localized contrast stretching within the input, and set the upper and lower quantile points. Linear stretching to the full dynamic range yields a neutral heatmap. .

[0055] based on make sure and After pixel alignment, connected component labeling is performed. Let the set of connected components be... Extract the shape backbone for each connected component, and denote it as the length. The bounding rectangle has a width and height of 1000. Circumference is The area is Define the aspect ratio of the circumscribed rectangle. , The aspect ratio is defined; the percentage of the length of the main body of the shape is defined. , The ratio of the main stem length to the perimeter; the number of holes is... The connectivity level is denoted by the parent-child nesting depth. The boundary chain code histogram quantizes the directions into 8 directions and normalizes the frequency vector. Concentration in the main direction .

[0056] Boundary transition density is defined as ,in This represents the number of boundary direction changes. The binary texture pattern uses 8-neighborhood LBP encoding, and the histogram is denoted as... High frequency ratio , This is a set of high-frequency codes. (Through...) Establish a two-channel entry-level correspondence. For any Find the overlapping ones Define the overlap ratio .

[0057] when Create entries at that time. Thermal factors include the proportion of thermal zones. Thermal boundary continuity is expressed as the ratio of thermal boundary length to total boundary length. ,in The number of boundary pixels falling on the hot-cold interface. Total number of boundary pixels; thermal interface density ,in This represents the number of hot and cold boundary pixels detected within a 1-pixel width of the boundary band. This represents the area of ​​the band. After obtaining the neutral contrast map and neutral heatmap, a candidate set is generated. Calculate the histogram, and the threshold for the lowest point between the second peak and the first peak is: Segmentation to obtain contrast candidate masks Perform an opening operation (structuring element radius 2 pixels) and remove elements with an area smaller than 2 pixels. The region of pixels; for by Segmentation of thermal candidate masks Similarly, perform the opening operation and remove areas smaller than 100. The region of pixels. and Project onto the same coordinate system and calculate each candidate region. Only retain And the aspect ratio of the circumscribed rectangle The region is used to obtain the candidate set. In this example, the number of candidate regions is 6.

[0058] Consistency determination generates a configuration signature for each candidate region. , which includes Let the decision threshold be: the first aspect ratio threshold. Length threshold Chaincode concentration threshold Boundary transition density range High-frequency ratio range of binary texture mode hot zone percentage range Density range at the junction of hot and cold zones And requires thermal boundary continuity The judgment rule is to pass one of three criteria: geometric judgment, boundary texture judgment, and thermal feature judgment.

[0059] The following provides the numerical values ​​and calculation process for two specific candidate regions, demonstrating the judgment and output.

[0060] Candidate Region A: Width and height of the circumscribed rectangle are measured. Pixels Pixels Skeleton length Pixels, Perimeter Pixels Number of holes Connectivity level The proportion of the main peak in the boundary chain code histogram. Boundary transition count , LBP high-frequency code count percentage . and the corresponding hot connected domain Hot zone percentage Thermal boundary continuity Density at the thermal interface According to geometric determination: and It does not meet the requirement (because 1.037 < 1.2), but the proportion of the main body length in the shape is... Number of holes The overall geometric determination failed. Boundary texture determination: Not satisfied, but This category still fails. Thermal characteristic determination: , Thermal feature determination passed. Based on the three-choice strategy, candidate region A passed consistency determination. The shape of the main trunk was determined to be cross-shaped, with four branches of similar length and a similar aspect ratio to the circumscribed rectangle. The value is close to 1, so the output recognition result is a multi-rotor drone.

[0061] Candidate Region B: Width and Height of the Circulating Rectangle Pixels Pixels Skeleton length Pixels, Perimeter Pixels Number of holes Connectivity level Chain code principal direction concentration Boundary transition density (Boundary transition count 5), high frequency ratio of binary texture mode . and the corresponding hot connected domain Hot zone percentage Thermal boundary continuity Density at the thermal interface Geometric determination: Not satisfied (because) (Too large), but satisfy, , According to the geometric determination of the present invention, the three conditions are combined (in this example, the combination of "within a set range, greater than a first threshold, length percentage exceeding the threshold, and number of holes not greater than 1" is used). Because the first sub-condition is not met, the overall geometric determination fails. Boundary texture determination: satisfy, Slightly below the lower limit of 0.015 does not meet the requirement. The condition is met, therefore this class also fails. Thermal feature determination: , , Thermal feature determination passed. Based on the three-choice strategy, candidate region B passed consistency determination. The shape of the main skeleton was then analyzed; it was found to be elongated and slender with sharp ends, thus the output recognition result was a fixed-wing UAV.

[0062] After calculating the six candidate regions one by one, four in this example passed the consistency test. Two of them met the criteria of having a cross or star-shaped main body and an aspect ratio of the bounding rectangle close to 1, and were output as multi-rotor drones. The other two were elongated and pointed at both ends, and were output as fixed-wing drones. To provide a confidence metric, the number of sub-judgments satisfied by each passing entry was accumulated. And normalize ,in This represents the confidence level for the entry. In this example, candidate region A satisfies Class 1. Candidate region B satisfies Class 1. If it is continuous in the time series If a frame detects the same type of result at the same spatial location, a stable label is output. Temporal consistency is achieved by ensuring that the frames are centered at the same bounding rectangle. radius Pixel neighborhood tracking is complete, if Then adjacent frames are considered to match, where For the first Frame timestamp, The frame period is in milliseconds. In actual testing, the two multi-rotor entries and one fixed-wing entry maintained stable output for 5 consecutive frames, and finally reported the recognition results and trajectory initialization of the two UAV multi-rotor and one UAV fixed-wing in the scene.

[0063] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive target recognition method for unmanned aerial vehicles based on optical detection, characterized in that, The method includes the following steps: Step 1: Acquire a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and then perform alignment preprocessing on the image frame pair to obtain an aligned frame pair; Step 2: Based on the aligned frame pairs, generate neutral contrast maps corresponding to visible light image frames and neutral heat maps corresponding to infrared image frames, and generate topological manifold maps based on the neutral contrast maps and neutral heat maps. Step 3: Perform threshold segmentation on the neutral contrast map and the neutral heatmap respectively, and filter the results by combining geometric and spatial constraints to obtain a candidate set; Multi-dimensional features are extracted for each candidate region in the candidate set, and consistency is determined according to preset rules. If the determination is successful, the recognition result is output based on its shape and main features.

2. The adaptive target recognition method for UAVs based on optical detection as described in claim 1, characterized in that, In step 1, the alignment preprocessing includes: performing timestamp alignment on the visible light image frame and the infrared image frame, and then performing a 3x3 median filter on the visible light image frame and the infrared image frame respectively.

3. The adaptive target recognition method for UAVs based on optical detection as described in claim 2, characterized in that, The alignment preprocessing process specifically includes: on the visible light image frame, edge detection is used to obtain a binary edge map; on the binary edge map, corner point detection is used to obtain a set of visible light corner points; and on the infrared image frame, threshold segmentation is used to obtain a binary map of the thermal target; morphological opening and closing operations are performed on the binary map of the thermal target to extract an infrared contour set; and contour corner points are uniformly sampled on the infrared contour to obtain an infrared corner point set; matching pairs are generated based on the parallelism, approximate collinearity, and proximity of endpoints between the visible light segments and the infrared contour; random sampling consistency screening is used to remove outliers from the matching pairs, and geometric transformations between the two image frames are fitted, prioritizing perspective transformations; if a stable fit is not possible, it degenerates into an affine transformation; the fitted perspective or affine transformation is used to perform geometric correction on the infrared image frame, aligning the infrared image frame and the visible light image frame in the same pixel coordinate system, generating aligned frame pairs, and generating a bidirectional reversible mapping table for coordinate mapping.

4. The adaptive target recognition method for UAVs based on optical detection as described in claim 3, characterized in that, The process of generating a neutral contrast map in step 2 includes: performing the following process on the visible light image frames of the aligned frame pairs: using top-hat and bottom-hat operations to suppress isolated strong reflections and deep shadows; performing contrast-limited histogram equalization on the processed image to enhance the contrast of medium brightness areas; and using guided filtering to perform edge-preserving smoothing in strong reflection candidate areas to weaken reflection peaks while preserving edge positions, thereby obtaining a neutral contrast map.

5. The adaptive target recognition method for UAVs based on optical detection as described in claim 4, characterized in that, The process of generating a neutral thermal image in step 2 includes the following steps on the infrared image frames of the aligned frame pairs: obtaining a binary mask of hot and cold areas by threshold segmentation; performing morphological reconstruction on the hot area mask to remove pseudo heat sources with too small an area; performing edge-preserving smoothing on the reconstructed hot areas and performing local contrast stretching on the cold areas to obtain a neutral thermal image.

6. The adaptive target recognition method for UAVs based on optical detection as described in claim 5, characterized in that, Step 2, which generates the topological manifold map, includes: ensuring pixel alignment between the neutral contrast map and the neutral heatmap based on a bidirectional reversible mapping table; labeling connected components in both the neutral contrast map and the neutral heatmap; extracting the shape backbone for each connected component and calculating its hole count, connectivity level, boundary chain code histogram, boundary inflection point sequence, and binary texture pattern histogram; and establishing an entry-level correspondence between spatially overlapping connected components using a bidirectional reversible mapping table to form the topological manifold map.

7. The adaptive target recognition method for UAVs based on optical detection as described in claim 6, characterized in that, Step 3, the process of obtaining the candidate set includes: on the neutral contrast map, segmenting the data using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a contrast candidate mask, and then removing burrs and discarding areas with an area smaller than a first set area threshold through an opening operation; on the neutral heatmap, segmenting the data using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a heat candidate mask, and then removing burrs and discarding areas with an area smaller than a second set area threshold through an opening operation; projecting the heat candidate mask and the contrast candidate mask onto the same coordinate system, calculating their overlap ratio, and retaining only areas that meet the following conditions as candidate areas: the overlap ratio is not lower than a set lower overlap limit, and the aspect ratio of its bounding rectangle falls within a set aspect ratio range; all candidate areas together constitute the candidate set.

8. The adaptive target recognition method for UAVs based on optical detection as described in claim 7, characterized in that, In step 3, the consistency determination process includes: for each candidate region, obtaining the corresponding entry from the topological manifold graph and generating a configuration signature. The configuration signature includes: the aspect ratio of the bounding rectangle, the proportion of the shape backbone length, the number of holes, the connectivity level, the concentration of the main direction of the chain code histogram, the boundary transition density, the high-frequency proportion of the binary texture pattern, the proportion of hot areas, the continuity of hot boundaries, and the density of hot-cold junctions. A candidate region is deemed to pass when it meets at least one of the following criteria: Geometric determination passes: the aspect ratio of the candidate region is within the set aspect ratio range, is greater than the set first aspect ratio threshold, the proportion of the shape backbone length exceeds the set length threshold, and the number of holes is not greater than one; Boundary texture determination passes: the main direction of the chain code of the candidate region is concentrated, the boundary transition density is within the set density range, and the high-frequency proportion of the binary texture pattern is within the set proportion range; Thermal feature determination passes: the proportion of hot areas of the candidate region is within the set hot area range, the hot boundary is continuous, and the density of hot-cold junctions is within the set density range.

9. The adaptive target recognition method for UAVs based on optical detection as described in claim 8, characterized in that, The process of outputting the recognition result includes: if a candidate region passes the consistency judgment, then its shape backbone features are further judged: if the shape backbone is cross or star-shaped, and the aspect ratio of its circumscribed rectangle is close to 1, then the output recognition result is a multi-rotor drone; if the shape backbone is long and thin and sharp at both ends, then the output recognition result is a fixed-wing drone.

10. An optically-based adaptive target recognition system for unmanned aerial vehicles (UAVs) for implementing the method of any one of claims 1 to 9, characterized in that, The system includes: an acquisition unit, used to acquire a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and then perform alignment preprocessing on the image frame pair to obtain an aligned frame pair; an image processing unit, used to generate a neutral contrast map corresponding to the visible light image frame and a neutral heat map corresponding to the infrared image frame based on the aligned frame pair, and generate a topological manifold map based on the neutral contrast map and the neutral heat map; and an identification unit, used to perform threshold segmentation on the neutral contrast map and the neutral heat map respectively, and perform screening based on geometric and spatial constraints to obtain a candidate set; extract multi-dimensional features for each candidate region in the candidate set, and perform consistency judgment according to preset rules. If the judgment passes, the identification result is output according to its shape backbone features.

Images