Unmanned aerial vehicle adaptive target recognition method and system based on optical detection

Abstract

The present application relates to the technical field of optical identification, and more particularly to a method and system for adaptive target identification of unmanned aerial vehicles based on optical detection, which comprises the following steps: Step 1: a pair of visible light image frames and infrared image frames are collected at the same time point to form an image frame pair, and then alignment preprocessing is performed on the image frame pair to obtain an aligned frame pair; Step 2: based on the aligned frame pair, a neutral contrast map corresponding to the visible light image frame and a neutral heat map corresponding to the infrared image frame are respectively generated, and a topological manifold atlas is generated based on the neutral contrast map and the neutral heat map; Step 3: threshold segmentation is performed in the neutral contrast map and the neutral heat map respectively, and screening is performed in combination with geometric and spatial constraints to obtain a candidate set; if it is determined to pass, a recognition result is output according to the shape stem features thereof. The present application significantly improves the accuracy, robustness and practical value of identification in complex backgrounds and dynamic environments.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of optical detection, and particularly relates to an unmanned aerial vehicle adaptive target recognition method and system based on optical detection. BACKGROUND

[0002] In the development of unmanned aerial vehicle target detection and recognition technology, with the wide application of unmanned aerial vehicles in civil and military fields, the demand for reliable detection and classification of unmanned aerial vehicles in complex environments is increasingly urgent. Existing unmanned aerial vehicle detection and recognition methods mainly include single sensor imaging-based methods and multi-sensor information fusion-based methods. Single sensor-based detection methods include visible light image analysis and infrared imaging analysis. Visible light imaging depends on natural light conditions, and through the extraction and analysis of edge, texture, and color features of video streams or static images, the recognition of unmanned aerial vehicle targets is realized. This kind of method has high recognition accuracy in the environment with sufficient light and simple background, but in the case of low light, backlight, shadow shielding or complex background, the contrast and edge definition of the image will decrease significantly, resulting in a sharp decrease in detection performance. The infrared imaging method relies on the thermal radiation characteristics of the unmanned aerial vehicle body, power system and electronic equipment, and can still work stably at night or in low light environment. However, the resolution of infrared image is usually low, the detail information is insufficient, and it is easily affected by environmental temperature changes, background heat source interference and weather conditions (such as fog, rain), resulting in unstable thermal characteristics. In addition, infrared imaging has limited ability to distinguish the shape of small targets at long distances, and it is difficult to accurately distinguish the type of unmanned aerial vehicle only by relying on thermal distribution.

[0003] To overcome the limitations of a single sensor, the prior art has developed a fusion method of visible light and infrared imaging in order to obtain complementary information under different light and background conditions. In this kind of method, the common idea is to align the visible light image and the infrared image through spatial registration, and then fuse them at the pixel level or feature level, for example, by using weighted superposition, feature splicing or deep learning multi-modal fusion model. However, the existing multi-sensor fusion method still faces several outstanding problems in practical engineering: first, the insufficient registration accuracy leads to the misplacement of cross-channel information. Due to the differences in optical system, installation position and imaging characteristics, there are inherent parallax and distortion between different waveband sensors, especially when imaging long-distance targets, slight registration errors will cause inconsistencies in boundaries, textures and hot areas, thereby affecting the fusion effect and subsequent recognition. Second, the suppression of environmental interference in the fusion process is insufficient. Visible light images are prone to false detection under strong reflection, shadow, complex background texture, etc., while infrared images will appear false targets when there are background heat sources. The existing method often simply superimposes or averages the two kinds of information, lacks targeted suppression strategy, and the false detection rate is still high. Third, the existing fusion detection method relies on single-channel saliency or simple geometric constraints based on the bounding rectangle in the candidate region screening and target determination, and lacks comprehensive utilization of multi-dimensional features such as target geometric structure, boundary shape and heat distribution pattern. Especially in the classification of unmanned aerial vehicles, the existing technology is difficult to accurately distinguish the structural features of multi-rotor and fixed-wing vehicles. Fourth, the existing method lacks robustness in complex background or dynamic scene. When there are moving objects (such as birds, wind-blown leaves, and cloud changes) or dynamic heat sources in the background, the detection system often has difficulty in stable tracking and accurate identification of unmanned aerial vehicle targets, especially when the background and target size are similar and the speed is similar, false detection and misclassification are prone to occur. SUMMARY

[0004] The main purpose of the present application is to provide an optical detection-based adaptive target recognition method and system for unmanned aerial vehicles, which synchronously acquires visible light and infrared image frames and performs high-precision alignment preprocessing, realizes accurate matching of cross-channel data in time and space, combines the construction of neutral contrast map and neutral heat map, effectively suppresses the interference of light, reflection and false heat sources, and realizes multi-dimensional structured feature fusion by using topological manifold atlas. In the candidate region screening and consistency determination process, multi-channel joint constraints of geometric, texture and heat features are adopted, which greatly reduces the false detection rate, and can automatically distinguish multi-rotor and fixed-wing unmanned aerial vehicles according to the shape stem features, thereby significantly improving the accuracy, robustness and practical value of recognition in complex background and dynamic environment.

[0005] To solve the above problems, the technical scheme of the present application is as follows:

[0006] An unmanned aerial vehicle adaptive target recognition method based on optical detection, the method comprising the following steps:

[0007] Step 1: Collecting a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and then performing alignment preprocessing on the image frame pair to obtain an aligned frame pair;

[0008] Step 2: Based on the aligned frame pair, generating a neutral contrast map corresponding to the visible light image frame and an infrared image frame corresponding to the neutral heat map, and generating a topological manifold atlas based on the neutral contrast map and the neutral heat map;

[0009] Step 3: Performing threshold segmentation in the neutral contrast map and the neutral heat map respectively, and screening combined with geometric and spatial constraints to obtain a candidate set; extracting multi-dimensional features for each candidate region in the candidate set, and performing consistency judgment according to a preset rule, if the judgment is passed, outputting a recognition result according to the shape stem feature thereof.

[0010] Further, in step 1, the alignment preprocessing process includes: performing timestamp alignment on the visible light image frame and the infrared image frame, and then performing 3x3 median filtering on the visible light image frame and the infrared image frame respectively.

[0011] Further, the alignment preprocessing process further includes: on the visible light image frame, using edge detection to obtain a binary edge map, using corner point detection to obtain a visible light corner point set on the binary edge map, and using straight line segment detection to obtain a visible light line segment set; on the infrared image frame, performing threshold segmentation to obtain a thermal target binary map, extracting an infrared contour set after morphological opening and closing operation on the thermal target binary map, and uniformly sampling contour corner points on the infrared contour to obtain an infrared corner point set; taking parallelism, approximate collinearity and endpoint proximity of the visible light line segment and the infrared contour as matching conditions, generating a matching pair; using random sample consensus screening to remove outliers from the matching pair, and fitting a geometric transformation between the two image frames, preferentially fitting a perspective transformation, and if stable fitting is not possible, degrading to an affine transformation; using the fitted perspective transformation or affine transformation to perform geometric correction on the infrared image frame, so that the infrared image frame and the visible light image frame are aligned in the same pixel coordinate system to generate an aligned frame pair, and a bidirectional reversible mapping table for coordinate mapping is generated.

[0012] Further, in step 2, the process of generating the neutral contrast map includes: on the visible light image frame of the aligned frame pair, performing the following processes: using top-hat operation and bottom-hat operation to suppress isolated strong reflection and deep shadow; performing contrast-limited histogram equalization on the processed image to enhance the contrast of the medium brightness region; using guided filtering to perform edge-preserving smoothing in the strong reflection candidate region to weaken the reflection peak while preserving the edge position, to obtain the neutral contrast map.

[0013] Further, the process of generating the neutral thermal map in step 2 includes: on the aligned infrared image frames of the frame pair, performing the following process: threshold segmentation to obtain a hot region and a cold region binary mask; morphological reconstruction on the hot region mask to remove small false heat sources; edge-preserving smoothing on the reconstructed hot region and local contrast stretching on the cold region to obtain the neutral thermal map.

[0014] Further, the process of generating the topological manifold atlas in step 2 includes: ensuring that the neutral contrast map and the neutral thermal map are pixel-aligned based on a bidirectional reversible mapping table; separately labeling the connected domains of the neutral contrast map and the neutral thermal map; extracting the shape stem of each connected domain and calculating its hole number, connectivity level, boundary chain code histogram, boundary turning point sequence, and binary texture pattern histogram; through the bidirectional reversible mapping table, establishing an entry-level correspondence between the two spatially overlapping connected domains to form the topological manifold atlas.

[0015] Further, in step 3, the process of obtaining the candidate set includes: on the neutral contrast map, threshold segmentation using the lowest point between the second peak and the first peak of its histogram to obtain a contrast candidate mask, followed by opening operation to remove burrs and eliminate regions with an area less than a first set area threshold; on the neutral thermal map, threshold segmentation using the lowest point between the second peak and the first peak of its histogram to obtain a thermal candidate mask, followed by opening operation to remove burrs and eliminate regions with an area less than a second set area threshold; projecting the thermal candidate mask and the contrast candidate mask into the same coordinate system, calculating their overlap ratio, and only retaining regions that meet the following conditions as candidate regions: the overlap ratio is not less than a set lower overlap limit, and the aspect ratio of its bounding rectangle falls within a set aspect ratio range; all candidate regions together constitute the candidate set.

[0016] Further, in step 3, the process of consistency determination includes: for each candidate region, obtaining the corresponding entry from the topological manifold atlas to generate a configuration signature, which includes: aspect ratio of the bounding rectangle, length proportion of the shape stem, number of holes, connectivity level, main direction concentration of the chain code histogram, boundary turning density, high frequency proportion of the binary texture pattern, thermal region proportion, thermal boundary continuity, and thermal-cold junction density; when the candidate region meets at least one of the following determinations, it is determined to pass: geometric determination passes: the aspect ratio of the candidate region is within the set aspect ratio range, greater than the set first aspect ratio threshold, the length proportion of the shape stem exceeds the set length threshold, and the number of holes is not greater than one; boundary texture determination passes: the main direction of the candidate region's chain code is concentrated, the boundary turning density is within the set density interval, and the high frequency proportion of the binary texture pattern is within the set proportion interval; thermal feature determination passes: the thermal region proportion of the candidate region is within the set thermal region interval, the thermal boundary is continuous, and the thermal-cold junction density is within the set thermal-cold junction density interval.

[0017] Further, the process of outputting the recognition result comprises: if a candidate region passes the consistency determination, further judging the shape stem feature thereof; if the shape stem is cross-shaped or star-shaped, and the aspect ratio of the circumscribed rectangle thereof is close to 1, outputting the recognition result as a multi-rotor unmanned aerial vehicle; if the shape stem is in the form of an elongated strip and the two ends are sharp, outputting the recognition result as a fixed-wing unmanned aerial vehicle.

[0018] The adaptive target recognition system for unmanned aerial vehicles based on optical detection comprises: a collection unit configured to collect a pair of visible light image frames and infrared image frames at the same time point to form an image frame pair, and perform alignment preprocessing on the image frame pair to obtain an aligned frame pair; a graph processing unit configured to generate a neutral contrast graph corresponding to the visible light image frame and a neutral thermal graph corresponding to the infrared image frame based on the aligned frame pair, and generate a topological manifold atlas based on the neutral contrast graph and the neutral thermal graph; and a recognition unit configured to perform threshold segmentation in the neutral contrast graph and the neutral thermal graph respectively, and perform screening in combination with geometric and spatial constraints to obtain a candidate set; extract multi-dimensional features for each candidate region in the candidate set, and perform consistency determination according to a preset rule, and if the determination passes, output a recognition result according to the shape stem feature thereof.

[0019] The unmanned aerial vehicle adaptive target recognition method and system based on optical detection have the following beneficial effects: visible light image frames and infrared image frames are collected at the same time point, and high-precision alignment preprocessing is performed, so that strict synchronization of multispectral data in time and space is ensured, and the influence of cross-channel information misplacement on subsequent recognition is effectively eliminated.On this basis, the neutral contrast map is constructed for the visible light channel to suppress isolated strong reflection and deep shadow interference and enhance the contrast of the medium brightness area; the neutral thermal map is constructed for the infrared channel to remove pseudo-thermal sources with small area and enhance the detail performance of the effective thermal area, so that the effective target saliency of the two channels in complex environments is improved respectively.Further, the bidirectional reversible mapping table is used to realize pixel accurate alignment of the neutral contrast map and the neutral thermal map, and multidimensional topological and texture features are extracted to construct a topological manifold atlas, so as to provide a unified structured description for cross-channel target association and consistency determination.In the candidate region screening stage, the non-target interference area is effectively eliminated, and the false detection rate is significantly reduced by combining histogram peak-valley threshold segmentation, morphological operation, area constraint, overlap ratio calculation and length-width ratio range screening.In the consistency determination stage, the fine screening of the target is realized by the multi-channel joint constraint of geometric features, boundary texture features and thermal features, and on this basis, the unmanned aerial vehicle type is output according to the shape stem feature, and the automatic differentiation of multi-rotor and fixed-wing is realized.This design not only maintains a high recognition accuracy in a static complex background, but also has stability and robustness in a dynamic scene, can effectively adapt to complex conditions such as light change, background interference and multi-target scene, and thus greatly improves the reliability and practical value of the unmanned aerial vehicle target recognition system in practical application. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The method flowchart of the unmanned aerial vehicle adaptive target recognition method based on optical detection provided by the embodiment of the present application is shown in the figure.

[0021] Figure 2 The histogram bimodal distribution threshold segmentation analysis principle diagram in the embodiment of the present application is shown in the figure.

[0022] Figure 3 The shape stem feature determination analysis principle diagram in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0023] In order to make the person skilled in the art better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the protection scope of the present application.

[0024] Reference Figure 1 An unmanned aerial vehicle adaptive target recognition method based on optical detection, the method comprising the following steps:

[0025] Step 1: A pair of visible light image frames and infrared image frames are collected at the same time point to form an image frame pair, and time stamp alignment is used to ensure that the image frame pair reflects the scene information at the same observation moment. Time stamp alignment is used to correct the micro time sequence deviation caused by the sampling and transmission delay of different sensors, and to eliminate abnormal frames or remap the adjacent frames, so that the moving targets in the two channels are consistent in time. After completing the time domain synchronization, the visible light image frames and the infrared image frames are subjected to 3x3 median filtering respectively to suppress transient noise, small speckle and salt and pepper interference, while trying to keep the edges and textures from being excessively smoothed, thereby creating conditions for stable extraction of geometric elements.

[0026] On this basis, in order to realize spatial domain alignment, edge detection is performed on the visible light image frames to generate a binary edge map, and then corner detection is performed on the binary edge map to obtain a visible light corner set, and straight line segment detection is performed to obtain a visible light line segment set; the infrared image frames are then threshold segmented to obtain a thermal target binary map, and then morphological opening and closing operations are performed on the thermal target binary map to remove isolated false heat sources and correct regular boundaries, and then the corrected infrared contour set is uniformly sampled to obtain an infrared corner set. Using the geometric correlation between the visible light line segment and the infrared contour, the matching pairs are generated with the above parallel, approximately collinear and endpoint adjacent as matching conditions, and the outlier elimination is performed on the matching pairs by random sample consensus to improve the robustness of estimating the geometric relationship.

[0027] The geometric transformation between two image frames is fitted under the support of effective matching pairs, and a perspective transformation is preferentially fitted to adapt to the projection difference between the imaging planes; when the scene structure or element distribution is insufficient to stabilize the perspective model, it is degraded to an affine transformation to maintain the stability and realizability of alignment. Then the fitted perspective transformation or affine transformation is used to geometrically correct the infrared image frame, map the infrared image frame into the pixel coordinate system of the visible light image frame, align the 2-channel data in the same pixel coordinate system, and generate an aligned frame pair. In the geometric correction process, a bidirectional reversible mapping table for coordinate mapping is simultaneously generated, which records the forward and reverse correspondence between the pixels of the visible light image frame and the infrared image frame, and can identify interpolation, occlusion and invalid area, facilitating subsequent accurate pixel-level association and area projection between the neutral contrast map and the neutral heat map.

[0028] Step 2: Based on the aligned frame pair and bidirectional reversible mapping table obtained in step 1, ensure that the visible light image frame and the infrared image frame can be bidirectionally mapped and consistently accessed at each pixel position. In the visible light image frame channel, to construct a neutral contrast map, first perform top-hat operation and bottom-hat operation on the original brightness to suppress the intensity bias caused by isolated strong reflection and deep shadow, and then perform contrast-limited histogram equalization on the overall image after the residual non-uniform illumination is suppressed, to highlight the weak contrast in the medium brightness area, so that structures such as slender fuselages, paddle outlines, and arms have more balanced local contrast performance in the background.

[0029] For the strong reflection candidate regions that may still exist, guided filtering is used for edge-preserving smoothing, which maintains the contour sharpness at the edges by using the structural prior of the original image, while weakening the reflection peaks inside the region, thereby reducing the interference of high light saturation on boundary detection and connected component labeling, and finally forming a neutral contrast map that still has robust boundary performance in strong light, shadow, and complex texture background. In the infrared image frame channel, to construct a neutral heat map, first segment the heat area and cold area binary mask according to the threshold, remove the false heat sources with small area, irregular shape and lack of spatial continuity with the real heat target through morphological reconstruction, and perform edge-preserving smoothing on the reconstructed heat area to reduce the broken boundaries caused by heat noise and local temperature fluctuations, while maintaining the continuity of the heat boundaries of possible corresponding parts such as the body, wings, rotor motor or tail jet; for the cold area, perform local contrast stretching to flatten the slow gradient of the background temperature field and highlight the subtle intensity transition at the hot-cold junction, thereby obtaining a neutral heat map with more stable saliency for real heat targets in complex heat field scenes.

[0030] After the neutralization enhancement of the two channels, a bidirectional reversible mapping table is used to ensure strict alignment of the neutral contrast map and the neutral heat map at the pixel level, and connected component labeling is performed on each channel to obtain a number of candidate connected components with closed boundaries and internal consistency. For each connected component, the shape backbone is extracted to depict its elongation and principal axis connectivity, and the number of holes and connectivity levels are calculated to reflect the possible topological features caused by the paddle gap, arm intersection or mounting structure inside the target; at the boundary level, the boundary chain code histogram and boundary turning point sequence are calculated to record the main direction distribution and turning density of the contour, thereby forming a scale and rotationally stable description of the cross structure of the multi-rotor, the elongated edge of the fixed wing, and the connection between the fuselage and the auxiliary components; at the texture level, the binary texture pattern histogram is counted to express the proportion of binary texture patterns inside and outside the connected component, helping to distinguish false contours caused by background texture or feature details from real structures produced by unmanned aerial vehicles.

[0031] The two channels are then spatially overlapped or adjacent to each other through a bidirectional reversible mapping table to establish a entry-level correspondence between the connected components, forming a topological manifold atlas containing cross-channel paired entries, geometric attributes, texture statistics and thermal feature elements. In this topological manifold atlas, each entry can be regarded as a unified description of the same spatial entity under visible light and infrared observation, containing information such as contour clarity, shape backbone length ratio, boundary turning density from the neutral contrast map, and thermal information such as thermal area ratio, thermal boundary continuity and thermal cold junction density from the neutral heat map; the adjacent, containing and contact relationships between entries constitute a connectivity level and spatial relationship network, which reflects the structural organization between the target and the background, and the target components in a complex scene.

[0032] The topological manifold atlas generated by the above process, on the one hand, complements the advantages of visible light image frames and infrared image frames into a unified data structure, reducing the ambiguity produced by single-channel in strong light reflection, shadow obstruction, uneven heat source distribution or complex background texture; on the other hand, it provides a direct basis for subsequent steps to perform threshold segmentation result projection, geometric and spatial constraint filtering and configuration signature generation under the same reference coordinate and entry semantics, so that the unmanned aerial vehicle target can still be recognized stably through entry-level consistency under cross-spectral condition changes and scene disturbances.

[0033] Step 3: The neutral contrast map and the neutral thermal map generated by the aligned frame pair are used to complete the screening, measurement and judgment of the target entry in a unified pixel coordinate system, and the recognition result is output according to the shape stem feature after passing the consistency judgment. First, threshold segmentation is performed in the neutral contrast map, and the lowest point between the second peak and the first peak of its histogram is taken as the threshold to obtain a contrast candidate mask. The burrs are removed by an opening operation, and regions with an area smaller than a first set area threshold are removed to constrain false responses caused by reflection, shadow or fine texture. Subsequently, threshold segmentation with the same strategy is performed in the neutral thermal map, and the lowest point between the second peak and the first peak of its histogram is taken as the threshold to obtain a thermal candidate mask. Similarly, burrs are removed by an opening operation, and regions with an area smaller than a second set area threshold are removed to reduce the interference of isolated false heat sources or noise heat spots.

[0034] The thermal candidate mask and the contrast candidate mask are projected into the same coordinate system, the overlap ratio of the two in space is calculated, and only the regions that meet the set overlap lower limit and whose aspect ratio of the circumscribed rectangle falls within the set aspect ratio range are retained as candidate regions. All candidate regions together constitute a candidate set. To maintain semantic consistency in cross-channel information, when extracting multi-dimensional features from each candidate region in the candidate set, directly use the entry-level correspondence established by the topological manifold atlas to retrieve the two-channel connected domain entries that spatially overlap with the candidate region, and comprehensively obtain geometric, boundary and thermal elements. In the feature construction process, a configuration signature is generated for consistency analysis. The configuration signature includes the aspect ratio of the circumscribed rectangle, the shape stem length ratio, the number of holes, the connectivity level, the chain code histogram main direction concentration, the boundary turning density, the high frequency proportion of the binary texture pattern, the thermal area proportion, the thermal boundary continuity and the thermal cold junction density, etc. The boundary and shape information from the neutral contrast map and the thermal distribution information from the neutral thermal map are associated and expressed under the same entry.

[0035] When consistency judgment is performed according to the preset rule, the judgment is performed around three types of rules, i.e., geometric judgment, boundary texture judgment and thermal feature judgment: when the candidate region is in the geometric judgment, the aspect ratio is in the set aspect ratio range and is greater than the set first aspect ratio threshold, and the length proportion of the shape stem is greater than the set length threshold and the number of holes is not greater than 1, it can be considered as passing; when the candidate region is in the boundary texture judgment, the chain code main direction is concentrated, the boundary turning density is in the set density interval, and the high frequency proportion of the binary texture mode is in the set proportion interval, it can be considered as passing; when the candidate region is in the thermal feature judgment, the hot area proportion is in the set hot area interval, the thermal boundary is continuous, and the hot-cold junction density is in the set hot-cold junction density interval, it can also be considered as passing. As long as any one of the above-mentioned preset rules is satisfied, it is judged as passing the consistency judgment. For the candidate region passing the consistency judgment, the recognition result is further output according to the shape stem feature thereof: when the shape stem is in the shape of cross or star and the aspect ratio of the circumscribed rectangle is close to 1, the recognition result is output as a multi-rotor unmanned aerial vehicle; when the shape stem is in the shape of a slender strip and the two ends are sharp, the recognition result is output as a fixed-wing unmanned aerial vehicle.

[0036] Further, the target of the alignment preprocessing is to ensure the consistency of the visible light image frame and the infrared image frame in the time domain and the spatial domain at the same time, so that the two channel data can be jointly analyzed in the same pixel coordinate system and form an aligned frame pair, and then provide reliable input for subsequent generation of neutral contrast maps, neutral thermal maps and topological manifold maps. First, time stamp alignment is performed to eliminate the trigger error and delay difference caused by different imaging links in the acquisition, caching and transmission process. The time stamp alignment is based on the time mark of each pair of image frames, and the adjacent frames across channels are matched. If necessary, abnormal frames that do not meet the requirement of the same time point are removed or a resampling strategy of adjacent time points is adopted, so that the time sequence position of the same moving target in the two channels remains consistent, avoiding the time shift across channels caused by the relative motion of the aircraft and the background. After completing the time domain synchronization, the visible light image frame and the infrared image frame are respectively subjected to a 3x3 median filter, and the median operator is used to suppress pulse noise, speckle and random brightness mutation, while trying to keep the edges and details from being excessively smoothed, providing conditions for the stable extraction of subsequent geometric elements. Subsequently, the structured element construction and cross-channel constraint matching stage in the spatial domain is entered. On the visible light image frame, first, the edge detection is used to obtain a binary edge map, which can highlight the structure outline without relying on the absolute value of the illumination; the visible light corner point set is obtained through corner point detection on the binary edge map, so as to capture the local curvature or strong direction change position, and enhance the repeatability and distinguishability of the features; and the visible light line segment set is obtained through line segment detection, so as to extract the straight line elements formed by the arm, fuselage edge, blade projection or ground structure, providing a basis for the subsequent geometric relationship constraint.

[0037] On the infrared image frame, a thermal target binary image is obtained by threshold segmentation to highlight the thermal regions possibly caused by power systems, electronic loads or friction heat; a set of infrared contours is extracted after a morphological opening-closing operation on the thermal target binary image, the opening operation helps to remove pseudo-thermal sources with too small areas, and the closing operation helps to repair the broken boundaries caused by thermal noise, making the infrared contours more consistent with the shape continuity of real objects; and the infrared contour is uniformly sampled on the infrared contour to obtain a set of infrared corner points, thereby forming discrete structure anchor points corresponding to the set of visible light corner points in thermal expression. Based on the above two-channel element sets, an initial cross-channel association is established: a matching pair is generated by taking the parallelism, approximate collinearity and endpoint proximity of the visible light line segment and the infrared contour as the matching condition. This matching condition takes advantage of the consistency of the visible light edge and the infrared contour in macroscopic geometric trend, while the endpoint proximity constraint suppresses the false matching caused by accidental parallelism. To improve the robustness of matching, the random sample consensus screening is used to exclude outliers from the matching pairs, and the inlier set that best explains most of the matching pairs is selected from several random subset evaluations, reducing false associations caused by noise, local occlusion and non-rigid background. Based on the valid matching pairs after consistency screening, the geometric transformation between the two image frames is fitted, and the perspective transformation is preferentially fitted to adapt to the projection differences caused by imaging geometry, view angle change and scene planarization assumption; when the scene structure, feature distribution or disparity condition is insufficient to stably support the perspective model, it is degenerated into an affine transformation to ensure that a stable spatial mapping can still be obtained in the case of sparse features or minimal disparity.

[0038] The fitted perspective transformation or affine transformation is used to geometrically correct the infrared image frame, map the infrared image frame into the pixel coordinate system of the visible light image frame, realize pixel-by-pixel alignment of the two channels in the same pixel coordinate system, and generate an aligned frame pair. During the geometric correction process, in order to ensure the explainability and efficiency of subsequent cross-channel data interaction, a bidirectional reversible mapping table for coordinate mapping is generated. The bidirectional reversible mapping table records the forward mapping from the visible light image frame to the infrared image frame and the inverse mapping from the infrared image frame to the visible light image frame, including the correspondence of pixel coordinates, the identification of interpolation method, the label of invalid area caused by boundary extrapolation or disparity occlusion, and the local registration quality indication, so that reliable regional projection, connected component correspondence and feature aggregation can be performed between the two channels in subsequent processing.

[0039] Further, the purpose of generating the neutral contrast image on the aligned visible light image frames of the frame pair is to suppress the intensity bias caused by illumination and material and enhance the separability of structural boundaries, so as to have a stable contrast basis for subsequent connected component labeling, configuration signature construction and entry-level correspondence in the unified pixel coordinate system. In the specific implementation process, first, top-hat operation and bottom-hat operation are sequentially applied on the aligned visible light image frames of the frame pair. The top-hat operation is used to weaken the local highlight protrusions caused by isolated strong reflection, so that the abnormally high light generated by the mirror reflection, wet surface or metal part no longer dominates the local intensity distribution. The bottom-hat operation is used to fill the dark pits and boundary collapse caused by deep shadows, and reduce the low brightness tail caused by backlight, occlusion or ground object relief, so as to realize more balanced appearance of background brightness without changing the real geometric structure. After the above morphological shaping, the global and local brightness imbalance of the image is significantly alleviated, but there may still be insufficient contrast in the medium brightness region. Therefore, the processed image is subjected to contrast-limited histogram equalization, and through block histogram redistribution and excessive amplification suppression mechanism, the gradient visibility of the medium brightness region is significantly enhanced while keeping the bright part and the dark part from being oversaturated, so that the slender arm, paddle root, fuselage contour and support structure in the complex background texture obtain clearer gray scale transition.

[0040] Considering that the inevitable strong reflection in the environment may still remain locally and cause interference to boundary detection and subsequent connected component labeling, further edge-preserving smoothing is performed in the strong reflection candidate region by using guided filtering. The strong reflection candidate region can be obtained jointly according to the brightness peak value distribution and the neighborhood gradient stability, forming a mask for limiting the processing range. The guided filtering takes the visible light image frame as the guide, and by performing edge-preserving smoothing inside the candidate region, the reflection peak value of the highlight region is uniformly weakened without crossing the real edge, while the sharpness of the gray scale transition at the edge position is kept from being destroyed, thereby avoiding the boundary expansion and connected component virtual connection caused by highlight tailing. This strategy ensures that the high-contrast region will not cause edge drift due to simple smoothing, and also avoids over-enhancement of the low-contrast region, so that the image forms a kind of contrast expression that is relatively neutral to illumination on the whole, that is, the neutral contrast image. The generation of the neutral contrast image emphasizes two points. First, after suppressing isolated strong reflection and deep shadow, the background intensity distribution of the scene is pulled back to a scale comparable to the real target structure. Second, while enhancing the medium brightness region, the edge-preserving strategy is adopted to ensure that the slender structure, cross connection and local gap related to the unmanned aerial vehicle will not be blurred or misconnected.

[0041] Since the aligned frame pair has been strictly aligned with the infrared channel through the bidirectional reversible mapping table, the neutral contrast map can not only be directly used for edge extraction and connected domain labeling in this channel, but also provide reliable geometric contour basis when performing entry-level registration with the neutral heat map in the subsequent stage, so that the saliency from the heat distribution and the shape information from the visible light boundary can be complementary in the same pixel coordinate system. To ensure the stability of the downstream decision-making link, the neutral contrast map can be directly used for threshold segmentation after being generated. Its histogram structure is more bimodal and separable after contrast-limited histogram equalization, which is conducive to segmentation by taking the lowest point between the second peak and the first peak as the threshold to obtain a contrast candidate mask with less noise. At the same time, edge-preserving smoothing suppresses the reflection peak value in the strong reflection candidate region, so that these regions no longer form unnecessary false large connected domains in the segmentation and morphological processing stage.

[0042] Further, the purpose of generating a neutral heat map on the infrared image frame of the aligned frame pair is to form an intensity basis that has a stable expression of thermal saliency and can be accurately corresponded with the visible light channel, so that the subsequent connected domain labeling, entry-level correspondence and configuration signature construction can be reliably performed in the same pixel coordinate system. In specific implementation, first, relying on the aligned frame pair and the bidirectional reversible mapping table, it is ensured that each pixel position of the infrared image frame has a determined spatial correspondence relationship when cross-channel processing. Then, the threshold segmentation is performed to obtain a heat zone and cold zone binary mask, so that the heat sources potentially caused by dynamic systems, electronic components or aerodynamic friction are concentrated in the heat zone mask, while the environment background, sky area or ground object shadow are distinguished in the cold zone mask, thereby preliminarily completing the foreground and background division of the heat distribution. Considering that infrared imaging is easily affected by random thermal noise, isolated hot spots and sensor drift, directly using the initial heat zone mask may easily produce a large number of pseudo heat sources with small area, irregular shape and no spatial continuity in the subsequent connected domain analysis, therefore, the heat zone mask is subjected to morphological reconstruction, by taking the original mask as a constraint and performing connected consistency recovery on the reliable structure, the pseudo heat sources with small area are deleted and the small cracks caused by noise are repaired, so that the reconstructed heat zone has better overall integrity and boundary integrity in line with the target physical form. After obtaining the reconstructed heat zone, in order to balance the heat boundary continuity and internal texture smoothness, edge-preserving smoothing is performed on the reconstructed heat zone, which suppresses the high-frequency noise and brightness fluctuation caused by internal temperature fluctuations without crossing the boundary, so as to ensure that the regions related to the unmanned aerial vehicle, such as the fuselage, wings, rotor motor or heating accessories, present a continuous, stable and easily connected domain labeled intensity distribution in the thermal aspect.

[0043] Meanwhile, to improve the visibility of the subtle gradient at the thermal-cold interface, a local contrast stretch is performed on the cold region, which enhances the expression of the thermal-cold interface density by stretching the gray scale between partitions, making the small intensity change of the boundary transition zone more easily separated in the subsequent boundary detection and threshold segmentation, reducing the boundary adhesion and false merging caused by slow temperature gradient. The neutral thermal map obtained through the above operation has stronger distinguishability of thermal region proportion, higher thermal boundary continuity and more stable thermal-cold interface density performance in complex background compared with the original infrared image frame. When the neutral thermal map is matched with the neutral contrast map at the entry level, the pixel-to-pixel accurate projection can be realized through the bidirectional reversible mapping table, so as to provide structural evidence from the thermal dimension for each entry in the topological manifold atlas, including information such as thermal region spatial distribution, boundary consistency and local intensity gradient at the interface. In the downstream processing, the neutral thermal map directly participates in threshold segmentation and connected component labeling, and the connectivity after morphological reconstruction and the boundary stability after edge-preserving smoothing significantly reduce the interference of false heat sources, thermal spot fragmentation and boundary sawtooth on the candidate region generation; the local contrast stretch makes the cold region background present a more balanced intensity field, which facilitates obtaining a clearer double-peak structure when the lowest point between the second peak and the first peak is used as the threshold for segmentation, thereby improving the accuracy of candidate region screening.

[0044] Further, the implementation process of generating the topological manifold atlas first relies on the strict registration of the neutral contrast map and the neutral thermal map in the same pixel coordinate system, so the bidirectional reversible mapping table is used as the only basis for pixel-level association, and the uncertain pixels with occlusion, extrapolation or interpolation are marked to avoid pollution of subsequent statistics. After confirming the pixel alignment, the neutral contrast map and the neutral thermal map are respectively labeled with connected components, and the regions with closed boundaries and internal consistency are divided into several connected components, and the bounding rectangle, area, perimeter and adjacent relationship of each connected component are recorded, while the original pixel index is kept to directly interact with the bidirectional reversible mapping table. Then the shape backbone of each connected component is extracted, which is used to describe the connected skeleton of slender and branched structures, and can stably represent the projection of unmanned aerial vehicle arms, fuselage and blades; the number of holes is calculated to reflect the internal cavities caused by internal cavities, blade gaps or component occlusions; the connected level is calculated to represent the inclusion and inclusion relationship, contact relationship and adjacency order between connected components, forming a topological sequence composed of parent-child relationship and brotherhood. Further, the boundary chain code histogram of each connected component is calculated based on the boundary and texture features, which is used to statistically represent the boundary main direction distribution and direction stability; the boundary turning point sequence is calculated to record the position and order of the direction mutation on the boundary, and to describe the angle folding degree and folding rhythm of the shape; the binary texture mode histogram is calculated to statistically represent the frequency of the binary texture mode in the connected domain and its boundary neighborhood, reflecting the binary structure difference caused by material, blade modulation or background texture.

[0045] After the completion of the respective connected component representation of the two channels, the entry-level correspondence between the spatially overlapped connected components of the neutral contrast map and the neutral thermal map is established by means of a bidirectional reversible mapping table: taking the pixel mapping and spatial overlap as the criterion, the connected components in the neutral contrast map are taken as the candidate entry subjects, and the connected components in the neutral thermal map that have effective overlap or adjacent boundaries with them are paired to generate entries containing the attributes of the two channels; when one-to-many or many-to-one situations occur, the entries are split or merged according to the overlap area, boundary consistency and hot-cold boundary distribution, and the corresponding relationship and conflict resolution strategy are explicitly recorded in the entries to ensure the traceability and explainability of the entries. The entry-based set constitutes the topological manifold atlas, the content of which includes the geometric elements (shape stem, hole number, connected level) of the entry body, the boundary elements (boundary chain code histogram, boundary turning point sequence) and the texture elements (binary texture pattern histogram), as well as the overlap ratio, boundary distribution and pixel alignment quality of the two channels. The mutual relationship between the entries is stored in the form of a graph structure, recording the edge information of adjacency, containment, contact and cross-channel mapping, thereby providing a unified geometric-thermal joint representation under the same pixel coordinate system. The topological manifold atlas can be directly retrieved in the subsequent candidate region generation and consistency judgment stage: for any candidate region, the entries and associated entries that are spatially consistent with it can be found in the atlas, and the calculated features such as shape stem, hole number, connected level, boundary chain code histogram, boundary turning point sequence and binary texture pattern histogram can be quickly retrieved, and combined with the thermal zone distribution and hot-cold boundary information inherited from the neutral thermal map for comprehensive evaluation.

[0046] Further, the brightness distribution of the neutral contrast image is analyzed and a histogram is constructed, and a threshold is determined as the minimum point between the second peak and the first peak of the histogram to segment the contrast candidate mask. The threshold is selected to minimize the aliasing between the background and the structure boundary in a statistical sense. Since the neutral contrast image has been suppressed by the top-hat operation and the bottom-hat operation to suppress strong reflections and deep shadows, and has been enhanced by the contrast limited histogram equalization to enhance the medium brightness region, the contrast candidate mask presents a good bimodal separability in the whole, but may still contain small-scale burrs and isolated spots caused by fine texture, residual reflections or weak shadows. Therefore, an opening operation is performed on the mask to remove the burrs, and regions with an area smaller than a first set area threshold are removed to retain the responses with geometric integrity and boundary continuity. Simultaneously, the same threshold determination mechanism is performed on the neutral heat image to segment the heat candidate mask with the minimum point between the second peak and the first peak of the histogram of the neutral heat image as the threshold. The neutral heat image has been reconstructed by morphological reconstruction to remove the pseudo heat sources with an area smaller than a set area threshold, and has been smoothed inside the heat region to retain the edge. Therefore, the heat candidate mask has high connectivity and boundary stability for the real heat region. To further suppress the noise connected domains caused by the background temperature fluctuation, the opening operation is also performed on the heat candidate mask to remove the burrs, and regions with an area smaller than a second set area threshold are removed to avoid misjudging the random heat spots or edge jaggies as effective targets. After the two-channel independent candidate mask construction is completed, the heat candidate mask and the contrast candidate mask are projected into the same coordinate system for spatial consistency constraint. The projection relies on the aligned frame pair and the bidirectional reversible mapping table established in the previous step to realize pixel-by-pixel mapping, ensuring that the two masks are strictly comparable in geometry.

[0047] For each connected domain existing in the contrast candidate mask, the overlap ratio with the heat candidate mask in space is calculated. To avoid accidental mismatches caused by boundary jitter or interpolation, the mapping effectiveness and occlusion markers are filtered during the overlap calculation, and only the pixels with effective mapping are counted. Then, the connected domains are screened according to a preset overlap lower limit. Only when the overlap ratio is not lower than the set overlap lower limit, the region is considered to have both geometric contrast and thermal significance. To constrain the false detection regions with abnormal shapes, the aspect ratio of the bounding rectangle of the region passing the overlap screening is calculated, and whether it falls within the set aspect ratio range is checked. This step is used to exclude extremely thin or extremely flat connected domains formed by ground texture, linear reflections or temperature strips, while retaining geometric shapes consistent with the drone arm, body or blade projection. For adjacent connected domains with one-to-many or many-to-one relationship, they are preferentially merged or split according to the overlap ratio, boundary consistency and bounding rectangle compatibility, and the aspect ratio of the bounding rectangle and the overlap ratio of the merged or split regions are recalculated to ensure that each retained region satisfies the overlap ratio not lower than the overlap lower limit and the aspect ratio of the bounding rectangle falls within the aspect ratio range.

[0048] All the candidate regions obtained via the above steps collectively constitute a candidate set. The candidate set, under the joint action of statistical threshold, morphological constraint, cross-channel overlap and geometric shape constraint, significantly reduces false positives and false negatives caused by single-channel noise, local strong reflection, random hot spot and background texture, providing a moderate scale and reliable quality input for subsequent multi-dimensional feature extraction and consistency determination. Since each candidate region in the candidate set can establish a direct correspondence with the entry in the topological manifold atlas under the support of the bidirectional reversible mapping table, the shape stem, hole number, connectivity level, boundary chain code histogram, boundary turning point sequence and binary texture pattern histogram of the candidate region can be directly read in the subsequent decision-making, and the hot area proportion, hot boundary continuity and hot-cold junction density are jointly evaluated, thereby improving the robustness and generalizability of the recognition while maintaining the computational efficiency, laying a reliable foundation for the final recognition result based on the shape stem feature output.

[0049] Further, for any candidate region, first obtain the corresponding entry in the topological manifold atlas through its spatial coverage relationship, and generate a configuration signature by aggregating joint elements from the neutral contrast map and the neutral heat map within the entry. The configuration signature is composed of the aspect ratio of the circumscribed rectangle, the shape stem length proportion, the hole number, the connectivity level, the chain code histogram main direction concentration, the boundary turning density, the binary texture pattern high frequency proportion, the hot area proportion, the hot boundary continuity and the hot-cold junction density, etc. The aspect ratio of the circumscribed rectangle is used to measure the length of the overall shape of the candidate region and the direction stability, the shape stem length proportion is used to reflect the dominant degree of the elongated structure and the branch structure in the region, the hole number and the connectivity level are used to express the internal topological organization caused by the blade gap, the part occlusion or the support structure, the chain code histogram main direction concentration and the boundary turning density are used to describe the consistency and the angle folding complexity of the boundary direction, the binary texture pattern high frequency proportion is used to distinguish the structure texture near the real target boundary and the background noise texture, and the hot area proportion, the hot boundary continuity and the hot-cold junction density are used to represent the spatial coverage, the boundary coherence and the gradient stability of the target heating site from the thermal dimension. The configuration signature generation preserves the mapping reference from pixels to entries, entries to candidate regions, and labels any local uncertainty caused by interpolation, occlusion or parallax, ensuring that the uncertainty contribution is limited in subsequent decision-making.

[0050] In the consistency determination according to the preset rule, the satisfaction of the three types of criteria is independently calculated for each candidate region, and at least one type of criterion is satisfied, that is, the strategy of passing is determined to enhance the robustness under complex scenes and cross-channel disturbance. The condition for passing the geometric determination is that the aspect ratio of the candidate region is within the set aspect ratio range and is greater than the set first aspect ratio threshold, the length proportion of the shape stem is greater than the set length threshold, and the number of holes is not greater than 1. This combination of conditions is used to filter out wide and thick block-shaped regions and irregular fragmented regions caused by ground feature textures or shadow boundaries, so that the remaining candidate regions are more consistent with the geometric prior of the unmanned aerial vehicle arm, body or airfoil. The condition for passing the boundary texture determination is that the chain code main direction of the candidate region is concentrated, the boundary turning density is within the set density interval, and the high frequency proportion of the binary texture pattern is within the set proportion interval. This set of conditions is used to suppress the boundary jitter and high-frequency false texture caused by noise, reflection or background strip, and preferentially selects solid profiles with stable main direction and moderate angle folding, which conforms to the boundary statistical characteristics of multi-rotor cross structure or fixed-wing slender edge. The condition for passing the thermal feature determination is that the hot area proportion of the candidate region is within the set hot area interval, the thermal boundary is continuous, and the hot-cold junction density is within the set hot-cold junction density interval. This set of conditions is used to ensure that the candidate region presents continuous and non-occasional thermal saliency in the infrared channel, and maintains continuous and separable intensity gradient on the boundary and junction band, so as to exclude isolated hot spots, drifting thermal noise or false responses caused by background temperature variation.

[0051] In actual decision-making, the three types of criteria are evaluated at the same time, and as long as any one type of criterion is satisfied, it can be marked as passing the consistency determination; if multiple types of criteria are satisfied at the same time, the priority level is marked in the entry record for subsequent conflict resolution and type output reference; if only the boundary texture or thermal feature level is satisfied and the geometric determination is not satisfied, the candidate region is retained and time consistency is sought in adjacent frames or multi-view to reduce the risk of missed judgment caused by instantaneous occlusion or local parallax. For the candidate region that passes the consistency determination, the process of outputting the recognition result is entered, and the shape stem feature is directly used as the basis for type discrimination. Under the premise of the same pixel coordinate system and entry level mapping, the consistency between the overall topological form of the shape stem and the aspect ratio of the circumscribed rectangle is preferentially checked: if the shape stem is cross-shaped or star-shaped, and the aspect ratio of its circumscribed rectangle is close to 1, the recognition result is output as a multi-rotor unmanned aerial vehicle. This situation indicates that there are multiple approximately orthogonal or radial stem branches inside the candidate region, which conforms to the cross support pattern formed by the multi-rotor arms under the condition of top view or oblique view, and in the neutral contrast map, it shows a profile intersecting in multiple directions, and in the neutral thermal map, it is often accompanied by multiple point or ring-shaped hot area distribution and continuous thermal boundary. If the shape stem is in a slender strip shape and the two ends are sharp, the recognition result is output as an unmanned aerial vehicle fixed wing.

[0052] The case corresponds to the elongated projection of the fixed-wing body or wing in the line-of-sight direction, the boundary main direction is concentrated and the boundary turning density is low, the circumscribed rectangle length-width ratio deviates significantly from 1, and in the neutral heat map, it may present linear or strip-shaped heat zones distributed along the main axis direction, and form a continuous and uniform gradient zone at the cold-heat junction. To further reduce misjudgment, a consistency cross-check is performed on the adjacent items of the candidate region before the type output, if there are significantly overlapping items with different shape main stem types, the main item is selected according to the connected level and overlap degree of the item in the topological manifold atlas, and the secondary item is marked as an auxiliary structure or background interference; for the case of local main stem incoherence in the same item, the consistency is corrected by referring to the chain code histogram main direction concentration and heat boundary continuity, and if necessary, the components affected by the obstruction in the configuration signature are reduced in weight. After outputting the recognition result, the configuration signature components used in this decision and the threshold satisfaction are registered in the item history, so as to be used as a stability measure in the time sequence association of the subsequent frames, if the same shape main stem type and similar circumscribed rectangle length-width ratio are maintained for continuous multiple frames, the recognition confidence of the target is improved; if there is a transition from cross or star to elongated strip shape in the continuous frames and accompanied by a monotonic change in the circumscribed rectangle length-width ratio, the recognition category is updated or maintained according to the time sequence consistency rule. Through the above consistency determination and output recognition result, the geometric, boundary texture and thermal characteristics are uniformly expressed at the item level, and the shape main stem feature is used as the key evidence for final type distinction, so as to maintain stable distinction and reliable output of the multi-rotor and fixed-wing unmanned aerial vehicles under the conditions of complex background, light change and thermal field disturbance.

[0053] Figure 2The core algorithm principle of threshold selection based on histogram analysis is described in detail. The standard two-dimensional coordinate system is adopted in the histogram, the horizontal coordinate represents the gray value, the value range is 0 to 255, and the vertical coordinate represents the pixel number, the unit is individual, and the value range is 0 to 4500 pixels. The histogram curve shown in the figure presents a typical bimodal distribution characteristic, which is an important characteristic shared by neutral contrast maps and neutral heat maps. The first peak is located at the gray value of 56, corresponding to 2200 pixels, marked with red and indicated by a vertical dashed line; the second peak is located at the gray value of 180, corresponding to 3800 pixels, also marked with red. There is a clear valley between the two peaks, which is located at the gray value of 118, corresponding to 600 pixels, emphasized by blue and marked with a solid blue line. According to the threshold segmentation algorithm, the system automatically identifies the first peak and the second peak of the histogram, and then finds the lowest point between the two peaks as the segmentation threshold. The specific calculation formula is: T=argmin(H(x))forx∈[first peak position, second peak position], where H(x) is the pixel number corresponding to the gray value x. The image is divided into two regions by the threshold: the background region with a gray value less than 118 and the contrast candidate region or heat candidate region with a gray value greater than or equal to 118. The technical advantage of this bimodal distribution threshold segmentation method is that it can adaptively determine the optimal segmentation point, effectively separate the target region from the background region, and lay a solid image preprocessing foundation for subsequent candidate region extraction and geometric constraint screening. Experimental verification shows that this threshold selection algorithm can maintain stable segmentation effect under various lighting conditions and thermal environment.

[0054] Figure 3This paper demonstrates the technical principle of UAV type recognition based on shape backbone features. The analysis graph uses aspect ratio as the horizontal axis, ranging from 0.1 to 2.1, and the proportion of shape backbone length as the vertical axis, ranging from 0% to 90%, establishing a two-dimensional feature determination space. The graph shows the distribution characteristics of two types of UAV targets: multi-rotor UAV samples are marked in red, mainly distributed in the aspect ratio range of 0.8 to 1.4, with the corresponding shape backbone length proportion concentrated in the range of 65% to 85%; fixed-wing UAV samples are marked in blue, mainly distributed in the aspect ratio range of 1.5 to 2.0, with the corresponding shape backbone length proportion concentrated in the range of 45% to 70%. Shape backbone feature determination includes several key thresholds: the first aspect ratio threshold is set to 1.5, indicated by an orange dashed line; the length threshold is set to 60%, indicated by a green dashed line. For the identification of multi-rotor drones, the criteria are: the aspect ratio of the candidate region is within the set range and close to 1.0±0.2; the length of the main shape segment accounts for more than 65%; and the main shape segment is cross-shaped or star-shaped. For the identification of fixed-wing drones, the criteria are: the aspect ratio is greater than 1.5; the length of the main shape segment accounts for more than 45%; and the main shape segment is long and thin with sharp ends. Experimental results show that for the 7 multi-rotor drone samples, the average aspect ratio is 0.98±0.15, and the average length segment accounts for 72.3%±8.7%; for the 7 fixed-wing drone samples, the average aspect ratio is 1.67±0.18, and the average length segment accounts for 58.6%±11.2%. This feature-based identification method effectively distinguishes between the two types of drones and provides a reliable basis for the main shape segment features for subsequent identification results.

[0055] This example uses a dual-channel imaging system, with a visible light camera and an infrared camera mounted on the same pan-tilt unit. The visible light camera has a resolution of 1280×720 and a frame rate of 30, while the infrared camera has a resolution of 640×512 and a frame rate of 30. The system is triggered by a unified time source with a timestamp accuracy of 1 millisecond.

[0056] First, acquire a pair of visible light image frames and infrared image frames at the same time point, denoted as follows: and .in Represents a visible light image frame. Represents an infrared image frame. Timestamp alignment uses time difference. Criterion, if Then it is considered aligned, where The difference between the timestamps of the two frames. This is the time threshold; in this example, we take... Milliseconds. This data was collected in this instance. Milliseconds, satisfying the alignment condition. Then, respectively... and Perform a 3×3 median filter once to obtain the denoised image. and The median filter kernel size is denoted as... .

[0057] In spatial domain alignment, first in Perform Canny edge detection, and take low and high thresholds respectively. (Grayscale range 0 to 255) to obtain a binary edge map .exist The Shi-Tomasi method was used to extract the corner set. Maximum number of corner points Minimum mass fraction minimum distance Pixels; and extract the set of line segments using probabilistic Hough transform. Minimum line length Pixels, maximum gap Pixels. In Based on the bimodal shape of the grayscale histogram, the lowest point between the two peaks is taken as the threshold. In this example, the actual measurement yielded... Threshold segmentation yields a binary image of the thermal target. .right Perform morphological opening and closing operations, with the structuring element being the radius. A circular shape yields a set of infrared contours with better connectivity. Sampling of corner points at equal arc length intervals along each infrared profile, with a step size of [missing information]. Pixels, to obtain the infrared corner point set .

[0058] Construct cross-channel matching pairs based on geometric relationships. and Pair elements that are approximately parallel, approximately collinear, and whose endpoints are close together. Set an endpoint proximity threshold. Pixels, collinear angle threshold Degree. Obtain the initial set of matching pairs. In this example .right Random Sampling Consensus (RANSAC) is used, with an in-point criterion pixel geometric error threshold. Pixels, maximum number of iterations Obtain the inner point subset In this example .exist Supports prioritizing the fitting of perspective transformation matrices (Hotopy matrix), if ill-conditioned, it degenerates into an affine transformation. This example has a stable fit, yielding... .

[0059] in Used to transfer infrared coordinates Mapped to visible light coordinates The mapping relationship is as follows: , For homogeneity metric. right Geometric correction was performed to obtain And generate a bidirectional invertible mapping table for coordinate mapping. ,in At the same time, invalid and interpolation flags are recorded.

[0060] A neutral contrast map was constructed in the visible light channel. Apply top-hat and bottom-hat operations sequentially, structuring element radius Pixels, to obtain the image after brightness shaping Then, contrast-limited histogram equalization (CLAHE) is performed, with the block grid size set. pruning threshold , to obtain the enhanced image For candidate regions with strong reflections, guided filtering is used to preserve edges and smooth the edges. Candidate regions are then selected based on a threshold value. Combined with local gradient instability determination, guiding the filter radius Regular terms Output neutral contrast chart .

[0061] Construct a neutral thermal map in the infrared channel. By threshold The hot and cold regions are segmented to obtain a binary mask. and .right Perform morphological reconstruction and mark the minimum effective area threshold. Pixels, removing false heat sources, to obtain the reconstructed heat area. .exist The inner execution edge is preserved and smoothed using bilateral filtering and spatial standard deviation. Strength standard deviation Output .exist Perform localized contrast stretching within the input, and set the upper and lower quantile points. Linear stretching to the full dynamic range yields a neutral heatmap. .

[0062] based on make sure and After pixel alignment, connected component labeling is performed. Let the set of connected components be... Extract the shape backbone for each connected component, and denote it as the length. The bounding rectangle has a width and height of 1000. Circumference is The area is Define the aspect ratio of the circumscribed rectangle. , The aspect ratio is defined; the length percentage of the main body of the shape is defined. , The ratio of the main stem length to the perimeter; the number of holes is... The connectivity level is denoted by the parent-child nesting depth. The boundary chain code histogram quantizes the directions into 8 directions and normalizes the frequency vector. Concentration in the main direction .

[0063] Boundary transition density is defined as ,in This represents the number of boundary direction changes. The binary texture pattern uses 8-neighborhood LBP encoding, and the histogram is denoted as... High frequency ratio , A set of high-frequency codes. (Through...) Establish a two-channel entry-level correspondence. For any Find the overlapping ones Define the overlap ratio .

[0064] when Create entries at that time. Thermal factors include the proportion of thermal zones. Thermal boundary continuity is expressed as the ratio of thermal boundary length to total boundary length. ,in The number of boundary pixels falling on the hot-cold interface. Total number of boundary pixels; thermal interface density ,in This represents the number of hot and cold boundary pixels detected within a 1-pixel width of the boundary band. This represents the area of ​​the band. After obtaining the neutral contrast map and neutral heatmap, a candidate set is generated. Calculate the histogram, and the threshold for the lowest point between the second peak and the first peak is: Segmentation to obtain contrast candidate masks Perform an opening operation (structuring element radius 2 pixels) and remove elements with an area smaller than 2 pixels. The region of pixels; for by Segmentation of thermal candidate masks Similarly, perform the opening operation and remove areas smaller than 100. The region of pixels. and Project onto the same coordinate system and calculate each candidate region. Only retain and the aspect ratio of the circumscribed rectangle , obtaining a candidate set . This example obtains 6 candidate regions.

[0065] The consistency determination generates a configuration signature for each candidate region , which contains . Set the determination threshold: the first aspect ratio threshold , the length threshold , the chain code concentration threshold , the boundary turning density interval , the high-frequency proportion of binary texture pattern interval , the hot area proportion interval , the hot-cold junction density interval , and the hot boundary continuity . The determination rule is one of the three: geometric determination, boundary texture determination, and hot feature determination.

[0066] The numerical values and calculation processes of two specific candidate regions are given below to show the determination and output.

[0067] Candidate region A: the width and height of the circumscribed rectangle are measured pixels, pixels, . The skeleton length pixels, the perimeter pixels, . The number of holes , the connectivity level . The proportion of the main peak of the boundary chain code histogram . The boundary turning count , . The proportion of LBP high-frequency code count . The . The hot area proportion , the hot boundary continuity , the hot-cold junction density . According to the geometric determination: and do not meet (because 1.037 < 1.2), but the shape main stem length proportion , the number of holes . The geometric determination as a whole does not pass. The boundary texture determination: does not meet, but , which still does not pass. The hot feature determination: , , the hot feature determination passes. According to the one-of-three strategy, the consistency determination of candidate region A passes. For type judgment of the shape main stem, the skeleton is cross-shaped, the lengths of the four branches are approximately equal, and the aspect ratio of the circumscribed rectangle Close to 1, so the output recognition result is a multi-rotor drone.

[0068] Candidate region B: circumscribed rectangle width-height Pixels, Pixels, Skeleton length Pixels, perimeter Pixels, Hole number Connected level Chain code main direction concentration Boundary turning density Boundary turning count 5, high frequency proportion of binary texture pattern Corresponding thermal connected domain Thermal zone proportion Thermal boundary continuity Thermal-cold junction density Geometric determination: Not satisfied (because Too large), but Satisfied, , According to the geometric determination three conditions of the application (the combination of "in the set range, and greater than the first threshold, and the length proportion exceeds the threshold, and the hole number is not greater than 1" is adopted in this example), because the first sub-condition is not satisfied, the geometric determination as a whole does not pass. Boundary texture determination: Satisfied, Not satisfied, slightly lower than the lower limit 0.015, Satisfied, so this kind also does not pass. Thermal feature determination: , , Thermal feature determination passes. According to the three-to-one strategy, the consistency determination of candidate region B passes. The type of the shape main stem is judged, the skeleton is in the form of a long strip and the two ends are sharp, so the output recognition result is a fixed-wing drone.

[0069] After calculating the 6 candidate regions one by one, there are 4 that pass the consistency determination, of which 2 satisfy that the shape main stem is in the form of a cross or a star and the length-width ratio of the circumscribed rectangle is close to 1, and are output as multi-rotor drones, and the other 2 are in the form of a long strip and the two ends are sharp, and are output as fixed-wing drones. In order to give a confidence measure, the number of sub-determinations satisfied by each passing item is accumulated and normalized , wherein is the determination confidence of the item. In this example, candidate region A satisfies class 1, ; candidate region B satisfies class 1, . If consecutive If a frame detects the same type of result at the same spatial location, a stable label is output. Temporal consistency is achieved by ensuring that the frames are centered at the same bounding rectangle. radius Pixel neighborhood tracking is complete, if Then adjacent frames are considered to match, where For the first Frame timestamp, The frame period is in milliseconds. In actual testing, the two multi-rotor entries and one fixed-wing entry maintained stable output for 5 consecutive frames, and finally reported the recognition results and trajectory initialization of the two UAV multi-rotor and one UAV fixed-wing in the scene.

[0070] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An unmanned aerial vehicle adaptive target recognition method based on optical detection, characterized in that, The method comprises the following steps: Step 1: A pair of visible light image frames and infrared image frames are collected at the same time point to form an image frame pair, and alignment preprocessing is performed on the image frame pair to obtain an aligned frame pair; Step 2: Based on the aligned frame pair, a neutral contrast map corresponding to the visible light image frame and a neutral heat map corresponding to the infrared image frame are respectively generated, and a topological manifold atlas is generated based on the neutral contrast map and the neutral heat map; Step 3: Threshold segmentation is respectively performed in the neutral contrast map and the neutral heat map, and screening is performed in combination with geometric and spatial constraints to obtain a candidate set; Multi-dimensional features are extracted for each candidate region in the candidate set, and consistency determination is performed according to a preset rule, and if the determination is passed, an identification result is output according to the shape stem feature thereof; In step 1, the process of alignment preprocessing comprises: performing timestamp alignment on the visible light image frame and the infrared image frame, and then performing 3x3 median filtering on the visible light image frame and the infrared image frame respectively; the process of alignment preprocessing further comprises: on the visible light image frame, an edge detection is used to obtain a binary edge map, a corner point detection is used on the binary edge map to obtain a visible light corner point set, and a straight line segment detection is used to obtain a visible light line segment set; on the infrared image frame, a threshold segmentation is performed to obtain a thermal target binary map, a morphological opening and closing operation is performed on the thermal target binary map to extract an infrared contour set, and contour corner points are uniformly sampled on the infrared contour to obtain an infrared corner point set; parallelism, approximate collinearity and endpoint proximity of the visible light line segment and the infrared contour are taken as matching conditions to generate a matching pair; wherein, when the spatial distance between the endpoints of the visible light line segment and the corresponding positions of the infrared contour is not more than 8 pixels, the endpoint proximity is determined, and when the angle deviation between the direction of the visible light line segment and the local tangent direction of the infrared contour is not more than 5 degrees, the approximate collinearity is determined; the random sample consensus screening is used to remove outliers from the matching pair, and a geometric transformation between the two image frames is fitted, the perspective transformation is preferentially fitted, and if stable fitting cannot be achieved, the affine transformation is degenerated; the fitted perspective transformation or affine transformation is used for geometric correction of the infrared image frame, so that the infrared image frame and the visible light image frame are aligned in the same pixel coordinate system to generate an aligned frame pair, and a bidirectional reversible mapping table for coordinate mapping is generated; in step 2, the process of generating the neutral contrast map comprises: on the visible light image frame of the aligned frame pair, the following processes are performed: top-hat operation and bottom-hat operation are used to suppress isolated strong reflection and deep shadow; a contrast limited histogram equalization is performed on the processed image to enhance the contrast of the medium brightness region; guided filtering is used in the strong reflection candidate region to perform edge-preserving smoothing, so as to weaken the reflection peak value while retaining the edge position, and obtain the neutral contrast map; in step 2, the process of generating the neutral heat map comprises: on the infrared image frame of the aligned frame pair, the following processes are performed: threshold segmentation is performed to obtain a hot area and a cold area binary mask; morphological reconstruction is performed on the hot area mask to remove pseudo-thermal sources with too small area; edge-preserving smoothing is performed on the reconstructed hot area, and local contrast stretching is performed on the cold area to obtain the neutral heat map. 2.The unmanned aerial vehicle adaptive target recognition method based on optical detection of claim 1, wherein, The process of generating the topological manifold atlas in step 2 includes: ensuring that the neutral contrast map and the neutral heat map are pixel-aligned based on a bidirectional reversible mapping table; performing connected component labeling on the neutral contrast map and the neutral heat map respectively; extracting the shape stem of each connected component and calculating the number of holes, the connected level, the boundary chain code histogram, the boundary turning point sequence and the binary texture pattern histogram; through the bidirectional reversible mapping table, establishing an entry-level correspondence between the two channels of the connected components that overlap in space, and forming the topological manifold atlas. 3.The method of claim 2, wherein, In step 3, the process of obtaining the candidate set includes: segmenting the neutral contrast map using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a contrast candidate mask, and then removing burrs by opening operation and removing regions with an area less than a first set area threshold; segmenting the neutral heat map using the lowest point between the second peak and the first peak of its histogram as a threshold to obtain a heat candidate mask, and then removing burrs by opening operation and removing regions with an area less than a second set area threshold; projecting the heat candidate mask and the contrast candidate mask into the same coordinate system, calculating the overlap ratio of the two, and only retaining regions that meet the following conditions as candidate regions: the overlap ratio is not less than a set lower overlap limit, and the aspect ratio of its bounding rectangle falls within a set aspect ratio range; all candidate regions together constitute the candidate set. 4.The method of claim 3, wherein, In step 3, the process of consistency determination includes: for each candidate region, obtaining the corresponding entry from the topological manifold atlas to generate a configuration signature, which includes: the aspect ratio of the bounding rectangle, the length ratio of the shape stem, the number of holes, the connected level, the main direction concentration of the chain code histogram, the boundary turning density, the high frequency proportion of the binary texture pattern, the hot area proportion, the hot boundary continuity and the hot-cold junction density; when the candidate region meets at least one of the following conditions, it is determined to pass: geometric determination passes: the aspect ratio of the candidate region is within the set aspect ratio range, is greater than the set first aspect ratio threshold, the length ratio of the shape stem exceeds the set length threshold, and the number of holes is not greater than one; boundary texture determination passes: the main direction of the candidate region is concentrated, the boundary turning density is within the set density interval, and the high frequency proportion of the binary texture pattern is within the set proportion interval; hot feature determination passes: the hot area proportion of the candidate region is within the set hot area interval, the hot boundary is continuous, and the hot-cold junction density is within the set hot-cold junction density interval. 5.The method of claim 4, wherein, The process of outputting the recognition result includes: if a candidate region passes the consistency determination, further judge its shape stem feature: if the shape stem is cross-shaped or star-shaped, and the aspect ratio of its bounding rectangle is within the set aspect ratio range 1.0±0.2, then output the recognition result as a multi-rotor unmanned aerial vehicle; if the shape stem is in the form of an elongated strip and the two ends are sharp, then output the recognition result as a fixed-wing unmanned aerial vehicle.

6. An optical detection based UAV adaptive target recognition system for implementing the method of any one of claims 1 to 5, characterized in that, The system comprises: an acquisition unit, configured to acquire a pair of visible light image frames and infrared image frames at the same time point, form an image frame pair, and perform alignment preprocessing on the image frame pair to obtain an aligned frame pair; a graph processing unit, configured to generate a neutral contrast graph corresponding to the visible light image frame and a neutral thermal graph corresponding to the infrared image frame based on the aligned frame pair, and generate a topological manifold atlas based on the neutral contrast graph and the neutral thermal graph; and an identification unit, configured to perform threshold segmentation in the neutral contrast graph and the neutral thermal graph respectively, and perform screening in combination with geometric and spatial constraints to obtain a candidate set; extract multi-dimensional features for each candidate region in the candidate set, and perform consistency determination according to a preset rule, and if the determination is passed, output an identification result according to a shape stem feature thereof.

Images