Unmanned aerial vehicle cross-spectral re-identification method and system based on short-time kinematic estimation and attitude response modeling

Abstract

The application discloses a kind of short-time kinematic estimation and attitude response modeling-based unmanned aerial vehicle cross-spectrum re-identification method and system, it is related to unmanned aerial vehicle target monitoring technical field, including: to the visible light and infrared dual-spectrum image of synchronous acquisition, short-time kinematic state estimation is carried out without tracking dependence, the attitude increment and confidence of unmanned aerial vehicle in short-time window are obtained;Based on this, the cross-spectrum stable structure correspondence between dual-spectrum is constructed, and attitude-structure response model is established;Further, generate attitude-aware cross-spectrum identity descriptor;Finally, the similarity between visible light and infrared target is calculated according to the descriptor, and cross-spectrum identity matching is completed.The application does not need to rely on long-time tracking, uses short-time kinematic information to enhance attitude robustness, effectively alleviates the appearance change brought by spectral difference, significantly improves the re-identification accuracy and stability of unmanned aerial vehicle under cross-spectrum condition.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) target monitoring technology, and in particular to a method and system for cross-spectral re-identification of UAVs based on short-time kinematic estimation and attitude response modeling. Background Technology

[0002] With the development of the low-altitude economy, the number of drones has grown rapidly. Utilizing ground-based visible light and infrared dual-spectrum systems for all-weather monitoring of drone targets has become an important technical means in the field of low-altitude security. In multi-target monitoring scenarios, cross-spectral target re-identification is a key technology for maintaining target identity consistency and an important foundation for long-term continuous tracking.

[0003] Existing technologies typically employ deep neural networks to extract target appearance features from visible light and infrared images separately, and reduce feature differences between different spectra through shared feature spaces or adversarial learning methods to perform cross-spectral target matching. Some technical solutions use the target's motion trajectory, speed, and flight attitude obtained from long-term tracking as auxiliary information to improve the stability of identity association.

[0004] However, the above-mentioned technical solutions have the following shortcomings: Kinematic information is highly dependent on long-term target tracking. Existing methods typically require first completing long-term continuous target tracking, and then calculating kinematic information such as speed, direction of motion, and attitude changes based on the trajectory. Long-term tracking itself relies on accurate identity re-identification results. When the initial identity association is incorrect, subsequent trajectories will continuously deviate from the true target, and the kinematic information calculated from the trajectory will become invalid, forming a closed-loop problem of interdependence between identity association and motion estimation, reducing the robustness of the system in complex scenarios. The transspectral response caused by UAV attitude changes is ignored. During flight, the UAV's pitch, roll, and yaw attitudes constantly change. Different attitudes alter the visibility of structural areas such as motors, heat dissipation vents, battery compartments, and fuselage, thus changing the visible light texture distribution and infrared radiation distribution. Due to individual differences in motor layout, heat dissipation structure, and airframe design among different UAVs, the transspectral response caused by attitude changes also has individual specificity. Existing technologies typically only utilize a single frame of static appearance for transspectral matching, without establishing a correlation model between attitude and transspectral response, resulting in limited ability to distinguish the identities of UAVs of the same model.

[0005] Therefore, how to fully utilize the transspectral response characteristics induced by UAV attitude to construct a more stable identity description without relying on long-term target tracking is a technical problem that urgently needs to be solved in the field of UAV transspectral re-identification. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract and title of the invention. Such simplifications or omissions shall not be used to limit the scope of the present invention.

[0007] In view of the problems of existing UAV cross-spectral re-identification methods, such as kinematic information dependence on long-term tracking and lack of correlation modeling between attitude and cross-spectral response, this invention is proposed.

[0008] Therefore, the problem to be solved by this invention is how to construct an attitude-aware identity description by utilizing the differences in cross-spectral structural response caused by UAV attitude changes without relying on long-term target tracking, thereby improving the cross-spectral identity differentiation capability between UAVs of the same model.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a cross-spectral re-identification method for unmanned aerial vehicles (UAVs) based on short-time kinematics estimation and attitude response modeling, comprising, Short-time kinematic state estimation without tracking dependence was performed on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images. Based on attitude increment and attitude estimation confidence, a cross-spectral stable structure correspondence of UAV targets in bispectral images is constructed, and an attitude-structure response model is established. Based on the cross-spectral stable structure correspondence and attitude-structure response model, attitude perception cross-spectral identity descriptors of UAV targets in bispectral images are generated; Based on the attitude-aware cross-spectral identity descriptor, the cosine similarity between the UAV target in the visible light image and the UAV target in the infrared image is calculated. The cross-spectral identity matching result is obtained by using bipartite graph matching methods such as the Hungarian algorithm, and the cross-spectral re-identification of the UAV target is completed.

[0010] Secondly, embodiments of the present invention provide a cross-spectral re-identification system for unmanned aerial vehicles (UAVs) based on short-time kinematics estimation and attitude response modeling, comprising: The short-time kinematics estimation module is used to perform tracking-independent short-time kinematic state estimation on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images. The attitude response modeling module constructs the cross-spectral stable structural correspondence of the UAV target in the dual-spectral image based on attitude increment and attitude estimation confidence, and establishes an attitude-structure response model. The identity descriptor generation module generates attitude-aware cross-spectral identity descriptors for UAV targets in bispectral images based on cross-spectral stable structural correspondences and attitude-structure response models. The cross-spectral identity matching module is used to calculate the cosine similarity between UAV targets in visible light images and UAV targets in infrared images based on attitude-aware cross-spectral identity descriptors. It then uses bipartite graph matching methods such as the Hungarian algorithm to obtain the cross-spectral identity matching results, thus completing the cross-spectral re-identification of UAV targets.

[0011] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any step of the above-described method for cross-spectral re-identification of unmanned aerial vehicles based on short-time kinematics estimation and attitude response modeling.

[0012] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the above-described method for cross-spectral re-identification of unmanned aerial vehicles based on short-time kinematics estimation and attitude response modeling.

[0013] Compared with existing technologies, the advantages of this invention are as follows: It employs a detection-driven short-term kinematic state estimation mechanism to obtain the attitude increment of the UAV target without requiring long-term trajectory and cross-frame identity preservation, avoiding the closed-loop problem of mutual dependence between identity re-identification and tracking-dependent re-identification, thus improving the robustness of identity association in complex scenarios. It uses the UAV's flight attitude as an incentive factor for the manifestation of identity features, inducing changes in the visibility of different structural regions through attitude, and establishing a correlation model between attitude and cross-spectral structural response, extending identity representation from a static appearance to a dynamic attitude response mode, thereby improving the ability to distinguish identities between UAVs of the same model. It utilizes attitude response consistency to dynamically correspond cross-spectral features of different structural regions, making identity encoding conform to the physical imaging laws during actual UAV flight, thus improving the identity consistency between visible light targets and infrared targets. Since the attitude-structure response model can utilize the dynamic identity information generated during UAV movement, this invention can obtain more stable identity association results in scenarios with target maneuvering flight, frequent attitude changes, and multi-target intersections, providing support for subsequent continuous multi-target tracking. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 The flowchart shows a method for cross-spectral re-identification of unmanned aerial vehicles based on short-time kinematics estimation and attitude response modeling. Figure 2 This is a structural diagram of a UAV cross-spectral re-identification system based on short-time kinematic estimation and attitude response modeling. Detailed Implementation

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0016] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0017] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0018] As mentioned in the background section, existing UAV cross-spectral re-identification methods heavily rely on long-term target tracking for kinematic information and neglect the cross-spectral response patterns caused by UAV attitude changes, resulting in limited ability to distinguish between UAVs of the same model. To address these issues, this invention provides a UAV cross-spectral re-identification method and system based on short-time kinematic estimation and attitude response modeling.

[0019] Example 1

[0020] Reference Figure 1 , Figure 1 This is a flowchart illustrating a cross-spectral re-identification method for unmanned aerial vehicles (UAVs) based on short-time kinematics estimation and attitude response modeling, according to an embodiment of the present invention. Figure 1 As shown, in a UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling, the UAV target identity association in a ground-based visible / infrared dual-spectral monitoring system includes: This embodiment employs a fixed dual-spectrum imaging device to observe UAV targets in the airspace. The dual-spectrum imaging device includes a visible light camera and an infrared camera. The optical axes of the visible light and infrared cameras are calibrated to establish a corresponding target region relationship between the visible light and infrared images. The visible light camera is used to acquire texture, contour, and edge structure information of the UAV target, while the infrared camera is used to acquire thermal radiation and infrared structural response information of the UAV target.

[0021] This embodiment also includes a computing device for performing target detection, local structural key point extraction, short-time kinematics estimation, attitude response modeling, cross-spectral structure mapping, and identity matching processing. Visible light and infrared images are acquired simultaneously, with the image frame rate set to, for example, 25–30 frames per second. For UAV targets entering the monitoring area, target detection boxes are obtained in both the visible light and infrared images using the target detection module.

[0022] S1: Perform short-time kinematic state estimation without tracking dependence on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images.

[0023] S1.1: The short-term window starts with the detection of a UAV target by the target detector in frame t. After frame t, N frames are continuously acquired to form a short-term analysis window. Local candidate regions are established inside the detection box, with the center of the detection box as the reference.

[0024] Specifically, the YOLO target detector is used to obtain the detection bounding boxes of the UAV target in the visible light image and infrared image of the t-th frame, respectively. ,in and Indicates the center position of the detection box. and These represent the width and height of the detection box, respectively. Local candidate regions are established within the detection box, using its center as a reference. Subsequent local structural keypoint extraction and local correspondence calculations are performed only within these local candidate regions. The length of the short-time analysis window can be adjusted based on the target's flight speed, imaging frame rate, and target size.

[0025] In an optional embodiment, the value of N ranges from 8 to 20, and in this embodiment, N is 12; when the target detection box size is 64×48 pixels, 32×32 pixels are selected as a local candidate region in the center area of ​​the detection box.

[0026] S1.2: The ORB feature point detection method is used to extract local structural key points in local candidate regions, and the set of local structural key points is obtained by filtering according to the feature response intensity. .

[0027] In an optional embodiment, when 25 feature points are detected in a local candidate region, 10 to 15 of them are selected for subsequent processing based on feature response intensity, local texture stability, and edge response.

[0028] S1.3: For adjacent frames t and t+1 within the short-time analysis window, select the set of local structural key points within the local candidate region. Local matching is performed on each local structural key point to obtain the two-dimensional displacement of the i-th local structural key point. The specific formula is as follows: ; in, The displacement of the i-th structural point along the x-axis of the image at frame t is given. The displacement of the i-th structural point along the y-axis of the image at frame t.

[0029] Furthermore, for the matching results, obvious erroneous matches are removed based on the matching distance, local correlation peaks, and consistency of position between consecutive frames.

[0030] S1.4: Two-dimensional displacement based on multiple key points of local structure after screening The median statistical method is used to estimate the common translational components of the UAV target in the image plane, and the two-dimensional displacement of each local structural key point is calculated. After removing common translation components, the structural relative displacements that characterize the relative motion of the internal structure of the UAV target are obtained. .

[0031] Furthermore, in monitoring scenarios where multiple UAV targets coexist, for the current detection box, only local structural keypoints located within the effective area of ​​the current detection box and satisfying local spatial constraints are used for motion estimation. When two detection boxes partially overlap, local structural keypoints falling into the overlapping area of ​​both detection boxes are marked as low-reliability points and are not included in the attitude increment solution. When two UAV targets are partially occluded, local structural keypoints in the unoccluded area are used for motion estimation. When the current UAV target is completely occluded by another UAV target, and a sufficient number of local structural keypoints cannot be obtained, the attitude increment update of the current short-term analysis window is stopped. After the current UAV target is re-detected, the short-term kinematic estimation window is re-triggered based on the new detection results. Thus, the current short-term kinematic estimation does not depend on the target identity of the previous short-term analysis window, nor on the long-term trajectory.

[0032] S1.5: Combine the local kinematic mapping relationships corresponding to multiple local structural key points in the set of local structural key points to obtain an overdetermined set of equations, and use the weighted least squares method to solve for the attitude increment.

[0033] Specifically, the formulas for the overdetermined system of equations are as follows: ; in, Let represent the local kinematic relationship between the local relative displacement of the M-th structure point in frame t and the attitude change of the UAV. Let be the local projection sensitivity matrix of the structure point in frame t with respect to the three-axis attitude change. Let be the image measurement error and model error of the structural point in frame t.

[0034] Furthermore, a local kinematic mapping relationship between the relative displacement of the structure and the attitude increment is established, with the specific formula as follows:

[0035] ; in, Let represent the local kinematic relationship between the local relative displacement of the i-th structure point in frame t and the attitude change of the UAV. These represent the changes in Roll, Pitch, and Yaw. Let be the local projection sensitivity matrix of the i-th local structural keypoint to the three-axis attitude change. The increment of the roll angle for the UAV target between frame t and frame t+1. The pitch angle increment of the UAV target between frame t and frame t+1. The yaw angle increment of the UAV target between frame t and frame t+1. Let be the image measurement error and model error of the structural point in frame t.

[0036] For cases with obvious anomalous matching points, RANSAC or robust least squares methods are used to solve the problem, reducing the impact of erroneous local structural keypoints on the attitude increment. The above calculation is repeated for N consecutive frames within the short-time analysis window to obtain... as well as This allows us to obtain the target's roll angle within a short-term analysis window. Pitch angle and yaw angle The change in quantity.

[0037] S1.6: Calculate the attitude estimation confidence based on the number and spatial distribution of the local structural key points involved in the solution, the condition number of the attitude observation matrix, and the attitude fitting residuals.

[0038] Specifically, the structure distribution index is calculated based on the spatial covariance or coverage of the local structural key points participating in the attitude increment solution within the local candidate region; the attitude observability index is calculated based on the ratio of the minimum singular value to the maximum singular value of the attitude observation matrix or its condition number; the model fitting index is calculated based on the residual between the structural point displacement obtained by reprojecting the attitude increment estimation results and the actual observed displacement; and the attitude estimation confidence Cpose is obtained based on the weighted combination or product of the various indices.

[0039]

[0040] in, As a structural distribution index, it calculates the dispersion, coverage, or spatial covariance of each reliable structural point in the target local coordinate system in the horizontal and vertical directions. As an attitude observability index, an attitude observation matrix is ​​established based on all reliable structural points, and the sensitivity of this observation matrix to three-axis attitude changes is analyzed. Evaluation can be performed by calculating the singular values, eigenvalues, or condition numbers of the attitude observation matrix. To fit the residual calculation model, the estimated roll, pitch, and yaw increments are re-introduced into the local kinematics model to calculate the theoretically expected displacement of each structural point under the attitude change. The predicted displacement is then compared with the actual measured displacement of the structural points in the image, and the residual is calculated. In an optional embodiment, the attitude observation moments project the three-axis attitude change into the relative displacement of the structural points. : ; in, This represents the local projection sensitivity matrix of key points in the local structure to three-axis attitude changes. Let Roll, Pitch, and Yaw be the changes, and K be the local kinematic relationship between the local relative displacement and the UAV attitude change.

[0041] By calculating the eigenvalues ​​of the attitude observation matrix And sort them: ......; Defineable attitude observability metrics: ; in, and , respectively, are the minimum and maximum singular values ​​in the eigenvalues ​​of the attitude observation matrix, and ε is a preset adjustment constant. The smaller the attitude observability index, the more easily the attitude increment is affected by image measurement noise. The larger the attitude observability index, the easier it is for the three-axis attitude to be stably distinguished.

[0042] Preferably, when the attitude estimation confidence Cpose is higher than a preset threshold, the current attitude increment ΔPt and the attitude estimation confidence Cpose are used for subsequent processing; when the attitude estimation confidence Cpose is lower than the preset threshold, the attitude increment corresponding to the current short-time analysis window is stopped, and the short-time kinematic estimation window is retried based on the new detector detection results; the preset threshold is adjusted through engineering practice. The usual practice is to select a set of test scenarios that conform to the actual situation, adjust the parameters of the test scenarios, and retain the value or a range of values ​​that have the best effect.

[0043] S2: Based on attitude increment and attitude estimation confidence, construct the cross-spectral stable structure correspondence of UAV targets in bispectral images and establish an attitude-structure response model.

[0044] It should be noted that even if different individuals of the same model of UAV have highly similar static appearances, the relative relationships, exposure / occlusion relationships, and local morphological changes between their observable structural regions under different attitudes can form attitude-conditional structural patterns with individual identification value. Therefore, this step independently extracts local structural key points from visible light and infrared images, without requiring that the local structural key points in the two spectral images be completely identical in pixel position, texture, or saliency response. Instead, cross-spectral correspondences between local structural key points in the two spectral images are established through subsequent attitude changes and consistency of local motion responses.

[0045] S2.1: Extract local structural regions from the bispectral image.

[0046] Furthermore, for the UAV target region obtained by the detector in frame t, the corresponding target region is extracted from the visible light image and the infrared image respectively. Using a set of local structural key points, the positions of each local structural key point are transformed to represent them in the target normalized coordinate system to reduce the impact of target scale changes and overall target translation on the structural position description. A fixed-size local structural region is extracted from both the visible light image and the infrared image, centered on each local structural key point.

[0047] S2.2: Construct short-time attitude response.

[0048] For adjacent frames t and t+1 within the short-time analysis window, based on the pose increment, respectively in the visible light image... and infrared images The image motion of the i-th local structural region is calculated using the following formula: ; in, This represents the image motion of the local structural region under spectrum m. Let be the position of the i-th local structural region in frame t and spectrum m. Let m be the position of the structural region in the (t+1)th frame and under spectrum m.

[0049] Furthermore, to reduce the impact of the overall target translation on the local motion response, the common translation component of the target is removed from the image motion, resulting in the relative motion of the local structural region mainly caused by the UAV attitude change. .

[0050] Based on this, the short-time attitude response is constructed for the i-th local structural region, and the specific formula is as follows: ; Among them, short-time attitude response Describes when a drone target undergoes a specific attitude increment When, the local motion response generated by the i-th local structural region under spectrum m.

[0051] S2.3: Define pose response consistency for the i-th local structure region in the visible light image and the candidate local structure region j in the infrared image. .

[0052] It should be noted that, since the imaging mechanisms of visible light images and infrared images are different, the same physical structure region may exhibit completely different textures or radiation responses under the two spectra. Therefore, this step does not directly compare the static appearance similarity of the two local regions, but rather compares their local motion responses under the same pose increment.

[0053] Specifically, attitude response consistency The specific formula is as follows: ; Where sim(·) is the similarity measurement function, which can be implemented by normalized correlation coefficient, cosine similarity or exponential mapping of Euclidean distance, etc. Visible light and Infrared regions represent the attitude increments of the i-th visible light local structure region and the j-th infrared candidate local structure region. Local relative motion below.

[0054] Furthermore, consider short-term analysis windows. Within multiple time frames, the attitude response consistency between the i-th visible light local structure region and the j-th infrared candidate local structure region is accumulated, and the attitude estimation confidence Cpose is used to weight the accumulated cross-spectral structure response consistency to obtain the weighted attitude response consistency within the short time window.

[0055] S2.4: For the i-th local structural region in the visible light image, select the region with the highest weighted attitude response consistency from the candidate local structural regions in the infrared image as the cross-spectral structural correspondence.

[0056] It should be noted that the region corresponding to the cross-spectral stable structure does not need to be predefined as a specific UAV component such as a motor, heat sink, or fuselage. It only needs to have a stable and consistent correspondence between visible light and infrared images.

[0057] S2.5: After determining the correspondence of transspectral stable structures, extract local structural features from visible light images and infrared images respectively.

[0058] Preferably, local structural features include local gradient direction histograms, edge direction distributions, contour descriptors, frequency domain amplitude features, or feature vectors obtained through feature extraction models.

[0059] S2.6: Jointly represent the local structural features with their corresponding attitude states and response information to obtain the set of attitude perception structural features of the UAV target under spectrum m, and establish the attitude-structure response model of the local structural region under the target attitude change condition through multiple short-time analysis windows.

[0060] Furthermore, based on the attitude condition structure observations obtained from multiple short-time analysis windows, a correspondence is established between the target attitude state and the local structural features, local motion response, and visibility of the stable structural region, forming an attitude-structure response model of the stable structural region under spectrum m. The attitude-structure response model is used to describe the appearance structure, local motion, and visibility changes of the stable structural region under the corresponding spectrum when the UAV is in different attitudes or undergoes different attitude changes.

[0061] It should be noted that the attitude-structure response model describes the appearance and local motion response changes of the stable structural region under different UAV attitudes and in a specific spectrum, thus forming a cross-spectral attitude response relationship.

[0062] S3: Based on the cross-spectral stable structure correspondence and attitude-structure response model, generate attitude-aware cross-spectral identity descriptors of UAV targets in bispectral images.

[0063] Specifically, based on the target attitude state obtained in the current short-term analysis window, the effective local structure observations under the current attitude conditions are determined from the attitude-structure response models corresponding to each stable structural region, and the local structural characteristics, local motion response, visibility or observation reliability of each stable structural region are obtained.

[0064] Furthermore, based on the confidence level of the current attitude estimation and the reliability of the structural region observation under the current attitude, the feature weights of each stable structural region are determined, and the local structural features and attitude response features of multiple stable structural regions are weighted and aggregated according to the feature weights to obtain the attitude-aware structural description of the target under spectrum m.

[0065] Finally, the attitude-aware structural description is jointly encoded with the target's global or local appearance features to form an attitude-aware cross-spectral identity descriptor under spectrum m. The identity descriptors under the visible and infrared spectra adopt a consistent structural region organization method according to the aforementioned cross-spectral stable structural correspondence, thereby enabling comparison of the identity information of corresponding structural regions under the two spectra.

[0066] S4: Calculate the cosine similarity between the UAV target in the visible light image and the UAV target in the infrared image based on the attitude-aware cross-spectral identity descriptor, and use bipartite graph matching methods such as the Hungarian algorithm to obtain the cross-spectral identity matching result, thus completing the cross-spectral re-identification of the UAV target.

[0067] S4.1: For UAV target A in the visible light image and UAV target B in the infrared image that need to be identified, obtain the attitude-aware cross-spectral identity descriptors corresponding to them respectively, map the two attitude-aware cross-spectral identity descriptors to a unified identity representation space, and calculate the similarity between the two attitude-aware cross-spectral identity descriptors after mapping to obtain the cross-spectral identity matching score.

[0068] In an optional embodiment, during the calculation of the cross-spectral identity matching score, the contribution weight of the attitude-aware cross-spectral identity descriptor in the cross-spectral identity matching score is adjusted based on the attitude estimation confidence Cpose: when the attitude estimation confidence Cpose is high, the contribution weight of the attitude-aware cross-spectral identity descriptor to the cross-spectral identity matching score is increased; when the attitude estimation confidence Cpose is low, the contribution weight of the attitude-aware cross-spectral identity descriptor to the cross-spectral identity matching score is decreased.

[0069] S4.2: Based on the comprehensive identity matching score, determine whether the UAV target in the visible light image and the UAV target in the infrared image belong to the same identity, output the cross-spectral identity matching result, and complete the cross-spectral re-identification of UAV targets.

[0070] Example 2

[0071] Based on the teachings of the above embodiments, other aspects of the present invention also propose a cross-spectral re-identification system for unmanned aerial vehicles (UAVs) based on short-time kinematics estimation and attitude response modeling, such as... Figure 2 As shown, it includes: The short-time kinematics estimation module is used to perform tracking-independent short-time kinematic state estimation on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images. The attitude response modeling module constructs the cross-spectral stable structural correspondence of the UAV target in the dual-spectral image based on attitude increment and attitude estimation confidence, and establishes an attitude-structure response model. The identity descriptor generation module generates attitude-aware cross-spectral identity descriptors for UAV targets in bispectral images based on cross-spectral stable structural correspondences and attitude-structure response models. The cross-spectral identity matching module is used to calculate the similarity between UAV targets in visible light images and UAV targets in infrared images based on attitude-aware cross-spectral identity descriptors, obtain cross-spectral identity matching results, and complete cross-spectral re-identification of UAV targets.

[0072] Specifically, when multiple UAV targets exist simultaneously in the monitoring area, and some targets overlap or occlude each other during flight, the computing device independently triggers a short-time kinematics estimation window for each UAV target: For UAV targets whose detection boxes do not overlap, local structural key points extraction and attitude increment solution are performed according to steps S1.1 to S1.6 of Example 1; For UAV targets whose detection boxes partially overlap, local structural key points that fall into the overlapping area of ​​two detection boxes are marked as low reliability points, and only local structural key points located within the effective area of ​​the current detection box and satisfying local spatial constraints are used to participate in the attitude increment solution; For UAV targets that are partially occluded, only local structural key points in the unoccluded area are used to solve the attitude increment; For UAV targets that are completely occluded, the attitude increment update of the current short-time analysis window is stopped, and the short-time kinematics estimation window is re-triggered based on the new detection results after the UAV target is re-detected by the detector.

[0073] In the above processing, the short-term kinematic state estimation of each UAV target is independent of each other, without depending on the identity state of other UAV targets or their respective long-term tracking trajectories. Based on this, steps S2 to S4 of Example 1 are executed for each UAV target to obtain the attitude perception cross-spectral identity descriptor of each UAV target in visible light and infrared images, and the cross-spectral identity matching score is calculated to complete the cross-spectral target identity association in multi-UAV intersection scenarios.

[0074] In summary, by performing short-time kinematic state estimation on simultaneously acquired visible light and infrared images without tracking dependence, the attitude increment and attitude estimation confidence of the UAV target are obtained, avoiding the dependence of identity re-identification on long-term target tracking. Based on the attitude increment, short-time attitude responses of local structural regions in the visible light and infrared images are constructed, and a cross-spectral stable structural correspondence is established through the consistency of attitude responses, thus forming an attitude-structure response model. The local structural features corresponding to the cross-spectral stable structural correspondence are jointly represented with the attitude state and short-time attitude response to generate attitude-aware cross-spectral identity descriptors. The contribution weight of the attitude-aware cross-spectral identity descriptors in the identity matching process is adjusted based on the attitude estimation confidence, which improves the stability of cross-spectral identity association of the same type of UAV in complex maneuvering flight and multi-target intersection scenarios.

[0075] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A cross-spectral re-identification method for unmanned aerial vehicles (UAVs) based on short-time kinematics estimation and attitude response modeling, characterized in that, include: Short-time kinematic state estimation without tracking dependence was performed on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images. Based on the attitude increment and the attitude estimation confidence, the cross-spectral stable structure correspondence of the UAV target in the bispectral image is constructed, and an attitude-structure response model is established. Based on the cross-spectral stable structure correspondence and the attitude-structure response model, an attitude-aware cross-spectral identity descriptor for the UAV target in the bispectral image is generated. The cosine similarity between the UAV target in the visible light image and the UAV target in the infrared image is calculated based on the attitude-aware cross-spectral identity descriptor. The cross-spectral identity matching result is obtained using bipartite graph matching methods such as the Hungarian algorithm, thus completing the cross-spectral re-identification of the UAV target.

2. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 1, characterized in that, The acquisition of the attitude estimation confidence includes: The structural distribution index is calculated based on the spatial covariance or coverage of the local structural key points involved in the attitude increment solution within the local candidate region. The attitude observability index is calculated based on the ratio of the minimum singular value to the maximum singular value of the attitude observation matrix or its condition number. The model fitting index is calculated based on the residual between the structural point displacement obtained by reprojection of the attitude increment estimation results and the actual observed displacement. The attitude estimation confidence is obtained by weighted combination or product of the indices.

3. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 2, characterized in that, The acquisition of the attitude increment includes: The detection of a drone target by the target detector in frame t is taken as the starting frame of the short-term window. N frames of images are continuously acquired after frame t to form the short-term analysis window. Local candidate regions are established inside the detection box based on the center of the detection box. The ORB feature point detection method is used to extract local structural key points within the local candidate region, and the set of local structural key points is obtained by filtering according to the feature response intensity. ; For adjacent frames t and t+1 within the short-time analysis window, local matching is performed on each local structural key point in the set of local structural key points within the local candidate region to obtain the two-dimensional displacement of the i-th local structural key point. ; Two-dimensional displacement based on multiple key points of local structure after screening The median statistical method is used to estimate the common translational components of the UAV target in the image plane, and the two-dimensional displacement of each local structural key point is calculated. After removing the common translation component, the structural relative displacement, which characterizes the relative motion of the internal structure of the UAV target, is obtained. ; The local kinematic mapping relationships corresponding to multiple local structural key points in the set of local structural key points are combined to obtain an overdetermined set of equations, and the attitude increment is solved by weighted least squares method.

4. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 3, characterized in that, The two-dimensional displacement The specific formula is as follows: ； in, The displacement of the i-th structural point along the x-axis of the image at frame t is given by... The displacement of the i-th structural point along the y-axis of the image at frame t.

5. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 3, characterized in that, The specific formulas for the overdetermined system of equations are as follows: ； in, Let represent the local kinematic relationship between the local relative displacement of the M-th structure point in frame t and the attitude change of the UAV. Let be the local projection sensitivity matrix of the structure point in frame t with respect to the three-axis attitude change. Let be the image measurement error and model error of the structural point in frame t.

6. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 1, characterized in that, The establishment of the attitude-structure response model includes: After determining the correspondence of cross-spectral stable structures, local structural features were extracted from visible light and infrared images, respectively. The local structural features are jointly represented with their corresponding attitude states and response information to obtain the set of attitude perception structural features of the UAV target under spectrum m. After passing through multiple short-time analysis windows, an attitude-structure response model of the local structural region under target attitude change conditions is established.

7. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 6, characterized in that, The construction of the cross-spectral structure correspondence includes: For adjacent frames t and t+1 within the short-time analysis window, based on the pose increment, respectively in the visible light image... and infrared images Calculate the image motion of the i-th local structural region; For the i-th local structure region in the visible light image and the candidate local structure region j in the infrared image, the pose response consistency is defined. ; For the i-th local structural region in the visible light image, the region with the highest weighted attitude response consistency is selected from the candidate local structural regions in the infrared image as the cross-spectral structural correspondence.

8. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 7, characterized in that, The local structural region includes: For the UAV target region obtained by the detector in frame t, the corresponding target regions are extracted from the visible light image and the infrared image respectively, and the positions of each local structural key point are transformed into the target normalized coordinate system using the set of local structural key points. Using each key point of the local structure as the center, a fixed-size local structural region is cropped from both the visible light image and the infrared image.

9. The UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in claim 8, characterized in that, The image motion is specifically formulated as follows: ； in, This represents the image motion of the local structural region under spectrum m. Let be the position of the i-th local structural region in frame t and spectrum m. Let m be the position of this structural region in frame t+1 and spectrum m. For light images, This is an infrared image.

10. A UAV cross-spectral re-identification system based on short-time kinematics estimation and attitude response modeling, based on the UAV cross-spectral re-identification method based on short-time kinematics estimation and attitude response modeling as described in any one of claims 1 to 9, characterized in that, include: The short-time kinematics estimation module is used to perform tracking-independent short-time kinematic state estimation on the synchronously acquired bispectral images to obtain the attitude increment and attitude estimation confidence of the UAV target within a short time window. The bispectral images include visible light images and infrared images. The attitude response modeling module constructs the cross-spectral stable structure correspondence of the UAV target in the bispectral image based on the attitude increment and the attitude estimation confidence, and establishes an attitude-structure response model. The identity descriptor generation module generates an attitude-aware cross-spectral identity descriptor for the UAV target in the bispectral image based on the cross-spectral stable structure correspondence and the attitude-structure response model. The cross-spectral identity matching module is used to calculate the cosine similarity between the UAV target in the visible light image and the UAV target in the infrared image based on the attitude-aware cross-spectral identity descriptor, and use bipartite graph matching methods such as the Hungarian algorithm to obtain the cross-spectral identity matching result, thereby completing the cross-spectral re-identification of the UAV target.

Images