An unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification

Abstract

The application discloses a kind of unmanned plane autonomous reconnaissance load control method, device and equipment integrated target detection identification, by using the dynamic mapping mode of pre-constructed height-Field of View angle controls unmanned plane to execute wide-area investigation mode;When the target priority parameter of dispatch queue is greater than the first preset value, trigger detailed investigation state;Flight height is input into the second formula, and the current zoom ratio is obtained, and the three-dimensional characteristics of multiple targets are collected under the current zoom ratio using multiple sensors, based on the preset multi-modal feature fusion network processing each target three-dimensional characteristics, to obtain comprehensive verification score, the target of comprehensive verification score reaching second preset value is regarded as detailed investigation target;Incremental target list is constructed to update detailed investigation target;When unconfirmed target proportion is less than third preset value and new target rate is less than third preset value, detailed investigation state is switched back to wide-area investigation mode using intelligent state machine, and the application can realize reconnaissance mode dynamic switching.

Description

Technical Field

[0001] This application relates to the field of target detection, and in particular to a method, apparatus and equipment for controlling autonomous reconnaissance payloads of unmanned aerial vehicles that incorporates target detection and recognition. Background Technology

[0002] Currently, the field of UAV optoelectronic reconnaissance faces three major technical bottlenecks: First, insufficient field-of-view adaptation capability. For example, existing technologies employ fixed optical zoom magnification, requiring manual intervention to adjust the field of view when flight altitude changes, resulting in a target tracking loss rate exceeding 35%. Second, inefficient reconnaissance mode switching. According to measured data from "Advances in UAV Optoelectronic Reconnaissance Technology," switching from wide-area reconnaissance to detailed target investigation relies on ground station commands, with an average delay of 8-12 seconds. Third, a lack of situational awareness maintenance mechanisms. The target list lacks real-time dynamic updates, causing intelligence timeliness to decay to below 60% within 10 minutes of generation. These deficiencies severely restrict the effectiveness of autonomous UAV reconnaissance and urgently require technological innovation to resolve. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device and equipment for controlling the autonomous reconnaissance payload of UAVs that integrates target detection and recognition. It aims to break through the technical bottleneck of existing UAV optoelectronic reconnaissance systems, construct a four-stage closed-loop workflow of "wide-area scanning - intelligent detection - precise detailed investigation - situation update", and realize dynamic switching of reconnaissance mode through an autonomous decision engine.

[0004] To achieve the above objectives, this application provides a method for controlling an autonomous reconnaissance payload of a UAV that integrates target detection and recognition. The method includes: controlling the UAV to execute a wide-area reconnaissance mode using a pre-constructed altitude-field-of-view dynamic mapping pattern, wherein the parameters of the UAV in the wide-area reconnaissance mode include the optimal field-of-view of the sensor, the instantaneous field-of-view of the payload gimbal's automatic zoom, and the corrected small target detection confidence level; processing the corrected small target detection confidence level using a spatiotemporal joint priority algorithm to obtain a scheduling queue, and triggering a detailed investigation state when the target priority parameter in the scheduling queue is greater than a first preset value; wherein the spatiotemporal joint priority algorithm is determined based on the product of the corrected detection confidence level and the timeliness coefficient, and the target priority parameter in the scheduling queue is determined based on the corrected detection confidence level and the timeliness coefficient. The product of confidence level and exponential decay factor determines the target priority parameter; the flight altitude is input into a preset second formula to obtain the current zoom ratio, and multiple sensors are used to collect the three-dimensional features of multiple targets at the current zoom ratio. The three-dimensional features of each target are processed by a preset multimodal feature fusion network to obtain a comprehensive verification score. Targets with a comprehensive verification score that reaches a second preset value are designated as targets for detailed investigation; an incremental target list is constructed using ProtoBuf binary encoding combined with an LRU update strategy to update the targets for detailed investigation; when the proportion of unconfirmed targets is less than a third preset value and the rate of new targets is less than a third preset value, an intelligent state machine is used to switch the detailed investigation state back to wide-area reconnaissance mode.

[0005] Optionally, the step of controlling the UAV to perform wide-area reconnaissance mode using a pre-constructed altitude-field-of-view dynamic mapping model includes: processing the acquired flight altitude data using a field-of-view calculation expression to obtain the optimal field-of-view; controlling the photoelectric payload gimbal to automatically zoom so that the instantaneous field-of-view is not greater than the optimal field-of-view, and correcting the confidence of small target detection through a confidence compensation expression, wherein the confidence of small target detection is determined by the YOLO model.

[0006] Optionally, the expression for calculating the field of view is:

[0007] in, Indicates the minimum size of the target. This represents the sensor resolution coefficient of the drone. H Indicates the drone's flight altitude; The expression for the corrected small target detection confidence score for each small target is:

[0008] in, This represents the corrected confidence level for small target detection. Indicates the basic detection confidence level. Indicates highly adjusted weights, This represents the attenuation coefficient.

[0009] Optionally, the data structure of the scheduling queue adopts a complete binary tree structure.

[0010] Optionally, the timeliness coefficient is the reciprocal of the difference between the current time of small target detection and the last detection time.

[0011] Optionally, it also includes: obtaining a subtrahend term based on the difference between the current number of detailed investigation tasks and the maximum concurrent detailed investigation capacity; obtaining a first multiplier term based on 1 minus the subtrahend term; determining a second multiplier term based on the negative value of the detected target's unupdated time and the time-decrease constant; determining an exponent term based on the product of the first multiplier term and the second multiplier term; and determining an exponential decay factor based on the exponent term and using natural numbers as the base.

[0012] Optionally, the multimodal feature fusion network includes: a feature extraction layer, used to process visible light images using a ResNet-50 backbone network and feature pyramid to obtain visible light feature vectors, and to process infrared images using a CNN backbone network and thermal radiation normalization algorithm to obtain thermal features; a cross-modal attention fusion layer, used to interact thermal features and multi-scale morphological features in the cross-attention fusion layer, using the visible light feature vector as the query vector to weight the key-value pairs of the infrared light feature vector to obtain illumination-adaptive fusion features; a feature enhancement layer, used to reduce the dimensionality of the illumination-adaptive fusion features and then aggregate features in the spatial-temporal dimensions through 3D convolution to obtain aggregated features, and then use the GRU gating mechanism to process bimodal features to obtain motion trajectory continuity features; and a fully connected classification layer, using a three-head fully connected layer to process motion trajectory continuity features and aggregated features to obtain morphological matching scores. Optionally, the morphological matching score includes visible light morphological matching similarity score, motion pattern matching score, and thermal radiation feature confidence score; after processing the motion trajectory continuity feature and aggregation feature using a three-head fully connected layer to obtain the morphological matching score, the method further includes: calculating the comprehensive verification score of each target using the weighted visible light morphological matching similarity score, motion pattern matching score, and thermal radiation feature confidence score.

[0013] To achieve the above objectives, this application also provides a UAV autonomous reconnaissance payload control device integrating target detection and recognition, comprising: a wide-area scanning module, used to control the UAV to execute a wide-area reconnaissance mode using a pre-constructed altitude-field-of-view dynamic mapping mode, wherein the parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal auto-zoom, and the corrected small target detection confidence; a detailed investigation task determination module, used to process the corrected small target detection confidence using a spatiotemporal joint priority algorithm to obtain a scheduling queue, and trigger a detailed investigation state when the target priority parameter of the scheduling queue is greater than a first preset value; wherein, the spatiotemporal joint priority algorithm is determined based on the product of the corrected detection confidence and the timeliness coefficient, and the timeliness coefficient is determined based on the product of the corrected detection confidence and the timeliness coefficient. The product of exponential decay factors determines the target priority parameter; the detailed investigation execution module is used to input the flight altitude into a preset second formula to obtain the current zoom ratio, and at the current zoom ratio, multiple sensors are used to collect the three-dimensional features of multiple targets respectively. The three-dimensional features of each target are processed by a preset multimodal feature fusion network to obtain a comprehensive verification score. Targets with comprehensive verification scores reaching a second preset value are designated as detailed investigation targets; the detailed investigation target update module is used to construct an incremental target list using ProtoBuf binary encoding combined with an LRU update strategy to update the detailed investigation targets; the detailed investigation exit module is used to switch the detailed investigation state back to wide-area reconnaissance mode using an intelligent state machine when the proportion of unconfirmed targets is less than a third preset value and the rate of new targets is less than a third preset value.

[0014] To achieve the above objectives, this application also provides a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the UAV autonomous reconnaissance payload control method incorporating target detection and recognition provided in the above embodiments.

[0015] To achieve the above objectives, this application also provides an electronic device, which includes: at least one processor, a memory, and an input / output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the UAV autonomous reconnaissance payload control method integrating target detection and recognition provided in any of the foregoing embodiments.

[0016] This application proposes a method, apparatus, and device for controlling an autonomous reconnaissance payload of a UAV that integrates target detection and recognition. By utilizing a pre-constructed altitude-field-of-view dynamic mapping mode, the method controls the UAV to execute a wide-area reconnaissance mode, addressing the issue of target loss rate exceeding 35% due to altitude changes. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal's automatic zoom, and the corrected small target detection confidence. A spatiotemporal joint priority algorithm is used to process the corrected small target detection confidence, resulting in a scheduling queue. A detailed investigation state is triggered when the target priority parameter in the scheduling queue exceeds a first preset value. Specifically, the spatiotemporal joint priority algorithm is determined based on the product of the corrected detection confidence and the timeliness coefficient, and the target priority parameter is determined based on the product of the corrected detection confidence and the exponential decay factor. The flight altitude is input into a preset second formula to obtain the current zoom magnification. At the current zoom magnification, multiple sensors are used to collect the three-dimensional features of multiple targets. The three-dimensional features of each target are processed by a preset multimodal feature fusion network to obtain a comprehensive verification score. Targets with a comprehensive verification score that reaches a second preset value are designated as targets for detailed investigation. An incremental target list is constructed using ProtoBuf binary encoding combined with an LRU update strategy to update the targets for detailed investigation, ensuring that the intelligence timeliness remains >90% within 60 minutes of continuous operation, completely solving the defect of timeliness decay under the traditional list maintenance mechanism. When the proportion of unconfirmed targets is less than a third preset value and the rate of new targets is less than a third preset value, the intelligent state machine is used to switch the detailed investigation state back to the wide-area reconnaissance mode, realizing a seamless switch between wide-area reconnaissance and target detailed investigation modes. Attached Figure Description

[0017] Figure 1 A flowchart is provided for an embodiment of the UAV autonomous reconnaissance payload control method incorporating target detection and recognition in this application; Figure 2 This is a system block diagram provided for an embodiment of the UAV autonomous reconnaissance payload control method incorporating target detection and recognition, as described in this application. Figure 3 This application provides a mode switching process based on a finite state machine (FSM) for an embodiment of the UAV autonomous reconnaissance payload control method incorporating target detection and recognition. Figure 4 This application provides a target state transition diagram for an embodiment of the UAV autonomous reconnaissance payload control method incorporating target detection and recognition. Figure 5 This application provides a dual-modal feature fusion network structure for an embodiment of the UAV autonomous reconnaissance payload control method incorporating target detection and recognition.

[0018] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0020] Figure 2 This application provides a system block diagram of an embodiment of an UAV autonomous reconnaissance payload control method incorporating target detection and recognition, with reference to... Figure 2 The system's UAVs acquire their flight altitude in real time via altitude sensors and calculate the optimal field of view at the current altitude based on the altitude-field-of-view dynamic mapping model within the intelligent mission control system. This ensures that the ground target imaging size is ≥15 pixels to meet the recognition requirements of the intelligent mission control system. The intelligent mission control system autonomously generates focusing commands and sends them to the optoelectronic payload, driving the visible light camera or infrared thermal imager to complete the field of view adjustment. The adjusted optoelectronic payload transmits a reconnaissance video stream back to the UAV. The intelligent mission control system performs target detection and recognition on the video frames, outputting the target's three-dimensional coordinates (longitude, latitude, and elevation), confidence level, and detection timestamp (millisecond-level accuracy). When the intelligent mission control system determines that the target priority is >0.8, it automatically generates a target list and transmits it to the ground control station via a wireless data link. After receiving the list, the ground control station performs parsing and extracts structured intelligence fields: target number, target type, target location, confidence level, and status marker.

[0021] This invention aims to overcome the technical bottlenecks of existing UAV optoelectronic reconnaissance systems, construct a four-stage closed-loop workflow of "wide-area scanning - intelligent detection - precise detailed investigation - situation update", and achieve dynamic switching of reconnaissance modes through an autonomous decision engine. The following three innovative mechanisms achieve intelligent upgrades: ① Constructing a dynamic mapping model of altitude and field of view, dynamically calculating the optimal field of view based on real-time flight altitude (θ=2×arctan(W / (2H×k))), solving the problem of target loss rate >35% caused by altitude changes, and achieving autonomous and precise adaptation of the payload's field of view; ② Establishing a closed-loop decision mechanism for the entire process of "detection-identification-update", using an intelligent state machine to achieve seamless switching between wide-area reconnaissance and target detailed investigation modes (switching delay <1 second), eliminating the 8-12 second operation delay caused by ground station intervention in traditional solutions; ③ Designing an incremental target list coding protocol, using ProtoBuf binary coding combined with an LRU update strategy to ensure that intelligence timeliness is maintained >90% within 60 minutes of continuous operation, completely solving the defects of timeliness decay under the traditional list maintenance mechanism.

[0022] Reference Figure 1 , Figure 1This is a flowchart of a UAV autonomous reconnaissance payload control method incorporating target detection and recognition, provided in the first embodiment of this application. This application is applied to an intelligent mission control system. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition may include the following execution process: S10. Use a pre-built altitude-field-of-view dynamic mapping model to control the UAV to execute a wide-area reconnaissance mode. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. In one embodiment of this application, the process of controlling a UAV to perform a wide-area reconnaissance mode using a pre-constructed altitude-field-of-view dynamic mapping model may include the following: S101. Process the acquired flight altitude data using the field of view angle calculation expression to obtain the optimal field of view angle; S102. Control the automatic zoom of the photoelectric payload gimbal to ensure that the instantaneous field of view is not greater than the optimal field of view, and correct the confidence of small target detection through a confidence compensation expression, wherein the confidence of small target detection is determined by the YOLO model.

[0023] Specifically, the processor can obtain the drone's flight altitude in real time. (Unit: m), Dynamically calculate the optimal field of view. Controlling the automatic zoom of the photoelectric payload gimbal to adjust the instantaneous field of view. To ensure ground target imaging Pixel.

[0024] The expression for calculating the field of view angle is as follows:

[0025] In the above formula, Indicates the minimum size of the target (unit: m, default value 0.3, configurable). This indicates the sensor resolution coefficient of the drone ( ), H Indicates the drone's flight altitude; The expression for the corrected small target detection confidence score for each small target is:

[0026] in, This represents the corrected confidence level for small target detection. This represents the basic detection confidence level (output of models such as YOLO). This indicates a highly adjusted weight (empirical value 0.15). This represents the attenuation coefficient (empirical value 0.005).

[0027] Clearly, by dynamically calculating the optimal field of view based on real-time flight altitude, the problem of target loss rate exceeding 35% caused by altitude changes can be solved, enabling autonomous and precise adaptation of the payload's field of view.

[0028] S20. The spatiotemporal joint priority algorithm is used to process the corrected small target detection confidence, and a scheduling queue is obtained. When the target priority parameter of the scheduling queue is greater than the first preset value, a detailed investigation state is triggered. The spatiotemporal joint priority algorithm is determined based on the product of the modified detection confidence and the timeliness coefficient, and the target priority parameter is determined based on the product of the modified detection confidence and the exponential decay factor. The data structure of the scheduling queue adopts a complete binary tree structure, which is a min-heap structure.

[0029] Specifically, the processor schedules detailed query tasks based on a priority queue, where the priority calculation formula for the priority queue scheduling is as follows: Priority = Target confidence level × Timeliness coefficient.

[0030] Wherein, the timeliness coefficient is the reciprocal of the difference between the current time of small target detection and the last detection time; The priority parameter formula is as follows:

[0031] in, This indicates the time that target i has not been updated, in seconds; This indicates that the time-decrease constant is adaptive based on the scenario, with a default value of 300 seconds. Indicates the current number of detailed investigation tasks; This indicates the maximum concurrent detailed query capability; (determined by hardware performance).

[0032] Based on this, this application employs a spatiotemporal joint priority algorithm to schedule detailed target searches, and the scheduling queue is sorted according to the calculated target priority parameters. And dynamically generate and update the target list when When the target is triggered, a detailed investigation process is initiated, and the target is added to the detailed investigation queue. The queue scheduling adopts a min-heap structure.

[0033] After determining the scheduling queue, the next step for the processor is to control the drone to perform zoom control and target detailing.

[0034] S30. Input the flight altitude into the preset second formula to obtain the current zoom ratio, and use multiple sensors to collect the three-dimensional features of multiple targets at the current zoom ratio. The preset multimodal feature fusion network processes the three-dimensional features of each target to obtain a comprehensive verification score. The target whose comprehensive verification score reaches the second preset value is taken as the target for detailed investigation. Specifically, the drone can perform multiple orbital flights, and adjust its altitude accordingly. Automatically matches zoom magnification, acquires target 3D features, and maintains a constant target imaging size. (5% error).

[0035]

[0036] in, Reference zoom magnification (in) (Height measured) Current flight altitude; The initial detection height is for the target.

[0037] Next, the processor processes the three-dimensional features of each target based on a preset multimodal feature fusion network to obtain a comprehensive verification score. The multimodal feature fusion network may include: The feature extraction layer is used to process visible light images using the ResNet-50 backbone network and feature pyramid to obtain multi-scale morphological features, and to process infrared images using the CNN backbone network and thermal radiation normalization algorithm to obtain thermal features. The cross-modal attention fusion layer is used to interact thermal features and multi-scale morphological features in the cross-attention fusion layer. The key-value pairs of the infrared light feature vector are weighted using the visible light feature vector as the query vector to obtain the illumination adaptive fusion features. The feature enhancement layer is used to reduce the dimensionality of the illumination adaptive fusion features and then aggregate them in the spatial-temporal dimensions through 3D convolution to obtain aggregated features. Then, the GRU gating mechanism is used to process the dual-modal features to obtain the motion trajectory continuity features. The fully connected classification layer uses a three-head fully connected layer to process the continuity and aggregation features of the motion trajectory to obtain the morphological matching score.

[0038] refer to Figure 5It is worth noting that the dual-modal feature fusion network structure is a core module of the target detailed investigation system, solving the target confirmation problem in complex environments. After receiving the image input, the visible light branch first performs dynamic data augmentation (rotation ±15°, brightness / contrast ±20% adjustment), then extracts hierarchical features through the ResNet50 backbone network, and outputs multi-scale morphological features via the Feature Pyramid (FPN). The infrared branch performs thermal radiation value normalization (ambient temperature compensation) on the input image, and extracts thermal features using a lightweight CNN network (5 layers of depthwise separable convolutions). The dual-modal features interact at the cross-attention fusion layer: using visible light features as the query vector, infrared feature key-value pairs are dynamically weighted to achieve illumination-adaptive fusion. The fused features are then dimensionality-reduced by 1×1 convolution and input into the spatiotemporal joint optimization module—aggregating features in the spatial-temporal dimensions through 3D convolution, and then modeling the continuity of motion trajectories through a GRU gating mechanism. Finally, a three-head fully connected layer outputs a morphological matching score (dominated by visible light) and a thermal feature score (dominated by infrared) in parallel. These scores are then combined with an external motion analysis score to generate a weighted comprehensive verification score (Ver_score), which drives the target state transition decision. This architecture achieves end-to-end inference in 65ms through hardware acceleration (Jetson AGX Orin), improving the camouflage target recognition rate from 67% to 92.7%.

[0039] Specifically, refer to Figure 5 S40. An incremental target list is constructed using ProtoBuf binary encoding combined with an LRU update strategy to update the targets for detailed investigation; S50. When the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value, the intelligent state machine is used to switch the detailed investigation state back to the wide-area reconnaissance mode.

[0040] refer to Figure 3 In the diagram, the wide-area reconnaissance state is used to dynamically adjust the field of view of the optoelectronic payload, ensuring clear target imaging through high-frequency focusing control and infrared assistance, and outputting the target's preliminary three-dimensional coordinates and compensated confidence level; the target detailed investigation state initiates multi-circle flight for high-priority targets (priority > 0.8), adaptively adjusts the zoom ratio, and integrates three-modal verification of visible light morphology matching, motion trajectory analysis, and thermal radiation characteristics. Targets that achieve the required score are marked as confirmed targets; the emergency state is activated when the target trajectory deviation exceeds the limit or multiple consecutive frames are lost, immediately terminating the current mission and restarting wide-area scanning, while simultaneously sending an anomaly alarm to the ground station; state transition logic: The system uses an intelligent state machine to dynamically switch between wide-area reconnaissance, detailed target investigation, and emergency response modes. Specifically, the state change from wide-area to detailed investigation occurs when the UAV detects a high-priority target and the detailed investigation resources are available. Detailed investigation → Wide-area status changes are returned when the proportion of unconfirmed targets is less than 15% and the rate of new targets is less than the threshold; The state change from arbitrary state to emergency is a forced transition when trajectory prediction is abnormal or communication interruption times out, and resets to the wide-area state after the abnormality is resolved.

[0041] refer to Figure 4 The target starts in an "uncertain" state. If it scores more than 0.9 in the multimodal verification, it is promoted to a "determined target". If an uncertain target is not detected for 5 consecutive frames (0.5 seconds), it is automatically downgraded to an "invalid target". If a determined target does not update its information within 120 seconds, it is also converted to an invalid target.

[0042] This process forms a closed-loop state transition path, specifically including: a promotion path for information reinforcement: uncertain target → detailed verification (score > 0.9) → confirmed target (confidence level forcibly increased to 0.95). Failure paths are used for information attenuation: Rapid failure: Uncertain target → 5 consecutive frames lost → Invalid target (relevant resources released immediately) Timeout Failure: Target identified → No update after 120 seconds → Invalid target (send failure notification) In one embodiment of this application, the specific execution process of step SXX can be as follows: Based on the above embodiments, this application also provides an autonomous reconnaissance payload control device for unmanned aerial vehicles (UAVs) that integrates target detection and recognition, characterized in that it includes: The wide-area scanning module is used to control the UAV to perform a wide-area reconnaissance mode using a pre-built altitude-field-of-view dynamic mapping mode. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. The detailed investigation task determination module is used to process the corrected small target detection confidence using a spatiotemporal joint priority algorithm, obtain a scheduling queue, and trigger the detailed investigation state when the target priority parameter in the scheduling queue is greater than the first preset value. Among them, the spatiotemporal joint priority algorithm is determined based on the product of the modified detection confidence and the timeliness coefficient, and the target priority parameter is determined based on the product of the modified detection confidence and the exponential decay factor. The detailed investigation execution module is used to input the flight altitude into a preset second formula to obtain the current zoom ratio, and to use multiple sensors to collect the three-dimensional features of multiple targets at the current zoom ratio. The three-dimensional features of each target are processed by a preset multimodal feature fusion network to obtain a comprehensive verification score. The target whose comprehensive verification score reaches the second preset value is used as the detailed investigation target. The detailed target update module is used to construct an incremental target list by combining ProtoBuf binary encoding with an LRU update strategy, so as to update the detailed targets. The detailed investigation exit module is used to switch the detailed investigation state back to wide-area reconnaissance mode using an intelligent state machine when the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value.

[0043] A second embodiment of this application also provides an electronic device, characterized in that the electronic device includes: at least one processor, a memory, and an input / output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the aforementioned UAV autonomous reconnaissance payload control method incorporating target detection and recognition.

[0044] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for controlling the autonomous reconnaissance payload of an unmanned aerial vehicle (UAV) integrating target detection and recognition, characterized in that, include: The UAV is controlled to perform a wide-area reconnaissance mode by using a pre-built altitude-field-of-view dynamic mapping mode. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. The spatiotemporal joint priority algorithm is used to process the corrected small target detection confidence, and a scheduling queue is obtained. When the target priority parameter in the scheduling queue is greater than the first preset value, a detailed investigation state is triggered. The spatiotemporal joint priority algorithm includes: using the product of the corrected small target detection confidence and the timeliness coefficient as the spatiotemporal joint priority, and using the product of the corrected small target detection confidence and the exponential decay factor as the target priority parameter; the timeliness coefficient is related to the time difference of target detection, and the exponential decay factor is related to the target not updated time and the current number of detailed investigation tasks; The flight altitude is input into the preset second formula to obtain the current zoom ratio. At the current zoom ratio, multiple sensors are used to collect the three-dimensional features of multiple targets. The three-dimensional features of each target are processed based on the preset multimodal feature fusion network to obtain a comprehensive verification score. Targets whose comprehensive verification scores reach the second preset value are used as targets for detailed investigation. An incremental target list is constructed using ProtoBuf binary encoding combined with an LRU update strategy to update the targets for detailed investigation; When the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value, the intelligent state machine is used to switch the detailed investigation mode back to the wide-area reconnaissance mode.

2. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The method of controlling the UAV to perform wide-area reconnaissance using a pre-constructed altitude-field-of-view dynamic mapping mode includes: The acquired flight altitude data is processed using the field of view calculation expression to obtain the optimal field of view. The gimbal for controlling the photoelectric payload automatically zooms to ensure that the instantaneous field of view is no greater than the optimal field of view, and the confidence of small target detection is corrected by a confidence compensation expression, wherein the confidence of small target detection is determined by the YOLO model.

3. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 2, characterized in that, The expression for calculating the field of view is: in, Indicates the minimum size of the target. This represents the sensor resolution coefficient of the drone. H Indicates the drone's flight altitude; The expression for the corrected small target detection confidence score for each small target is: in, This represents the corrected confidence level for small target detection. Indicates the basic detection confidence level. Indicates highly adjusted weights, This represents the attenuation coefficient.

4. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The data structure of the scheduling queue adopts a complete binary tree structure.

5. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The timeliness coefficient is the reciprocal of the difference between the current time of small target detection and the last detection time.

6. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, Also includes: The subtrahend term is obtained based on the difference between the current number of detailed investigation tasks and the maximum concurrent detailed investigation capacity; The first multiplicative term is obtained by subtracting the subtrahend term from 1. The second multiplier term is determined based on the target's unupdated time and the negative value of the time decay constant. Determine the exponent term based on the product of the first and second multiplier terms; The exponential decay factor is determined based on the natural number base and the exponential term.

7. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The multimodal feature fusion network includes: The feature extraction layer is used to process visible light images using the ResNet-50 backbone network and feature pyramid to obtain visible light feature vectors, and to process infrared images using the CNN backbone network and thermal radiation normalization algorithm to obtain thermal features. The cross-modal attention fusion layer is used to interact thermal features and multi-scale morphological features in the cross-attention fusion layer. The key-value pairs of the infrared light feature vector are weighted using the visible light feature vector as the query vector to obtain the illumination adaptive fusion features. The spatiotemporal feature enhancement module is used to reduce the dimensionality of the illumination adaptive fusion features and then aggregate them in the spatial-temporal dimensions through 3D convolution to obtain aggregated features; then, the GRU gating mechanism is used to process the dual-modal features to obtain the motion trajectory continuity features. The fully connected classification layer uses a three-head fully connected layer to process the continuity and aggregation features of the motion trajectory to obtain the morphological matching score.

8. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 7, characterized in that, The morphological matching score includes visible light morphological matching similarity score, motion pattern matching score, and thermal radiation feature confidence score. After processing the continuity and aggregation features of the motion trajectory using a three-head fully connected layer to obtain the morphological matching score, the method further includes: The comprehensive verification score of each target is calculated using the weighted visible light morphology matching similarity score, motion pattern matching score, and thermal radiation feature confidence score.

9. A drone autonomous reconnaissance payload control device integrating target detection and recognition, characterized in that, include: The wide-area scanning module is used to control the UAV to perform a wide-area reconnaissance mode using a pre-built altitude-field-of-view dynamic mapping mode. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. The detailed investigation task determination module is used to process the corrected small target detection confidence using a spatiotemporal joint priority algorithm, obtain a scheduling queue, and trigger the detailed investigation state when the target priority parameter in the scheduling queue is greater than the first preset value. The spatiotemporal joint priority algorithm includes: using the product of the corrected small target detection confidence and the timeliness coefficient as the spatiotemporal joint priority, and using the product of the corrected small target detection confidence and the exponential decay factor as the target priority parameter; the timeliness coefficient is related to the time difference of target detection, and the exponential decay factor is related to the target not updated time and the current number of detailed investigation tasks; The detailed investigation execution module is used to input the flight altitude into the preset second formula to obtain the current zoom ratio, and to use multiple sensors to collect the three-dimensional features of multiple targets at the current zoom ratio. Based on the preset multimodal feature fusion network, the three-dimensional features of each target are processed to obtain a comprehensive verification score. Targets whose comprehensive verification scores reach the second preset value are used as detailed investigation targets. The detailed target update module is used to construct an incremental target list by combining ProtoBuf binary encoding with an LRU update strategy, so as to update the detailed targets. The detailed investigation exit module is used to switch the detailed investigation state back to wide-area reconnaissance mode using an intelligent state machine when the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value.

10. An electronic device, characterized in that, The electronic device includes: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the UAV autonomous reconnaissance payload control method incorporating target detection and recognition according to any one of claims 1 to 8.

Images