Unmanned aerial vehicle autonomous reconnaissance load control method integrated with target detection and identification

Abstract

The invention discloses an unmanned aerial vehicle autonomous reconnaissance load control method, device and equipment integrated with target detection and identification. An unmanned aerial vehicle is controlled to execute a wide-area reconnaissance mode by using a pre-constructed height-field angle dynamic mapping mode; when the target priority parameter of the scheduling queue is greater than a first preset value, triggering a detailed investigation state; inputting the flight height into a preset second formula to obtain a current zoom magnification, collecting respective three-dimensional features of a plurality of targets by adopting a plurality of sensors under the current zoom magnification, processing the three-dimensional features of each target based on a preset multi-modal feature fusion network to obtain a comprehensive verification score, and verifying the comprehensive verification score according to the comprehensive verification score. Taking the target with the comprehensive verification score reaching a second preset value as a detailed investigation target; constructing an incremental target list to update the detailed investigation target; when the unconfirmed target ratio is smaller than a third preset value and the newly added target rate is smaller than a third preset value, the detailed investigation state is switched back to the wide-area reconnaissance mode through the intelligent state machine, and dynamic switching of the reconnaissance mode can be achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and particularly relates to a UAV autonomous reconnaissance payload control method, device and equipment integrated with target detection and identification. BACKGROUND

[0002] There are three technical bottlenecks in the current UAV photoelectric reconnaissance field: first, the field of view adaptive ability is insufficient, for example, the scheme disclosed in the prior art adopts a fixed optical zoom ratio, and when the flight height changes, manual intervention is required to adjust the field of view angle, resulting in a target tracking loss rate of more than 35%; second, the reconnaissance mode switching efficiency is low, according to the measured data in "UAV Photoelectric Reconnaissance Technology Progress", it takes 8-12 seconds on average to switch from wide-area reconnaissance to target detailed investigation depending on the ground station command; third, the situation maintenance mechanism is missing, and the target list lacks real-time dynamic updating function, resulting in intelligence timeliness decaying to less than 60% after 10 minutes of generation. The above defects seriously restrict the effectiveness of UAV autonomous reconnaissance, and urgent technical innovation is needed to solve the problem. SUMMARY

[0003] The main purpose of the present application is to provide a UAV autonomous reconnaissance payload control method, device and equipment integrated with target detection and identification, aiming to break through the technical bottleneck of the existing UAV photoelectric reconnaissance system, and to build a "wide-area scanning-intelligent detection-precise detailed investigation-situation updating" four-order closed loop working flow, and to realize the dynamic switching of the reconnaissance mode through the autonomous decision engine.

[0004] To achieve the above object, the application provides a UAV autonomous reconnaissance payload control method integrated with target detection and identification, comprising: controlling the UAV to perform a wide-area reconnaissance mode by using a pre-constructed height-FOV dynamic mapping mode, wherein the parameters of the UAV in the wide-area reconnaissance mode include an optimal FOV of a sensor, an instantaneous FOV of a payload gimbal automatic zoom, and a corrected small target detection confidence; processing the corrected small target detection confidence by using a time-space joint priority algorithm to obtain a scheduling queue, and triggering a detailed investigation state when a target priority parameter of the scheduling queue is greater than a first preset value; wherein the time-space joint priority algorithm is determined based on the product of the corrected detection confidence and a time-effect coefficient, and the target priority parameter is determined based on the product of the corrected detection confidence and an exponential decay factor; inputting the flight height into a preset second formula to obtain a current zoom ratio, and collecting the three-dimensional features of a plurality of targets by using a plurality of sensors under the current zoom ratio, processing the three-dimensional features of each target based on a preset multi-modal feature fusion network to obtain a comprehensive verification score, and taking a target with a comprehensive verification score reaching a second preset value as a detailed investigation target; constructing an incremental target list by using ProtoBuf binary encoding combined with an LRU update strategy to update the detailed investigation target; and switching the detailed investigation state back to the wide-area reconnaissance mode by using an intelligent state machine when the proportion of unconfirmed targets is less than a third preset value and the rate of new targets is less than the third preset value.

[0005] Optionally, the controlling the UAV to perform the wide-area reconnaissance mode by using the pre-constructed height-FOV dynamic mapping mode comprises: processing the obtained flight height data by using a FOV calculation expression to obtain an optimal FOV; controlling the photoelectric payload gimbal automatic zoom to make the instantaneous FOV not greater than the optimal FOV, and correcting the confidence of small target detection by using a confidence compensation expression, wherein the confidence of small target detection is determined by using a YOLO model.

[0006] Optionally, the FOV calculation expression is:

[0007] wherein, represents the minimum size of the target, represents the sensor resolution coefficient of the UAV, H represents the flight height of the UAV; the expression of the corrected small target detection confidence of each small target is:

[0008] wherein, represents the corrected small target detection confidence, represents the basic detection confidence, represents the height correction weight, represents the decay coefficient.

[0009] Optionally, the data structure of the scheduling queue adopts a complete binary tree structure.

[0010] Optionally, the time coefficient is the reciprocal of the difference between the current time and the last detection time of the small target detection.

[0011] Optionally, the method further comprises: determining a subtraction item based on the difference between the current detailed investigation task number and the maximum concurrent detailed investigation capability; determining a first multiplier item based on 1 minus the subtraction item; determining a second multiplier item based on the negative value of the time decay constant and the non-updated time of the detected target; determining an exponential item according to the product of the first multiplier item and the second multiplier item; and determining an exponential decay factor based on the natural number and the exponential item.

[0012] Optionally, the multi-modal feature fusion network comprises: a feature extraction layer, configured to process a visible light image by using a ResNet-50 backbone network and a feature pyramid to obtain a visible light feature vector, and process an infrared image by using a CNN backbone network and a thermal radiation normalization algorithm to obtain a thermal feature; a cross-modal attention fusion layer, configured to interact the thermal feature and a multi-scale shape feature in a cross-attention fusion layer, and weight a key-value pair of the infrared light feature vector by taking the visible light feature vector as a query vector to obtain a light adaptation fusion feature; a feature enhancement layer, configured to aggregate features in a space-time dimension by 3D convolution after dimension reduction of the light adaptation fusion feature to obtain aggregated features, and process the dual-modal features by using a GRU gating mechanism to obtain a motion trajectory continuity feature; and a full-connection classification layer, configured to process the motion trajectory continuity feature and the aggregated features by using a three-head full-connection layer to obtain a shape matching score. Optionally, the shape matching score comprises a visible light shape matching similarity score, a motion pattern matching degree score, and a thermal radiation feature confidence score. After the three-head full-connection layer processes the motion trajectory continuity feature and the aggregated features to obtain the shape matching score, the method further comprises: calculating a comprehensive verification score of each target by using the weighted visible light shape matching similarity score, the motion pattern matching degree score, and the thermal radiation feature confidence score.

[0013] To achieve the above object, the application further provides an unmanned aerial vehicle autonomous reconnaissance payload control device integrated with target detection and identification, comprising: a wide-area scanning module configured to control the unmanned aerial vehicle to perform a wide-area reconnaissance mode by using a pre-constructed height-field-of-view angle dynamic mapping mode, wherein the parameters of the unmanned aerial vehicle in the wide-area reconnaissance mode include an optimal field of view angle of a sensor, a momentary field of view angle of a payload pan-tilt automatic zoom, and a corrected small target detection confidence; a detailed investigation task determination module configured to process the corrected small target detection confidence by using a time-space joint priority algorithm to obtain a scheduling queue, and trigger a detailed investigation state when a target priority parameter of the scheduling queue is greater than a first preset value; wherein the time-space joint priority algorithm is determined based on a product of the corrected detection confidence and a time effectiveness coefficient, and the target priority parameter is determined based on a product of the corrected detection confidence and an exponential decay factor; a detailed investigation execution module configured to input a flight height into a preset second formula to obtain a current zoom ratio, and collect three-dimensional features of a plurality of targets by using a plurality of sensors under the current zoom ratio, process the three-dimensional features of each target based on a preset multi-modal feature fusion network to obtain a comprehensive verification score, and take a target with the comprehensive verification score reaching a second preset value as a detailed investigation target; a detailed investigation target updating module configured to construct an incremental target list by using ProtoBuf binary encoding combined with an LRU updating strategy to update the detailed investigation target; and a detailed investigation exit module configured to switch the detailed investigation state back to the wide-area reconnaissance mode by using an intelligent state machine when an unconfirmed target proportion is less than a third preset value and a new target rate is less than the third preset value.

[0014] To achieve the above object, the application further provides a computer readable storage medium comprising instructions which, when executed on a computer, cause the computer to perform the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification provided by the above embodiments.

[0015] To achieve the above object, the application further provides an electronic device, comprising: at least one processor, a memory, and an input-output unit; wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program stored in the memory to perform the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification provided by any of the above embodiments.

[0016] The unmanned aerial vehicle autonomous reconnaissance payload control method, device and equipment integrated with target detection and identification provided by the embodiment of the application solve the problem of target loss rate > 35% caused by height change by using a pre-constructed height-FOV dynamic mapping mode to control the unmanned aerial vehicle to perform a wide-area reconnaissance mode, wherein the parameters of the unmanned aerial vehicle in the wide-area reconnaissance mode include the optimal FOV of the sensor, the instantaneous FOV of the payload gimbal automatic zoom, and the corrected small target detection confidence; the time-space joint priority algorithm is used to process the corrected small target detection confidence to obtain a scheduling queue, and the detailed investigation state is triggered when the target priority parameter of the scheduling queue is greater than a first preset value; wherein the time-space joint priority algorithm is determined based on the product of the corrected detection confidence and the time coefficient, and the target priority parameter is determined based on the product of the corrected detection confidence and the exponential decay factor; the flight height is input into a preset second formula to obtain the current zoom ratio, and the three-dimensional features of multiple targets are collected by multiple sensors under the current zoom ratio, the preset multi-modal feature fusion network is used to process the three-dimensional features of each target to obtain a comprehensive verification score, and the target whose comprehensive verification score reaches a second preset value is taken as a detailed investigation target; the incremental target list is constructed by using the ProtoBuf binary encoding combined with the LRU update strategy to update the detailed investigation target, so that the intelligence timeliness is maintained > 90% within 60 minutes of continuous operation, and the defect of timeliness decay under the traditional list maintenance mechanism is completely solved; when the unconfirmed target proportion is less than a third preset value and the new target rate is less than the third preset value, the detailed investigation state is switched back to the wide-area reconnaissance mode by using the intelligent state machine, and the seamless switching of the wide-area reconnaissance and the target detailed investigation mode is realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The flowchart provided for the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification of the application; Figure 2 The system block diagram provided for the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification of the application; Figure 3 The mode switching process based on the finite state machine (FSM) provided for the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification of the application; Figure 4 The target state transition diagram provided for the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification of the application; Figure 5 The dual-modal feature fusion network structure provided for the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and identification of the application.

[0018] The implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are merely exemplary and do not limit the application.

[0020] Figure 2 The application incorporates the system block diagram provided by the unmanned aerial vehicle autonomous reconnaissance payload control method of the target detection and identification embodiment, refer to Figure 2 , The unmanned aerial vehicle in the system acquires the flight height in real time through the height sensor, and calculates the optimal view field angle at the current height based on the height-view field angle dynamic mapping model in the intelligent task control system, to ensure that the ground target imaging size is greater than or equal to 15 pixels to meet the identification requirements of the intelligent task control system. The intelligent task control system autonomously generates a focusing instruction and issues it to the photoelectric payload, to drive the visible light camera or the infrared thermal imager to complete the view field adjustment. The adjusted photoelectric payload returns the reconnaissance video stream to the unmanned aerial vehicle, and the intelligent task control system performs target detection and identification on the video frame, and outputs the results including target three-dimensional coordinates (longitude, latitude, and elevation), confidence, and detection timestamp (millisecond level accuracy); when the intelligent task control system judges that the target priority is greater than 0.8, a target list is automatically generated and transmitted to the ground control station through the wireless data link. After receiving the list, the ground control station performs analysis and extracts the structured intelligence fields: target number, target type, target location, confidence, and state marker.

[0021] The application aims to break through the technical bottleneck of the existing unmanned aerial vehicle photoelectric reconnaissance system, to build a four-order closed loop working flow of "wide area scanning-intelligent detection-precise detailed investigation-situation updating", to realize the dynamic switching of the reconnaissance mode through the autonomous decision engine, and to realize the intelligent upgrading through the following three innovation mechanisms: ① build a height-view field angle dynamic mapping model, to dynamically calculate the optimal view field angle (θ=2×arctan(W / (2H×k)) based on the real-time flight height, to solve the problem of target loss rate>35% caused by height change, and to realize the autonomous precise adaptation of the payload view field; ② establish a "detection-identification-update" whole process closed loop decision mechanism, to realize the seamless switching (switching delay<1 second) of the wide area reconnaissance and target detailed investigation mode by using the intelligent state machine, and to eliminate the 8-12 second operation delay caused by the intervention of the ground station in the traditional scheme; ③ design an incremental target list coding protocol, to use the ProtoBuf binary coding combined with the LRU update strategy, to ensure that the timeliness of the intelligence is maintained>90% within 60 minutes of continuous operation, and to completely solve the defect of timeliness decay under the traditional list maintenance mechanism.

[0022] Refer to Figure 1 , Figure 1The flow chart of the unmanned aerial vehicle autonomous reconnaissance payload control method provided by the first embodiment of the present application and integrated with target detection and recognition is applied to an intelligent task control system. The unmanned aerial vehicle autonomous reconnaissance payload control method integrated with target detection and recognition can include the following execution process: S10, controlling the unmanned aerial vehicle to perform a wide-area reconnaissance mode by using a pre-constructed height-field-of-view angle dynamic mapping module, wherein the parameters of the unmanned aerial vehicle in the wide-area reconnaissance mode include an optimal field of view angle of a sensor, an instantaneous field of view angle of a payload gimbal automatic zoom, and a corrected small target detection confidence; In an embodiment of the present application, the execution process of controlling the unmanned aerial vehicle to perform the wide-area reconnaissance mode by using the pre-constructed height-field-of-view angle dynamic mapping module can include the following: S101, processing the obtained flight height data by using a field-of-view angle calculation expression to obtain the optimal field of view angle; S102, controlling the photoelectric payload gimbal automatic zoom to make the instantaneous field of view angle not greater than the optimal field of view angle, and correcting the confidence of small target detection by using a confidence compensation expression, wherein the confidence of small target detection is determined by using a YOLO model.

[0023] Specifically, the processor can obtain the flight height of the unmanned aerial vehicle in real time (unit: m), dynamically calculate the optimal field of view angle , control the photoelectric payload gimbal automatic zoom to make the instantaneous field of view angle , and ensure that the ground target is imaged with a pixel.

[0024] The field-of-view angle calculation expression is as follows:

[0025] In the above formula, represents the minimum size of the target (unit: m, default value 0.3, configurable), represents the sensor resolution coefficient of the unmanned aerial vehicle , H represents the flight height of the unmanned aerial vehicle; The expression of the corrected small target detection confidence of each small target is as follows:

[0026] Among them, represents the corrected small target detection confidence, represents the basic detection confidence (model output such as YOLO), represents the height correction weight (empirical value 0.15), represents the attenuation coefficient (empirical value 0.005).

[0027] Obviously, based on real-time flight height, the optimal field of view angle is dynamically calculated, which can solve the problem of target loss rate > 35% caused by height change, and realize the self-precision adaptation of the load field of view.

[0028] S20, the space-time joint priority algorithm is used to process the corrected small target detection confidence, a scheduling queue is obtained, and a detailed checking state is triggered when the target priority parameter of the scheduling queue is greater than a first preset value; Based on the product of the corrected detection confidence and the time decay factor, the space-time joint priority algorithm is determined, and based on the product of the corrected detection confidence and the exponential decay factor, the target priority parameter is determined. Among them, the data structure of the scheduling queue adopts a complete binary tree structure, that is, a minimum heap structure.

[0029] Specifically, the processor schedules the detailed checking task based on the priority queue, wherein the priority calculation formula of the priority queue scheduling is: Priority = target confidence x time decay factor.

[0030] Among them, the time decay factor is the reciprocal of the difference between the current time and the last detection time of the small target detection; The priority parameter formula is as follows:

[0031] Among them, Indicates the non-updated time of target i, in seconds; Indicates the time decay constant according to scene adaptation, and the default is 300 seconds; Indicates the current detailed checking task number; Indicates the maximum concurrent detailed checking capability; (hardware performance determines).

[0032] Based on this, the space-time joint priority algorithm is used to schedule the target detailed checking, the scheduling queue is sorted according to the target priority parameter calculated , and the updated target list is dynamically generated , the target detailed checking process is triggered when , the target detailed checking queue is added, and the queue scheduling adopts a minimum heap structure.

[0033] After determining the scheduling queue, the next step is that the processor controls the unmanned aerial vehicle to perform zoom control of target detailed checking and execution of target detailed checking.

[0034] S30, input the flight height into a preset second formula to obtain the current zoom ratio, and use multiple sensors to collect the three-dimensional features of multiple targets respectively under the current zoom ratio, the preset multi-modal feature fusion network is used to process the three-dimensional features of each target to obtain a comprehensive verification score, and the target with a comprehensive verification score reaching a second preset value is taken as a detailed checking target; Specifically, the unmanned aerial vehicle can perform multi-circle surrounding flight, and according to the height Automatic matching of zoom ratio, obtaining target three-dimensional features, keeping target imaging size constant 5% error.

[0035]

[0036] Wherein, The reference zoom ratio (measured at The height measured); The current flight height; The target first detection height.

[0037] Next, the processor processes each target three-dimensional feature based on a preset multi-modal feature fusion network to obtain a comprehensive verification score. The multi-modal feature fusion network can include: The feature extraction layer is used to process the visible light image by using the ResNet-50 backbone network and the feature pyramid to obtain multi-scale morphological features, and to process the infrared image by using the CNN backbone network and the thermal radiation normalization algorithm to obtain thermal features; The cross-modal attention fusion layer is used to interact the thermal features and the multi-scale morphological features in the cross-attention fusion layer, and to weight the key-value pair of the infrared light feature vector by taking the visible light feature vector as the query vector to obtain the illumination adaptive fusion features; The feature enhancement layer is used to reduce the dimension of the illumination adaptive fusion features, aggregate them in the space-time dimension by 3D convolution to obtain aggregated features, and then process the dual-modal features by using the GRU gating mechanism to obtain motion trajectory continuity features; The fully connected classification layer is used to process the motion trajectory continuity features and the aggregated features by using the three-head fully connected layer to obtain morphological matching scores.

[0038] Reference Figure 5, it is worth noting that the dual-modal feature fusion network structure belongs to the core module of the target detailed investigation system, solving the target confirmation problem in complex environments. After the visible light branch receives the image input, dynamic data enhancement (rotation ± 15°, brightness / contrast ± 20% adjustment) is performed, and then the hierarchical features are extracted through the ResNet50 backbone network, and multi-scale morphological features are output through the feature pyramid FPN; the infrared branch performs thermal radiation value normalization (environment temperature compensation) on the input image, and extracts thermal features through a lightweight CNN network (5 layers of depth separable convolution). The dual-modal features interact in the cross-attention fusion layer: the visible light features are used as the query vector, and the infrared feature key-value pairs are dynamically weighted to achieve adaptive fusion of light. After the fused features are reduced in dimension by 1x1 convolution, they are input into the spatio-temporal joint optimization module: the features are aggregated in the space-time dimension through 3D convolution, and then the GRU gating mechanism is used to model the continuity of the motion trajectory. Finally, the morphological matching score (visible light dominant) and the thermal feature score (infrared dominant) are output in parallel by three fully connected layers, and the comprehensive verification score (Ver_score) is generated by combining the external motion analysis score, which drives the target state migration decision. This structure realizes 65ms end-to-end inference through hardware acceleration (Jetson AGX Orin), and improves the camouflage target recognition rate from 67% to 92.7%.

[0039] Specifically, referring to Figure 5 S40, an incremental target list is constructed by using ProtoBuf binary encoding combined with LRU update strategy to update the detailed investigation target; S50, when the unconfirmed target proportion is less than a third preset value and the new target rate is less than a third preset value, the intelligent state machine is used to switch the detailed investigation state back to the wide-area reconnaissance mode.

[0040] Referring to Figure 3 , in the figure, the wide-area reconnaissance state is used to dynamically adjust the field of view angle of the photoelectric payload, and through high-frequency focusing control and infrared assistance, clear imaging of the target is ensured, and the preliminary three-dimensional coordinates and the compensated confidence of the target are output; the target detailed investigation state starts multi-circle around flight for high-priority targets (priority > 0.8), and adaptively adjusts the zoom ratio, fuses visible light morphological matching, motion trajectory analysis and thermal radiation feature three-modal verification, and marks the target as a confirmed target after the score meets the standard; the emergency state is used to activate when the target trajectory deviation is out of limit or continuous multiple frames are lost, immediately terminate the current task and restart the wide-area scanning, and send an abnormal alarm to the ground station at the same time; state transition logic: The system realizes the dynamic switching of the wide-area reconnaissance state, the target detailed investigation state and the emergency state through the intelligent state machine, Specifically, the state change from wide-area to detailed investigation is switched when the unmanned aerial vehicle detects a high-priority target and the detailed investigation resource is idle; The detailed investigation → wide-area state change is returned when the unconfirmed target proportion is lower than 15% and the new target rate is lower than the threshold value; The arbitrary state → emergency state change is forced to cut in when the trajectory prediction is abnormal or the communication interruption timeout is forced to cut in, and the abnormality is removed to reset to the wide-area state.

[0041] Reference Figure 4 The target starts from the "uncertain" state, and if the score exceeds 0.9 in the multi-modal verification, it is promoted to the "confirmed target"; when the uncertain target is not detected for 5 frames (0.5 seconds) in succession, it is automatically degraded to the "invalid target"; and the confirmed target is also converted into the invalid target if it does not update information within 120 seconds.

[0042] The flow forms a closed-loop state migration path, specifically including: a promotion path for information strengthening: uncertain target → detailed investigation verification (score > 0.9) → confirmed target (confidence is forced to be promoted to 0.95) The failure path is used for information attenuation: Fast failure: uncertain target → 5 frames lost in succession → invalid target (resources are immediately released) Timeout failure: confirmed target → 120 seconds without updating → invalid target (send failure notification) In an embodiment of the present application, the specific execution process of step SXX can be: Based on the above embodiment, the present application further provides an unmanned aerial vehicle autonomous reconnaissance load control device integrated with target detection and identification, characterized by comprising: A wide-area scanning module is configured to control the unmanned aerial vehicle to perform a wide-area reconnaissance mode by using a pre-constructed height-visual field angle dynamic mapping mode, wherein the parameters of the unmanned aerial vehicle in the wide-area reconnaissance mode include the optimal visual field angle of the sensor, the instantaneous visual field angle of the load holder automatic zoom, and the corrected small target detection confidence; A detailed investigation task determination module is configured to process the corrected small target detection confidence by using a time-space joint priority algorithm to obtain a scheduling queue, and trigger a detailed investigation state when the target priority parameter of the scheduling queue is greater than a first preset value; The time-space joint priority algorithm is determined based on the product of the corrected detection confidence and the time coefficient, and the target priority parameter is determined based on the product of the corrected detection confidence and the exponential decay factor; A detailed investigation execution module is configured to input the flight height into a preset second formula to obtain a current zoom ratio, and collect the three-dimensional features of a plurality of targets by using a plurality of sensors under the current zoom ratio, process the three-dimensional features of each target based on a preset multi-modal feature fusion network to obtain a comprehensive verification score, and take the target with the comprehensive verification score reaching a second preset value as a detailed investigation target; The detailed investigation target updating module is configured to update the detailed investigation target by using ProtoBuf binary coding combined with LRU updating strategy to construct an incremental target list. The detailed investigation exit module is configured to switch the detailed investigation state back to the wide-area reconnaissance mode by using the intelligent state machine when the unconfirmed target proportion is less than the third preset value and the new target rate is less than the third preset value.

[0043] The second embodiment of the present application further provides an electronic device, characterized in that the electronic device comprises at least one processor, a memory and an input-output unit; wherein the memory is configured to store a computer program, and the processor is configured to call the computer program stored in the memory to execute the unmanned aerial vehicle autonomous reconnaissance payload control method integrated with the target detection and identification.

[0044] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for controlling the autonomous reconnaissance payload of an unmanned aerial vehicle (UAV) integrating target detection and recognition, characterized in that, include: The UAV is controlled to perform a wide-area reconnaissance mode by using a pre-built altitude-field-of-view dynamic mapping model. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. The spatiotemporal joint priority algorithm is used to process the corrected small target detection confidence, and a scheduling queue is obtained. When the target priority parameter in the scheduling queue is greater than the first preset value, a detailed investigation state is triggered. Among them, the spatiotemporal joint priority algorithm is determined based on the product of the modified detection confidence and the timeliness coefficient, and the target priority parameter is determined based on the product of the modified detection confidence and the exponential decay factor. The flight altitude is input into the preset second formula to obtain the current zoom ratio. At the current zoom ratio, multiple sensors are used to collect the three-dimensional features of multiple targets. The three-dimensional features of each target are processed based on the preset multimodal feature fusion network to obtain a comprehensive verification score. Targets whose comprehensive verification scores reach the second preset value are used as targets for detailed investigation. An incremental target list is constructed using ProtoBuf binary encoding combined with an LRU update strategy to update the targets for detailed investigation; When the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value, the intelligent state machine is used to switch the detailed investigation mode back to the wide-area reconnaissance mode.

2. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The method of controlling the UAV to perform wide-area reconnaissance using a pre-constructed altitude-field-of-view dynamic mapping mode includes: The acquired flight altitude data is processed using the field of view calculation expression to obtain the optimal field of view. The gimbal for controlling the photoelectric payload automatically zooms to ensure that the instantaneous field of view is no greater than the optimal field of view, and the confidence of small target detection is corrected by a confidence compensation expression, wherein the confidence of small target detection is determined by the YOLO model.

3. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 2, characterized in that, The expression for calculating the field of view is: in, Indicates the minimum size of the target. This represents the sensor resolution coefficient of the drone. H Indicates the drone's flight altitude; The expression for the corrected small target detection confidence score for each small target is: in, This represents the corrected confidence level for small target detection. Indicates the basic detection confidence level. Indicates highly adjusted weights, This represents the attenuation coefficient.

4. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The data structure of the scheduling queue adopts a complete binary tree structure.

5. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The timeliness coefficient is the reciprocal of the difference between the current time of small target detection and the last detection time.

6. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, Also includes: The subtrahend term is obtained based on the difference between the current number of detailed investigation tasks and the maximum concurrent detailed investigation capacity; The first multiplicative term is obtained by subtracting the subtrahend term from 1. The second multiplier term is determined based on the target's unupdated time and the negative value of the time decay constant. Determine the exponent term based on the product of the first and second multiplier terms; The exponential decay factor is determined based on the natural number base and the exponential term.

7. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 1, characterized in that, The multimodal feature fusion network includes: The feature extraction layer is used to process visible light images using the ResNet-50 backbone network and feature pyramid to obtain visible light feature vectors, and to process infrared images using the CNN backbone network and thermal radiation normalization algorithm to obtain thermal features. The cross-modal attention fusion layer is used to interact thermal features and multi-scale morphological features in the cross-attention fusion layer. The key-value pairs of the infrared light feature vector are weighted using the visible light feature vector as the query vector to obtain the illumination adaptive fusion features. The spatiotemporal feature enhancement module is used to reduce the dimensionality of the illumination adaptive fusion features and then aggregate them in the spatial-temporal dimensions through 3D convolution to obtain aggregated features; then, the GRU gating mechanism is used to process the dual-modal features to obtain the motion trajectory continuity features. The fully connected classification layer uses a three-head fully connected layer to process the continuity and aggregation features of the motion trajectory to obtain the morphological matching score.

8. The UAV autonomous reconnaissance payload control method incorporating target detection and recognition as described in claim 7, characterized in that, The morphological matching score includes visible light morphological matching similarity score, motion pattern matching score, and thermal radiation feature confidence score. After processing the continuity and aggregation features of the motion trajectory using a three-head fully connected layer to obtain the morphological matching score, the method further includes: The comprehensive verification score of each target is calculated using the weighted visible light morphology matching similarity score, motion pattern matching score, and thermal radiation feature confidence score.

9. A drone autonomous reconnaissance payload control device integrating target detection and recognition, characterized in that, include: The wide-area scanning module is used to control the UAV to perform a wide-area reconnaissance mode using a pre-built altitude-field-of-view dynamic mapping mode. The parameters of the UAV in the wide-area reconnaissance mode include the optimal field of view of the sensor, the instantaneous field of view of the payload gimbal autofocus, and the corrected small target detection confidence. The detailed investigation task determination module is used to process the corrected small target detection confidence using a spatiotemporal joint priority algorithm, obtain a scheduling queue, and trigger the detailed investigation state when the target priority parameter in the scheduling queue is greater than the first preset value. Among them, the spatiotemporal joint priority algorithm is determined based on the product of the modified detection confidence and the timeliness coefficient, and the target priority parameter is determined based on the product of the modified detection confidence and the exponential decay factor. The detailed investigation execution module is used to input the flight altitude into the preset second formula to obtain the current zoom ratio, and to use multiple sensors to collect the three-dimensional features of multiple targets at the current zoom ratio. Based on the preset multimodal feature fusion network, the three-dimensional features of each target are processed to obtain a comprehensive verification score. Targets whose comprehensive verification scores reach the second preset value are used as detailed investigation targets. The detailed target update module is used to construct an incremental target list by combining ProtoBuf binary encoding with an LRU update strategy, so as to update the detailed targets. The detailed investigation exit module is used to switch the detailed investigation state back to wide-area reconnaissance mode using an intelligent state machine when the proportion of unconfirmed targets is less than the third preset value and the rate of new targets is less than the third preset value.

10. An electronic device, characterized in that, The electronic device includes: At least one processor, memory, and input / output unit; The memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute the UAV autonomous reconnaissance payload control method incorporating target detection and recognition according to any one of claims 1 to 8.

Images