Unmanned aerial vehicle autonomous target identification and strike method, device, equipment and medium

Abstract

The invention provides an unmanned aerial vehicle autonomous target identification and strike method, device and equipment and a medium, airborne high-dynamic target detection is realized through a lightweight model, and tracking robustness is improved in combination with a time sequence verification mechanism; innovatively designing a multi-coordinate system coupling transformation chain, and resolving image coordinates into a body direction vector in real time; a neural network model is adopted to replace manual rule decision, and an intelligent strike instruction is output; accurate strike in a dynamic environment is realized through pod-flight control double-layer PID cooperative control. The problems that an existing system is low in recognition precision, high in decision delay and high in communication dependence are solved, and full-process edge autonomous closed-loop combat is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle autonomous target identification and attack method, device, equipment and medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and unmanned system technology, unmanned aerial vehicle platforms have been widely used in military reconnaissance, battlefield monitoring and special attack tasks. However, the current mainstream target identification and attack system still mainly relies on remote ground stations for image processing and decision control, and the overall process highly depends on manual operation, resulting in insufficient system autonomy, large response delay, and being easily affected by communication interruption, electromagnetic interference and other factors in complex battlefield environments, which seriously restricts the real-time performance and combat effectiveness.

[0003] To improve the above problems, some research attempts to move the target identification and simplified decision function to the flight platform, such as embedding a preliminary autonomous task processing through embedded vision algorithms or rule engines, so as to improve the response speed and off-chain operation ability. However, such solutions still have significant shortcomings:

[0004] Limited computing power, insufficient recognition accuracy: due to the resource bottleneck of the airborne edge platform, existing solutions are difficult to deploy complex deep neural networks, resulting in limited target recognition performance, especially in strong background interference, multiple target density or small target size.

[0005] Strong dependence on human, lack of high autonomy: key identification results and attack decisions still need to be confirmed by operators, resulting in high dependence on human systems, and the inability to truly achieve end-side autonomous closed-loop decision-making.

[0006] Simple attack logic, lack of intelligence: existing attack judgments are mostly based on manually set threshold rules, lacking deep understanding of target value and context sensitivity, making it difficult to adapt to the diversity and dynamics of battlefield environments, and prone to false or missed attacks.

[0007] Identification and attack are disconnected: current systems often design target identification and attack control as separate modules, lack a unified closed-loop linkage mechanism, and cannot achieve an autonomous combat process of "discovery-identification-tracking-decision-attack".

[0008] Severe dependence on communication links, poor anti-interference performance: system combat functions rely on high-frequency data interaction with ground stations, and once the communication link is interrupted, it will lead to identification lag or control failure, and does not have the ability to complete the task independently in a communication-limited environment.

[0009] Therefore, an integrated solution integrating edge intelligence, autonomous decision-making and precise control is urgently needed to realize high-precision target identification, low-delay intelligent decision-making and high-stable cooperative control closed-loop combat capability under limited computing power. SUMMARY

[0010] The technical problem to be solved by the present application is to provide a UAV autonomous target identification and attack method, device, equipment and medium, aiming to realize high-precision target identification, low-delay intelligent decision-making and high-stable cooperative control closed-loop combat capability under limited computing power.

[0011] To solve the above technical problems, the technical scheme adopted by the present application is as follows: a UAV autonomous target identification and attack method, comprising the following steps:

[0012] S1, real-time processing of video stream collected by photoelectric pod through airborne edge computing platform, detecting dynamic target in image by using lightweight deep learning model, and outputting target center coordinates with highest confidence;

[0013] S2, continuously determining target in continuous frames based on time sequence verification mechanism, screening optimal tracking target and outputting stable coordinates thereof;

[0014] S3, real-time solving of two-dimensional image coordinates of target into three-dimensional direction vector under UAV body coordinate system through four-level coordinate system transformation chain;

[0015] S4, extracting target features and inputting into neural network model, outputting attack probability score and comparing with dynamic threshold value, and generating attack authorization instruction;

[0016] S5, generating pod yaw / pitch angle velocity instruction based on image pixel offset, driving pod to visually lock target, and generating UAV attitude angle instruction based on body coordinate system direction vector deviation, driving body to dynamically align target;

[0017] S6, under attack authorization state, controlling UAV to complete accurate attack on target.

[0018] The present application also provides a UAV autonomous target identification and attack device, comprising:

[0019] A high-dynamic target identification module is configured to real-time process video stream collected by photoelectric pod through airborne edge computing platform, detect dynamic target in image by using lightweight deep learning model, and output target center coordinates with highest confidence;

[0020] A single target stable tracking module is configured to continuously determine target in continuous frames based on time sequence verification mechanism, screen optimal tracking target and output stable coordinates thereof;

[0021] A multi-coordinate system coupling positioning module is configured to convert a two-dimensional image coordinate of a target into a three-dimensional direction vector in a UAV body coordinate system through a four-stage coordinate system conversion chain in real time.

[0022] An intelligent attack decision module is configured to input target features into a neural network model, output an attack probability score, compare the attack probability score with a dynamic threshold, and generate an attack authorization instruction.

[0023] A double-layer cooperative control module is configured to generate a pod yaw / pitch angle velocity instruction based on an image pixel offset to drive the pod to visually lock the target, and generate a UAV attitude angle instruction based on a body coordinate system direction vector deviation to drive the body to dynamically align the target.

[0024] An autonomous attack execution module is configured to control the UAV to complete accurate attack on the target in an attack authorization state.

[0025] The application further provides a computer device including a memory and a processor, the memory storing a computer program, and the processor implementing the UAV autonomous target identification and attack method when executing the computer program.

[0026] The application further provides a storage medium storing a computer program, and the computer program being executable by a processor to implement the UAV autonomous target identification and attack method.

[0027] The application has the advantages that target identification and decision are completed by relying on an airborne edge platform, remote backhaul delay is eliminated, and response speed is improved; image positioning error is compressed through a four-stage coordinate conversion chain, dynamic tracking precision is improved in combination with double-layer cooperative control; single-target stable tracking suppresses most false detection interference, and the intelligent decision model still autonomously executes an attack task when communication is interrupted. Ultimately, a perception-positioning-decision-attack integrated closed loop is formed, combat effectiveness is obviously improved compared with a traditional system, and artificial intervention requirement is reduced to zero. BRIEF DESCRIPTION OF DRAWINGS

[0028] The specific structure of the application will be described in detail below with reference to the drawings.

[0029] Figure 1 A flowchart of the UAV autonomous target identification and attack method of the application embodiment is shown in the figure.

[0030] Figure 2 A system software architecture diagram of the UAV autonomous target identification and attack method of the application embodiment is shown in the figure.

[0031] Figure 3 A target frame center point schematic diagram of the application embodiment is shown in the figure.

[0032] Figure 4A single-target stable tracking algorithm principle schematic diagram of an embodiment of the present application;

[0033] Figure 5 A single-target stable tracking algorithm flow chart of an embodiment of the present application;

[0034] Figure 6 A four-level coordinate system transformation schematic diagram of an embodiment of the present application;

[0035] Figure 7 A mathematical modeling schematic diagram of an image coordinate system of an embodiment of the present application;

[0036] Figure 8 A camera coordinate system schematic diagram of an embodiment of the present application;

[0037] Figure 9 A pod coordinate system rotation axis schematic diagram of an embodiment of the present application;

[0038] Figure 10 A target vector in a pod coordinate system pitch angle schematic diagram of an embodiment of the present application;

[0039] Figure 11 An unmanned aerial vehicle body coordinate system schematic diagram of an embodiment of the present application;

[0040] Figure 12 An AI attack decision model structure diagram of an embodiment of the present application;

[0041] Figure 13 An AI attack decision model training result diagram of an embodiment of the present application;

[0042] Figure 14 A pod layer and flight control layer collaborative control logic diagram of an embodiment of the present application;

[0043] Figure 15 A pod layer and flight control layer collaborative control communication structure diagram of an embodiment of the present application;

[0044] Figure 16 A target pixel offset error schematic diagram of an embodiment of the present application;

[0045] Figure 17 A target direction and error vector in a body coordinate system schematic diagram of an embodiment of the present application;

[0046] Figure 18 A YOLO model improved structure diagram of an embodiment of the present application;

[0047] Figure 19 A symmetric linear signed quantization schematic diagram of an embodiment of the present application;

[0048] Figure 20A real-time inference frame rate comparison chart of a model of an embodiment of the application on a Jetson NX development board is shown in FIG. 1;

[0049] Figure 21 A block diagram of an unmanned aerial vehicle autonomous target identification and attack device of an embodiment of the application is shown in FIG. 2;

[0050] Figure 22 A schematic block diagram of a computer device of an embodiment of the application is shown in FIG. 3. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the application.

[0052] It should be understood that, when used in the specification and the appended claims, the terms "comprise" and "include" indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0053] It should also be understood that the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification and the appended claims of the application, the singular forms "a", "an" and "the" are intended to include the plural forms, unless the context clearly indicates otherwise.

[0054] It should be further understood that the term "and / or" used in the specification and the appended claims of the application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0055] As shown in FIG. 4, to implement the system software architecture employed in the application, the following components are included: Figure 2

[0056] Target detection and tracking node: integrates target identification and tracking algorithms, and outputs high-confidence target image coordinates in real time to provide visual information support for subsequent calculation and control.

[0057] Pod attitude acquisition node: receives pod Yaw / Pitch angle data at a high frequency based on the UDP protocol, which is an important input for coordinate transformation and target direction calculation.

[0058] ​Gimbal control node: According to the pixel deviation of the target and the image center, real-time yaw / pitch angle velocity control instructions are generated, and the gimbal is driven to automatically track the target through UDP, realizing visual locking of the gimbal.

[0059] Coordinate solution and decision control node: Fusion of image and attitude data, real-time solution of target direction based on multi-coordinate system coupling algorithm, realization of space positioning. Combined with AI attack decision model, the target value is comprehensively evaluated, the attack judgment result is output, and the attitude and attack control instructions are generated, which drives the unmanned aerial vehicle to complete autonomous tracking and attack in a closed loop.

[0060] As shown in Figure 1 , the embodiment of the present application is: an unmanned aerial vehicle autonomous target identification and attack method, comprising the following steps:

[0061] S1, real-time processing of video stream collected by airborne optical pod through airborne edge computing platform, detecting dynamic target in image by using lightweight deep learning model, and outputting target center coordinates with highest confidence;

[0062] In step S1, the lightweight deep learning model is constructed by the following method:

[0063] A high-resolution feature layer is added to the YOLO architecture to enhance the detection ability of small targets, a deep redundant semantic layer is cropped to compress the calculation amount, and the backbone network is replaced by a lightweight convolutional network.

[0064] Based on the application scene of small target observation image of unmanned aerial vehicle at high altitude, and considering the accuracy and real-time performance of edge target detection, an improved YOLOv8n model is used.

[0065] The YOLOv8n model outputs five corresponding size feature maps, the P2 layer is located in the shallow layer of the backbone network, and the feature map has higher spatial resolution (160x160), so it can retain more edge, texture and position information of small targets, but the semantic expression ability is relatively weak. The P5 layer is located at the bottom of the feature pyramid, and the feature map resolution is the lowest (20x20), but after deeper network extraction, the semantic information is extremely rich.

[0066] A detection head P2 that is more sensitive to small targets is added to the original algorithm, and considering that the output of the P5 related detection head does not match the actual application scene, the Conv, Concat and C2f layers connected with the P5 detection head are cropped, but the related semantic feature layers transmitted from the P5 layer to the shallow layer are retained to avoid missing detection of some small targets due to lack of high-level semantic information. In this way, the detection ability of small targets is maintained, the semantic perception ability of small targets is retained, the recall rate of small target detection is enhanced, and the model inference amount is greatly compressed.

[0067] At the same time, replacing the original model's backbone network with a more lightweight FasterNet can effectively compress the feature map size while preserving as much spatial information as possible, significantly reducing the model's parameter and computational complexity, and improving inference efficiency and edge deployment adaptability.

[0068] Finally, the improved model not only has better small target detection capability, but also considers the real-time and lightweight requirements of the model, making it particularly suitable for unmanned aerial vehicle high-altitude micro target perception tasks.

[0069] The final model structure is shown in Figure 18

[0070] As shown in Table 1, during the system development process, multiple groups of model comparison designs were designed and carried out around the trade-off between lightweight and small target detection performance. Specifically, the following five types of improved models were included: original YOLOv8n, YOLOv8n+P2 with P2 detection head, YOLOv8+P2+P5 full-cut version with P5 all detection layers, YOLOv8+P2+P5 local-cut version with only P5 detection head reserved semantic transfer, and YOLOv8+P2+P5 local-cut+FasterNet model with FasterNet backbone network introduced on the basis of the above.

[0071] Table 1 Model naming and improvement scheme comparison table

[0072]

[0073] Note: FullCut: full-cut, SemCut: semantic cutting, FN: FasterNet lightweight backbone

[0074] As shown in Table 2, based on the public unmanned aerial vehicle small target detection dataset VisDrone, the performance of each lightweight improved model in the actual application scenario was systematically tested and compared with ablation experiment evaluation.

[0075] Table 2 Ablation experiment training parameters

[0076]

[0077] As shown in Table 3, after multiple comparison and ablation experiments, the results show that although P2_SemCutP5_FN (YOLOv8+P2+P5 semantic cutting+FasterNet) is suboptimal in terms of parameter quantity and GFLOPs indicators, and the mAP@0.5 precision is second, it maintains good detection performance under the premise of high lightweight.

[0078] Table 3 Ablation experiment results

[0079] ​

[0080] In the process of YOLOv8 algorithm training and inference, floating point 32-bit data (FP32) is used to represent the model's weight, bias, activation value, etc. High-precision data representation improves model performance while greatly increasing model computation overhead and memory requirements, resulting in slower inference speed on edge computing platforms.

[0081] Using INT8 symmetric linear signed quantization type, the model parameter Float32 (FP32) data range: -3.4x10 Λ 38~+3.4x10 Λ 38 is linearly quantized to INT8 range: -128~127.

[0082] As Figure 19 shown, symmetric signed linear quantization can be represented by the following mathematical expression:

[0083]

[0084] Where:

[0085] X represents the original FP32 value;

[0086] Z represents the mapped zero point (Zero Point), used to adjust the offset, which is 0 in symmetric quantization;

[0087] S represents the scaling factor derived from the Min-Max calibration algorithm for the data set;

[0088] The clip function ensures that the quantized value does not exceed the INT8 type representation range -128~127:

[0089]

[0090] X int represents the quantized INT8 integer value.

[0091] As Figure 20 shown, five quantized models were deployed and tested on JetsonNX, and their inference frame rates (FPS) were recorded. The final selected model P2_SemCutP5_FN (i.e., introducing P2 micro target detection head based on YOLOv8, retaining P5 layer for partial semantic pruning, and replacing it with a lightweight FasterNet backbone network) achieved real-time inference performance of about 140FPS while maintaining high detection accuracy, meeting the rapid target detection requirements in high-dynamic unmanned aerial vehicle scenarios.

[0092] To ensure the system to achieve autonomous recognition and continuous locking of ground targets in complex environments, the system designs and integrates a "high dynamic target detection and stable tracking algorithm", whose core processes include three key modules: target detection, confidence sorting and filtering, and image coordinate system modeling, to ensure that each frame outputs stable and optimal target center point coordinates.

[0093] (1)YOLO model multi-target detection result extraction and structure analysis

[0094] This scheme adopts a single-stage target detection algorithm based on deep convolutional neural network, references the structural framework of YOLO series, and optimizes the model precision and lightens the model for micro target recognition tasks to realize high-precision detection and stable tracking of ground micro targets.

[0095] The YOLO model provides N (N≥0) target detection frame results in the output of each image frame, each target frame including: target frame coordinate information (xmin, ymin, xmax, ymax) of the target; target class number Classes; target unique identifier ID: Target_id; detected target: corresponding confidence Confidence.

[0096] (2) Optimal target screening strategy

[0097] The system sorts the N candidate targets output by YOLO for each frame in descending order according to the confidence value Confidence, and selects the target with the highest confidence as the candidate target of the current frame, to improve the stability and tracking reliability of the target. This strategy ensures that the system always focuses on the only most prominent target, effectively avoids redundant target disturbance, and is more suitable for subsequent single target tracking and attack.

[0098] (3) Image plane coordinate modeling

[0099] To realize accurate calculation of the direction of the visual target, the system establishes a standard two-dimensional image coordinate system model for the YOLO output image, and extracts the image center position of the target based on the coordinate system as the input basis for subsequent pod control and target pointing calculation.

[0100] As shown in Figure 3 , on the image plane, the center point of the image is defined as the reference point, denoted as (c x ,c y ), where:

[0101]

[0102] W and H represent the width and height of the YOLO output image, respectively.

[0103] Each target output by the YOLO model contains the coordinate information (xmin, ymin, xmax, ymax) of its bounding box.min ,y min ,x max ,y max ), the image pixel center coordinates (u, v) of the target with the highest confidence in the current frame can be calculated, and the calculation formula is as follows:

[0104]

[0105] Thus, the coordinate information of the target with the highest confidence in each frame image is obtained.

[0106] S2, continuously determining the target in the continuous frames based on a time sequence verification mechanism, screening an optimal tracking target and outputting stable coordinates of the optimal tracking target;

[0107] The time sequence verification mechanism of step S2 comprises:

[0108] The continuous frame number threshold verification is started for the first appearing target, the delay confirmation counter is started for the target ID switching, the reinitialization process is started for the target loss, and the double threshold fault tolerance mechanism is used to suppress false detection interference.

[0109] The present application aims at unstable situations such as false detection, missed detection or target switching that may occur in the target detection process, as shown in the accompanying drawings, a single target stable tracking algorithm is proposed, and the maximum likelihood target in the continuous frame image time is stably selected and continuously tracked through the target persistence detection and confidence evaluation. Figure 4

[0110] The method comprises the following steps:

[0111] The algorithm effectively improves the fault tolerance ability of the system to target switching and temporary false detection by introducing a double threshold mechanism, time sequence verification and state driven update strategy. The overall process is shown in the accompanying drawings, and mainly comprises the following four stages: Figure 5

[0112] (1) initialization detection

[0113] After the system is started, the main loop schedules the YOLOv8 detection and tracking module, performs real-time processing on each frame of input image, and extracts potential target information.

[0114] (2) target screening

[0115] The target with the highest confidence in the current frame is screened as a candidate, and the priority is sorted according to the confidence score mechanism output by YOLOv8.

[0116] (3) target existence verification

[0117] ​​If the target first appears: if a new target is detected in the current frame, start the initial verification counter, and only when the preset frame threshold is continuously met, the target can be confirmed as the best_obj.

[0118] If the target ID changes: if the target ID is inconsistent with the last frame, start the continuous verification counter, and replace the current tracking target after a certain number of frames meet the condition.

[0119] If the target ID is consistent: if the target ID is continuous and consistent, directly update the target position coordinates without additional verification.

[0120] If the target is lost: if the target is not detected in consecutive frames, the system will clear the current tracking state and re-enter the initialization phase.

[0121] (4) Output stage

[0122] The optimal target (best_obj) that passes the verification is published with its target box center coordinate information to the downstream coordinate system conversion module. If there is no effective optimal target detection result for a short time, the system will publish the optimal target bounding box coordinates of the last frame for a short time to ensure that the coordinate flow is not interrupted due to temporary missed detection.

[0123] S3, the two-dimensional image coordinates of the target are converted into three-dimensional direction vectors in the unmanned aerial vehicle body coordinate system through a four-level coordinate system conversion chain in real time;

[0124] The four-level coordinate system conversion chain of step S3 is executed in order:

[0125] The space vector conversion of the image coordinate system, the camera coordinate system, the pod coordinate system, and the unmanned aerial vehicle body coordinate system, wherein:

[0126] The image coordinate system to the camera coordinate system through the focal length parameter normalized direction vector;

[0127] The camera coordinate system to the pod coordinate system through the pod attitude angle rotation matrix compensation;

[0128] The pod coordinate system to the body coordinate system through the coordinate system alignment mapping to realize the identity conversion.

[0129] As shown in Figure 6 , a four-level coordinate system conversion chain of image coordinates → camera coordinates → pod coordinates → aircraft body coordinates is constructed, and the core is the space direction calculation under the multi-coordinate fusion, which ensures that the system has real-time, accurate, and stable target direction output capability in dynamic flight state.

[0130] Specifically, the following steps are included:

[0131] (1) Image coordinates → camera coordinate system

[0132] The image output by YOLO is established in a plane coordinate system, as shown in Figure 7 A mathematical model is constructed: taking the center of the image as the reference point, denoted as (c x ,c y ), and the target center coordinates are denoted as (u, v).

[0133] As shown in Figure 8 , the camera coordinate system sets the optical axis direction of the camera on the pod as the positive direction of the x-axis, the pitch direction as the positive direction of the y-axis, and the yaw direction as the positive direction of the z-axis, and embeds the two-dimensional image coordinate system into the three-dimensional camera coordinate system. Specifically, the image plane is set at a unit distance in front of the camera optical center (i.e., at the position of x = 1) and is parallel to the y-z plane. At this time, the image center point corresponds to the point (1, 0, 0) in the three-dimensional coordinate system. Then the coordinates of the target in the camera three-dimensional coordinate system are:

[0134]

[0135] Taking the optical center of the camera on the pod as the coordinate origin, the direction vector of the target is:

[0136]

[0137] This vector represents the extension of the camera optical center in the direction through the image pixel point (u, v), reflecting the direction information of the target in the coordinate system space.

[0138] This direction vector takes the optical center of the camera as the starting point and points to the three-dimensional space direction corresponding to the target point in the image plane. It is normalized in the subsequent steps and used for coordinate transformation.

[0139]

[0140] ON: objectnormalize

[0141] where f x ,f y represent the effective focal lengths of the camera in the horizontal and vertical directions (in pixels), which can be calculated from the camera field of view (FOV) and the image resolution:

[0142]

[0143] where w and h are the width and height (pixels) of the image, respectively, and FOV x , FOV y are the horizontal and vertical field of view angles (in radians) of the camera, respectively.

[0144] (1) Camera coordinates → pod coordinate system

[0145] In the unmanned aerial vehicle autonomous target tracking system, the pitch and yaw of the pod have two degrees of freedom, and the attitude angle will directly affect the accuracy of the target direction estimation in the body coordinate system. Therefore, the unit direction vector extracted from the image coordinates needs to be compensated for the attitude angle to ensure the spatial consistency of the target direction.

[0146] As shown in Figure 9 , the yaw and pitch (Yaw / Pitch) rotation of the pod complies with the right-hand screw rule:

[0147] The rotation angle around the z-axis is θ, which is the yaw angle θ of the pod;

[0148] The rotation angle around the y-axis is ψ, which is the pitch angle ψ of the pod;

[0149] The rotation angle around the x-axis is φ, which is the roll angle φ of the pod.

[0150] Compensation around the z-axis (yaw angle θ):

[0151]

[0152] Compensation around the y-axis (pitch angle ψ):

[0153]

[0154] Compensation around the x-axis (roll φ):

[0155]

[0156] As shown in Figure 10 , when controlling the pod to track the target, only the yaw and pitch of the pod are controlled, so R x (-φ)=R x (0)=I3.

[0157] Then the total rotation matrix is:

[0158]

[0159] Vector in the pod coordinate system:

[0160] (2) Pod coordinate system → unmanned aerial vehicle body coordinate system

[0161] As shown in Figure 11 , the body coordinate system generally used by the flight control system complies with the NED (North-East-Down) inertial navigation system, and the body coordinate system is defined as follows:

[0162] x-axis: pointing to the front of the aircraft, controlling the roll angle (Roll) of the aircraft;

[0163] y-axis: points to the right of the aircraft, controls the pitch angle of the aircraft;

[0164] z-axis: points to the down of the aircraft, controls the yaw angle of the aircraft.

[0165] Similarly, the rotation conforms to the right-hand screw rule.

[0166] In order to convert the target direction vector from the pod coordinate system to the unmanned aerial vehicle body coordinate system, a transformation matrix between the two needs to be constructed. The coordinate axes of the pod in the system are defined in accordance with the body coordinate system adopted by the flight control (both are x forward, y right, and z downward), and the two coordinate systems are completely aligned in direction, so the transformation matrix is a unit matrix, and the conversion process can be simplified as an identity mapping:

[0167]

[0168] That is, the normalized direction vector of the target in the unmanned aerial vehicle body coordinate system is constructed. Subsequently, the vector direction is used to drive the execution body to complete the attitude adjustment, so as to realize the dynamic alignment and stable tracking of the unmanned aerial vehicle to the target direction.

[0169] S4, inputting the target feature into a neural network model to output a strike probability score and compare it with a dynamic threshold to generate a strike authorization instruction;

[0170] The neural network model of step S4 is a multi-layer perceptron, and the input features include:

[0171] At least five of the target category, confidence, continuous frame number, relative distance, flight height, flight speed, battery capacity, region code, task level, and image clarity.

[0172] To solve the problems of traditional strike decision relying on artificial rule making, response lag, subjective judgment, etc., the present application proposes an AI autonomous strike decision model based on neural network. The model learns a large number of annotated task scenarios and strike historical data, induces the key factors affecting strike judgment, and establishes a generalizable deep learning inference system to realize end-to-end fitting from perception data to decision instruction.

[0173] Technical effects:

[0174] Replace rule programming: the model learns from past decision experience and no longer relies on artificial static rule making;

[0175] Strong non-linear modeling capability: can handle complex interaction between input features and adapt to multiple scenarios;

[0176] High generalization: supports flexible adjustment of the model in different environments, targets, and states.

[0177] (1) Model input feature construction

[0178] The model takes each frame of task data as an input sample, extracts several core features related to the target attack value, including:

[0179] Ten relevant features are selected as input variables, including target category, target confidence, number of consecutive frames of target appearance, relative distance, flight height, flight speed, battery remaining capacity, region code, task level, and image clarity. After standardization, they are input into the neural network model.

[0180] (2) Model structure design

[0181] As shown in Figure 12 , the present application adopts a multi-layer perceptron (MLP) to construct an attack decision model, and the network structure is as follows:

[0182] Input layer: 10-dimensional task features;

[0183] Hidden layer: two layers of fully connected network, containing 64 and 32 neurons respectively, and the activation function is ReLU;

[0184] Output layer: Sigmoid function, outputting target attack probability score between 0 and 1.

[0185] (3) Model training and optimization

[0186] The training data includes:

[0187] Images + flight control + artificially annotated attack decision samples from real flight tasks;

[0188] Large-scale data automatically annotated from simulation platform (Gazebo + ROS environment)

[0189] During the training process, Binary Cross Entropy is used as the loss function, and Adam optimizer is used for iterative training. The final output probability value is used as the attack confidence score.

[0190] As shown in Figure 13 , the decision model training result can be seen. The fitting accuracy of the model trained using the MLP fully connected network can be as high as 97%, indicating that the network has good fitting effect on non-linear factors.

[0191] (4) Decision output logic

[0192] The system compares the score with the dynamic attack threshold:

[0193] If the score ≥ threshold value: determine that the target can be attacked, generate attack authorization instructions, and switch the UAV guidance mode (Guide);

[0194] If the score < threshold value: continue to track the target, and do not trigger the attack.

[0195] S5, based on the image pixel offset, generate a gondola yaw / pitch angle velocity instruction to drive the gondola to lock the target; based on the deviation of the body coordinate system direction vector, generate a UAV attitude angle instruction to drive the body to dynamically align the target;

[0196] In step S5, the gondola control is achieved by the following way:

[0197] Calculate the pixel deviation vector of the target center and the image center, and input the PID controller to generate the angular velocity instruction;

[0198] The flight control in step S5 is achieved by the following way:

[0199] According to the yaw angle and pitch angle deviation of the body coordinate system direction vector, input the PID controller to generate the attitude angle instruction.

[0200] As shown in Figure 14 To achieve stable tracking and high-precision attack on the enemy target, a gondola-flight control double-layer collaborative fusion control algorithm is proposed. The algorithm constructs a two-level control mechanism: the first-level control is responsible for real-time adjustment of the gondola attitude under visual error, and the second-level control is responsible for correction of the UAV body attitude and adjustment of the flight trajectory under target direction guidance, so as to realize the precise attack closed loop in the three-dimensional dynamic scene.

[0201] As shown in Figure 15 When the attack decision of the attack decision model is obtained, the double-layer fusion control algorithm in the control link coordinates the two independent but complementary PID control loops of the gondola and the flight control. The two-level controls are mutually compensated, the gondola provides fast small-angle compensation, and the UAV body provides large-range heading adjustment, so as to realize the high efficiency and precision of target tracking and attack. The two PID controllers run independently to improve the system robustness. Specifically:

[0202] As shown in Figure 16 The first layer PID control (gondola layer) is based on the pixel offset of the target in the image, constructs the error vector between the target and the image center, and inputs the error into the gimbal control PID loop. The system adjusts the yaw angle and pitch angle of the gimbal in real time to make the camera field of view continuously align with the target, realizes stable visual lock and autonomous fine tracking of the target, and provides high-precision perception support for subsequent navigation and attack tasks.

[0203] The target center point coordinates are denoted as (u, v), and the image center point is denoted as (c x ,cy ), construct error vector e img

[0204]

[0205] The error vector reflects the degree of pixel deviation of the target from the image center, wherein e u For driving the yaw angle adjustment of the gimbal, e v For driving the pitch angle correction. The error is input into the PID controller to generate the angular velocity command, generating:

[0206]

[0207] ω gimbal-yaw And ω gimbal-pitch The target angular velocity command is issued to the pod driving module through the UDP control protocol to realize the rapid and stable alignment of the image to the target.

[0208] As shown in Figure 17 , the second layer PID control (flight control layer) calculates the angle error between the current flight attitude and the target direction based on the directional vector deviation of the target in the unmanned aerial vehicle body coordinate system. The error is dynamically adjusted through the attitude control PID loop to generate the expected attitude angle command of the unmanned aerial vehicle, drive the flight control system to control the attitude angle of the unmanned aerial vehicle body to dynamically adjust, thereby realizing the continuous and stable tracking of the target and the adaptive optimization of the flight path.

[0209] The converted target unit directional vector in the unmanned aerial vehicle body coordinate system is:

[0210]

[0211] Construct the directional error index:

[0212]

[0213] Wherein:

[0214]

[0215] The e vec Error vector reflects the angle deviation between the target direction and the current head direction of the unmanned aerial vehicle, wherein e yaw For driving the yaw attitude control of the unmanned aerial vehicle, e pitch For driving the pitch attitude adjustment. Input it into the PID controller to generate:

[0216]

[0217] S6, in the strike authorization state, control the unmanned aerial vehicle to complete the accurate strike on the target.

[0218] Wherein, the strike execution of step S6 satisfies:

[0219] The gondola layer control and the flight control layer control run in parallel, the gondola layer executes high-frequency small-angle correction, and the flight control layer executes large-range heading adjustment, forming a cooperative tracking closed loop.

[0220] Finally, the calculated theta aircraft-yaw and theta aircraft-pitch The target attitude angle instruction is sent to the flight control platform through the ROS+MAVROS communication mechanism, and the rudder surface adjustment is automatically completed by the built-in controller of the flight control, so as to adjust the attitude of the unmanned aerial vehicle body to align with the target direction, thereby realizing dynamic alignment and stable tracking of the unmanned aerial vehicle to the target direction.

[0221] Through the organic combination of the two control loops, the algorithm realizes hierarchical utilization of visual information and dynamic fusion of control actions, improves the robustness and response speed of the unmanned aerial vehicle target capture, and guarantees the stability and precision of the system in complex environments.

[0222] Compared with the prior art, the present application has the following technical effects:

[0223] Autonomous closed-loop control: a five-in-one closed loop of "discovery-identification-tracking-decision-strike" is constructed to realize autonomous operation in the whole process.

[0224] Edge intelligent deployment: high-precision identification and intelligent judgment are completed through the edge platform, without relying on a remote ground station, and the delay is significantly reduced.

[0225] Multi-coordinate system coupling modeling: a multi-coordinate solving link based on images, gondolas and unmanned aerial vehicle attitudes is proposed to improve the target pointing and path control precision.

[0226] Nonlinear intelligent strike model: a neural network model is used to fit the strike strategy, and the traditional rule-based strike logic is converted into intelligent research and decision-making to adapt to actual combat environments.

[0227] Gondola flight control double-layer cooperative control algorithm: the gondola layer controls the gondola angular velocity for visual locking through image pixel deviation, and the flight control layer controls the unmanned aerial vehicle body attitude angle for dynamic adjustment according to the vector deviation in the body coordinate system of the target interpretation model, so that the unmanned aerial vehicle dynamically aligns and stably tracks the target.

[0228] As shown in Figure 21 The embodiment of the present application also provides an unmanned aerial vehicle autonomous target identification and strike device, which comprises:

[0229] A high-dynamic target identification module 10 is used for real-time processing of video streams collected by an optical gondola through an airborne edge computing platform, detecting dynamic targets in images by using a lightweight deep learning model, and outputting the target center coordinates with the highest confidence.

[0230] A single-target stable tracking module 20 is configured to make a persistence determination on the target in consecutive frames based on a timing verification mechanism, filter out an optimal tracking target, and output stable coordinates of the target;

[0231] A multi-coordinate system coupling positioning module 30 is configured to convert two-dimensional image coordinates of the target into a three-dimensional direction vector in a UAV body coordinate system through a four-stage coordinate system conversion chain in real time.

[0232] An intelligent attack decision module 40 is configured to input target features into a neural network model, output an attack probability score, compare the attack probability score with a dynamic threshold, and generate an attack authorization instruction.

[0233] A double-layer cooperative control module 50 is configured to generate a pod yaw / pitch angle velocity instruction based on image pixel offset, drive the pod to visually lock the target, and generate a UAV attitude angle instruction based on a body coordinate system direction vector deviation, drive the body to dynamically align the target.

[0234] An autonomous attack execution module 60 is configured to control the UAV to complete accurate attack on the target in an attack authorization state.

[0235] It should be noted that the specific implementation process of the UAV autonomous target recognition and attack device can be clearly understood by those skilled in the art, and can refer to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be described here.

[0236] The UAV autonomous target recognition and attack device can be implemented in the form of a computer program, which can run on a computer device as shown in Figure 22 .

[0237] Please refer to Figure 22 , Figure 22 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a terminal or a server, wherein the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, a wearable device, and an electronic device with a communication function. The server can be a stand-alone server or a server cluster composed of multiple servers.

[0238] Referring to Figure 22 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0239] The nonvolatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions which, when executed, can cause the processor 502 to perform the method of autonomous target identification and attack of a UAV.

[0240] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0241] The nonvolatile storage medium 503 provides an environment for the computer program 5032 stored therein to be executed by the processor 502, which can cause the processor 502 to perform the method of autonomous target identification and attack of a UAV.

[0242] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that the network interface 505 can be configured to perform wired or wireless communication. Figure 22 The structure shown in FIG. 5 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. Specifically, the computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0243] The processor 502 is configured to execute the computer program 5032 stored in the memory to implement the method of autonomous target identification and attack of a UAV as described above.

[0244] It should be understood that, in the embodiments of the present application, the processor 502 can be a central processing unit (CPU), and the processor 502 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0245] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiments of the method can be completed by a computer program instructing related hardware. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the above-described embodiments of the method.

[0246] Therefore, the present application also provides a storage medium. The storage medium can be a computer readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. The program instructions are executed by a processor to enable the processor to perform the method for autonomous target identification and attack of a UAV as described above.

[0247] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0248] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0249] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0250] The steps in the method embodiments of the present application can be adjusted, combined and reduced in sequence according to actual needs. The units in the device embodiments of the present application can be combined, divided and reduced according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0251] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for enabling a computer device (which can be a personal computer, a terminal or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.

[0252] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for autonomous target recognition and attack by a UAV, characterized in that: The following steps are involved: S1. The airborne edge computing platform processes the video stream collected by the optoelectronic pod in real time, uses a lightweight deep learning model to detect dynamic targets in the image, and outputs the target center coordinates with the highest confidence. S2: Based on the timing verification mechanism, the target in the continuous frame is continuously judged, the optimal tracking target is selected and its stable coordinates are output; S3, the two-dimensional image coordinates of the target are converted into three-dimensional direction vectors in the UAV body coordinate system in real time through a four-level coordinate system transformation chain; S4: Extract target features and input them into the neural network model, output a strike probability score, compare it with the dynamic threshold, and generate a strike authorization instruction; S5. Generate yaw / pitch angular velocity instructions for the pod based on the image pixel offset to drive the pod to visually lock onto the target; generate attitude angle instructions for the drone based on the body coordinate system direction vector deviation to drive the drone to dynamically align with the target; S6. Under the strike authorization status, control the UAV to carry out precise strikes on the target.

2. The autonomous target recognition and attack method of a UAV according to claim 1, characterized in that: The lightweight deep learning model in step S1 is constructed in the following way: A high-resolution feature layer is added to the YOLO architecture to enhance the ability to detect small targets, deep redundant semantic layers are pruned to compress the amount of computation, and the backbone network is replaced with a lightweight convolutional network.

3. The autonomous target recognition and attack method of UAV according to claim 1, characterized in that: The timing verification mechanism of step S2 includes: The continuous frame number threshold verification is started for the first appearance of the target, the delayed confirmation counter is started for the target ID switch, the reinitialization process is started for the target loss, and the dual-threshold fault tolerance mechanism is used to suppress false detection interference.

4. The autonomous target recognition and attack method of a UAV according to claim 1, characterized in that: The four-level coordinate system transformation chain of step S3 is executed in sequence: Space vector transformation of image coordinate system, camera coordinate system, pod coordinate system, and drone body coordinate system, where: The direction vector from the image coordinate system to the camera coordinate system is normalized by the focal length parameter; The camera coordinate system to the pod coordinate system is compensated by the pod attitude angle rotation matrix; The pod coordinate system is transformed into the body coordinate system through coordinate system alignment mapping.

5. The autonomous target recognition and attack method of a UAV according to claim 1, characterized in that: The neural network model in step S4 is a multi-layer perceptron, and the input features include: At least five of the following: target category, confidence, continuous frame number, relative distance, flight altitude, flight speed, battery capacity, area coding, mission level, and image clarity.

6. The autonomous target recognition and attack method of a UAV according to claim 1, characterized in that: The pod control in step S5 is achieved by: Calculate the pixel deviation vector between the target center and the image center and input it into the PID controller to generate the angular velocity command; The flight control in step S5 is achieved by: The yaw angle and pitch angle deviation are calculated according to the direction vector of the body coordinate system, and input into the PID controller to generate the attitude angle command.

7. The autonomous target recognition and attack method of a UAV according to claim 1, characterized in that: The attack execution of step S6 satisfies: The pod layer control and the flight control layer control run in parallel. The pod layer performs high-frequency small-angle corrections, and the flight control layer performs large-scale heading adjustments, forming a collaborative tracking closed loop.

8. An autonomous target recognition and attack device for unmanned aerial vehicles, characterized in that: include: A high-dynamic target recognition module, which processes the video stream collected by the optoelectronic pod in real time through an onboard edge computing platform. It uses a lightweight deep learning model to detect dynamic targets in the image and outputs the target center coordinates with the highest confidence. The single target stable tracking module is used to perform continuous target determination in consecutive frames based on a timing verification mechanism, select the optimal tracking target and output its stable coordinates; The multi-coordinate system coupling positioning module is used to convert the target's two-dimensional image coordinates into three-dimensional direction vectors in the UAV body coordinate system in real time through a four-level coordinate system transformation chain; The intelligent strike decision module is used to extract target features and input them into the neural network model, output a strike probability score, compare it with the dynamic threshold, and generate a strike authorization instruction; The dual-layer collaborative control module is used to generate pod yaw / pitch velocity commands based on image pixel offsets, driving the pod to visually lock onto the target; and to generate drone attitude angle commands based on the body coordinate system direction vector deviation, driving the body to dynamically align with the target; The autonomous strike execution module is used to control the UAV to complete precise strikes on targets under the strike authorization state.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the autonomous target recognition and attack method of the unmanned aerial vehicle according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, can implement the autonomous target recognition and attack method for a drone according to any one of claims 1 to 7.

Images