Target recognition system and method for image / television guided aircraft with stable attitude and image
By integrating electronic stable image and YOLO algorithm models on image/TV guided aircraft, combining particle filtering submodules and damping loops, the problems of aircraft posture jitter and target loss are solved, and stable posture stable image and accurate target recognition are achieved, which improves the System stability and recognition accuracy.
Patent Information
- Application Number
- CN202210391446.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-04-14
AI Technical Summary
During the flight, existing image/TV guided aircraft have strong posture jitter due to factors such as aerodynamic disturbances, model uncertainty and target maneuverability, and the phenomenon of target loss occurs from time to time. It lacks autonomous target recognition and locking functions, making it difficult to achieve stable performance. Stable posture.
A stable-position stable-image target recognition system on an image/TV guided aircraft was designed, and images were collected in real time through the TV camera, and motion compensation and target detection were performed using electronic stable-image technology and YOLO algorithm model, combining particle filtering submodule and damping loop , realize the stability of the aircraft attitude and the precise identification and tracking of targets.
It realizes stable posture and image image sequences on high-speed motion platforms, improves the accuracy of target recognition and system stability, meets the real-time detection needs in complex environments, and reduces the risk of targets leaving the field of view.
Smart Images

Figure CN114967715B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of aircraft guidance and control, and in particular to an image / television-guided target recognition and aircraft control solution. Background Art
[0002] "Intelligent" image / TV guided aircraft should have a series of functions such as autonomous target identification, locking, guidance, control, etc. However, at present, image / TV guided aircraft still have people in the loop and do not have the function of autonomous target identification and locking; considering the influence of factors such as aerodynamic disturbances, model uncertainty, and target maneuverability, during the flight, the aircraft attitude jitters strongly and the target is lost from time to time, which brings severe challenges to the autonomous target identification of image / TV guided aircraft.
[0003] Based on the above problems, the inventors have conducted in-depth research on image / television guided aircraft in the hope of designing a target recognition system and method for image / television guided aircraft with stable attitude and image that can solve the above problems. Summary of the invention
[0004] In order to overcome the above-mentioned problems, the inventors have conducted intensive research and designed a target recognition system and method for image / television-guided aircraft with stable attitude and image. In the system, a television camera is used to collect a target area image sequence in real time, and electronic image stabilization technology is used to achieve high-precision image stabilization of motion-shifted image sequences. The damping loop component design improves the attitude stability of the image / television-guided aircraft. The processed image sequence is input into a deep learning algorithm model to obtain real-time, effective and accurate target location information, and autonomous target detection and tracking are performed, thereby achieving accurate recognition and tracking of the target, thereby completing the present invention.
[0005] Specifically, the object of the present invention is to provide a target recognition system for stabilizing attitude and image on an image / television guided aircraft, characterized in that the system comprises:
[0006] Image processing module 1 and guidance control module 2,
[0007] The image processing module 1 is used to identify and lock the target, obtain the relative position information between the target and the aircraft,
[0008] The guidance control module 2 is used to adjust the attitude of the aircraft through the steering gear so that the aircraft can fly smoothly towards the target.
[0009] The image processing module 1 includes a television camera 11, an electronic image stabilization submodule 12, an image guidance head 13 and a particle filter submodule 14;
[0010] Wherein, the television camera 11 is used to collect the target area image in real time;
[0011] The electronic image stabilization submodule 12 performs motion compensation on the image signal input by the television camera 11 to stabilize the image sequence;
[0012] The image seeker 13 uses the YOLO algorithm model to detect and identify the image, thereby locking the target;
[0013] The particle filter submodule 14 is used to obtain the distance r, the sight line deflection angle θ and the sight line inclination angle φ of the aircraft relative to the target according to the position information of the target in the image provided by the image seeker 13 .
[0014] Wherein, the YOLO algorithm model includes a feature extraction unit 131, a feature fusion unit 132 and a detection output unit 133;
[0015] The feature extraction unit 131 extracts feature information from the image.
[0016] The feature fusion unit 132 fuses the extracted feature information into feature maps of three scales.
[0017] The detection output unit 133 detects the target position using the feature map to obtain target position information.
[0018] The guidance control module 2 includes an inertial guidance submodule 21, a microprocessor 22 and a steering gear 23;
[0019] The inertial navigation module 21 is used to obtain the attitude information of the aircraft in real time.
[0020] The microprocessor 22 obtains the required overload in real time based on the relative position information between the target and the aircraft and the attitude information of the aircraft, and further obtains the rudder instruction;
[0021] The steering is performed by the steering engine 23 according to the rudder instruction.
[0022] Wherein, a damping circuit is also provided in the guidance control module 2, and the stability of the aircraft is improved by the damping circuit.
[0023] The damping loop is adjusted in real time according to the desired damping μ d Gain K g ;
[0024] The required overload obtained by the microprocessor 22 and the gain K g The corrected required overload is obtained by superposition, and a rudder command is generated based on the corrected required overload.
[0025] Among them, the gain K g Obtained by the following formula:
[0026]
[0027] Medium, μ d represents the expected damping;
[0028] T m represents the inherent time constant of the projectile;
[0029] μ m It represents the inherent damping of the projectile;
[0030] T i represents the attitude time constant;
[0031] k s Indicates the servo gain;
[0032] represents the attitude angular rate gain;
[0033] A represents the intermediate variable, which is obtained by the following formula:
[0034] A=μ d 2 T i 2 +μ d 4 T m 2 -2T m μ m T i μ d 2 -T i 2 (μ m 2 -μ d 2 ).
[0035] The present invention also provides a method for target recognition with stable posture and image on an image / television guided aircraft, the method comprising the following steps:
[0036] Step 1, collecting the target area image in real time through the TV camera 11;
[0037] Step 2, performing motion compensation on the image signal input by the television camera 11 through the electronic image stabilization submodule 12 to stabilize the image sequence;
[0038] Step 3, detecting and identifying the image by using the YOLO algorithm model, thereby locking the target;
[0039] Step 4: Obtain the distance r, sight line deflection angle θ and sight line inclination angle φ of the aircraft relative to the target through the particle filter submodule 14.
[0040] Step 5, obtaining the attitude information of the aircraft through the inertial navigation module 21,
[0041] Step 6, obtaining the required overload in real time through the microprocessor 22,
[0042] Step 7, generate a steering command through the steering engine 23 and execute it.
[0043] In step 6, after obtaining the required overload, the microprocessor 22 adds the required overload to the gain K obtained by the damping circuit. g The corrected required overload is obtained by superposition, so that the steering engine generates a rudder instruction based on the corrected required overload and executes it.
[0044] The beneficial effects of the present invention include:
[0045] (1) According to the target recognition system for stabilizing the attitude and image on the image / television guided aircraft provided by the present invention, in order to solve the problem that when the television camera is mounted on a high-speed moving platform, the acquired image sequence will change rapidly with the platform, an electronic image stabilization submodule in the image processing module and a damping circuit in the guidance and control component are added, thereby achieving stabilization of the attitude and image of the motion-shifted image sequence.
[0046] (2) According to the target recognition system for stabilizing attitude and image on the image / TV-guided aircraft provided by the present invention, the image seeker is loaded with a YOLO algorithm model to solve the problem that it is difficult for the image seeker to recognize the target during the strapdown image guidance process: compared with the traditional target recognition and detection methods, which are only applicable to situations with obvious abstract features and simple backgrounds and cannot be applied to complex actual environments, the YOLO algorithm model is subjected to deep learning to extract rich features of the same target, so that the algorithm has stronger generalization ability and is easier to be applied to actual scenes. The YOLO algorithm model has high detection accuracy and can also ensure the model detection speed, meeting the requirements of real-time detection;
[0047] (3) According to the target recognition system for stabilizing attitude and image on the image / television guided aircraft provided by the present invention, the relative position information of the target and the aircraft is obtained through a particle filter submodule. Compared with the Kalman filter and the H∞ filter, the particle filter submodule has relatively minimum requirements on the system, and does not require the system model equation to be linear, nor does it require the system noise to be Gaussian distributed, and has better algorithm scalability and universality. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 The overall logic diagram of the target recognition system for stabilizing attitude and image on an image / television guided aircraft according to a preferred embodiment of the present invention is shown;
[0049] Figure 2A curve diagram showing the variation of accuracy in an embodiment of the present invention is shown;
[0050] Figure 3 A curve diagram showing a change in recall rate in an embodiment of the present invention is shown;
[0051] Figure 4 A curve diagram showing a change in the average precision mean in an embodiment of the present invention is shown;
[0052] Figure 5a Shows how the bounding box loss of the training set changes with the number of training rounds;
[0053] Figure 5b Shows how the target loss of the training set changes with the number of training rounds;
[0054] Figure 5c Shows how the validation set bounding box loss changes with training epochs;
[0055] Figure 5d Shows how the validation set target loss changes with the number of training rounds;
[0056] Figure 6a Shows how the accuracy of detecting targets changes with the number of training rounds;
[0057] Figure 6b Shows how the recall changes with the number of training rounds;
[0058] Figure 6c Shows how map_0.5 changes with training rounds;
[0059] Figure 6d Shows how mAP_0.5:0.95 changes with the number of training rounds;
[0060] Figure 7 A schematic diagram showing the target capture result in an embodiment of the present invention;
[0061] Figure 8 An aircraft trajectory diagram is shown in an embodiment of the present invention;
[0062] Fig. 9 A curve diagram showing a change in pitch angle rate in Comparative Example 1 of the present invention is shown;
[0063] Fig.10 A curve diagram showing the change of overload instructions in Comparative Example 1 of the present invention is shown;
[0064] Fig.11 A curve diagram showing the change of overload instructions in Comparative Example 1 of the present invention is shown;
[0065] Fig.12 The trajectory diagrams of the aircraft in Example 2 of the present invention and Comparative Example 2 are shown.
[0066] 1- Image processing module
[0067] 11-TV Camera
[0068] 12-Electronic image stabilization module
[0069] 13-Image Seeker
[0070] 131-Feature Extraction Department
[0071] 132-Feature Fusion Department
[0072] 133-Detection output unit
[0073] 14-Particle filter submodule
[0074] 2-Guidance Control Module
[0075] 21-Inertial navigation module
[0076] 22-Microprocessor
[0077] 23-Servo DETAILED DESCRIPTION
[0078] The present invention will be further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present invention will become more clear and distinct.
[0079] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.
[0080] According to the present invention, the target recognition system for stabilizing posture and image on the image / television guided aircraft is as follows: Figure 1 As shown in , the system includes: an image processing module 1 and a guidance control module 2, wherein the image processing module 1 is used to identify and lock the target, and obtain the relative position information between the target and the aircraft in real time; the guidance control module 2 is used to adjust the attitude of the aircraft through the servo, so that the aircraft flies smoothly to the target. Since the guidance control module 2 can make the aircraft more stable during flight, it ensures that the target position obtained by the image processing module 1 remains stable, that is, the image sequence is output stably, and the occurrence of unexpected situations such as the target leaving the field of view due to factors such as jitter is reduced, ensuring that the aircraft flies smoothly and continuously to the target and finally collides with the target.
[0081] In the present application, the relative position information between the target and the aircraft includes the distance r of the aircraft relative to the target, the line of sight deflection angle θ and the line of sight inclination angle φ.
[0082] In a preferred embodiment, the image processing module 1 includes a television camera 11, an electronic image stabilization submodule 12, an image seeker 13 and a particle filter submodule 14;
[0083] The TV camera 11 is used to collect the target area image in real time. The target area mentioned in this application refers to the approximate area where the target is located, which is generally a spatial area with a radius of 100 meters centered on the target. When the TV camera is just started, it scans all visible areas and obtains images, and sends the images to the image seeker 13 for motion compensation. After the image seeker 13 finds the target in the image, it feeds back the acquisition command to the TV camera 11, and the TV camera continues to collect the target area image.
[0084] The electronic image stabilization submodule 12 performs motion compensation on the image signal input by the TV camera 11 to stabilize the image sequence; the image sequence in this application refers to a sequence of pictures collected by the TV camera in the target area frame by frame. In the electronic image stabilization submodule 12, the motion vector of the TV camera is estimated by using the difference in image content between frames, so as to perform motion compensation. For the specific operation process of the motion compensation, please refer to Wang Zhimin, Xu Xiaogang. Overview of Electronic Image Stabilization Technology [J]. Journal of Image and Graphics, 2010, 15(03): 470-480.
[0085] The image seeker 13 uses the YOLO algorithm model to detect and identify the image obtained by the TV camera 11, thereby locking the target. The YOLO algorithm model can read each frame of the image obtained, and quickly determine whether there is a target in each frame of the image and the position of the target in the image.
[0086] In a preferred embodiment, the YOLO algorithm model includes a feature extraction unit 131, a feature fusion unit 132 and a detection output unit 133. Preferably, the feature extraction unit is a Backbone feature extraction unit, the feature fusion unit is a Neck feature fusion unit, and the detection output unit is a Prediction detection output unit.
[0087] Preferably, the feature extraction unit 131 extracts feature information from the image.
[0088] The feature fusion unit 132 fuses the extracted feature information into feature maps of three scales.
[0089] The target position is detected by the feature map through the detection output unit 133 to obtain the target position information. An input end is also provided in the YOLO algorithm model of the present application; the input end processes the image through operations such as Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling, thereby enhancing the detection effect and improving the model calculation speed; the Backbone feature extraction unit in the present application uses the Focus structure to slice the input image, and applies the CSP1_X network structure to extract feature information from the input image; the Neck feature fusion unit uses the FPN feature pyramid network combined with the PAN structure to fuse the extracted feature information: the FPN is top-down, and the high-level feature information is transmitted and fused by upsampling. On this basis, two PAN structures are used for downsampling to form a bottom-up feature pyramid, thereby generating feature maps of three scales. In this application, the FPN layer conveys strong semantic features from top to bottom, and the feature pyramid conveys strong positioning features from bottom to top. Parameters of different detection layers are aggregated from different backbone layers to achieve better feature fusion. The Prediction detection output unit needs to perform post-processing of target detection, use CIOU_Loss as the loss function of the Bounding box, perform non-maximum suppression nms operation for the screening of many target boxes, and finally output multi-scale feature maps and relevant indicators of the detection results such as confidence, loss value, accuracy and recall rate.
[0090] In a preferred embodiment, the feature extraction unit 131 in the YOLO algorithm model includes a Focus structure, and the original 608*608*3 image is input into the Focus structure, and a slicing operation is adopted to first convert it into a 304*304*12 feature map, and then a convolution operation of 32 convolution kernels is performed to finally convert it into a 304*304*32 feature map; the feature extraction unit 131 also includes four CBL components, three CSP1_X structures and an SPP spatial pyramid pooling layer, and the CBL component is composed of a Conv+Bn+Leaky_relu activation function;
[0091] The feature fusion unit 132 includes four tensor concatenation layers and two upsampling layers; it also includes four CBL components and five CSP2_X structure layers;
[0092] The detection output unit 133 includes three convolutional layers, which are used to output multi-scale feature maps of 76×76×255, 38×38×255 and 19×19×255 respectively.
[0093] The CSP1_X structure and CSP2_X described in this application are both designed with CSP structures in the backbone network. CSP1_X consists of a convolutional layer and X Res unit (residual modules) Concate (tensor splicing); CSP2_X consists of a convolutional layer and 2X Resunit modules Concate (tensor splicing). For details, please refer to Wang CY, Liao H, Wu YH, et al. CSPNet: A New Backbone that can Enhance Learning Capability of CNN [C] / / 2020 IEEE / CVFConference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2020.
[0094] In a preferred embodiment, the YOLO algorithm model is injected into the image seeker before the aircraft takes off, and the training process of the YOLO algorithm model includes the following steps:
[0095] Step a, obtain data samples and preprocess the data samples; specifically, use a drone equipped with a TV camera to collect 1000 256×256 pixel images containing target objects and backgrounds on site to form a database, and use the labelimg tool to preprocess the database image samples, and the training set, validation set, and test set are divided in a ratio of 6:3:1.
[0096] Step b, train the YOLO algorithm model with sample data; preferably, it is installed on the deep learning framework PyTorch and runs in the integrated development environment PyCharm. Specifically, the parameters must be configured first, and the training batch size is set to 8. The larger the dimension, the higher the model learning rate, but the larger the memory usage, and the higher the performance required of the device. Therefore, if the hardware conditions permit, adjust the batch size as large as possible. The training batch described in this application refers to a group of 8 data, and the number of pictures for each network training is 8.
[0097] The training round is 100 epochs, and the training round refers to the number of training iterations.
[0098] During training, each image in the dataset is traversed. After each round of training, information such as round number, video memory usage, and loss function value are output in the command line.
[0099] During training, the loss function is IoU stands for intersection over union, which reflects the detection effect of the predicted detection box and the real detection box; Represents the metric of the similarity of the detection box aspect ratio, which is used to measure the similarity of the aspect ratio, w gt Indicates the width of the real detection box, h gt represents the height of the real detection box, w represents the width of the predicted detection box, and h represents the height of the predicted detection box; α is the weight function; b, b gt Represent the center points of the predicted box and the real box respectively; ρ represents the Euclidean distance between the two center points.
[0100] After the training set is trained, the validation set is used to output the training model's indicator results such as precision, recall, mAP, and mAP@0.5.
[0101] Among them, the evaluation indicators of the YOLO algorithm model are measured by accuracy, recall and average precision, where the accuracy is obtained by the following formula:
[0102]
[0103] The recall rate is obtained by the following formula:
[0104]
[0105] Among them, Precision refers to the accuracy rate; it indicates the proportion of real target targets contained in the samples detected as target targets, reflecting the accuracy of the detection results;
[0106] Recall stands for recall rate; it indicates the proportion of correctly detected targets in all target image samples, reflecting whether the detected targets are complete;
[0107] TP indicates the number of verification samples whose targets are true targets and whose detection results are also true targets;
[0108] FP means the target is not the real target, and the number of verification samples whose detection results are the real target;
[0109] FN indicates the number of validation samples where the target is the true target and the detection result is not the true target;
[0110] The precision mean AP is the area of the curve enclosed by the precision rate Precision and the recall rate Recall; the PR curve is a two-dimensional curve with the precision rate Precision as the horizontal axis and the recall rate Recall as the vertical axis. There is a restrictive relationship between the precision rate Precision and the recall rate Recall. The values of the precision rate and the recall rate can be adjusted by changing the detection threshold. The change of the threshold will obtain different combinations of Precision and Recall, thereby drawing the PR curve. The PR curve reflects the change in the relationship between different recall rates and the maximum precision rate under the corresponding recall rate.
[0111] The mean average precision (mAP) is the average of the mean precision AP of the target category being learned.
[0112] mAP@0.5 means that when the intersection-over-union (IOU) is set to 0.5, the AP of all image datasets of each category is calculated, and then the average of all APs is calculated.
[0113] Step 3: After the set 100 epochs are trained, the training is completed and the training result of the model is obtained. The YOLO algorithm model obtained at this time can be loaded into the image guidance head 13.
[0114] In a preferred embodiment, the particle filter submodule 14 is used to obtain the distance r, the sight line deflection angle θ and the sight line inclination angle φ of the aircraft relative to the target according to the position information of the target in the image provided by the image seeker 13 .
[0115] The particle filter (PF) described in the present application is a Bayesian filter algorithm based on Monte Carlo simulation, which estimates the identified parameters by calculating the sample mean of a particle set, and adopts an approximate estimation method of particle sampling simulation for the state space at any time.
[0116] In this application, the process of obtaining the distance r of the aircraft relative to the target, the sight line deflection angle θ and the sight line inclination angle φ specifically includes the following steps:
[0117] (1) Input the state of the target at time k0 and the target posterior probability distribution function. The posterior probability at time k0 is the initial state at time k0.
[0118] (2) Determine the target state transfer function and observation model, and initialize the particle swarm;
[0119] (3) Resampling: The particle effectiveness sampling coefficient N is obtained by the following formula eff :
[0120]
[0121] (4) Set the validity sampling threshold N th , if it is less than a given threshold N th :N eff <N th , then resampling obtains a new particle set
[0122] (5) Use importance sampling from the proposed distribution function to approximate the posterior probability density distribution and recursively calculate the particle weight at time k;
[0123] (6) Normalized particle weights;
[0124] (7) Obtain an approximate estimate of the posterior probability distribution of the target state k at the current moment:
[0125]
[0126] In the formula N k Denote the particle state, weight and total number at time k, respectively, and δ(·) is the Dirac delta function. Through the above seven steps, the target tracking problem based on particle filtering can be realized. For details, please refer to Jiang Tao, Fan Bin, Qin Wutao. Trajectory tracking method based on genetic resampling particle filter [J]. Modern Radar, 2022, 44(2): 58-63.
[0127] In a preferred embodiment, the guidance control module 2 includes an inertial guidance submodule 21, a microprocessor 22 and a steering gear 23;
[0128] The inertial navigation module 21 is used to obtain the attitude information of the aircraft in real time. The inertial navigation module 21 includes at least two MEMS gyroscopes, which respectively measure the pitch angle of the aircraft. and yaw angle ψ, thereby sensing and acquiring the aircraft’s attitude information in real time.
[0129] The microprocessor 22 obtains the required overload in real time based on the relative position information between the target and the aircraft and the attitude information of the aircraft, and further obtains the rudder instruction; the rudder instruction is the steering angle and steering time of the servo.
[0130] The steering is performed according to the rudder command by the steering gear 23. Preferably, an amplifier converter is also provided in the steering gear, through which the rudder command signal output by the microprocessor is amplified and converted into a signal form required by the steering gear. The steering gear can be a cold air steering gear, which uses high-pressure air or helium stored in a high-pressure cold air bottle as an air source to control the movement of the rudder surface.
[0131] In a preferred embodiment, a damping circuit is further provided in the guidance control module 2, and the stability of the aircraft is improved by the damping circuit.
[0132] Preferably, the damping circuit is adjusted in real time according to the desired damping μ d Gain K g ;
[0133] The required overload obtained by the microprocessor 22 and the angular rate feedback The corrected required overload is obtained by superposition, and a rudder command is generated based on the corrected required overload.
[0134] In the UAV, the microprocessor 22 can generate the required overload through proportional guidance, such as
[0135] Preferably, the gain K g Obtained by the following formula:
[0136]
[0137] Among them, μ d Indicates the desired damping; its value is preferably 0.5, which can take into account both response speed and posture stability.
[0138] T m represents the inherent time constant of the projectile;
[0139] μ m It represents the inherent damping of the projectile;
[0140] T i represents the attitude time constant;
[0141] k s Indicates the servo gain, which is calculated through the missile body parameters and aerodynamic parameters;
[0142] It represents the attitude angular rate gain, which is calculated by the missile body parameters and aerodynamic parameters;
[0143] A represents the intermediate variable, which is obtained by the following formula:
[0144] A=μ d 2 T i 2 +μ d 4 T m 2 -2T m μ m T i μ d 2 -T i 2 (μ m 2 -μ d2 ).
[0145] The present application also provides a method for target recognition with stable attitude and image on an image / television guided aircraft, the method comprising the following steps:
[0146] Step 1, collecting the target area image in real time through the TV camera 11;
[0147] Step 2, performing motion compensation on the image signal input by the television camera 11 through the electronic image stabilization submodule 12 to stabilize the image sequence; wherein the motion vector of the television camera is estimated by using the difference in image content between frames, thereby performing motion compensation;
[0148] Step 3, detecting and identifying the image by using the YOLO algorithm model, thereby locking the target;
[0149] Step 4: Obtain the distance r, sight line deflection angle θ and sight line inclination angle φ of the aircraft relative to the target through the particle filter submodule 14.
[0150] Step 5, obtaining the attitude information of the aircraft through the inertial navigation module 21,
[0151] Step 6, obtaining the required overload in real time through the microprocessor 22,
[0152] Preferably, the microprocessor obtains the required overload by the following formula: in is the line of sight angular velocity, N1 and N2 are proportional coefficients, V is the aircraft speed, Projectile pitch angle, is the desired pitch angle.
[0153] In step 6, after obtaining the required overload, the microprocessor 22 adds the required overload to the gain K obtained by the damping circuit. g The superposition obtains the correction required for overload;
[0154] Further preferably, a damping circuit is also provided on the aircraft, and the damping circuit obtains the gain K in real time through the following formula: g :
[0155]
[0156] Among them, μ d Indicates the desired damping; its value is preferably 0.5;
[0157] T m represents the inherent time constant of the projectile;
[0158] μ m It represents the inherent damping of the projectile;
[0159] T i represents the attitude time constant;
[0160] k s Indicates the servo gain;
[0161] represents the attitude angular rate gain;
[0162] A represents the intermediate variable, which is obtained by the following formula:
[0163] A=μ d 2 T i 2 +μ d 4 T m 2 -2T m μ m T i μ d 2 -T i 2 (μ m 2 -μ d 2 )
[0164] The gain K obtained by the damping loop in real time g Superimpose with the demand overload in real time to obtain the corrected demand overload;
[0165] Step 7, generating a rudder instruction based on the corrected required overload by the servo 23, controlling the servo to steer by the rudder instruction, adjusting the flight attitude of the aircraft, and flying toward the target.
[0166] Embodiment 1:
[0167] Train the YOLO algorithm model on the deep learning framework PyTorch:
[0168] The YOLO algorithm model includes a Backbone feature extraction unit, a Neck feature fusion unit, and a Prediction detection output unit;
[0169] The Backbone feature extraction unit includes a Focus structure. In the Focus structure, a slicing operation is first used to convert the original 608*608*3 image into a 304*304*12 feature map, and then a convolution operation of 32 convolution kernels is performed to finally convert it into a 304*304*32 feature map; the Backbone feature extraction unit also includes four CBL components, three CSP1_X structures and an SPP spatial pyramid pooling layer;
[0170] The Neck feature fusion unit includes four tensor concatenation layers and two upsampling layers; it also includes four CBL components and five CSP2_X structure layers;
[0171] The prediction detection output part includes three convolutional layers, which are used to output multi-scale feature maps of 76×76×255, 38×38×255 and 19×19×255 respectively.
[0172] The specific training process is:
[0173] Step a: Use a drone equipped with a TV camera to collect 1000 256×256 pixel images containing the target and background on site to form a database. Use the labelimg tool to pre-process the database image samples. The training set, validation set, and test set are divided into a ratio of 6:3:1.
[0174] Step b, train the YOLO algorithm model using sample data, with the training batch set to 8 and the training rounds to 100 epochs;
[0175] After 100 rounds of training, the accuracy, recall and average precision of the model obtained by training are statistically obtained as follows: Figure 2 , Figure 3 and Figure 4 As shown in;
[0176] The YOLO algorithm model has a higher accuracy and recall rate, and the higher the recall rate, the better. The extreme point is (1,1). Figure 2 and Figure 3 It can be seen that the overall curve tends to (1.1), which can indicate that the overall performance of the model is good; Figure 4 The curve shows that the overall performance of the model is very good.
[0177] During training, the loss function is IoU represents intersection over union; υ represents the measure of similarity in the aspect ratio of the detection box; α is the weight function; b, b gt Represent the center points of the predicted box and the real box respectively; ρ represents the Euclidean distance between the two center points.
[0178] The target and prediction box loss functions in the training set and validation set gradually decrease as the number of training rounds increases during the training process, and eventually tend to 0. Figure 5a The figure shows how the bounding box loss of the training set changes with the number of training rounds. Figure 5b The figure shows how the target loss of the training set changes with the number of training rounds. Figure 5c The change of the validation set bounding box loss over the training rounds is shown in Figure 5dThe figure shows how the validation set target loss changes with the number of training rounds. After 25 rounds, the loss function drops below 0.02, which shows that the model has local rapid convergence. As the number of training rounds increases, the loss becomes smaller and smaller, and finally approaches 0. At the same time, Figure 6a It shows that the accuracy of detecting the target is close to 1. Figure 6b It shows that the recall rate is close to 1. Figure 6c It shows that map_0.5 approaches 1, Figure 6d It shows that mAP_0.5:0.95 gradually approaches 1 as the number of training rounds increases; the mean average precision determined by the object detection accuracy and recall rate also tends to be stable, indicating the accuracy of the model. mAP_0.5: mean Average Precision (IoU = 0.5), that is, when IoU is set to 0.5, the AP of all images in each category is calculated, and then the average of all categories is calculated; mAP_0.5:0.95: represents the mAP at different IoU thresholds, namely 0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, and 0.95.
[0179] After the YOLO algorithm model is trained, an image containing the target position is input into it. The model can accurately capture the target in the image. The capture result is as follows: Figure 7 As shown in , according to the results, it can be seen that the trained YOLO algorithm model has good target capture capability.
[0180] Example 2
[0181] The YOLO algorithm model trained in Example 1 is loaded into an image seeker, and then the image seeker is mounted on an aircraft, wherein the aircraft is also provided with a television camera, an electronic image stabilization submodule, a particle filter submodule, an inertial guidance submodule, a microprocessor and a steering gear;
[0182] After the aircraft is launched and takes off, target identification and aircraft control are performed by the following method:
[0183] Step 1, collecting the target area image in real time through a TV camera;
[0184] Step 2, performing motion compensation on the image signal input by the television camera through the electronic image stabilization submodule to stabilize the image sequence, wherein the motion vector of the television camera is estimated by using the difference in image content between frames, thereby performing motion compensation;
[0185] Step 3, identify the target position in the image through the YOLO algorithm model in the image seeker;
[0186] Step 4, obtaining the relative position information between the target and the aircraft through the particle filter submodule;
[0187] Step 5, obtaining the attitude information of the aircraft in real time through the inertial navigation module;
[0188] Step 6, obtaining the required overload in real time through the microprocessor based on the relative position information between the target and the aircraft and the attitude information of the aircraft, wherein the required overload is obtained in the microprocessor through the following formula: in is the line of sight angular velocity, N1 and N2 are proportional coefficients, V is the aircraft speed, Projectile pitch angle, is the desired pitch angle;
[0189] A damping circuit is also provided on the aircraft, and the damping circuit obtains the gain K in real time through the following formula: g :
[0190]
[0191] Among them, μ d Indicates the expected damping; its value is 0.5;
[0192] T m represents the inherent time constant of the projectile;
[0193] μ m It represents the inherent damping of the projectile;
[0194] T i represents the attitude time constant;
[0195] k s Indicates the servo gain;
[0196] represents the attitude angular rate gain;
[0197] A represents the intermediate variable, which is obtained by the following formula:
[0198] A=μ d 2 T i 2 +μ d 4 T m 2 -2T m μ m T i μ d 2 -T i 2 (μ m 2 -μ d 2 )
[0199] The gain K obtained by the damping loop in real time g Superimpose with the demand overload in real time to obtain the corrected demand overload;
[0200] Step 7, the servo generates a rudder command based on the corrected overload, controls the servo to steer through the rudder command, adjusts the flight attitude of the aircraft, and flies to the target.
[0201] The target position information and flight trajectory obtained by the aircraft in real time are as follows: Figure 8 As shown in Figure 12, the target is finally hit accurately, where the solid line represents the flight trajectory of the aircraft and the dotted line represents the motion trajectory of the target. The change of the pitch rate of the aircraft is shown in Fig. 9 As shown in , the changes of overload instructions are as follows Fig.10 As shown in , the change of the angular rate of the field of view is as follows Fig.11 as shown in .
[0202] Comparative Example 1
[0203] An aircraft substantially the same as that in Example 2 is selected, except that the aircraft is not provided with a damping circuit. After the aircraft is launched and takes off, target identification and aircraft control are performed by the following method:
[0204] Step 1, collecting the target area image in real time through a TV camera;
[0205] Step 2, performing motion compensation on the image signal input by the television camera through the electronic image stabilization submodule to stabilize the image sequence, wherein the motion vector of the television camera is estimated by using the difference in image content between frames, thereby performing motion compensation;
[0206] Step 3, identifying the target position in the image through the YOLO algorithm model in the image seeker; the YOLO algorithm model is trained in Example 1;
[0207] Step 4, obtaining the relative position information between the target and the aircraft through the particle filter submodule;
[0208] Step 5, obtaining the attitude information of the aircraft in real time through the inertial navigation module;
[0209] Step 6, obtaining the required overload in real time through the microprocessor based on the relative position information between the target and the aircraft and the attitude information of the aircraft, wherein the required overload is obtained in the microprocessor through the following formula: The sight angle is N1 and N2 are proportional coefficients, V is the missile speed, Projectile pitch angle, is the desired pitch angle;
[0210] Step 7, generating a rudder command based on the required overload by the servo, controlling the servo to steer by the rudder command, adjusting the flight attitude of the aircraft, and flying toward the target;
[0211] The pitch angle rate without damping loop and the pitch angle rate with damping loop are as follows: Fig. 9 As shown in , the overload instruction without setting the damping circuit and the overload instruction with setting the damping circuit are as follows Fig.10 As shown in , the field of view angular rate without setting the damping loop and the field of view angular rate with setting the damping loop are as follows Fig.11 As shown in Fig. 9 Fig.10 As can be seen from Figure 11, the damping circuit can significantly improve the stability of the aircraft and stabilize the flight attitude;
[0212] It can be seen from Example 2 and Comparative Example 1 that the target position information obtained by the aircraft equipped with a damping circuit is smooth and continuous. The damping circuit can reduce the fluctuations of the attitude angle and the field of view angle, improve the stability of the projectile, and thus improve the accuracy of obtaining the target image information, thereby ultimately improving the hit accuracy of the aircraft.
[0213] Comparative Example 2
[0214] An aircraft substantially the same as that in Example 2 is selected, except that the image seeker on the aircraft does not have a deep learning YOLO algorithm model, but uses an image seeker using a template matching method; after the aircraft is launched and takes off, the target is identified and the aircraft is controlled by the following method:
[0215] Step 1, collecting the target area image in real time through a TV camera;
[0216] Step 2, performing motion compensation on the image signal input by the television camera through the electronic image stabilization submodule to stabilize the image sequence, wherein the motion vector of the television camera is estimated by using the difference in image content between frames, thereby performing motion compensation;
[0217] Step 3, identifying the target position in the image by an image seeker using a template matching method;
[0218] Step 4, obtaining the relative position information between the target and the aircraft through the particle filter submodule;
[0219] Step 5, obtaining the attitude information of the aircraft in real time through the inertial navigation module;
[0220] Step 6, obtaining the required overload in real time through the microprocessor based on the relative position information between the target and the aircraft and the attitude information of the aircraft, wherein the required overload is obtained in the microprocessor through the following formula: in is the line of sight angular velocity, N1 and N2 are proportional coefficients, V is the aircraft speed, Projectile pitch angle, is the desired pitch angle;
[0221] A damping circuit is also provided on the aircraft, and the damping circuit obtains the gain K in real time through the following formula: g :
[0222]
[0223] Among them, μ d Indicates the expected damping; its value is 0.5;
[0224] T m represents the inherent time constant of the projectile;
[0225] μ m It represents the inherent damping of the projectile;
[0226] T i represents the attitude time constant;
[0227] k s Indicates the servo gain;
[0228] represents the attitude angular rate gain;
[0229] A represents the intermediate variable, which is obtained by the following formula:
[0230] A=μ d 2 T i 2 +μ d 4 T m 2 -2T m μ m T i μ d 2 -T i 2 (μ m 2 -μ d 2 )
[0231] The gain K obtained by the damping loop in real time g Superimpose with the demand overload in real time to obtain the corrected demand overload;
[0232] Step 7, the servo generates a rudder command based on the corrected overload, controls the servo to steer through the rudder command, adjusts the flight attitude of the aircraft, and flies to the target.
[0233] The aircraft trajectory obtained in Example 2 is as follows Fig.12As shown in , the aircraft trajectory corresponding to the template matching image seeker in Comparative Example 2 is a blue trajectory, and the aircraft trajectory in Example 2 is a red trajectory. It can be seen that the red trajectory eventually hits the target, and the blue trajectory misses the target;
[0234] By comparing Experimental Example 2 with Comparative Example 2, it can be shown that the detection process of using the traditional template matching method for target detection is relatively complicated, the algorithm calculation time is long, the method has large limitations, and it does not meet the rapidity of the seeker to identify the target, which ultimately causes the aircraft to miss the target and cannot accurately intercept the target; the deep learning YOLO algorithm is built by a convolutional neural network, with high detection accuracy and fast detection speed, which meets the requirements of the image seeker to accurately identify the target in real time, and ultimately enables the aircraft to hit the target.
[0235] The present invention has been described above in conjunction with preferred embodiments, but these embodiments are only exemplary and serve only as an illustration. On this basis, the present invention may be subjected to a variety of substitutions and improvements, all of which fall within the scope of protection of the present invention.
Claims
1. Target recognition system for image / TV-guided aircraft with stable attitude and image, It is characterized in that The system includes: An image processing module (1) and a guidance control module (2), The image processing module (1) is used to identify and lock the target, obtain the relative position information between the target and the aircraft, The guidance control module (2) is used to adjust the attitude of the aircraft through the steering gear so that the aircraft can fly smoothly towards the target. The guidance control module (2) comprises an inertial guidance submodule (21), a microprocessor (22) and a steering gear (23); The inertial navigation module (21) is used to obtain the attitude information of the aircraft in real time. The microprocessor (22) obtains the required overload in real time based on the relative position information between the target and the aircraft and the attitude information of the aircraft, and further obtains the rudder instruction; The steering engine (23) performs steering according to the rudder instruction, A damping circuit is also provided in the guidance control module (2), and the stability of the aircraft is improved by the damping circuit. The damping loop is adjusted in real time according to the desired damping μ d Gain K g ; The required overload obtained by the microprocessor (22) and the gain K g The required overload is corrected by superposition, and the rudder command is generated based on the corrected required overload. The gain K g Obtained by the following formula: Among them, μ d represents the expected damping; T m represents the inherent time constant of the projectile; μ m It represents the inherent damping of the projectile; T i represents the attitude time constant; k s Indicates the servo gain; represents the attitude angular rate gain; A represents the intermediate variable, which is obtained by the following formula:
2. The target recognition system for image / television guided aircraft with stable attitude and image according to claim 1, It is characterized in that The image processing module (1) comprises a television camera (11), an electronic image stabilization submodule (12), an image guidance head (13) and a particle filter submodule (14); Wherein, the television camera (11) is used to collect images of the target area in real time; The electronic image stabilization submodule (12) performs motion compensation on the image signal input by the television camera (11) to stabilize the image sequence; The image seeker (13) uses a YOLO algorithm model to detect and identify the image, thereby locking the target; The particle filter submodule (14) is used to obtain the distance r, the sight line deflection angle θ and the sight line inclination angle φ of the aircraft relative to the target according to the position information of the target in the image given by the image guidance head (13).
3. The target recognition system for image / television guided aircraft with stable attitude and image according to claim 2, It is characterized in that The YOLO algorithm model includes a feature extraction unit (131), a feature fusion unit (132) and a detection output unit (133); wherein the feature extraction unit (131) extracts feature information from the image, The feature fusion unit (132) fuses the extracted feature information into feature maps of three scales. The detection output unit (133) uses the characteristic map to detect the target position and obtain target position information.
4. A method for target recognition using the attitude and image stabilization target recognition system on an image / television guided aircraft according to any one of claims 1 to 3, It is characterized in that The method comprises the following steps: Step 1, collecting the target area image in real time through the TV camera (11); Step 2, performing motion compensation on the image signal input by the television camera (11) through the electronic image stabilization submodule (12) to stabilize the image sequence; Step 3, detecting and identifying the image by using the YOLO algorithm model, thereby locking the target; Step 4, obtaining the distance r, sight angle θ and sight inclination φ of the aircraft relative to the target through the particle filter submodule (14); Step 5, obtaining the attitude information of the aircraft through the inertial navigation module (21), Step 6, obtaining the required overload in real time through the microprocessor (22), Step 7, generating a steering command through the steering gear (23) and executing it.
5. The method for target recognition with stable attitude and image on an image / television guided aircraft according to claim 4, It is characterized in that In step 6, after obtaining the required overload, the microprocessor (22) compares the required overload with the gain K obtained through the damping loop. g The corrected required overload is obtained by superposition, so that the steering engine generates a rudder instruction based on the corrected required overload and executes it.
Citation Information
Patent Citations
Control method for realizing overload autopilot by using accelerometer
CN102425980A
Integrated analyzing and setting method for control parameters of three-loop automatic pilot
CN103076806A
Drone indoor positioning and multi-target tracking method completely based on vision information
CN109682378A
Unmanned aerial vehicle airborne target detection system and method
CN111736190A
Unmanned aerial vehicle target recognition method and device and unmanned aerial vehicle
CN112380933A
