A method for aircraft target recognition based on neural network coverage

Abstract

This invention belongs to the field of computer vision technology, and particularly relates to an aircraft target recognition method based on neural network coverage. This invention analyzes the test results of the target detection model, prunes the neurons and connections between neurons in the deep neural network-based detection model, and adaptively adjusts the model structure to optimize target detection accuracy, while accelerating the convergence of the "intelligent testing + machine learning" closed loop. The invention quantifies the activation level of the deep neural network using neural network coverage, and trains the model using collected aerial image data, eliminating the need to generate candidate boxes based on image data. This approach reduces the time cost of training the target recognition model. Through backpropagation and parameter fine-tuning, an intelligent model applicable to aerial image detection tasks can be efficiently obtained to meet the high real-time and high maneuverability requirements of the aviation field.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a method for aircraft target recognition based on neural network coverage. Background Technology

[0002] Artificial intelligence technology is currently widely used in the aviation field, for example, to assist intelligent aircraft in target detection, intelligent decision-making, and aircraft engine fault prediction and health management. Among these, the purpose of target detection is mainly to locate and classify the collected image data. The detection results serve as the basis for the aircraft's intelligent decision-making, and the accuracy of the detection results directly affects, and even determines, the reliability and stability of the aircraft.

[0003] Researchers have conducted extensive studies on how to improve the accuracy of object detection. Based on whether candidate boxes need to be generated in advance, current research is mainly divided into two-stage methods and one-stage methods. Two-stage methods mainly include R-CNN, Fast-RCNN, and Faster-RCNN; one-stage methods include the YOLO series of methods.

[0004] Two-stage object detection methods: These methods mainly consist of two stages: candidate box generation and prediction result generation. Taking R-CNN as an example, in the candidate box generation stage, a selective search algorithm is used to cluster pixels in the image based on the similarity of features such as color and texture, generating pixel blocks. Based on these pixel blocks, candidate boxes are generated through multiple iterations. The prediction result generation stage mainly aims to generate classification and localization results. Specifically, it uses features extracted by a convolutional neural network and supports vector machines and linear regression to predict classification and localization results.

[0005] One-stage object detection methods: These methods do not require pre-generating candidate boxes; instead, they directly generate classification results and localization boxes using neural networks. First, convolutional kernels are used to divide the image into feature maps. Then, a deep network is built directly based on the anchor boxes to generate predicted localization boxes and classification results. To alleviate the overfitting problem in machine learning, the YOLO series of models incorporates improvements such as Dropout model regularization. Dropout refers to discarding hidden or visible units in a neural network. Specifically, it randomly removes some neurons from the network, along with all their input and output connections.

[0006] The efficiency and accuracy of existing target detection technologies still need improvement. Two-stage and one-stage target detection methods have the following limitations.

[0007] Two-stage object detection methods: These methods require generating a large number of candidate boxes, resulting in significant time costs and making them unsuitable for real-time object detection tasks in aviation scenarios. In fact, R-CNN not only needs to generate multiple candidate boxes but also transforms the multi-class classification problem into multiple binary classification problems during the classification process, requiring the training of multiple classifiers. These additional steps further increase the time cost. While Fast-R-CNN and Faster-R-CNN use neural networks for direct multi-class classification, they still struggle to alleviate the substantial time delay caused by generating a large number of candidate boxes, making lightweight deployment difficult.

[0008] One-stage object detection methods aim to reduce the runtime cost of object recognition while improving the accuracy of the object detection model. On one hand, the YOLO series of methods employs fixed anchor boxes, reducing the time cost of generating candidate boxes in two-stage methods. On the other hand, regularization techniques such as Dropout and DropBlock are used. These techniques randomly remove some neurons and convolutional modules from the deep neural network to reduce the complexity of the object detection model, thereby mitigating the overfitting problem caused by data sparsity in the aviation field.

[0009] However, random pruning can easily remove some highly activated neurons, reducing model performance and ultimately making it difficult for the model to accurately distinguish the category and location of targets during application. On the PASCAL VOC dataset (containing images of 20 categories such as airplanes, birds, and cars), although YOLO's recognition speed is improved compared to Fast R-CNN, its error rate is nearly three times that of Fast R-CNN. Therefore, how to prune the aviation target detection model according to the activation level of its neurons to improve recognition accuracy is a crucial problem that urgently needs to be solved. Summary of the Invention

[0010] To better address the aforementioned issues, a method for aircraft target recognition based on neural network coverage has been invented. This method can effectively balance the efficiency and performance requirements of aviation target detection tasks.

[0011] The technical solution of this invention: A target recognition method in the aviation field based on neural network coverage, comprising the following steps:

[0012] Step 1: Construct the training dataset. First, collect aerial image data, then perform data cleaning and labeling to build the training set;

[0013] Step 2: Train the target recognition model based on the training set constructed in the previous step: Set the hidden probability p according to the neural network coverage, and then use backpropagation Adam to solve for the target recognition model parameters.

[0014] Step 3: Based on the target recognition model parameters obtained in the previous step, i.e. the weights in the deep neural network, the neural network coverage is solved by forward propagation to provide an optimization scheme for the target recognition model training in the second step. After the number of iterations in steps 2 and 3 reaches a certain limit, the target recognition model trained in step 2 is transferred to step 4.

[0015] Step 4: Utilize the trained deep neural network-based target recognition model to perform automated target detection on the new aerial image data.

[0016] Step 2, target recognition model training, involves solving for model parameters using the training set, and is conducted in multiple rounds.

[0017] The target recognition model parameters are the weight values ​​of the links between neurons in a deep neural network.

[0018] Step 2 specifically involves:

[0019] Step 1: In a certain epoch of the model training phase, the neurons of the deep neural network are temporarily hidden randomly with a hiding probability p.

[0020] Step 2, transfer the aerial image x i The input is fed into the pruned deep neural network for forward propagation, followed by back propagation to calculate the gradient of the target recognition model's loss function with respect to the model parameters. Once the cumulative number of forward and back propagation iterations reaches a predefined batch size, the model parameters W are updated using gradient descent. (k) .

[0021] Step 3: Recover the neurons temporarily hidden in Step 1, and then perform Step 1 and Step 2 to obtain the target recognition model parameters.

[0022] The method for solving the neural network coverage in step 3 is as follows: In a certain epoch, the neural network coverage is calculated based on the ratio of the number of activated neurons in all inputs to the total number of neurons in the deep neural network.

[0023] The method for determining the number of activated neurons is as follows: if the absolute value of the neuron's output is greater than a certain threshold, then the neuron is considered to be activated.

[0024] The neural network coverage is defined as the ratio of the number of uniquely activated neurons across all inputs to the total number of neurons in the deep neural network.

[0025] The aerial image data is images collected by an airborne visible light camera.

[0026] Beneficial effects of this invention:

[0027] Unlike existing object detection methods that only involve a one-way path from machine learning to model testing, this invention proposes a closed loop of "intelligent testing + machine learning." In terms of intelligent testing, by analyzing the test results of the object detection model, the black box of deep neural networks can be opened, providing an interpretable solution for optimizing machine learning-based object detection models. In terms of machine learning, based on the intelligent testing results, the neurons and connections between neurons in the deep neural network-based detection model can be pruned (dropout), adaptively adjusting the model structure to optimize object detection accuracy and accelerate the convergence of the "intelligent testing + machine learning" closed loop.

[0028] In the intelligent testing phase, unlike the one-stage target detection method, this invention relates to an intelligent testing method based on neural network coverage. This method uses neural network coverage to quantify the activation level of a deep neural network, and performs model testing on the deep neural network based on the activation level of the model, providing an interpretable solution for further performance optimization of the target recognition model based on the deep neural network.

[0029] In the machine learning phase, unlike two-stage object detection methods, this invention relates to an efficient end-to-end machine learning framework that can directly train the model using the collected aerial image data without generating candidate boxes based on the image data. This approach reduces the time cost of training the object recognition model. Through backpropagation and parameter fine-tuning, an intelligent model applicable to aerial image detection tasks can be efficiently obtained to meet the high real-time and high mobility requirements of the aviation field. Attached Figure Description

[0030] Figure 1 This is a diagram of the overall closed-loop structure of the "intelligent testing + machine learning" of this invention;

[0031] Figure 2 This is a schematic diagram of neural network pruning in this invention;

[0032] Figure 3 This is a schematic diagram illustrating the calculation of neural network coverage using neuron forward propagation according to the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0034] This invention proposes an aircraft target recognition method based on neural network coverage, such as... Figure 1 As shown, it includes four steps:

[0035] Step 1: Construct the training dataset. This step includes three parts: aerial image data collection, data cleaning, and data annotation. Data annotation can be done using tools such as LabelImage. Step 2: Train a target recognition model, such as the YOLO series, based on the training set constructed in the previous step. Set the hyperparameter Dropout empirically, and then use backpropagation (Adam) to solve for the target detection model parameters. Step 3: Based on the model parameters obtained in the previous step, i.e., the weights in the deep neural network, use forward propagation to solve for the neural network coverage, providing an optimization scheme for the target detection model training in the second step. Step 4: After completing model training, use the trained target detection prediction model to automatically detect targets on new aerial image data.

[0036] Therefore, this method needs to consider the input, output, and training and testing of the object detection model. The input is the collected aerial image data, which can be obtained through an airborne visible light camera; the output is the trained object detection model and the detected aerial image data; the core of this method is the training and testing of the object detection model, which will be discussed in detail below.

[0037] The training module of the object detection model is mainly responsible for solving the model parameters using the training set, and this process is carried out in multiple rounds. As the number of training rounds increases, the model's fit to the data gradually improves. The training of the object recognition model mainly consists of the following steps.

[0038] First, define some symbols. For a given Epoch, let k∈{1,...,K} represent the index number of a certain layer of the network, l∈{1,...,L} represent the index number of the input aerial image, and v (k，l) This indicates that for aerial image x l The input to the k-th layer of the neural network. (k，l) This indicates that for aerial image x l The output of the k-th layer of the neural network, where u (0，l) This indicates that for aerial image x l The input W to layer 0 of the neural network(k) The model parameters of the k-th layer of the neural network are represented as follows: Our proposed neural network model can be represented as... Figure 2 .

[0039] (1) In a certain epoch of the model training phase, such as Figure 2 As shown, for the i-th neuron in any layer k, the neuron is temporarily hidden randomly with a probability of p.

[0040]

[0041]

[0042] in It is a vector of independent Bernoulli random variables, each of which has a probability of 1 for p.

[0043] (2) Transfer the aerial image x l The data is used as input for forward propagation on the pruned deep neural network, followed by backpropagation to calculate the gradient of the model's loss function with respect to the model parameters. Once the cumulative number of forward and backpropagation passes reaches a predefined batch size, the model parameters W are updated using gradient descent. (k) .like Figure 2 As shown, the forward propagation process can be represented as follows:

[0044]

[0045]

[0046] Where f represents the activation function, for any layer k, It is a vector of independent Bernoulli random variables, each with a hidden probability of p where each Bernoulli random variable is 1.

[0047] (3) Then, repeat the following process:

[0048] a) The temporarily hidden neurons are restored, and the parameters of the restored neurons remain unchanged, while the parameters of the unhidden neurons have been updated;

[0049] b) In the next epoch, from the k-th hidden layer, the neurons in the neural network are hidden with probability p;

[0050] c) For a batch of aerial images, the loss function is calculated by forward propagation, the gradient is calculated by backpropagation, and the parameters of the neurons that were not deleted are updated according to the stochastic gradient descent method, while the parameters of the hidden neurons remain unchanged.

[0051] Repeat steps (1), (2) and (3).

[0052] The testing module of the object detection model is mainly responsible for testing the object detection model trained in a certain epoch, and formulating optimization schemes for the model based on the test results. This paper tests the model based on the neural network coverage of the neural network model.

[0053] The testing process is conducted over multiple epochs. In each epoch, the neural network coverage is defined as the ratio of the number of uniquely activated neurons across all inputs to the total number of neurons in the deep neural network. During forward propagation, a neuron is considered activated if the absolute value of its output is greater than a certain threshold (e.g., 0.5).

[0054] This represents the input set of aerial image data. M represents the number of neurons in the neural network. For example... Figure 3 As shown, This means that for a given test input x l Let be the output of the i-th neuron in the k-th layer, and t represent the threshold for determining whether a neuron is activated. Then, the network coverage of a deep neural network can be expressed as:

[0055]

[0056] Where I(x, y) is the indicator function x > y, then I(x, y) = 1, otherwise it is 0.

[0057] The testing phase mainly consists of two steps:

[0058] 1. First, for a given set of aerial image data... The deep neural network coverage is calculated using the above formula.

[0059] 2. Calculate the probability p of network pruning based on the coverage of the deep neural network.

[0060]

[0061] 3. The parameters of the neural network can be adjusted according to the probability p using formula (1).

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for aircraft target recognition based on neural network coverage, characterized in that, Includes the following steps: Step 1, Constructing the training dataset: First, collect aerial image data, then clean and label the data to build the training set; Step 2: Train the deep neural network-based target recognition model using the training set built in the previous step: Set the hidden probability according to the neural network coverage. Then, backpropagation Adam is used to solve for the target recognition model parameters; Step 3: Based on the target recognition model parameters obtained in the previous step, i.e. the weights in the deep neural network, the neural network coverage is solved by forward propagation to provide an optimization scheme for the target recognition model training in the second step. After the number of iterations in steps 2 and 3 reaches a certain limit, the target recognition model trained in step 2 is transferred to step 4. Step 4: Use the trained target recognition model based on deep neural networks to perform automated target detection on the new aerial image data; The input to this method is the collected aerial image data, and the output is the trained target detection model and the aerial image data after detection. The core is the training and testing of the object detection model; The training module of the object detection model is mainly responsible for solving the model parameters using the training set, which is done in multiple rounds. The training of the object recognition model mainly consists of the following steps: First, define some symbols. For a given Epoch, let... This represents the index number of a certain layer of the network. This indicates the index number of the input aerial image. Indicates for aerial images The neural network The input of the layer. Indicates for aerial images Neural Network The output of the layer, where Indicates for aerial images The input to layer 0 of the neural network, The neural network model represents the first... Model parameters of layered networks: (1) In a certain epoch of the model training phase, firstly for any layer The 1 neuron, randomly selected The probability of temporarily hiding the neuron; （1） （2） in It is a vector of independent Bernoulli random variables, each of which has The probability is 1. (2) Take aerial images The data is used as input for forward propagation on the pruned deep neural network, followed by backpropagation to calculate the gradient of the model loss function with respect to the model parameters. Once the cumulative number of forward and backpropagation passes reaches a predefined batch size, the model parameters are updated using gradient descent. The forward propagation process is represented as follows: （3） （4） in, Represents the activation function, for any layer , It is a vector of independent Bernoulli random variables, where the hidden probability of each Bernoulli random variable being 1 is... ; (3) Then, repeat the following process: a) The temporarily hidden neurons are restored, and the parameters of the restored neurons remain unchanged, while the parameters of the unhidden neurons have been updated; b) In the next Epoch from the 1st A hidden layer, with neurons in the... The probability of hiding neurons in a neural network; c) For a batch of aerial images, first use forward propagation to find the loss function, then use backward propagation to find the gradient, and update the parameters of the neurons that were not deleted according to the stochastic gradient descent method. The parameters of the hidden neurons remain unchanged. Repeat steps (1), (2) and (3). The testing process is conducted over multiple epochs. In a given epoch, the neural network coverage is defined as the ratio of the number of uniquely activated neurons across all inputs to the total number of neurons in the deep neural network. During forward propagation, if the absolute value of a neuron's output is greater than a certain threshold, the neuron is considered activated. This represents the set of input aerial image data. This indicates the number of neurons in a neural network. This indicates that for a given test input , No. Layer The output of each neuron Let represent the threshold for determining whether a neuron is activated; then the network coverage of a deep neural network can be expressed as: （5） in For indicator functions ,but Conversely, it is 0.

2. The aircraft target recognition method based on neural network coverage as described in claim 1, characterized in that, Step 2, target recognition model training, involves solving for model parameters using the training set, and is conducted in multiple rounds.

3. The aircraft target recognition method based on neural network coverage as described in claim 2, characterized in that, The target recognition model parameters are the weight values ​​of the links between neurons in a deep neural network.

4. The aircraft target recognition method based on neural network coverage as described in claim 1, characterized in that, Step 2 specifically involves: Step 1: In a certain epoch during the model training phase, randomly assign hidden probabilities... Temporarily hide neurons in deep neural networks. Step 2, transfer the aerial images The data is fed into the pruned deep neural network for forward propagation, followed by backpropagation to calculate the gradient of the target recognition model's loss function with respect to the model parameters. Once the cumulative number of forward and backpropagation passes reaches a predefined batch size, the model parameters are updated using gradient descent. . Step 3: Recover the neurons temporarily hidden in Step 1, and then perform Step 1 and Step 2 to obtain the target recognition model parameters.

5. The aircraft target recognition method based on neural network coverage as described in claim 1, characterized in that, The method for solving the neural network coverage in step 3 is as follows: In a certain epoch, the neural network coverage is calculated based on the ratio of the number of activated neurons in all inputs to the total number of neurons in the deep neural network.

6. The aircraft target recognition method based on neural network coverage as described in claim 5, characterized in that, The method for determining the number of activated neurons is as follows: if the absolute value of the neuron's output is greater than a certain threshold, then the neuron is considered to be activated.

7. The aircraft target recognition method based on neural network coverage as described in claim 1, characterized in that, The coverage of a neural network is defined as the ratio of the number of uniquely activated neurons across all inputs to the total number of neurons in a deep neural network.

8. The aircraft target recognition method based on neural network coverage as described in claim 5, characterized in that, Aerial imagery data consists of images collected using airborne visible light cameras.

Images