A method and system for identifying aerial target maneuvering actions based on YOLO network
By improving the YOLOv5 neural network model, the problem of insufficient data segmentation capability in flight maneuver recognition is solved, adaptive recognition of maneuver actions of different lengths is achieved, and recognition accuracy and continuity are improved.
Patent Information
- Application Number
- CN202311644738.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-04
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-12-04
AI Technical Summary
The existing flight maneuver identification method ignores the segmentation of flight data, resulting in errors and discontinuities in the case of high noise. The fixed-length sliding window is difficult to handle non-fixed maneuver length, resulting in poor recognition effect.
Based on YOLOv5's improved neural network model, the input data range is narrowed through differential operations, the main part extracts features from different scales, the neck is fused with multi-scale information, and the head generates predicting the maneuver start time, end time and category probability, and uses non-extreme The value suppression algorithm filters the recognition results to realize adaptive segmentation and recognition of maneuvering operations of different lengths.
The problem that existing neural networks do not have segmentation capabilities is overcome, and the accurate segmentation and recognition of maneuvering operations in flight data is achieved, achieving single-sampling point recognition accuracy comparable to that of recurrent neural networks.
Smart Images

Figure CN117710787B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of battlefield situation analysis, and in particular to a method and system for identifying aerial target maneuvering actions based on a YOLO network. Background Art
[0002] In complex and dynamic air combat scenarios, the maneuvering performance of enemy aircraft directly reflects their combat intentions and decisions. By accurately and in real time recognizing maneuvers in flight data, enemy aircraft combat strategies can be analyzed and responded to in a timely manner, which is an urgent need in the field of air combat research. At present, the research on Flight Maneuver Recognition (FMR) tasks is mainly focused on four technical directions, namely, expert system-based, probabilistic graphical model-based, linear classification-based, and neural network-based. Among them, the neural network method has the advantages of high recognition rate, high accuracy, and low reliance on prior knowledge, but the existing models ignore the segmentation of flight data and regard FMR as just a classification task.
[0003] The movement of aerial targets is a continuous dynamic process, which includes non-maneuvering actions and may also include one or more maneuvering flight actions, and there is no clear dividing point between maneuvering actions or between maneuvering actions and non-maneuvering actions. For maneuvering actions in flight, the model is generally required to first determine the starting time point and the ending time point of a maneuvering action, and then classify the categories of maneuvering actions contained in the starting time point and the ending time point. That is, before identifying the category of maneuvering actions in flight data, it is necessary to segment and extract the flight data to exclude redundant flight data other than the maneuvering actions to be identified. Since the length of time required to complete a flight action is affected by the current flight speed of the aircraft, the maximum acceleration of the aircraft, the type of action, the completion of the flight action, and some human or external factors, the flight data of the same type of action often have different lengths. This also requires the identifier to be able to identify flight data of different lengths.
[0004] Since the existing neural networks used for flight maneuver recognition tasks ignore the segmentation of flight data, some problems will be encountered in practical applications. 1. The recurrent neural network (RNN) recognizes the category of the data of a single sampling point of the flight data. When the noise of the flight data is large, the category recognition error of individual sampling points in the maneuver will occur, resulting in discontinuity in the category recognition result of a complete maneuver. When extracting maneuvers, the original complete maneuver will be extracted into multiple fragmented maneuvers. 2. The subsequences of the flight data time series are intercepted by a sliding data window of a fixed length, and the one-dimensional convolutional neural network (CNN1d) classifies the intercepted subsequences. Since the time length of a complete maneuver is not fixed, the flight data subsequence intercepted by the sliding data window of a fixed window length either contains only incomplete maneuver flight data or non-maneuver flight data, which has a poor effect on the interception of complete maneuvers. In addition, the redundancy of non-maneuver flight data and the lack of maneuver data in the sliding data window will affect the recognition results of the intercepted flight data subsequence category. Summary of the invention
[0005] Purpose of the invention: In view of the problems existing in the existing flight maneuver recognition methods, the present invention proposes a method and system for identifying aerial target maneuver actions based on the YOLO network. By improving YOLOv5, the neural network structure can adaptively segment and extract flight data of maneuver actions of different lengths, thereby overcoming the problems encountered by the neural network in the existing flight maneuver recognition tasks due to the lack of segmentation ability.
[0006] Technical solution: A method for identifying aerial target maneuvers based on a YOLO network, characterized in that it includes the following steps:
[0007] The track time series of the aerial target D = {d1, d2, ..., d t ,...d T} Input the neural network model improved based on YOLOv5, and train the neural network model according to the set training related parameters; where d t =(x t ,y t , z t , v t ,θ t , ψ t) is the flight parameter vector of the aerial target at time t, which are the spatial position coordinates x, y, z, speed v, inclination angle θ and azimuth angle ψ, and T is the length of the time series; the neural network model includes four parts: input, trunk, neck and head. The input part processes the input data by differential operation to reduce the flight parameters of the input network to a fixed interval; the trunk extracts features from input data of different scales; the neck is composed of two structures, Fpn and Pan, which integrate the information of multi-scale feature vectors; the head regularizes the multi-scale feature vector set of the neck by convolution to generate three feature vector sets; the input data is equally divided into N time intervals according to time. When the time center of the aerial target maneuver falls within the nth time interval, the corresponding feature vector Responsible for identifying maneuvers, and They represent the start time, end time, confidence level and probability of maneuver category j of the predicted maneuver respectively;
[0008] The track time series of the aerial target to be identified is input into the trained neural network model, and the neural network model gives multi-category recognition results. The multi-category recognition results of the neural network model are filtered using the non-maximum suppression algorithm, and the recognition results obtained after non-maximum suppression filtering are used as the final recognition results of the aerial target maneuvers.
[0009] Furthermore, the backbone of the neural network model consists of a focus module, a partial connection module, a cross-stage partial connection module and a spatial pyramid pooling module. The focus module downsamples the 6-channel sequence data of length T and expands the channels to generate a 32-channel, The partial connection module uses a convolution with a stride of 2 to reduce the length of the feature vector and increase the receptive field of the feature vector; the cross-stage partial connection module uses a bottleneck structure to build a residual architecture, increase the depth of the network, and enable the network to learn more abstract features; the spatial pyramid pooling module pools and concatenates multi-scale features, enabling the network to learn object features more comprehensively.
[0010] Furthermore, in the neck mechanism of the neural network model, Fpn upsamples and up-fuses the multi-scale feature vectors from the trunk and the neck, and passes the strong semantic features to the first-scale feature vector; Pan downsamples the multi-scale feature vector from Fpn, and then down-fuses it, and passes the strong position features to the second-scale feature vector.
[0011] Furthermore, the head of the neural network model generates three feature vector sets: and The lengths are and The number of channels is 9, and the feature vector set F i ={f1, f2, ..., f N} represents the prediction result of the i-th scale, and N is respectively and
[0012] Furthermore, the neural network model is improved based on YOLOv5, specifically including:
[0013] Replace CNN2d in the YOLOv5 network that processes two-dimensional image data with CNN1d to process the flight data of the control target, and modify the batch normalization operation BatchNorm, the connection operation Concat, and the upsampling operation Upsample originally used for two-dimensional feature vectors in the neural network model accordingly to apply them to one-dimensional feature vectors, making the network model suitable for one-dimensional time series data;
[0014] The three input channels of YOLOv5, corresponding to the red, green, and blue color channels of the input image, are expanded to six, corresponding to the six parameters of the detection data, namely, the spatial position coordinates x, y, z, velocity v, inclination angle θ, and azimuth angle ψ;
[0015] The difference method is set to preprocess the input track time series so that the flight parameters of the input network are reduced to a fixed interval.
[0016] Furthermore, during the training of the neural network model, the loss function of flight maneuver recognition is It consists of three parts: maneuver confidence loss Maneuver segmentation point regression loss and maneuver type classification loss
[0017]
[0018] Among them, the maneuver confidence loss Calculated by binary cross entropy, the regression loss of the mobile segmentation point Calculated by mean square error, maneuver type classification loss Calculated by binary cross entropy, λ1, λ2, and λ3 are the corresponding hyperparameters and weight coefficients respectively;
[0019] Maneuver Confidence Loss Calculated by binary cross entropy:
[0020]
[0021] in, and are two binary parameters used to indicate whether the time center of the maneuver falls within the nth time interval. If so, then is equal to 1, Equal to 0, otherwise is equal to 0, Equal to 1; the hyperparameter λnomnr is used to balance The values of the two items in ; BEC represents the binary cross entropy function;
[0022] Maneuver segmentation point regression loss Calculated by mean square error:
[0023]
[0024] where t n is the center time of the nth time interval, MSE stands for mean square error;
[0025] Maneuver type classification loss Calculated by binary cross entropy:
[0026]
[0027] Where c represents the number of maneuver categories.
[0028] Furthermore, the intersection-over-union ratio (IOU) is used to measure the true value start time t of the maneuvering time interval. e and end time t e Start time with predicted value and end time In the prediction results of the neural network model, those with IOU lower than the preset threshold are filtered out.
[0029] The present invention also provides an aerial target maneuvering action recognition system based on a YOLO network, comprising:
[0030] Model training module, used to transform the track time series D = {d1, d2, ..., d t ,...d T} Input the neural network model improved based on YOLOv5, and train the neural network model according to the set training related parameters; where d t =(x t ,y t , z t , v t ,θ t , ψ t) is the flight parameter vector of the aerial target at time t, which are the spatial position coordinates x, y, z, speed v, inclination angle θ and azimuth angle ψ, and T is the length of the time series; the neural network model includes four parts: input, trunk, neck and head. The input part processes the input data by differential operation to reduce the flight parameters of the input network to a fixed interval; the trunk extracts features from input data of different scales; the neck is composed of two structures, Fpn and Pan, which integrate the information of multi-scale feature vectors; the head regularizes the multi-scale feature vector set of the neck by convolution to generate three feature vector sets; the input data is equally divided into N time intervals according to time. When the time center of the aerial target maneuver falls within the nth time interval, the corresponding feature vector Responsible for identifying maneuvers, and They represent the start time, end time, confidence level and probability of maneuver category j of the predicted maneuver respectively;
[0031] The maneuver identification module is used to input the track time series of the aerial target to be identified into the trained neural network model, and the neural network model gives a multi-category recognition result. The multi-category recognition result of the neural network model is filtered using the non-maximum suppression algorithm, and the recognition result obtained after non-maximum suppression filtering is used as the final recognition result of the aerial target maneuver.
[0032] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the aerial target maneuvering action recognition method based on the YOLO network as described above are implemented.
[0033] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for identifying aerial target maneuvers based on the YOLO network as described above are implemented.
[0034] Beneficial effects: The present invention improves the YOLOv5 neural network for target detection and proposes a neural network structure that can simultaneously segment and classify one-dimensional time series data, thereby overcoming the recognition errors and other problems encountered by the existing neural network for flight maneuver recognition tasks due to the lack of segmentation capabilities. After the proposed neural network model is trained on the data set constructed from the corresponding flight data, it can achieve the segmentation and recognition of maneuvers in the flight data, and can achieve the same single sampling point recognition accuracy as the recurrent neural network (RNN). BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1It is a processing flow chart of the identification method of the present invention;
[0036] Figure 2 It is a structural diagram of a neural network model of the present invention;
[0037] Figure 3 It is the precision-recall curve under different IOU thresholds of the present invention. DETAILED DESCRIPTION
[0038] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0039] The target detector is used to locate and classify objects in the image. Due to the similarity between the target detection task and the task of identifying aerial target maneuvers, the target detector can be improved to identify aerial target maneuvers. YOLOv5 is a target detector with fast detection speed and high detection accuracy, which adopts the feature pyramid network (FPN) and path aggregation network (PAN) structure. It can be improved to extract and fuse flight data information of different lengths. The present invention is modified on the basis of YOLOv5 to adapt to the needs of processing one-dimensional time series data for flight maneuver identification tasks. For the convenience of description, the following refers to aerial targets as aircraft or aircraft. The flight data of an aircraft is the time series data of flight state parameters sampled during the flight, which is obtained by detection sensors. Since it generally characterizes the position and orientation of the aircraft over time, it is referred to as track time series in this article, and the two can be used interchangeably.
[0040] Reference Figure 1 In an embodiment of the present invention, a method for identifying aerial target maneuvers based on a YOLO network includes the following steps:
[0041] (1) Obtain flight data:
[0042] In the present invention, the flight data segment with a length of T is represented by D = {d1, d2, ..., d t ,...d T}, where d t =(x t ,y t , z t ,θ t , ψ t ) is the aircraft flight state parameter vector at time t, which are the aircraft's spatial position coordinates x, y, z, velocity v, inclination angle θ and azimuth angle ψ. When the aircraft performs a maneuver, the aircraft uses the corresponding flight control instructions to change its flight state, and then the aircraft's flight state parameters show different time change patterns. The aircraft's flight control instructions can be simplified and described as the following three control parameters: tangential overload n || , normal overload n ⊥, and roll angle γ.
[0043] As an example, for a fixed-wing aircraft, the relationship between the time series changes of the flight control parameters and the flight state parameters can be expressed by the 3DOF equation:
[0044]
[0045] The maneuver types of fixed-wing aircraft and their corresponding control parameters are shown in Table 1. Given the initial value d0 = (x0, y0, z0, υ0, θ0, ψ0) and the control parameter tangential overload n || , normal overload n ⊥ , and the roll angle γ, the flight data d at any time t is solved by numerical integration of the 3DOF equation t =(x t ,y t , z t , v t ,θ t , ψ t ), generating flight data for training and testing.
[0046] Table 1 Aircraft maneuver types and their corresponding control parameters
[0047]
[0048]
[0049] In the table, C1, C2, and C3 represent constants greater than 0, where C2 is less than 1.
[0050] It should be understood that the present invention can be applied to other types of aircraft, and the maneuvering types, control parameters and corresponding 3DOF equations for other types of aircraft are different. By training the neural network on a data set consisting of flight data of the corresponding type of aircraft, the neural network has the ability to identify the corresponding maneuvering type.
[0051] (2) Set up the network architecture:
[0052] The present invention improves the neural network based on YOLOv5, so that the neural network can simultaneously learn the segmentation and classification tasks of the time series for identifying flight maneuvers. The neural network is also called a neural network model or a network model, also referred to as a network. The specific modifications to the network architecture of the present invention are as follows:
[0053] First, replace CNN2d in the YOLOv5 network that processes two-dimensional image data with CNN1d to process flight data. And modify the batch normalization operation BatchNorm, the connection operation Concat, and the upsampling operation Upsample originally used for two-dimensional feature vectors in the network model accordingly, so that they are applied to one-dimensional feature vectors, making the network model suitable for one-dimensional time series data.
[0054] Secondly, the three input channels of YOLOv5 corresponding to the red, green and blue colors of the input image are expanded to 6 to correspond to the six state parameters of the flight data, namely the aircraft’s spatial position coordinates x, y, z, velocity v, inclination angle θ and azimuth angle ψ.
[0055] Finally, the color intensity value of the pixel of the digital image is fixed from 0 to 255, while the value range of the state parameters of the flight data is extremely large, especially the position coordinates x, y, and z. Therefore, the difference method is used to preprocess the flight data and reduce the flight parameters of the input network to a fixed range, so that the neural network can fit better and accelerate the convergence of training.
[0056] The modified neural network structure is as follows Figure 2 As shown, it consists of four parts: input, trunk, neck and head.
[0057] Input: D = {d1, d2, ..., d t ,...d T} represents the input flight data, where d t =(x t ,y t , z t , v t ,θ t , ψ t ) has 6 input channels corresponding to 6 state parameters. The flight data is processed by differential operation, so that the vector input to the neural network becomes D = {d′1, d′2, ..., d′ t , ...d′ T}.
[0058] Backbone: The backbone consists of the focus module Focus, the partial connection module CBS, the cross-stage partial connection module C3 and the spatial pyramid pooling module SPP, which extract features from flight data of different scales. Focus downsamples the 6-channel sequence data of length T and expands the channels to produce a 32-channel, The information is expanded from the time dimension to the channel dimension, and the receptive domain of the feature vector increases. CBS uses convolution with a stride of 2 to reduce the length of the feature vector and increase the receptive domain of the feature vector. C3 uses a bottleneck structure to build a residual architecture, which increases the depth of the network and enables the network to learn more abstract features. SPP pools and concatenates multi-scale features, enabling the network to learn object features more comprehensively.
[0059] Neck: The neck consists of two structures, Fpn and Pan, which fuse the information of multi-scale feature vectors. Fpn upsamples and fuses the multi-scale feature vectors from the backbone and neck, passing strong semantic features to large-scale feature vectors. Pan downsamples the multi-scale feature vectors from Fpn, then fuses them down, passing strong position features to small-scale feature vectors. Scale refers to the size of the receptive field of the feature vector in the network. The final feature vector output by the flight data YOLOv5 network is at three scales.
[0060] Head: The head regularizes the multi-scale feature vector set of the neck through convolution to produce three feature vector sets and The lengths are and The number of channels is 9. The feature vector set F i ={f1, f2, ..., f N} represents the prediction result of the i-th scale, and N is respectively and The input flight data is divided into N time intervals equally according to time. If the time center of the aircraft maneuver falls within the nth time interval, then the eigenvector Responsible for identifying maneuvers, where and The start time, end time, confidence level and probability of maneuver category j are predicted respectively. The final recognition result is based on the category corresponding to the highest probability value.
[0061] (3) Set the loss function:
[0062] The prediction result of the network is mapped to the [0,1] interval through the Sigmoid activation function, and then the error between it and the true value is estimated.
[0063] Loss function for flight maneuver identification It consists of three parts: maneuver confidence loss Maneuver segmentation point regression loss and maneuver type classification loss
[0064]
[0065] Among them, the maneuver confidence loss Calculated by binary cross entropy, the regression loss of the mobile segmentation point Calculated by mean square error, maneuver type classification loss It is calculated by binary cross entropy. The three λ correspond to hyperparameters, which are also weight coefficients.
[0066] Specifically, the maneuver confidence loss Calculated by binary cross entropy:
[0067]
[0068] Among them, if the time center of the maneuver falls within the nth time interval, is equal to 1, Equal to 0, otherwise is equal to 0, Equal to 1. Since most of the Equal to 1, set the hyperparameter λ nomnr To balance The values of the two terms in . BEC stands for binary cross entropy function.
[0069] Maneuver segmentation point regression loss Calculated by mean square error:
[0070]
[0071] where t n is the center time of the nth time interval. MSE is the mean square error. The variables with triangles in the formula represent the predicted values, and the variables without triangles represent the true values given by the data set.
[0072] Maneuver type classification loss Calculated by binary cross entropy:
[0073]
[0074] Where c represents the number of maneuver categories, which is 6 here.
[0075] (4) Perform model training:
[0076] Randomly generate the initial state parameters, start time and end time t of the aircraft maneuver s ,t e , and randomly generate the corresponding control parameter n in a reasonable range according to Table 1 || , n ⊥and γ, and the maneuvering flight data is solved by numerical integration. The flight data of the six flight maneuvers are inserted into the flight data of the uniform forward flight to obtain a flight data sample.
[0077] Each of the six types of flight maneuvers generates 100 samples, forming a dataset of 600 samples for one training cycle. The hyperparameters of the neural network are set as follows: λ1, λ2, λ3, and λ nomnr The neural network was trained with SDG, a learning rate of 0.0001, a batch size of 12, and 20 epochs.
[0078] The trained model is fed with flight data to be identified and multi-category identification results are given.
[0079] (5) Filter the recognition results:
[0080] The non-maximum suppression (NMS) algorithm is used to filter the multi-category recognition results of the neural network. The intersection over union (IOU) is used to measure the true value of the maneuvering time interval (start time t s and end time t e ) and predicted value (start time and end time ). The larger the intersection over union (IOU), the higher the correlation. In the neural network prediction results, the ones with an intersection over union (IOU) lower than the preset IOU threshold will be filtered out. The confidence threshold of the non-maximum suppression algorithm is set to 0.9. The recognition result obtained after non-maximum suppression filtering is used as the final recognition result of the flight maneuver.
[0081] In order to have a clearer understanding of the recognition results, we plot the precision-recall curves at different IOU thresholds. Figure 3 As shown in the figure, when the IOU threshold in the NMS algorithm is set to 0.7 or lower, the AUC value of the precision-recall curve is high, the recall rate of the model exceeds 0.8 and is close to 0.9, and the precision level is about 0.9. On the contrary, when the IOU threshold is set to be greater than 0.7, the AUC value of the precision-recall curve decreases as the IOU threshold increases. It can be seen that the improved neural network based on YOLOv5 can segment and identify flight maneuvers in flight data.
[0082] Since the existing neural network model for identifying flight maneuvers does not have segmentation capabilities, the recognition results of the maneuvering flight data segment of the neural network proposed in the present invention are converted into the recognition accuracy of each sampling point in the flight data and compared with the existing neural network model for identifying flight maneuvers. When the confidence threshold of the NMS algorithm is set to 0.9, the recognition results of the neural network are filtered and converted into the recognition accuracy of the sampling points, which are compared with the recognition accuracy of the three-layer LSTM neural network. The results are shown in Table 2. It can be seen that the recognition accuracy of the sampling points of the neural network proposed in the present invention is comparable to that of the three-layer LSTM model.
[0083] Table 2 Comparison of recognition accuracy of two models
[0084]
[0085]
[0086] The present invention designs a suitable neural network and loss function to allow the neural network to learn segmentation and classification tasks simultaneously. Training multiple tasks simultaneously can save time and computing resources while maintaining a comparable or even better task performance level.
[0087] The present invention also provides an aerial target maneuvering action recognition system based on a YOLO network, comprising:
[0088] Model training module, used to transform the track time series D = {d1, d2, ..., d t ,...d T} Input the neural network model improved based on YOLOv5, and train the neural network model according to the set training related parameters; where d t =(x t ,y t , z t , v t ,θ t , ψ t ) is the flight parameter vector of the aerial target at time t, which are the spatial position coordinates x, y, z, speed c, inclination angle θ and azimuth angle ψ, and T is the length of the time series; the neural network model includes four parts: input, trunk, neck and head. The input part processes the input data by differential operation to reduce the flight parameters of the input network to a fixed interval; the trunk extracts features from input data of different scales; the neck is composed of two structures, Fpn and Pan, which integrate the information of multi-scale feature vectors; the head regularizes the multi-scale feature vector set of the neck by convolution to generate three feature vector sets; the input data is equally divided into N time intervals according to time. When the time center of the aerial target maneuver falls within the nth time interval, the corresponding feature vector Responsible for identifying maneuvers, and They represent the start time, end time, confidence level and probability of maneuver category j of the predicted maneuver respectively;
[0089] The maneuver identification module is used to input the track time series of the aerial target to be identified into the trained neural network model, and the neural network model gives a multi-category recognition result. The multi-category recognition result of the neural network model is filtered using the non-maximum suppression algorithm, and the recognition result obtained after non-maximum suppression filtering is used as the final recognition result of the aerial target maneuver.
[0090] Among them, the specific implementation of the structural setting of the neural network model, the parameter setting during the training process, and the loss function setting can refer to the relevant description in the above method embodiment, which will not be repeated here.
[0091] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the aerial target maneuvering action recognition method based on the YOLO network as described above are implemented.
[0092] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for identifying aerial target maneuvers based on the YOLO network as described above are implemented.
[0093] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices (systems), computer equipment or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0094] The present invention is described with reference to a flowchart of a method according to an embodiment of the present invention. It should be understood that each process in the flowchart and a combination of processes in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.
[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.
[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.
Claims
1. A method for identifying aerial target maneuvers based on a YOLO network, characterized in that: The following steps are involved: The track time series of the aerial target D = {d1, d2, ..., d t ,...d T } Input the neural network model improved based on YOLOv5, and train the neural network model according to the set training related parameters; where d t =(x t ,y t , z t , v t ,θ t , ψ t ) is the flight parameter vector of the aerial target at time t, which are the spatial position coordinates x, y, z, speed v, inclination angle θ and azimuth angle ψ, and T is the length of the time series; the neural network model includes four parts: input, trunk, neck and head. The input part processes the input data by differential operation to reduce the flight parameters of the input network to a fixed interval; the trunk extracts features from input data of different scales; the neck is composed of two structures, Fpn and Pan, which integrate the information of multi-scale feature vectors; the head regularizes the multi-scale feature vector set of the neck by convolution to generate three feature vector sets; the input data is equally divided into N time intervals according to time. When the time center of the aerial target maneuver falls within the nth time interval, the corresponding feature vector Responsible for identifying maneuvers, and They represent the start time, end time, confidence level and probability of maneuver category j of the predicted maneuver respectively; The track time series of the aerial target to be identified is input into the trained neural network model, and the neural network model gives multi-category recognition results. The multi-category recognition results of the neural network model are filtered using the non-maximum suppression algorithm, and the recognition results obtained after non-maximum suppression filtering are used as the final recognition results of the aerial target maneuvers.
2. The method according to claim 1, characterized in that The backbone of the neural network model consists of a focus module, a partial connection module, a cross-stage partial connection module, and a spatial pyramid pooling module. The focus module downsamples the 6-channel sequence data of length T and expands the channels to generate a 32-channel, The length of the feature vector set; the partial connection module uses a convolution with a step size of 2 to reduce the length of the feature vector and increase the receptive field of the feature vector; the cross-stage partial connection module uses a bottleneck structure to build a residual architecture, increase the depth of the network, and enable the network to learn more abstract features; The spatial pyramid pooling module pools and concatenates multi-scale features, enabling the network to learn object features more comprehensively.
3. The method according to claim 1, characterized in that: In the neck mechanism of the neural network model, Fpn upsamples and fuses the multi-scale feature vectors from the trunk and neck, and passes the strong semantic features to the first-scale feature vector; Pan downsamples the multi-scale feature vector from Fpn, and then fuses it downward to pass the strong position features to the second-scale feature vector.
4. The method according to claim 1, characterized in that The head of the neural network model generates three feature vector sets: and The lengths are and The number of channels is 9, and the feature vector set F i ={f1, f2, ..., f N } represents the prediction result of the i-th scale, and N is and 5. The method according to claim 1, characterized in that The neural network model is improved based on YOLOv5, specifically including: Replace CNN2d in the YOLOv5 network that processes two-dimensional image data with CNN1d to process the flight data of the control target, and modify the batch normalization operation BatchNorm, the connection operation Concat, and the upsampling operation Upsample originally used for two-dimensional feature vectors in the neural network model accordingly to apply them to one-dimensional feature vectors, making the network model suitable for one-dimensional time series data; The three input channels of YOLOv5, corresponding to the red, green, and blue color channels of the input image, are expanded to six, corresponding to the six parameters of the detection data, namely, the spatial position coordinates x, y, z, velocity v, inclination angle θ, and azimuth angle ψ; The difference method is set to preprocess the input track time series so that the flight parameters of the input network are reduced to a fixed interval.
6. The method according to claim 1, characterized in that The loss function of flight maneuver recognition during neural network model training It consists of three parts: maneuver confidence loss Maneuver segmentation point regression loss and maneuver type classification loss Among them, the maneuver confidence loss Calculated by binary cross entropy, the regression loss of the mobile segmentation point Calculated by mean square error, maneuver type classification loss Calculated by binary cross entropy, λ1, λ2, and λ3 are the corresponding hyperparameters and weight coefficients respectively; Maneuver Confidence Loss Calculated by binary cross entropy: in, and are two binary parameters used to indicate whether the time center of the maneuver falls within the nth time interval. If so, then is equal to 1, Equal to 0, otherwise is equal to 0, Equal to 1; hyperparameter λ nomnr Used for balance The values of the two items in ; BEC represents the binary cross entropy function; Maneuver segmentation point regression loss Calculated by mean square error: where t n is the center time of the nth time interval, MSE stands for mean square error; Maneuver type classification loss Calculated by binary cross entropy: Where c represents the number of maneuver categories.
7. The method according to claim 1, characterized in that The intersection-over-union ratio (IOU) is used to measure the true value start time t of the maneuvering time interval. s and end time t e Start time with predicted value and end time In the prediction results of the neural network model, those with IOU lower than the preset threshold are filtered out.
8. An aerial target maneuvering action recognition system based on YOLO network, characterized in that: include: The model training module is used to transform the track time series D = {d1, d2, ..., d t , …d T } Input the neural network model improved based on YOLOv5, and train the neural network model according to the set training related parameters; where d t =(x t ,y t , z t , v t ,θ t , ψ t ) is the flight parameter vector of the aerial target at time t, which are the spatial position coordinates x, y, z, speed v, inclination angle θ and azimuth angle ψ, and T is the length of the time series; the neural network model includes four parts: input, trunk, neck and head. The input part processes the input data by differential operation to reduce the flight parameters of the input network to a fixed interval; the trunk extracts features from input data of different scales; the neck is composed of two structures, Fpn and Pan, which integrate the information of multi-scale feature vectors; the head regularizes the multi-scale feature vector set of the neck by convolution to generate three feature vector sets; the input data is equally divided into N time intervals according to time. When the time center of the aerial target maneuver falls within the nth time interval, the corresponding feature vector Responsible for identifying maneuvers, and They represent the start time, end time, confidence level and probability of maneuver category j of the predicted maneuver respectively; The maneuver identification module is used to input the track time series of the aerial target to be identified into the trained neural network model, and the neural network model gives a multi-category recognition result. The multi-category recognition result of the neural network model is filtered using the non-maximum suppression algorithm, and the recognition result obtained after non-maximum suppression filtering is used as the final recognition result of the aerial target maneuver.
9. A computer device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the aerial target maneuvering action recognition method based on the YOLO network as described in any one of claims 1-7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the aerial target maneuvering action recognition method based on the YOLO network as described in any one of claims 1-7 are implemented.
Citation Information
Patent Citations
Improved YOLO and SIFT combined multi-small-target detection and tracking method for unmanned aerial vehicle
CN111666871A
Human body behavior recognition method and system based on YOLOv3-SPP
CN113536885A
Lightweight target image recognition method and device and storage medium
CN115170926A
Method and system for training neural network for entity detection
US20230196748A1
