Ship classification and recognition method and system based on track characteristics and deep neural network MobileNet transfer training

Abstract

The present invention discloses a ship classification and recognition method and system based on track features and deep neural network MobileNet migration training; the method comprises: first, mapping the features such as speed, heading and acceleration mined from track data to RGB color space, and converting them into track feature image data; then, migrating the deep neural network MobileNet and its ImageNet pre-training weights, and training the network using the previously generated track feature image data; finally, obtaining a ship type recognition model to realize ship classification and recognition. The present invention has obvious advantages in recognition accuracy and speed, and can be effectively applied to the classification and recognition of ship targets.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition, and in particular relates to a ship classification and recognition method and system based on track features and deep neural network MobileNet migration training. Background Art

[0002] In the era of big data, we can use a large amount of track data to conduct in-depth research on maritime traffic laws and ship behavior patterns, and provide strong information support for maritime supervision, abnormal behavior identification and port facility construction. However, accurate identification of ship targets is a prerequisite for real-time monitoring and even further research. Especially in special sea areas with intertwined routes and complex sea conditions, targeted intervention measures can be taken to ensure maritime traffic safety by timely identifying unknown ship target types. In addition, the precision-guided weapons, surface unmanned ships and integrated maritime reconnaissance and attack equipment used in modern naval warfare have a particularly obvious demand for ship target classification, and target identification technology has become a key factor affecting combat effectiveness. Therefore, accurate identification of ship targets is of great significance for target intention identification, threat warning and tracking and monitoring.

[0003] Accurate target recognition is the prerequisite for further mining of track data. However, current research on track data mainly focuses on mining marine traffic or movement patterns, ship collision risk assessment and marine anomaly identification, and there is relatively little research on ship target type identification. At present, there are roughly the following types of ship target identification methods, including identification based on radiation noise signals, identification based on radar echo signals, identification based on satellite remote sensing or synthetic aperture radar, identification based on infrared or visible light images, and multi-information fusion identification technology that widely uses deep learning technology [Reference [1]: Ma Xiao, Shao Limin, Jin Xin, et al. Research progress in ship target recognition technology [J]. Science and Technology Review, 2019, 37(24): 65-78.]. From the perspective of trajectory data mining, the literature [2] [ZHU Y, ZHENG Y, ZHANG L, et al. Inferring Taxi Status Using GPS Trajectories [J]. Computer Science, 2012.] combines trajectory features with geographic semantic information to analyze the target motion pattern and then predict the vehicle type and destination, but the prediction accuracy is low, and the accuracy of the three-classification is only 72.1%. The target classification method based on regional trajectory clustering mainly focuses on regional clustering to find out whether most of the trajectories are in one area, with an accuracy of 84.4% and a recognition speed of 9 ships / second [Literature [3]: LEE J, HAN J, LI X, et al. TraClass: trajectory classification using hierarchical region-based and trajectory-based clustering [J]. Proceedings of the VLDB Endowment, 2008, 1 (1): 1081-1094.]. However, the above two methods are only applicable to fixed regular traffic patterns, without in-depth mining of trajectory features, and cannot identify target types through irregular ship track features. The support vector machine method first uses the piecewise linear segmentation method to determine the dynamic and static conditions, and then defines the similarity of the segmented ship motion trajectory as the kernel to classify the ship motion mode [Reference [4]: ​​DE VRIES GKD, van SOMERENM. Machine learning for vessel trajectories using compression, alignments and domain knowledge [J]. Expert Systems with Applications, 2012, 39(18): 13426-13439.].However, this method is only applicable to a small area of ​​sea. Under the premise of using a large amount of sample data for training, the recognition accuracy is only 76.3%, and only 2 ship targets can be recognized per second. In areas with less track data, the model performance improvement space is limited and the recognition efficiency is not high. Reference [5] [M.ELWAKDY MEA MA Novel Spatial Algorithm for Similar and Non-Similar Trajectories Classification of Different Objects [C] / / The 2015 International Conference on Frontiers in Education: Computer Science and Computer Engineering (FECS'15) 2015: 234-238.] extracts track features from ship track data through segmentation and polynomial fitting, and uses the ship motion shape to establish a classifier. Although this method has an accuracy of up to 99.5% in the binary classification experiment, its classification prediction relies on manual operation, the recognition efficiency is low, and the recognition delay is not comparable. The recognition accuracy of the classifier trained using the recurrent neural network reaches 88.4%, but the recurrent neural network cannot be parallelized using GPU, and the recognition speed is limited, and only 25 ship targets can be recognized per second. Literature [6][NGUYEN D, VADAINE R, HAJDUCH G, et al. A Multi-Task Deep Learning Architecture for Maritime Surveillance Using AIS Data Streams [C] / / 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA): IEEE, 2018: 331-340.]. Although literature [7][CHEN X, LIU Y, ACHUTHAN K, et al. A ship movement classification based on Automatic Identification System (AIS) data using Convolutional Neural Network [J]. Ocean Engineering, 2020, 218: 168-182.] provides a method for studying target motion patterns, the application of RGB color space in this paper provides a new idea for the present invention to extract track features more specifically.Accurate identification and real-time monitoring of ship targets require consideration of both accuracy and recognition speed. However, the above studies mainly focus on unilaterally improving the accuracy of ship target recognition, without considering the simultaneous importance of recognition speed and delay in the problem of ship target recognition. Summary of the invention

[0004] Aiming at the problem that it is difficult to meet the actual recognition needs of specific ship target types by directly utilizing the rich track data in hot spots, the present invention proposes a ship classification and recognition method and system based on track features and deep neural network MobileNet migration training, which transforms the ship type recognition problem based on track motion features into an image recognition problem, and improves the efficiency of training and using deep neural networks by migrating pre-trained weights, so as to realize the recognition of ships of different categories.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] On the one hand, the present invention proposes a ship classification and recognition method based on track features and deep neural network MobileNet migration training, comprising:

[0007] Step 1: Preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals;

[0008] Step 2: Extract the speed, heading and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively;

[0009] Step 3: Extract the longitude and latitude of the moving target as position attributes and project them to the corresponding track grid points. At the same time, synthesize the three components of R, G, and B mapped to the RGB color space in step 2. The color change of each track grid point is used to represent the change of the state attribute. Traverse the track grid points of different colors of the track data to form a complete track feature image, thereby realizing the conversion of track data to track feature image data;

[0010] Step 4: For the converted track feature image data, the method of transferring pre-trained weights is used to train a ship classification and recognition model based on the deep neural network MobileNet. The trained model is used to recognize the track feature image data converted from the real-time track data, and the ship type is output to achieve classification and recognition of the ship.

[0011] Furthermore, the step 1 comprises:

[0012] Step 1.1: Filter out the complete track data of a single ship target by ship type identification number;

[0013] Step 1.2: Delete the entire track with abnormal data to ensure that there is no noise interference in the subsequent model training. The abnormal data includes missing data points, too long interval time, too low speed for a long time, and no change in longitude and latitude.

[0014] Step 1.3: Sample the filtered track data at uniform time intervals to obtain the final number of track points.

[0015] Furthermore, the step 2 comprises:

[0016] Step 2.1: Traverse all track data and extract the heading, speed and acceleration corresponding to each track point in the track;

[0017] Step 2.2: Calculate the rate of change of velocity, heading and acceleration between two adjacent track points respectively;

[0018] Step 2.3: Normalize the rate of change of velocity, heading, and acceleration at each track point;

[0019] Step 2.4: Map the normalized rate of change to the three components R, G, and B of the RGB color space respectively.

[0020] Furthermore, the step 3 comprises:

[0021] Step 3.1: Traverse and extract the longitude and latitude information of the track to determine the target track;

[0022] Step 3.2: Determine the latitude and longitude range and the center point position of the track image, the latitude and longitude range of the target movement, and the track grid point size;

[0023] Step 3.3: Determine the track grid points that need to be marked on the map based on the latitude and longitude information of the track points; including:

[0024] Use equations (5) and (6) to calculate the common longitude and latitude center point where the track appears:

[0025]

[0026]

[0027] Among them, LAT i Indicates the latitude of the i-th track point, LOG i represents the longitude of the ith track point, T * Indicates the number of track points;

[0028] Use equations (7) and (8) to calculate the motion range of the track:

[0029] X log =Max(LOG)-Min(LOG) (7)

[0030] Y lat =Max(LAT)-Min(LAT) (8)

[0031] Among them, X log is the latitude range, Y lat is the longitude range; LAT represents the latitude set of all track points, and LOG represents the longitude set of all track points;

[0032] Use equations (9) and (10) to calculate the longitude and latitude range of a single track grid point:

[0033]

[0034]

[0035] Among them, P log Indicates the longitude range of the track grid points, Q lat Indicates the latitude range of the track grid points.

[0036] Step 3.4: After determining the track grid points that need to be marked, synthesize the three components of R, G, and B mapped to the RGB color space in step 2; then traverse and synthesize all points on the track to generate a complete track feature image, realizing the conversion of track data to track feature image data.

[0037] Furthermore, the step 4 comprises:

[0038] Step 4.1: Migrate the pre-trained weights from the ImageNet image dataset to the deep neural network MobileNet, so that MobileNet has the ability to recognize common features of image data;

[0039] Step 4.2: Input the track feature image data, train the ship classification and recognition model composed of MobileNet and the classifier, identify the track feature image through the trained ship classification and recognition model, and output the ship type.

[0040] Furthermore, the ship classification and recognition model is functionally divided into three parts: a data input layer, an intermediate layer and a classifier. Each part is composed of a stack of several convolutional layers. Multiple neurons form a convolutional layer, and the layers are connected to each other through the weights between neurons.

[0041] Furthermore, in step 4.2, the ship classification and recognition model composed of MobileNet and the classifier is trained, including:

[0042] 1) Input track feature image data, the neurons in each layer calculate the output vector in turn and pass it to the next layer for forward propagation, and the classifier finally outputs the ship type label value;

[0043] 2) Calculate the error between the label value output by the classifier and the actual ship type label value corresponding to the track feature image. When the error is greater than the set threshold, calculate the error of each neuron output vector layer by layer through the back propagation process, and update the neuron connection weights between layers according to the error;

[0044] 3) Repeat the forward propagation process until the label value output by the model output layer is less than or equal to the set threshold, and then the training ends;

[0045] 4) Update the connection weights of all neurons in the model according to the final label value error.

[0046] On the other hand, the present invention proposes a ship classification and recognition system based on track features and deep neural network MobileNet migration training, comprising:

[0047] The data preprocessing module is used to preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals;

[0048] The mapping module is used to mine and extract the speed, heading and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively;

[0049] The data conversion module is used to extract the longitude and latitude of the moving target as the position attribute and project it to the corresponding track grid point. At the same time, the three components of R, G, and B mapped to the RGB color space in the synthesis mapping module are used to represent the change of the state attribute through the color change of each track grid point. The track grid points of different colors of the track data are traversed to form a complete track feature image, thereby realizing the conversion of track data to track feature image data;

[0050] The model training and recognition module is used to train the ship classification and recognition model based on the deep neural network MobileNet for the converted track feature image data, adopt the method of migrating pre-trained weights, recognize the track feature image data converted from real-time track data through the trained model, output the ship type, and realize the classification and recognition of the ship.

[0051] Furthermore, the data preprocessing module is specifically used for:

[0052] Step 1.1: Filter out the complete track data of a single ship target by ship type identification number;

[0053] Step 1.2: Delete the entire track with abnormal data to ensure that there is no noise interference in the subsequent model training. The abnormal data includes missing data points, too long interval time, too low speed for a long time, and no change in longitude and latitude.

[0054] Step 1.3: Sample the filtered track data at uniform time intervals to obtain the final number of track points.

[0055] Furthermore, the mapping module is specifically used for:

[0056] Step 2.1: Traverse all track data and extract the heading, speed and acceleration corresponding to each track point in the track;

[0057] Step 2.2: Calculate the rate of change of velocity, heading and acceleration between two adjacent track points respectively;

[0058] Step 2.3: Normalize the rate of change of velocity, heading, and acceleration at each track point;

[0059] Step 2.4: Map the normalized rate of change to the three components R, G, and B of the RGB color space respectively.

[0060] Furthermore, the data conversion module is specifically used for:

[0061] Step 3.1: Traverse and extract the longitude and latitude information of the track to determine the target track;

[0062] Step 3.2: Determine the latitude and longitude range and the center point position of the track image, the latitude and longitude range of the target movement, and the track grid point size;

[0063] Step 3.3: Determine the track grid points that need to be marked on the map based on the latitude and longitude information of the track points; including:

[0064] Use equations (5) and (6) to calculate the common longitude and latitude center point where the track appears:

[0065]

[0066]

[0067] Among them, LAT i Indicates the latitude of the i-th track point, LOG i represents the longitude of the ith track point, T * Indicates the number of track points;

[0068] Use equations (7) and (8) to calculate the motion range of the track:

[0069] X log=Max(LOG)-Min(LOG) (7)

[0070] Y lat =Max(LAT)-Min(LAT) (8)

[0071] Among them, X log is the latitude range, Y lat is the longitude range; LAT represents the latitude set of all track points, and LOG represents the longitude set of all track points;

[0072] Use equations (9) and (10) to calculate the longitude and latitude range of a single track grid point:

[0073]

[0074]

[0075] Among them, P log Indicates the longitude range of the track grid points, Q lat Indicates the latitude range of the track grid points.

[0076] Step 3.4: After determining the track grid points that need to be marked, synthesize the three components of R, G, and B mapped to the RGB color space in step 2; then traverse and synthesize all points on the track to generate a complete track feature image, realizing the conversion of track data to track feature image data.

[0077] Furthermore, the model training and recognition module is specifically used for:

[0078] Step 4.1: Migrate the pre-trained weights from the ImageNet image dataset to the deep neural network MobileNet, so that MobileNet has the ability to recognize common features of image data;

[0079] Step 4.2: Input the track feature image data, train the ship classification and recognition model composed of MobileNet and the classifier, identify the track feature image through the trained ship classification and recognition model, and output the ship type.

[0080] Furthermore, the ship classification and recognition model is functionally divided into three parts: a data input layer, an intermediate layer and a classifier. Each part is composed of a stack of several convolutional layers. Multiple neurons form a convolutional layer, and the layers are connected to each other through the weights between neurons.

[0081] Furthermore, in step 4.2, the ship classification and recognition model composed of MobileNet and the classifier is trained, including:

[0082] 1) Input track feature image data, the neurons in each layer calculate the output vector in turn and pass it to the next layer for forward propagation, and the classifier finally outputs the ship type label value;

[0083] 2) Calculate the error between the label value output by the classifier and the actual ship type label value corresponding to the track feature image. When the error is greater than the set threshold, calculate the error of each neuron output vector layer by layer through the back propagation process, and update the neuron connection weights between layers according to the error;

[0084] 3) Repeat the forward propagation process until the label value output by the model output layer is less than or equal to the set threshold, and then the training ends;

[0085] 4) Update the connection weights of all neurons in the model according to the final label value error.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] The present invention first mines the track data to obtain the speed, heading and acceleration of the ship between each timestamp, and then uses the change rate of the three as the state attribute vector, and the longitude and latitude and timestamp information as the position attribute vector; then, the track feature extraction and conversion method based on the RGB color space proposed by the present invention is used to extract and convert the data to obtain track feature image data; finally, the track feature image is used as training data, the ImageNet pre-training weights are transferred, and a ship classification and recognition model is trained based on the deep neural network MobileNet. The real-time track feature image is recognized through the trained model to realize the classification and recognition of the ship type.

[0088] The present invention solves the problem of ship type recognition based on track features by using a method for identifying track feature images, and by migrating pre-trained weights, MobileNet directly obtains data features, thereby having recognition capabilities, greatly improving the efficiency of the method. Experiments have also verified that the accuracy and average recognition speed of the present invention have obvious advantages in the problem of ship target classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 This is a flow chart of a ship classification and recognition method based on track features and deep neural network MobileNet migration training according to an embodiment of the present invention;

[0090] Figure 2 A schematic diagram of RGB state attribute vector mapping of a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention;

[0091] Figure 3A schematic diagram of position attribute vector extraction of a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention;

[0092] Figure 4 A schematic diagram of a track feature image converted by a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention;

[0093] Figure 5 A flow chart of a migration training method for a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention;

[0094] Figure 6 A schematic diagram of model training of a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention;

[0095] Figure 7 This is a model training flow chart of a ship classification and recognition method based on track features and deep neural network MobileNet migration training in an embodiment of the present invention. DETAILED DESCRIPTION

[0096] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0097] like Figure 1 As shown, a ship classification and recognition method based on track features and deep neural network MobileNet migration training includes:

[0098] Step 1: Preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals to ensure the quality of data conversion, reduce the interference of data noise on model training, and improve the accuracy of model recognition;

[0099] Step 2: Extract the speed, heading and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively;

[0100] Step 3: Extract the longitude and latitude of the moving target as position attributes and project them to the corresponding track grid points. At the same time, synthesize the three components of R, G, and B mapped to the RGB color space in step 2. The color change of each track grid point is used to represent the change of the state attribute. Traverse the track grid points of different colors of the track data to form a complete track feature image, thereby realizing the conversion of track data to track feature image data;

[0101] Step 4: For the converted track feature image data, the method of transferring pre-trained weights is used to train a ship classification and recognition model based on the deep neural network MobileNet. The trained model is used to recognize the track feature image data converted from the real-time track data, and the ship type is output to achieve classification and recognition of the ship.

[0102] Furthermore, the step 1 comprises:

[0103] Step 1.1: Filter out the complete track data of a single ship target by ship type identification number;

[0104] Step 1.2: Delete the entire track with abnormal data to ensure that there is no noise interference in the subsequent model training. The abnormal data includes missing data points, too long interval time, too low speed for a long time, and no change in longitude and latitude.

[0105] Step 1.3: Sample the filtered track data at uniform time intervals to obtain the final number of track points T * :

[0106] T * =Sampling(N,△t) (1)

[0107] Wherein, N represents the number of original track points in the track data, and Δt represents the equal time interval after resampling the N track data.

[0108] Furthermore, the step 2 comprises:

[0109] Step 2.1: Traverse all track data and extract the heading, speed and acceleration corresponding to each track point in the track;

[0110] Step 2.2: Calculate the rate of change of velocity, heading and acceleration between two adjacent track points by using equations (2), (3) and (4) respectively;

[0111]

[0112]

[0113]

[0114] Among them, △Cog, △Sog, and △Acceleration are the rates of change of velocity, heading, and acceleration between two adjacent track points, respectively. i+1 、cog i They represent the heading at the i+1th moment and the ith moment respectively, △t represents the interval time between two adjacent track points, sog i+1 、sog i 、sogi-1 They represent the speed at the i+1th moment, the ith moment, and the i-1th moment respectively.

[0115] Step 2.3: Normalize the rate of change of velocity, heading, and acceleration at each track point;

[0116] Step 2.4: Map the normalized rate of change to the three components of R, G, and B in the RGB color space, such as Figure 2 As shown. R, G, and B represent the three primary colors of red, green, and blue, namely the three components in the RGB color space. Different combinations of components will synthesize different colors. Figure 3 The three-dimensional space coordinate axis in the color space marks the colors synthesized by some special RGB component combinations in the color space, and marks the component values. Each component has 256 values. The color space composed of three components is huge, and the state attribute changes of the target can be expressed by the change of color.

[0117] Furthermore, if Figure 4 As shown, the step 3 includes:

[0118] Step 3.1: Traverse and extract the longitude and latitude information of the track to determine the target track;

[0119] Step 3.2: Determine the latitude and longitude range and the center point position of the track image, the latitude and longitude range of the target movement, and the track grid point size. Figure 4 The grey hollow triangle in part (b) is the common center point of longitude and latitude of all tracks;

[0120] Step 3.3: Determine the track grid points that need to be marked on the map based on the latitude and longitude information of the track points. The gray squares in part (c) of Figure 4 are track grid points; including:

[0121] Use equations (5) and (6) to calculate the common longitude and latitude center point where the track appears:

[0122]

[0123]

[0124] Among them, LAT i Indicates the latitude of the i-th track point, LOG i represents the longitude of the ith track point, T * Indicates the number of track points;

[0125] Use equations (7) and (8) to calculate the motion range of the track:

[0126] X log =Max(LOG)-Min(LOG) (7)

[0127] Y lat =Max(LAT)-Min(LAT) (8)

[0128] Among them, X log is the latitude range, Y lat is the longitude range; LAT represents the latitude set of all track points, and LOG represents the longitude set of all track points;

[0129] Use equations (9) and (10) to calculate the longitude and latitude range of a single track grid point:

[0130]

[0131]

[0132] Among them, P log Indicates the longitude range of the track grid points, Q lat Indicates the latitude range of the track grid points.

[0133] Step 3.4: After determining the track grid points that need to be marked, synthesize the three components of R, G, and B mapped to the RGB color space in step 2; Figure 3 The gray track grid points marked in the figure will use different colors to represent the motion characteristics of the target at that point; then all points on the track are traversed and synthesized to generate a complete track feature image (such as Figure 4 As shown), the track data is converted into track characteristic image data.

[0134] Furthermore, if Figure 5 As shown, the present invention solves the problem of ship type recognition based on track features by using a method for identifying track feature images, and improves the efficiency of training and using deep neural networks by using a transfer learning method. The step 4 specifically includes:

[0135] Step 4.1: Migrate the pre-trained weights from the ImageNet large image dataset to the deep neural network MobileNet, so that MobileNet has the ability to recognize common features of image data;

[0136] Specifically, in existing methods, a large amount of data is needed to train deep neural networks, obtain feature information from the data, and then convert them into corresponding weights, but this will take a lot of time to train. The present invention enables MobileNet to directly obtain data features by migrating pre-trained weights, thereby having recognition capabilities, greatly improving the efficiency of the method. The migrated pre-trained weights are extracted from a neural network model trained with the ImageNet large image dataset, which is a large image dataset with 1,000 categories and more than 1.5 million samples, covering a wealth of common features of image data. Therefore, migrating the pre-trained weights can enable MobileNet to have the ability to recognize common features of track feature image data.

[0137] Step 4.2: Input the track feature image data, train the ship classification and recognition model composed of MobileNet and the classifier, identify the track feature image through the trained ship classification and recognition model, and output the ship type.

[0138] The purpose of training the model is to enable the model to obtain the individual characteristic information corresponding to the ship type, so as to obtain the ability to identify the track characteristic image data, so that it can accurately identify the track characteristic image. The ship classification and recognition model composed of MobileNet and classifier can be divided into three parts according to function: data input layer, intermediate layer and classifier. Each part is composed of several convolutional layers stacked together, such as Figure 6 As shown. Multiple neurons form a convolution layer, and the layers are connected to each other through the weights between neurons. During the training process, the process from the data input layer to the classifier output label value is called the forward propagation process. In this process, each convolution layer continuously calculates the output vector and passes it to the next layer, and finally the classifier outputs the corresponding label value; the back propagation process refers to the process from the classifier calculating the output label value error to the reverse direction to calculate the error of the output vector of each convolution layer. The error calculated by each layer continuously updates the neuron connection weights between the layers in the model, so that the label value output by the classifier in the next forward propagation process is more accurate.

[0139] The specific process of model training is as follows Figure 7As shown in the figure: First, input the track feature image data, the neurons of each layer calculate the output vector in turn and pass it to the next layer for forward propagation, and the classifier finally outputs the ship type label value; then, calculate the error between the label value output by the classifier and the real ship type label value corresponding to the track feature image. When the error is greater than the set threshold, calculate the error of the output vector of each neuron layer by layer through the back propagation process, and update the neuron connection weights between layers according to the error; then, repeat the forward propagation process until the label value output by the model output layer is less than or equal to the set threshold, and end the training; finally, update the connection weights of all neurons in the model according to the final label value error. At this point, the individual characteristics of the track feature image data are converted into the connection weight parameters of the model, and the model has the ability to classify and recognize track feature image data.

[0140] To verify the effect of the present invention, the present invention uses the Automatic Identification System (AIS) data set in the Gulf of Mexico in December 2019 for experimental verification. The Gulf of Mexico has dense shipping routes, rich types of ships and large numbers. The data set also records the longitude and latitude and timestamp information of all active ships in the sea area, which is suitable for ship classification and identification research. Data preprocessing includes four parts: first, the complete track data of a single ship target is filtered out by the ship type identification number; then, the track with abnormal data is deleted in its entirety, including data points missing, too long intervals, too low speed for a long time, and no changes in longitude and latitude; then, the filtered track data is sampled at uniform time intervals; finally, the track of each target is divided according to the research period.

[0141] In the experiment, the track feature extraction and conversion method based on RGB color space was first used to extract and convert AIS data, and finally 54,398 track feature images containing three types of ships were obtained. Figure 4 (a), (b), and (c) are the track feature images of a cargo ship, a passenger ship, and a towing ship. Then, using the TensorFlow deep learning framework, we obtain three deep neural network structures, Xception, Resnet-50, and MobileNet, and call the pre-trained weights from the ImageNet large image dataset, and input the training set data to train the model. The neural network parameter settings involved in the experiment are shown in Table 1.

[0142] Table 1 Neural network parameters involved in the experiment

[0143]

[0144]

[0145] The experimental equipment is a Lenovo Thinkpaid X1 computer with 16GB memory, CPU frequency of 2.21GHz, and a NVIDIA GeForce GTX 1050Ti graphics card with 8GB video memory. On the Win 10 / 64-bit operating system, the Python 3.8.5 programming language is used. Through experimental tests, the average accuracy and ship recognition speed of the three networks are shown in Table 2.

[0146] Table 2 Average accuracy and ship recognition speed of three networks

[0147]

[0148] The experimental results show that the average accuracy of the present invention is as high as 96.29%, which is 5.28% and 2.34% higher than the recognition accuracy of the models trained by Resnet-50 and Xception respectively. The average recognition speed is 363 ships / second, which is 2.8 times the speed of the model trained by the deep neural network Resnet-50.

[0149] Table 3 compares the present invention and existing research results in terms of recognition accuracy and recognition speed.

[0150] Table 3 Comparison with existing research results

[0151]

[0152] Although the accuracy of the trajectory classification algorithm (TCA) proposed in the literature [5] [M.ELWAKDY MEA MA Novel Spatial Algorithm for Similar and Non-Similar Trajectories Classification of Different Objects [C] / / The 2015 International Conference on Frontiers in Education: Computer Science and Computer Engineering (FECS'15) 2015: 234-238] is as high as 99.5%, this method can only be applied to binary classification problems, and the recognition speed depends on the speed of manual operation, which has great limitations. The present invention can flexibly increase the types of ships according to the track data, and the recognition accuracy in the three ship classification experiments is as high as 96.29%, and the average recognition speed is 14 times that of the second VRNN method. In summary, the accuracy and average recognition speed of the present invention have obvious advantages in the problem of ship target classification.

[0153] On the basis of the above embodiments, the present invention proposes a ship classification and recognition system based on track features and deep neural network MobileNet migration training, comprising:

[0154] The data preprocessing module is used to preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals;

[0155] The mapping module is used to mine and extract the speed, heading and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively;

[0156] The data conversion module is used to extract the longitude and latitude of the moving target as the position attribute and project it to the corresponding track grid point. At the same time, the three components of R, G, and B mapped to the RGB color space in the synthesis mapping module are used to represent the change of the state attribute through the color change of each track grid point. The track grid points of different colors of the track data are traversed to form a complete track feature image, thereby realizing the conversion of track data to track feature image data;

[0157] The model training and recognition module is used to train the ship classification and recognition model based on the deep neural network MobileNet for the converted track feature image data, adopt the method of migrating pre-trained weights, recognize the track feature image data converted from real-time track data through the trained model, output the ship type, and realize the classification and recognition of the ship.

[0158] Furthermore, the data preprocessing module is specifically used for:

[0159] Step 1.1: Filter out the complete track data of a single ship target by ship type identification number;

[0160] Step 1.2: Delete the entire track with abnormal data to ensure that there is no noise interference in the subsequent model training. The abnormal data includes missing data points, too long interval time, too low speed for a long time, and no change in longitude and latitude.

[0161] Step 1.3: Sample the filtered track data at uniform time intervals to obtain the final number of track points.

[0162] Furthermore, the mapping module is specifically used for:

[0163] Step 2.1: Traverse all track data and extract the heading, speed and acceleration corresponding to each track point in the track;

[0164] Step 2.2: Calculate the rate of change of velocity, heading and acceleration between two adjacent track points respectively;

[0165] Step 2.3: Normalize the rate of change of velocity, heading, and acceleration at each track point;

[0166] Step 2.4: Map the normalized rate of change to the three components R, G, and B of the RGB color space respectively.

[0167] Furthermore, the data conversion module is specifically used for:

[0168] Step 3.1: Traverse and extract the longitude and latitude information of the track to determine the target track;

[0169] Step 3.2: Determine the latitude and longitude range and the center point position of the track image, the latitude and longitude range of the target movement, and the track grid point size;

[0170] Step 3.3: Determine the track grid points that need to be marked on the map based on the latitude and longitude information of the track points; including:

[0171] Use equations (5) and (6) to calculate the common longitude and latitude center point where the track appears:

[0172]

[0173]

[0174] Among them, LAT i Indicates the latitude of the i-th track point, LOG i represents the longitude of the ith track point, T * Indicates the number of track points;

[0175] Use equations (7) and (8) to calculate the motion range of the track:

[0176] X log =Max(LOG)-Min(LOG) (7)

[0177] Y lat =Max(LAT)-Min(LAT) (8)

[0178] Among them, X log is the latitude range, Y lat is the longitude range; LAT represents the latitude set of all track points, and LOG represents the longitude set of all track points;

[0179] Use equations (9) and (10) to calculate the longitude and latitude range of a single track grid point:

[0180]

[0181]

[0182] Among them, P log Indicates the longitude range of the track grid points, Q lat Indicates the latitude range of the track grid points.

[0183] Step 3.4: After determining the track grid points that need to be marked, synthesize the three components of R, G, and B mapped to the RGB color space in step 2; then traverse and synthesize all points on the track to generate a complete track feature image, realizing the conversion of track data to track feature image data.

[0184] Furthermore, the model training and recognition module is specifically used for:

[0185] Step 4.1: Migrate the pre-trained weights from the ImageNet image dataset to the deep neural network MobileNet, so that MobileNet has the ability to recognize common features of image data;

[0186] Step 4.2: Input the track feature image data, train the ship classification and recognition model composed of MobileNet and the classifier, identify the track feature image through the trained ship classification and recognition model, and output the ship type.

[0187] Furthermore, the ship classification and recognition model is functionally divided into three parts: a data input layer, an intermediate layer and a classifier. Each part is composed of a stack of several convolutional layers. Multiple neurons form a convolutional layer, and the layers are connected to each other through the weights between neurons.

[0188] Furthermore, in step 4.2, the ship classification and recognition model composed of MobileNet and the classifier is trained, including:

[0189] 1) Input track feature image data, the neurons in each layer calculate the output vector in turn and pass it to the next layer for forward propagation, and the classifier finally outputs the ship type label value;

[0190] 2) Calculate the error between the label value output by the classifier and the actual ship type label value corresponding to the track feature image. When the error is greater than the set threshold, calculate the error of each neuron output vector layer by layer through the back propagation process, and update the neuron connection weights between layers according to the error;

[0191] 3) Repeat the forward propagation process until the label value output by the model output layer is less than or equal to the set threshold, and then the training ends;

[0192] 4) Update the connection weights of all neurons in the model according to the final label value error.

[0193] In summary, the present invention solves the problem of ship type recognition based on track features by using a method for identifying track feature images, and by migrating pre-trained weights, MobileNet directly obtains data features, thereby having recognition capabilities, greatly improving the efficiency of the method. Experiments have also verified that the accuracy and average recognition speed of the present invention have obvious advantages in the problem of ship target classification.

[0194] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A ship classification and recognition method based on track features and deep neural network MobileNet migration training, characterized in that: include: Step 1: Preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals; Step 2: Extract the speed change rate, heading change rate and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively; Step 3: Extract the longitude and latitude of the moving target as position attributes and project them to the corresponding track grid points. At the same time, synthesize the three components of R, G, and B mapped to the RGB color space in step 2. The color change of each track grid point is used to represent the change of the state attribute. Traverse the track grid points of different colors of the track data to form a complete track feature image, thereby realizing the conversion of track data to track feature image data; Step 4: For the converted track feature image data, the method of transferring pre-trained weights is used to train a ship classification and recognition model based on the deep neural network MobileNet. The trained model is used to recognize the track feature image data converted from the real-time track data, and the ship type is output to achieve classification and recognition of the ship.

2. The ship classification and recognition method based on track features and deep neural network MobileNet migration training according to claim 1 is characterized in that: The step 1 comprises: Step 1.1: Filter out the complete track data of a single ship target by ship type identification number; Step 1.2: Delete the entire track with abnormal data to ensure that there is no noise interference in the subsequent model training. The abnormal data includes missing data points, too long interval time, too low speed for a long time, and no change in longitude and latitude. Step 1.3: Sample the filtered track data at uniform time intervals to obtain the final number of track points.

3. The ship classification and recognition method based on track features and deep neural network MobileNet migration training according to claim 1 is characterized in that: The step 2 comprises: Step 2.1: Traverse all track data and extract the heading, speed and acceleration corresponding to each track point in the track; Step 2.2: Calculate the rate of change of velocity, heading and acceleration between two adjacent track points respectively; Step 2.3: Normalize the rate of change of velocity, heading, and acceleration at each track point; Step 2.4: Map the normalized rate of change to the three components R, G, and B of the RGB color space respectively.

4. The ship classification and recognition method based on track features and deep neural network MobileNet migration training according to claim 3 is characterized in that: The step 3 comprises: Step 3.1: Traverse and extract the longitude and latitude information of the track to determine the target track; Step 3.2: Determine the latitude and longitude range and the center point position of the track image, the latitude and longitude range of the target movement, and the track grid point size; Step 3.3: Determine the track grid points that need to be marked on the map based on the latitude and longitude information of the track points; including: Use equations (5) and (6) to calculate the common longitude and latitude center point where the track appears: Among them, LAT i Indicates the latitude of the i-th track point, LOG i represents the longitude of the ith track point, T * Indicates the number of track points; Use equations (7) and (8) to calculate the range of motion of the track: X log =Max(LOG)-Min(LOG) (7) Y lat =Max(LAT)-Min(LAT) (8) Among them, X log is the latitude range, Y lat is the longitude range; LAT represents the latitude set of all track points, and LOG represents the longitude set of all track points; Use equations (9) and (10) to calculate the longitude and latitude range of a single track grid point: Among them, P log Indicates the longitude range of the track grid points, Q lat Indicates the latitude range of track grid points; Step 3.4: After determining the track grid points that need to be marked, synthesize the three components of R, G, and B mapped to the RGB color space in step 2; then traverse and synthesize all points on the track to generate a complete track feature image, realizing the conversion of track data to track feature image data.

5. The ship classification and recognition method based on track features and deep neural network MobileNet migration training according to claim 1 is characterized in that: The step 4 comprises: Step 4.1: Migrate the pre-trained weights from the ImageNet image dataset to the deep neural network MobileNet, so that MobileNet has the ability to recognize common features of image data; Step 4.2: Input the track feature image data, train the ship classification and recognition model composed of MobileNet and the classifier, identify the track feature image through the trained ship classification and recognition model, and output the ship type.

6. The ship classification and identification method based on track features and deep neural network MobileNet migration training according to claim 5 is characterized in that: The ship classification and recognition model is divided into three parts according to its functions: a data input layer, an intermediate layer and a classifier. Each part is composed of a plurality of stacked convolutional layers. Multiple neurons form a convolutional layer, and the layers are connected to each other through the weights between neurons.

7. The ship classification and identification method based on track features and deep neural network MobileNet migration training according to claim 6 is characterized in that: In step 4.2, the ship classification and recognition model composed of MobileNet and the classifier is trained, including: 1) Input track feature image data, the neurons in each layer calculate the output vector in turn and pass it to the next layer for forward propagation, and the classifier finally outputs the ship type label value; 2) Calculate the error between the label value output by the classifier and the actual ship type label value corresponding to the track feature image. When the error is greater than the set threshold, calculate the error of each neuron output vector layer by layer through the back propagation process, and update the neuron connection weights between layers according to the error; 3) Repeat the forward propagation process until the label value output by the model output layer is less than or equal to the set threshold, and then the training ends; 4) Update the connection weights of all neurons in the model according to the final label value error.

8. A ship classification and recognition system based on track features and deep neural network MobileNet migration training, characterized in that: include: The data preprocessing module is used to preprocess the track data of the marked ship type, including deleting the entire abnormal track data and sampling the filtered track data at equal time intervals; The mapping module is used to mine and extract the speed change rate, heading change rate and acceleration change rate of the ship target track point in the track data as the state attribute vector, and map them to the three components of R, G and B in the RGB color space respectively; The data conversion module is used to extract the longitude and latitude of the moving target as the position attribute and project it to the corresponding track grid point. At the same time, the three components of R, G, and B mapped to the RGB color space in the synthesis mapping module are used to represent the change of the state attribute through the color change of each track grid point. The track grid points of different colors of the track data are traversed to form a complete track feature image, thereby realizing the conversion of track data to track feature image data; The model training and recognition module is used to train the ship classification and recognition model based on the deep neural network MobileNet for the converted track feature image data, adopt the method of migrating pre-trained weights, recognize the track feature image data converted from real-time track data through the trained model, output the ship type, and realize the classification and recognition of the ship.

Images