Sea surface target radar information processing method and system based on integrated machine learning

Abstract

According to the sea surface target information processing system and method based on integrated deep learning provided by the application, the radar echo data is processed by the preprocessing module to obtain processed radar echo data; the target detection module is used for target detection based on the preprocessed radar echo data, and an end-to-end convolutional neural network model is used for target detection to obtain a target track; the target tracking module is used for target track tracking based on the obtained target track, and an end-to-end convolutional neural network model is used for target track tracking to obtain multi-target track information; the correlation fusion module is used for target fusion based on multi-channel target track information, and an end-to-end convolutional neural network model is used for target fusion to obtain fused target track information; and the display control module is used for display control based on the fused target track information, and an end-to-end convolutional neural network model is used for display control.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-module radar information processing, in particular to a sea surface target radar information processing method and system based on integrated machine learning.

[0002] The present application relates to a multi-module radar information processing method, the system input: navigation radar echo; the information processing module adopts an improved machine learning method, combines the characteristics of radar echo, realizes an integrated new multi-module deep learning structure, and completes the target detection, tracking, correlation, fusion, display control and other information processing of radar echo in layers, adopts an information feedback structure and AIS target information supervised learning, and the system realizes intelligent information processing of targets in various sea surface environments, and realizes the detection, tracking, fusion, display control and other system functions of the targets in an integrated manner. BACKGROUND

[0003] Traditional radar detection algorithms face many difficulties, such as weak target detection in strong clutter, parameter adaptation in complex and variable environments, etc. However, these difficulties can be solved in theory. For example, for an environment with poor detection effect, adjust the algorithm parameters to the optimal effect for this environment; record the algorithm parameters at this time and the characteristics of the environment; when encountering such an environment again, call the recorded algorithm parameters; in theory, as long as the number of recorded and targeted processed environments is sufficient, parameter adaptation in complex and variable environments is realized. To this day, parameter adaptation is still a big problem, because the environmental conditions change rapidly, the above process is too tedious and time-consuming, and it is difficult to realize the theoretical exhaustion.

[0004] The root cause of the problem lies in the fact that the essence of traditional radar detection algorithms is an algorithm based on artificial feature operators (for example, the clutter map extraction algorithm in the detection algorithm is an artificial feature operator based on empirical rules), which usually performs poorly outside the theoretical application range of the artificial feature operator used. In other words, traditional radar detection algorithms do not have generalization ability. In addition, even within the theoretical application range, it is very difficult to accurately estimate the distribution model of noise and clutter and master its dynamic characteristics due to the limitations of simulation technology and theoretical level, and the artificial feature operator designed may not achieve good results.

[0005] Radar small target detection in sea clutter environment and target behavior estimation with incomplete information are world-wide problems. After decades of research, various clutter statistical models have been proposed, even nonlinear models and chaotic models, fractal theory, etc. All models and methods seem to have the answer in the shallow function relationship, and the actual target detection performance is not greatly improved. These technical improvements have basically reached the flat top of the improvement technology.

[0006] With the application of artificial intelligence and machine learning methods in game, image recognition, etc., especially the deep learning architecture using multi-layer convolutional neural network, a particularly complex (deep) function relationship is achieved, and some even have "latent knowledge" or "dark knowledge" that humans cannot understand. Machine learning is a new cognitive methodology that humans cannot truly understand but can be tested by practice. The belief that "practice is the standard of truth" has been greatly improved in wisdom. Building a community of destiny with human-machine complementarity, or insisting that machines must explain the deduction and induction process to humans, will face new choices.

[0007] Deep learning is a set of algorithms that model high complexity data through multiple nonlinear transformations, with the advantages of automatic feature extraction and strong migration and generalization capabilities. Deep learning has achieved excellent practical results in search technology, data mining, optical image recognition, speech recognition, natural language processing, personalized recommendation, and many other fields, far surpassing previous traditional technologies. Radar target detection is essentially a binary classification task of radar targets and clutter background, and classification and recognition tasks are the areas where deep learning excels.

[0008] Machine learning has made great progress in image processing, but there are many differences between radar information and image information, such as pulse detection method, fan resolution area, frame scanning, precision varying with distance, azimuth cycle period, unstable distance boundary, and huge target quantity. The formation of sea clutter is very complex, and intelligent improvement is needed in suppressing sea clutter. Target track tracking reflects the front-back correlation of target contour and center, and whether it conforms to the motion characteristics of the target. Multi-station and multi-sensor target track fusion is an effective method of comprehensive processing based on compensation of system errors. Display control and selective feedback of information are the display output of the system and the overall optimization of the system. Each function of the system can be implemented individually and then integrated. Since the relevant information output by each module needs to be fed back to the previous stage information, this cannot achieve system optimization and it is difficult to effectively solve various difficult problems. First, the calculation is too large to be real-time, which may be fatal to radar information processing. Since deep learning calculation needs to run on a parallel computer, this brings huge cost pressure to the entire system. It is necessary to run on a general industrial computer. It is necessary to compress the depth of each deep network, use a cascaded form, and supervise the training of AIS target track to greatly reduce the learning calculation amount. Then, refine the traditional information processing method to have a directional optimization of the network structure in learning and training. In short, it is very meaningful to reduce the calculation amount and optimize the system structure, and further improve the system performance.

[0009] Realize based on radar echo (including various clutter, reflection, noise, interference, various targets, etc.), utilize convolutional neural network and deep learning to carry out preliminary step screening to radar echo, preliminary separation target (focus) and other categories, introduce (target) attention mechanism (Attent ion), extract the feature of the echo similar to the target in image processing, and refine the quasi-target echo;Then utilize middle layer convolutional neural network and deep learning, refer to the motion characteristics of multiple frame echoes, the degree of echo aggregation, etc., filter out the quasi-target echo not belonging to the target, splice the quasi-target echoes of the front and rear frames into target tracks;Utilize upper layer convolutional neural network and deep learning, refer to other (such as AIS, photoelectricity, infrared, laser ranging, other radar information, etc.) target track information and sea channel and region information, effectively separate special targets such as targets close to the target, state conversion (motion to static, or static to motion, sudden acceleration or deceleration, turning, etc.), realize the correlation and fusion output of various (small ships, large ships, tugboats, speedboats, buoys, anchorage targets, etc.) target tracks, and the target tracks that cannot be effectively correlated and fused, the target tracks with particularly complex tracks and too close correlation degree to be distinguished need to feed back information for continuous learning, and the information is fed back to the previous learning network for track reference;Some parts in each learning module can introduce other external information, such as AIS, photoelectricity, feedback of the subsequent processing module, other radar target information, etc., as a reference for machine learning, and the results are labeled;The learning results are automatically inferred according to the network structure for the parts without reference. The whole processing system can automatically complete the system structure and parameter optimization under the human intervention, so that the system is in the best state.

[0010] The patent document CN109934088A (application number: 201910024532.1) discloses a sea surface ship identification method based on deep learning, comprising the following steps: step one, using a UAV to shoot a sea surface ship video as a training video;Step two, performing frame processing on the training video obtained in step one to obtain a plurality of ship pictures to form a training data set;Step three, training the Faster R-CNN algorithm to obtain a final Faster R-CNN model;Step four, using a UAV to shoot a sea surface ship video as a test video;Step five, using the Faster R-CNN model obtained in step three to detect the detection data set obtained in step four, and finally outputting the corresponding ship detection result. SUMMARY

[0011] In view of the defects in the prior art, the purpose of the present application is to provide a sea surface target radar information processing method and system based on integrated machine learning.

[0012] The application provides a sea surface target information processing system based on integrated deep learning, which comprises a preprocessing module, a target detection module, a target tracking module, a correlation fusion module and a display control module.

[0013] The preprocessing module is used for performing echo processing on radar echo data to obtain processed radar echo data and taking the preprocessed radar echo data as first feedback information.

[0014] The target detection module is used for performing target detection on the preprocessed radar echo data by using an end-to-end convolutional neural network model for target detection to obtain target point tracks and taking the obtained target point tracks as second feedback information.

[0015] The target tracking module is used for performing target point track tracking on the obtained target point tracks by using an end-to-end convolutional neural network model for target tracking to obtain multi-target track information and taking the multi-target track information as third feedback information.

[0016] The correlation fusion module is used for performing target fusion on the multi-channel target track information by using an end-to-end convolutional neural network model for fusion to obtain fused target track information and taking the fused target track information as fourth feedback information.

[0017] The display control module is used for performing display control on the fused target track information by using an end-to-end convolutional neural network model for display control, extracting part of target display control information as feedback information and feeding back the current feedback information to the first feedback information, the second feedback information, the third feedback information and the fourth feedback information respectively.

[0018] The fourth feedback information is used as part of the input of the end-to-end convolutional neural network model for fusion.

[0019] The fourth feedback information is introduced into the third feedback information and used as part of the input of the end-to-end convolutional neural network model for target tracking.

[0020] The third feedback information is introduced into the second feedback information and used as part of the input of the end-to-end convolutional neural network model for target detection.

[0021] The second feedback information is introduced into the first feedback information and used as part of the input of the preprocessing module.

[0022] Preferably, each feedback information is used as part of the input of each end-to-end convolutional neural network model, and each end-to-end convolutional neural network model is optimized by introducing an attention mechanism.

[0023] Preferably, a radar echo database is constructed, and the end-to-end convolutional neural network model for target detection is trained by using the radar echo database.

[0024] The constructing radar echo database comprises: acquiring radar echo original data, and performing detection processing on the radar echo original data, recording detection results and radar original echo images; based on the detection results, radar original echo images are labeled by using optical-electric image information and / or AIS target information as target references to obtain the radar echo database.

[0025] Preferably, the end-to-end convolutional neural network model for target detection adopts a single neural network structure, and the end-to-end convolutional neural network model for target detection is compressed.

[0026] Preferably, the training of the end-to-end convolutional neural network model for target detection by using the radar echo database comprises:

[0027] constructing a training set and a test set based on the radar echo database;

[0028] training the end-to-end convolutional neural network model for target detection by using the training set to obtain a trained end-to-end convolutional neural network model for target detection, testing the performance of the trained end-to-end convolutional neural network model for target detection by using the test set, and when a preset requirement is met, obtaining the trained end-to-end convolutional neural network model for target detection; when the preset requirement is not met, optimizing the end-to-end convolutional neural network model for target detection, and repeating the triggering until the preset requirement is met.

[0029] taking the current trained end-to-end convolutional neural network model for target detection as a teacher model, constructing a student model, and in the transfer learning, matching the spatial neuron activation distribution between the teacher model and the student model by minimizing the MMD distance between the teacher model and the student model, so as to realize the knowledge distillation from the teacher model to the student model, and finally obtain the end-to-end convolutional neural network model for target detection meeting the preset requirement.

[0030] Preferably, the target tracking module comprises:

[0031] combining the end-to-end convolutional neural network model for target tracking with AIS target data to obtain multi-target track information.

[0032] Preferably, the fused end-to-end convolutional neural network model comprises:

[0033] acquiring optical-electric and infrared images, performing target detection and target tracking based on the optical-electric and infrared images to obtain target track information, and performing multi-target track correlation on the current target track information and the target track information obtained based on the end-to-end convolutional neural network model for target tracking to obtain multi-channel target track information.

[0034] The application provides a sea surface target information processing method based on integrated deep learning, which comprises the following steps:

[0035] In step S1, a preprocessing module performs echo processing on radar echo data to obtain processed radar echo data, and the processed radar echo data is taken as first feedback information.

[0036] In step S2, a target detection module performs target detection on the processed radar echo data by using an end-to-end convolutional neural network model for target detection to obtain target point tracks, and the obtained target point tracks are taken as second feedback information.

[0037] In step S3, a target tracking module performs target point track tracking on the obtained target point tracks by using an end-to-end convolutional neural network model for target tracking to obtain multi-target track information, and the multi-target track information is taken as third feedback information.

[0038] In step S4, a correlation fusion module performs target fusion on the multi-channel target track information by using an end-to-end convolutional neural network model for fusion to obtain fused target track information, and the fused target track information is taken as fourth feedback information.

[0039] In step S5, a display control module performs display control on the fused target track information by using an end-to-end convolutional neural network model for display control, extracts part of target display control information as feedback information, and feeds back the current feedback information to the first feedback information, the second feedback information, the third feedback information and the fourth feedback information.

[0040] The fourth feedback information is taken as part of the input of the end-to-end convolutional neural network model for fusion.

[0041] The fourth feedback information is introduced into the third feedback information and taken as part of the input of the end-to-end convolutional neural network model for target tracking.

[0042] The third feedback information is introduced into the second feedback information and taken as part of the input of the end-to-end convolutional neural network model for target detection.

[0043] The second feedback information is introduced into the first feedback information and taken as part of the input of the preprocessing module.

[0044] Preferably, each feedback information is taken as part of the input of each end-to-end convolutional neural network model, and each end-to-end convolutional neural network model is optimized by introducing an attention mechanism.

[0045] Preferably, a radar echo database is constructed, and the end-to-end convolutional neural network model for target detection is trained by using the radar echo database.

[0046] The constructing radar echo database comprises: acquiring radar echo original data, and performing detection processing on the radar echo original data, recording detection results and radar original echo graphs; based on the detection results, radar original echo graphs are labeled by using optical-electric image information and / or AIS target information as target references to obtain a radar echo database;

[0047] The end-to-end convolutional neural network model for target detection adopts a single neural network structure, and the end-to-end convolutional neural network model for target detection is compressed;

[0048] The target tracking module comprises:

[0049] The end-to-end convolutional neural network model for target tracking is combined with AIS target data to obtain multi-target track information;

[0050] The fused end-to-end convolutional neural network model comprises:

[0051] Optical-electric and infrared images are acquired, target detection and target tracking are performed based on the optical-electric and infrared images, target track information is obtained, and the current target track information is related to the target track information obtained based on the end-to-end convolutional neural network model for target tracking to obtain multi-channel target track information.

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] 1. The present application solves the technical problems of sea surface radar target tracking fusion information processing by designing and implementing an integrated machine learning architecture, comprehensively realizes the deep complex function relationship and technical features of each processing module, realizes effective separation of radar targets and environment, strengthens the overall information technology effect, improves the overall intelligent level of sea surface radar information processing, and enhances the robustness and real-time performance of the system.

[0054] 2. The present application can effectively solve various system problems caused by separate learning of each information processing module, globally optimize the system, overcome some shortcomings of the system, and strengthen the effectiveness and real-time performance of the system to comprehensively improve the intelligent learning level of the learning architecture. BRIEF DESCRIPTION OF DRAWINGS

[0055] Other features, objects and advantages of the present application will become more apparent through reading the following detailed description of the non-limiting embodiments with reference to the accompanying drawings:

[0056] Figure 1 It is a schematic diagram of multi-station information processing module and information connection relationship.

[0057] Figure 2 It is a platform integrated learning module and architecture.

[0058] Figure 3 A layered architecture for an integrated machine.

[0059] Figure 4 A flowchart for building an automated labeling system.

[0060] Figure 5 A flowchart for designing and training a deep neural network model.

[0061] Figure 6 A flowchart for model optimization acceleration.

[0062] Figure 7 To monitor various noise interference in radar images.

[0063] Figure 8 For ship tracking trajectory measurement chart. DETAILED DESCRIPTION

[0064] The application will be described in detail below with specific embodiments. The following examples will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These are within the scope of the present application.

[0065] Example 1

[0066] According to the sea surface target information processing system and method based on integrated deep learning provided by the application, as shown in the figure, it includes: radar echo preprocessing, target detection, target tracking, multi-target correlation, track fusion, display control and other module cascade processing, each module uses convolutional neural network for integrated machine learning, realizes the integrated optimization of each module of the system level. Figures 1 to 8

[0067] The sea surface target information processing system based on integrated deep learning comprises:

[0068] The preprocessing module, the target detection module, the target tracking module, the correlation fusion module and the display control module;

[0069] The preprocessing module is configured to perform echo processing on the radar echo data to obtain processed radar echo data, and use the preprocessed radar echo data as first feedback information.

[0070] The target detection module is configured to perform target detection on the preprocessed radar echo data using an end-to-end convolutional neural network model for target detection to obtain target point tracks, and use the obtained target point tracks as second feedback information.

[0071] ​The target tracking module is configured to track the target point trail based on the acquired target point trail by using an end-to-end convolutional neural network model for target tracking, to obtain multi-target track information, and to take the multi-target track information as third feedback information.

[0072] The correlation fusion module is configured to fuse the multi-channel target track information by using an end-to-end convolutional neural network model for fusion to obtain fused target track information, and to take the fused target track information as fourth feedback information.

[0073] The display control module is configured to perform display control based on the fused target track information by using an end-to-end convolutional neural network model for display control, to extract part of target display control information as feedback information, and to feed back the current feedback information to the first feedback information, the second feedback information, the third feedback information, and the fourth feedback information, respectively.

[0074] The fourth feedback information is taken as part of the input of the end-to-end convolutional neural network model for fusion.

[0075] The fourth feedback information is introduced into the third feedback information and taken as part of the input of the end-to-end convolutional neural network model for target tracking.

[0076] The third feedback information is introduced into the second feedback information and taken as part of the input of the end-to-end convolutional neural network model for target detection.

[0077] The second feedback information is introduced into the first feedback information and taken as part of the input of the preprocessing module.

[0078] Specifically, each feedback information is taken as part of the input of each end-to-end convolutional neural network model, and each end-to-end convolutional neural network model is optimized by introducing an attention mechanism.

[0079] Specifically, a radar echo database is constructed, and the end-to-end convolutional neural network model for target detection is trained by using the radar echo database.

[0080] The construction of the radar echo database includes: acquiring radar echo raw data, performing detection processing on the radar echo raw data, and recording detection results and radar raw echo images; based on the detection results, the radar raw echo images are labeled by using information including photoelectric image information and / or AIS target information as target reference to obtain the radar echo database.

[0081] Specifically, the end-to-end convolutional neural network model for target detection adopts a single neural network structure, and the end-to-end convolutional neural network model for target detection is compressed.

[0082] Specifically, the end-to-end convolutional neural network model for target detection is trained by using the radar echo database, and the training comprises the following steps:

[0083] A training set and a test set are constructed based on the radar echo database.

[0084] The end-to-end convolutional neural network model for target detection is trained by using the training set to obtain a trained end-to-end convolutional neural network model for target detection, and the performance of the trained end-to-end convolutional neural network model for target detection is tested by using the test set.

[0085] The trained end-to-end convolutional neural network model for target detection is regarded as a teacher model, and a student model is constructed.

[0086] Specifically, the target tracking module comprises:

[0087] The end-to-end convolutional neural network model for target tracking is combined with AIS target data to obtain multi-target track information.

[0088] Specifically, the fused end-to-end convolutional neural network model comprises:

[0089] Optical and infrared images are acquired, and target detection and target tracking are performed based on the optical and infrared images to obtain target track information.

[0090] The application also provides a sea surface target information processing system based on integrated deep learning.

[0091] Embodiment 2

[0092] Embodiment 2 is a preferred example of embodiment 1

[0093] According to the application, a sea surface target information processing system based on integrated deep learning is provided, which combines the characteristics of radar sea surface echo, refers to the application scene, and needs to be completed in stages; the system has (later stage) information layered feedback, information layered addition, (AIS target) reference of main learning direction, and controllability of system global optimization. Three stages of research and development include: theoretical implementation simulation of overall deep learning, combination of background and system application promotion, and application improvement and perfection. System learning is open, and on-site adaptive learning can be performed on the basis of historical learning rules, and the overall performance of the system is gradually improved.

[0094] Among them, the end-to-end model neural network has fewer hidden layers and is relatively simple, but the flexibility is poor. The information feedback and prior information and reference information of each level are mainly considered, because these information cannot directly access the hidden layer of a large end-to-end network. The whole radar data processing system is divided into four end-to-end network models, an end-to-end convolutional neural network model for echo classification and target detection, an end-to-end convolutional neural network model for target tracking, an end-to-end convolutional neural network model for target fusion, and an end-to-end convolutional neural network model for display and control; an integrated optimization display and control module is externally connected; the structures of the end-to-end network models are basically the same, the knowledge convolution and learning target are different, and the reference information models are different.

[0095] Since the end-to-end neural network model for echo classification and target detection is a basic model, the learning degree of this part directly affects the subsequent network models, this part can be pre-learned according to the previous radar data to establish a basic architecture, and only a small amount of iteration is needed in actual system work, which can greatly resolve the huge amount of calculation and is very beneficial to the real-time performance of radar target learning.

[0096] Among them, the echo preprocessing part constructs an automatic labeling system of radar echo map;

[0097] For example Figure 4As shown, first, a large amount of radar echo raw data covering various environmental conditions is collected, and existing radar detection algorithms are used to process and detect the data, and the detection results and radar raw echo maps, such as the size and position of the target, are recorded. At this time, the radar detection algorithm should be adjusted to the optimal state under the condition according to experience. Then, the radar echo map is divided into a large number of sub-maps of consistent specifications (for example, 32*32 specifications). At this time, part of the sub-maps contains one or more radar targets, and part of the sub-maps does not contain any radar target. In the sub-map containing the radar target, according to the previously recorded position information and size of the target, the smallest rectangular frame containing the target is calculated, and the horizontal and vertical coordinates of the top left corner of the rectangular frame and the length and width of the rectangular frame are recorded. The sub-map without containing the radar target is regarded as empty target information. All sub-maps and their corresponding target information constitute the radar echo database. Next, the labeled database is cleaned. If conditions permit, other information sources such as AIS can be used to assist in obtaining target information, improve data quality, and expand data dimensions.

[0098] By then, the automatic labeling of the radar echo map under various environmental conditions is completed, and a general radar echo dataset (more than 100,000) in the radar field is established. In addition, based on this dataset, a standardized and automated radar detection effect evaluation system can also be established.

[0099] A deep neural network model suitable for radar target detection is designed and trained, such as Figures 5-6 as shown;

[0100] Due to the complex network structure and numerous calculation nodes of the deep neural network, it has the disadvantage of slow running speed. Therefore, a “one-stage” type detection algorithm is adopted. Compared with the “two-stage” detection algorithm, this algorithm discards the candidate region generation stage and directly identifies the object and calculates the position coordinate value, so it has a faster detection speed. The network structure is roughly composed of a basic feature extraction network, a multi-scale feature fusion layer, and an output layer.

[0101] The feature fusion layer considers using multiple features of different scales for target detection. The specific scale should be determined according to the actual range represented by a single pixel in the database. Since the radar target is relatively small, multiple scales such as 3*3, 7*7, 13*13, etc. can be selected to detect small, medium, and large types of radar sea surface targets. Then, a series of convolution layers and up-sampling are used to fuse the features of different scales. The output layer considers using a full convolution structure to collect the prediction content of each rectangular region to obtain target category and position information. The dataset can be divided into a training set and a test set in a ratio of 5:1. The training set is used to train the model, and the test set is used to evaluate the radar target detection effect, so as to obtain a deep neural network model that performs well on both the training set and the test set.

[0102] Optimizing and accelerating a trained deep neural network model

[0103] Although a "one-stage" type of detection algorithm is adopted, in order to achieve the best radar target detection capability under the environment, the complexity of the designed deep neural network should be relatively high to have the ability to learn more complex functions. This means that there may be many parts of the model that are rarely activated. These parts have little effect on the output results, but they consume more computing resources. Therefore, model compression is needed to further improve the running speed of the model. With reference to the subsequent feedback target information, with the help of target features and partially successful target models, the trained deep neural network model obtained is regarded as a teacher network; and a shallow network with relatively low complexity (this network still meets the aforementioned model design idea to be suitable for the radar detection field) is regarded as a student network.

[0104] In transfer learning, the distribution of space neuron activations between the teacher model and the student model is matched by minimizing the MMD (Maximum mean discrepancy) distance between them, so as to realize the knowledge distillation from the teacher network (high complexity) to the student network (low complexity), and finally obtain a radar target detection network model that meets the real-time requirement in processing speed.

[0105] The radar detection effect benchmark platform developed based on this database can provide convenience for detection effect evaluation. Usually, radar detection effect evaluation relies on manual discrimination of false alarm rate, missing rate and other indicators, which is time-consuming and laborious, and may have inconsistent scales and insufficient sample size. The radar detection effect benchmark platform directly compares the results output by the radar detection algorithm to be tested with the data annotations, which can quickly and accurately calculate various indicators under different environmental conditions, and there is no problem of inconsistent scales or insufficient sample size.

[0106] Because in the process of information processing, target track information may be fed back to the detection part in tracking, correlation, fusion, and multi-sensor (including radar, photoelectric, infrared, AIS) target information processing modules may also be fed back to the previous modules, so there is a deep coupling relationship between the modules. Using machine learning, any one module cannot make the system optimal. The preprocessing, detection, tracking, correlation, fusion, and display control modules should be considered as a whole. Deep learning is originally a hierarchical learning framework for a problem (temporarily called fine hierarchical learning). Each module is considered to be coarse hierarchical. The coarse and fine hierarchical systems are jointly learned. End-to-end network learning is fine hierarchical learning, which is used to represent deep function relationships. Information hierarchical transmission and on-demand feedback belong to coarse hierarchical, global optimization of integrated systems, which realizes the deep coupling of coarse and fine layers of the system.

[0107] In order to ensure stable tracking of small targets, the information of tracking needs to be fed back to the detection learning module, the attention mechanism is introduced, and the local optimum of the deep learning network is fully utilized (originally a defect, now an advantage for a specific target); The multi-level deep learning network uses feedback and reference information, and the parameters of each layer module are constantly learning and optimizing, and the structure and parameters are constantly adjusted to gradually realize the global optimization of the processing system. Make full use of information feedback, solve the problems of clutter, noise, interference, reflection, etc. and intelligent machine learning.

[0108] Embodiment 3

[0109] Embodiment 3 is a preferred example of embodiment 1

[0110] Using image thinking to process radar information

[0111] Traditionally, radar echo signals have been processed as one-dimensional signals, which reduces the correlation between signals at different times, different directions, and different frames. Directly considering radar echoes as images can take advantage of the correlation between signals, improving system processing effect.

[0112] In the radar image, there is no longer a distinction between noise, clutter, reflection, interference, etc. All backgrounds including targets are considered as echoes. The target is gradually separated from the background mainly based on target characteristics (motion, contour, track, reference, priori, etc.), and the target track is established and stable tracking is achieved. Through the characteristics, reference and correlation, feedback information of the target, further correlation fusion confirmation and extension of the target track and improvement of the reliability and accuracy of the target track are realized, and the global optimization of system performance is realized.

[0113] Introducing deep learning into the field of ship radar information

[0114] Due to the obvious differences between optical images and radar images, such as different scanning methods, resolution, granularity, etc., the method can be referenced. According to the scanning method used by the radar, the cyclic convolutional neural network is used in the cyclic azimuth direction, and the convolutional neural network is used in the distance direction. According to the frame calculation, the radar image detected and processed does not exceed 200 frames of radar scanning (can adapt to the rapid change of sea clutter and environment), and its influence decays frame by frame; The target track information of tracking and subsequent processing is not limited by 200 frames. Introducing the feedback mechanism of subsequent information as part of the input of the convolutional network, the (part) reference information can be used as the hidden layer parameters of the supervised training convolutional neural network.

[0115] Using a classifier to classify ships, directly using a classifier to separate sea clutter and other interference signals from targets;

[0116] The difference between the types of ships is distinguished by using multiple typical target characteristics and motion characteristics as the feature extraction image of the neural network convolution layer. A large sample size training method is used to distinguish high-quality, thereby providing high-quality services for the needs of users. In the traditional way, the sea clutter is filtered by capturing texture and other features, and the feature expression ability is not strong, and many details cannot be captured. Deep learning relies on its powerful analysis capability, which can gradually discover many details in the sea clutter signal that people cannot notice.

[0117] The reference tracking method combines deep learning to track the target track;

[0118] The traditional tracking problem has a matching and searching idea, and uses a "filtering + prediction + correlation" method. A new idea of deep learning and "prediction" is used, which can greatly improve the learning speed and efficiency. The target information of the previous frames (multi-step recursion) is sent to the deep convolutional neural network (tracking correlation module), and the position of the target track in the current frame is predicted (multi-step). In the current frame processing, the correlation and filtering method is used, the M / N detector is used as the correlation reference, the target track is tracked frame by frame, the target information that is not correlated is retained for continuous observation, and the target track after correlation and filtering is fed back to the previous convolutional neural network to reduce the learning time of the previous stage, and sent to the next convolutional neural network as the learning input layer. This is a great innovation to the traditional target tracking idea.

[0119] The reference correlation fusion method combines deep learning to track the target track;

[0120] The traditional fusion problem has a correlation and fusion idea, and combines other sensor (AIS, photoelectric, infrared, laser ranging, other radar, etc.) target track information (these information is incomplete or the measured coordinates are different, such as radar target position for distance, bearing; photoelectric and infrared for: bearing, elevation; laser ranging for distance; the size, outline, echo amplitude, measurement time, sampling period, target track number of targets of each sensor may be different), a new idea of deep convolutional neural network learning and "correlation" is used, which can explore the subtle features and deep function relationship between the different coordinate characteristics of the target track information of each sensor. The target track with several (different) coordinate characteristics can be considered as a correlated target. Through the correlation of multiple tracks, the correlation target track is fused to further confirm the target track and improve the performance index of the (fusion) track. The target track that does not pass the correlation can continue to learn. When there is evidence that the track cannot be correlated, the subsequent network learning is not performed. The fusion track can feed back the previous network learning module to reduce the calculation amount of the previous stage; the network information of the fusion layer is sent to the next convolutional neural network as the input. This is a great innovation to the traditional correlation fusion idea.

[0121] Reference system display control method, combined with deep learning for system optimization;

[0122] The traditional system display control mainly includes display and control. The display can include target echo (shape and amplitude size) and track information (track ID number, fusion ID number, target position, speed and heading, ship length, discovery target station and fusion station, tracking time, etc.) sent by the previous stage. The display of the target exists in special areas such as strong clutter area, multi-reflection area, radar blind area, echo shielding area, key attention area, multi-target channel, sea area anchorage, and alarm sea area. The control part includes three parts: evidence collection; the target focused on by the user (clicking with the mouse), guiding the photoelectric and infrared to the target for long-distance image evidence collection with the radar target position; or guiding the radar to the target for long-distance measurement and tracking with the photoelectric target position; guiding the unmanned aerial vehicle to take evidence near the target. Alarm; whether the radar target has AIS information (not open, intermittent, AIS number change), whether it needs to call available resources (other photoelectric, other radar, etc.) for evidence collection or authentication. Whether the target deviates from the channel and departs or docks, whether the behavior trajectory is normal, etc. Call historical track and record to comprehensively judge whether it is abnormal; when a target enters the alarm sea area and the previous evidence collection exceeds the alarm threshold (which can be further learned later), alarm to enable the user action team to take corresponding measures, and guide the unmanned aerial vehicle and unmanned ship to shout or drive away near the target. Optimization; the system interface adjusts the system parameters of the entire system (including radar, photoelectric, infrared, etc.) according to the needs of the user. The display control information also includes the type and characteristics of the target focused by the user, the position and area, and the system performance required by the user.

[0123] These display control information is input (display control level) convolutional neural network, mainly learning system optimization structure and parameters, controlling the input size and information feedback structure of each learning module, optimizing the learning depth and learning model parameters of each module, optimizing the control process, etc. The ultimate goal is to optimize the system to realize intelligent system performance in a limited time; mainly learning the characteristics of the target focused by the user, various information of the target, alarm area and threshold, etc. This point is related to the user, and different users need to learn again. The ultimate goal is to meet the needs of the user. This is a great innovation to the traditional display control idea.

[0124] Using multi-level convolutional neural network learning to replace (one-level deep) convolutional neural network

[0125] Modern first-level deep learning convolutional neural network, hidden layer up to more than 100 layers, although can learn very complex function relationship, but the calculation time and system processing flexibility will be greatly discounted. This is not suitable for radar information processing system, because the sea surface environment changes very quickly, that is, the background echo changes very quickly (calculated by 15 minutes), the radar scanning speed (2.5s per frame) is also fast, the system needs to track a large number of targets in real time (up to 2000 batches of targets), which greatly compresses the learning time and calculation amount, and the front and rear information exists feedback, the middle level input may add other sensor target track information, the whole system pursues the goal of (scanning) frame cycle recursive processing optimization, rather than batch (multi-frame per batch) processing optimization (with no internal adjustment and intermediate information addition). Considering the flexibility of the system, information feedback, global optimization, time requirements and other issues, it is not suitable to use a one-level (deep) convolutional network for deep learning, and a multi-level (relatively deep, about 7 levels) convolutional network for integrated (integrated) deep learning. Comprehensive processing can combine prior information and reference information of radar targets, adapt to feedback structure, realize global optimization of adaptive processing system, so that the system parameters set in the application are minimized, which brings great convenience to the system application. This is a great innovation to the machine learning approach.

[0126] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can be realized by logically programming the method steps to form a logic gate, a switch, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures within hardware components.

[0127] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A sea surface target information processing system based on integrated deep learning, characterized in that, include: Preprocessing module, target detection module, target tracking module, correlation fusion module and display control module; The preprocessing module is used to perform echo processing on the radar echo data to obtain processed radar echo data, and use the preprocessed radar echo data as the first feedback information; The target detection module is configured to perform target detection based on the preprocessed radar echo data using an end-to-end convolutional neural network model for target detection to obtain target traces, and use the obtained target traces as second feedback information; The target tracking module is used to track the target points based on the acquired target points using an end-to-end convolutional neural network model for target tracking, obtain multi-target track information, and use the multi-target track information as the third feedback information; The related fusion module is used to perform target fusion based on the multi-channel target track information using the fused end-to-end convolutional neural network model to obtain fused target track information, and use the fused target track information as the fourth feedback information; The display control module is configured to perform display control based on the fused target track information using an end-to-end convolutional neural network model of the display control; extract part of the target display control information as feedback information, and feed back the current feedback information to the first feedback information, the second feedback information, the third feedback information, and the fourth feedback information respectively; The fourth feedback information is used as part of the input of the fused end-to-end convolutional neural network model; Introducing the fourth feedback information into the third feedback information as part of the input of an end-to-end convolutional neural network model for target tracking; Introducing the third feedback information into the second feedback information as part of the input of an end-to-end convolutional neural network model for object detection; The second feedback information is introduced into the first feedback information and serves as a part of the input of a preprocessing module. 2.The sea surface target information processing system based on integrated deep learning according to claim 1, wherein, Each feedback information is used as part of the input of each end-to-end convolutional neural network model, and each end-to-end convolutional neural network model is optimized by introducing the attention mechanism. 3.The sea surface target information processing system based on integrated deep learning according to claim 1, characterized in that, Build a radar echo database and use it to train an end-to-end convolutional neural network model for target detection; The radar echo database is constructed by acquiring raw radar echo data, performing detection and processing on the raw radar echo data, and recording the detection results and the raw radar echo map; and annotating the raw radar echo map based on the detection results using photoelectric image information and / or AIS target information as a target reference to obtain the radar echo database. 4.The sea surface target information processing system based on integrated deep learning according to claim 1, wherein, The end-to-end convolutional neural network model for target detection adopts a single neural network structure and performs compression processing on the end-to-end convolutional neural network model for target detection. 5.The sea surface target information processing system based on integrated deep learning according to claim 1, wherein, The radar echo database is used to train an end-to-end convolutional neural network model for target detection, including: Construct training and test sets based on the radar echo database; The trained target detection end-to-end convolutional neural network model is obtained by training an end-to-end convolutional neural network model of target detection by using a training set, and the performance of the trained target detection end-to-end convolutional neural network model is tested by using a test set; when a preset requirement is met, the trained target detection end-to-end convolutional neural network model is obtained; when the preset requirement is not met, the end-to-end convolutional neural network model of target detection is optimized, and the triggering is repeated until the preset requirement is met; The current trained target detection end-to-end convolutional neural network model is regarded as a teacher model, and a student model is constructed; in the process of transfer learning, the spatial neuron activation distribution between the teacher model and the student model is matched by minimizing the MMD distance between the two, so as to realize knowledge distillation from the teacher model to the student model, and finally obtain the target detection end-to-end convolutional neural network model meeting the preset requirement. 6.The sea surface target information processing system based on integrated deep learning according to claim 1, wherein, The target tracking module comprises: The target tracking end-to-end convolutional neural network model is combined with the AIS target data to obtain multi-target track information. 7.The sea surface target information processing system based on integrated deep learning according to claim 1, wherein, The fused end-to-end convolutional neural network model comprises: Optical and infrared images are acquired, target detection and target tracking are performed based on the optical and infrared images, target track information is obtained, and multi-channel target track information is obtained by correlating the current target track information with the target track information obtained based on the target tracking end-to-end convolutional neural network model.

8. A sea surface target information processing method based on integrated deep learning, characterized in that, Comprise: Step S1: The pre-processing module performs echo processing on the radar echo data to obtain processed radar echo data, and the pre-processed radar echo data is taken as first feedback information; Step S2: The target detection module performs target detection based on the pre-processed radar echo data by using the target detection end-to-end convolutional neural network model to obtain target point tracks, and the obtained target point tracks are taken as second feedback information; Step S3: The target tracking module performs target point track tracking based on the obtained target point tracks by using the target tracking end-to-end convolutional neural network model to obtain multi-target track information, and the multi-target track information is taken as third feedback information; Step S4: The correlation fusion module performs target fusion based on the multi-channel target track information by using the fused end-to-end convolutional neural network model to obtain fused target track information, and the fused target track information is taken as fourth feedback information; Step S5: The display control module performs display control based on the fused target track information by using the display control end-to-end convolutional neural network model; part of the target display control information is extracted as feedback information, and the current feedback information is fed back to the first feedback information, the second feedback information, the third feedback information and the fourth feedback information respectively; The fourth feedback information is taken as part of the input of the fused end-to-end convolutional neural network model; The fourth feedback information is introduced into the third feedback information and taken as part of the input of the target tracking end-to-end convolutional neural network model; The third feedback information is introduced into the second feedback information and taken as part of the input of the target detection end-to-end convolutional neural network model; The second feedback information is introduced into the first feedback information and taken as part of the input of the pre-processing module. 9.The sea surface target information processing method based on integrated deep learning according to claim 8, characterized in that, The feedback information is taken as part of each end-to-end convolutional neural network model, and each end-to-end convolutional neural network model is optimized by introducing an attention mechanism. 10.The sea surface target information processing method based on integrated deep learning according to claim 8, characterized in that, A radar echo database is constructed, and the end-to-end convolutional neural network model for target detection is trained by using the radar echo database. The construction of the radar echo database comprises: obtaining radar echo raw data, detecting and processing the radar echo raw data, recording the detection results and the radar raw echo graph, labeling the radar raw echo graph based on the detection results by using the optical-electric image information and / or AIS target information as a target reference to obtain the radar echo database. The end-to-end convolutional neural network model for target detection adopts a single neural network structure, and the end-to-end convolutional neural network model for target detection is compressed. The target tracking module comprises: The end-to-end convolutional neural network model for target tracking is combined with the AIS target data to obtain multi-target track information. The fused end-to-end convolutional neural network model comprises: Optical-electric and infrared images are obtained, target detection and target tracking are performed based on the optical-electric and infrared images, target track information is obtained, and the current target track information is correlated with the target track information obtained based on the end-to-end convolutional neural network model for target tracking to obtain multi-channel target track information.

Images