A method and system for detecting and tracking sea vessels

Abstract

The present invention provides a method and system for detecting and tracking sea vessels, including: obtaining a sea vessel navigation video frame; inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; wherein the sea vessel target detection model is obtained by training a target detection model improved based on a multi-task branch based on a labeled video frame training set; post-processing the sea vessel detection result based on a sea level detection result to obtain a sea vessel detection and tracking result. The present invention improves the detection performance and obtains the appearance feature extraction capability by improving the anchor-free paradigm detection model, and agrees on the detection model to complete the sea level detection, and uses a dual matching strategy of large and small targets to make full use of the target appearance features for detection and tracking, thereby improving the accuracy of the sea vessel detection and tracking algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a method and system for detecting and tracking sea vessels. Background Art

[0002] With the development of artificial intelligence technology and computer vision technology, their applications in maritime surveillance scenarios are increasing, and the corresponding detection effect requirements are also getting higher and higher.

[0003] Ship multi-target video tracking refers to calculating the navigation track of each ship in a given ship video to assist in completing maritime supervision tasks. Usually, in important water source protection areas, the use of ship multi-target video tracking technology can effectively track and predict the trajectory of ships and promptly remind ships to leave the protection area. At present, the multi-target tracking algorithm is mainly based on a combination of detection and tracking. This method often has the problem of switching target IDs. Even if the appearance feature description model is introduced, the real-time performance of the algorithm will be reduced precisely because of the introduction of an additional model.

[0004] Therefore, how to overcome the above problems and improve the accuracy of ship multi-target video tracking has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a method and system for detecting and tracking sea vessels, which are used to solve the defects of low accuracy and high false detection rate when using traditional target detection methods to detect sea vessels in the prior art.

[0006] In a first aspect, the present invention provides a method for detecting and tracking a sea vessel, comprising:

[0007] Get the video frame of the ship sailing on the sea surface;

[0008] Inputting the sea surface ship navigation video frame into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by training a target detection model improved based on a multi-task branch based on a labeled video frame training set;

[0009] The sea surface vessel detection result is post-processed based on the sea horizon detection result to obtain a sea surface vessel detection and tracking result.

[0010] According to a method for detecting and tracking sea vessels provided by the present invention, the sea vessel target detection model is obtained by the following steps:

[0011] Obtaining the labeled video frame training set;

[0012] Determine an anchor-free paradigm detection model for anchor nodes, perform multi-task decomposition on the anchor-free paradigm detection model based on a decoupling head, and obtain multiple target detection branch models;

[0013] The multiple target detection branch models are trained based on the labeled video frame training set to obtain the sea surface ship target detection model.

[0014] According to a method for detecting and tracking sea vessels provided by the present invention, the step of obtaining the labeled video frame training set includes:

[0015] Acquire multiple sea surface navigation video frames at different time periods and / or different locations;

[0016] The sea level markings and the ship markings of the plurality of sea surface navigation video frames are determined, and the same ship between different video frames in the same video is determined as the same ship marking.

[0017] According to a method for detecting and tracking sea vessels provided by the present invention, the anchor-free paradigm detection model is determined, and the anchor-free paradigm detection model is multi-tasked decomposed based on a decoupling head to obtain multiple target detection branch models, including:

[0018] Obtaining a feature map of the anchor-free paradigm detection model;

[0019] A number of preset convolution kernels are added to the feature map to obtain a sea level position branch model, a target position branch model and a target appearance feature branch model.

[0020] According to a method for detecting and tracking sea vessels provided by the present invention, the plurality of target detection branch models are trained based on the sea horizon position information, the sea horizon extraction threshold and the annotated video frame training set to obtain the sea surface vessel target detection model, including:

[0021] Determine the coordinates of the upper left corner of the sea horizon detection frame in the sea horizon position information, as well as the width and height of the sea horizon detection frame;

[0022] Obtain each ship target detection result in the labeled video frame training set, and determine the coordinates of the lower right corner of the ship detection box of each ship target detection result;

[0023] Determine a preset protection threshold, and if it is determined that the sum of the ordinate of the lower right corner coordinate of the ship detection frame and the preset protection threshold is less than the ordinate of the upper left corner coordinate of the sea level detection frame, delete the current ship target detection result;

[0024] All the ship target detection results in the labeled video frame training set are traversed to obtain the sea surface ship target detection model.

[0025] According to a method for detecting and tracking sea surface vessels provided by the present invention, the sea surface vessel detection result is post-processed based on the sea level detection result to obtain the sea surface vessel detection and tracking result, including:

[0026] Determine a detection frame area threshold, and divide the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold;

[0027] Tracking and matching the preset large target detection set based on the appearance feature vector, and tracking and matching the preset small target detection set based on the template matching feature, to obtain a current detection box matching tracking object, put the current detection box matching tracking object into a matching success queue, and put an unmatched object into a matching failure queue;

[0028] Based on the intersection-over-union (IOU) ratio, the detection box in the failed matching queue is matched again with the unmatched tracking object. If the match is successful, the tracking object is updated; otherwise, the current tracking result is placed in the matching candidate queue;

[0029] If it is determined that the number of lost frames in the detection results in the matching failure queue continuously exceeds the preset frame threshold, deleting the corresponding tracking object;

[0030] If it is determined that the confidence of the detection result in the matching candidate queue exceeds a preset confidence threshold, and the corresponding number of frames exceeds a preset number of frames continuously, the corresponding tracking object is determined as a new tracking object;

[0031] The sub-images corresponding to each current video frame are stored in all tracked objects to obtain the position information of the next frame.

[0032] According to a method for detecting and tracking sea vessels provided by the present invention, the method tracks and matches the preset large target detection set based on the appearance feature vector, and tracks and matches the preset small target detection set based on the template matching feature, to obtain a current detection frame matching tracking object, including:

[0033] Based on the appearance feature vector, the detection result of the current frame in the preset large target detection set is compared with all the tracked objects stored in the previous frame one by one to obtain a feature vector comparison result. If the Euclidean distance of the feature vector comparison result is less than a preset distance threshold and is a minimum distance, the current detection frame matches the tracked object;

[0034] Based on the template matching feature, template matching is performed between the image of the area corresponding to the detection result of the current frame in the preset small target detection set and the image of the area corresponding to all the tracked objects stored in the previous frame to obtain the similarity between the images. If the similarity between the images is greater than the preset similarity threshold and is the best match, the current detection frame matches the tracked object.

[0035] In a second aspect, the present invention further provides a sea surface vessel detection and tracking system, comprising:

[0036] An acquisition module is used to acquire video frames of ships sailing on the sea surface;

[0037] A detection module, used for inputting the sea surface ship navigation video frame into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by training a target detection model improved based on a multi-task branch based on a labeled video frame training set;

[0038] The tracking module is used to post-process the sea surface ship detection result based on the sea level detection result to obtain the sea surface ship detection and tracking result.

[0039] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for detecting and tracking sea vessels are implemented.

[0040] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for detecting and tracking sea surface vessels.

[0041] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for detecting and tracking sea surface vessels.

[0042] The sea surface vessel detection and tracking method and system provided by the present invention improve the detection performance and obtain the appearance feature extraction capability by improving the anchor-free paradigm detection model, and use the agreed detection model to complete the sea horizon detection, and use the dual matching strategy of large and small targets to make full use of the target appearance features for detection and tracking, thereby improving the accuracy of the sea surface vessel detection and tracking algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 This is one of the flow charts of the sea surface vessel detection and tracking method provided by the present invention;

[0045] Figure 2 This is the second flow chart of the sea surface vessel detection and tracking method provided by the present invention;

[0046] Figure 3 It is a structural schematic diagram of a decoupling head of a sea surface ship detection model provided by the present invention;

[0047] Figure 4 It is a structural schematic diagram of the sea surface vessel detection and tracking system provided by the present invention;

[0048] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Figure 1 FIG. 1 is one of the flow charts of the method for detecting and tracking sea vessels provided by the present invention. Figure 1 As shown, including:

[0051] Step S1, obtaining a video frame of a ship sailing on the sea surface;

[0052] Step S2, inputting the sea surface ship navigation video frame into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by training a target detection model improved based on a multi-task branch based on a labeled video frame training set;

[0053] Step S3, post-processing the sea surface vessel detection result based on the sea level detection result to obtain the sea surface vessel detection and tracking result.

[0054] Specifically, Figure 2As shown, the sea surface ship navigation video frame that needs to be detected is determined from the sea surface navigation video, and the sea surface ship navigation video frame is input into the trained sea surface ship target detection model, that is, the target detection module, to obtain the sea surface ship detection result.

[0055] It should be noted that the sea surface ship target detection model is obtained by performing multiple annotations on multiple video frame training sets and adopting a decoupling head design to improve the target detection model with multi-task branches to obtain a multiple branch detection model to improve the detection model performance. The model training is completed by collecting and annotating the data set, and then the ship target detection results are post-processed according to the sea level detection results. That is, Figure 2 The detection results are input into the tracking module to complete the status update of the tracking object and add the new tracking object. The Nth frame image returned by the tracking module and the image after the tracking object is initialized are input into the target detection module again, and the detection box is introduced for large and small target classification detection. The dual-target matching strategy is adopted, and the appearance features of the target are used for detection and tracking.

[0056] The present invention improves the model performance by introducing a decoupling head to perform multi-task regression through an improved anchor-free paradigm detection model. The model is trained through the collected labeled data. The trained model has the capabilities of ship target detection, ship target appearance features and sea horizon detection. The ship detection results are post-processed through sea horizon position information to eliminate targets above the sea horizon, detect the false alarm rate, improve the detection performance, match the detected ship targets according to the designed tracking strategy, and finally complete the detection and tracking of sea surface ship targets.

[0057] Based on the above embodiment, the sea surface ship target detection model is obtained by the following steps:

[0058] Obtaining the labeled video frame training set;

[0059] Determine an anchor-free paradigm detection model for anchor nodes, perform multi-task decomposition on the anchor-free paradigm detection model based on a decoupling head, and obtain multiple target detection branch models;

[0060] The multiple target detection branch models are trained based on the labeled video frame training set to obtain the sea surface ship target detection model.

[0061] Specifically, the surface ship target detection model constructed by the present invention is based on the anchor-free paradigm detection model, and adopts a decoupling head to perform multi-branch task prediction of appearance features to obtain multiple prediction branches, thereby realizing prediction branch decoupling, and each branch model is responsible for predicting different features;

[0062] At the same time, the reasoning tasks of the model are set as ship target position detection, ship target appearance feature description and sea horizon detection. During the reasoning training process, based on the sea horizon position information and the sea horizon extraction threshold, the annotated video frame training set is used as the input of the model to train the above-mentioned improved anchor-free paradigm detection model and obtain the sea surface ship target detection model.

[0063] The present invention improves detection performance and obtains appearance feature extraction capability through an improved anchor-free paradigm detection model, and sets the detection model to simultaneously complete sea level detection to reduce the model false detection rate.

[0064] Based on any of the foregoing embodiments, the step of obtaining the labeled video frame training set includes:

[0065] Acquire multiple sea surface navigation video frames at different time periods and / or different locations;

[0066] The sea level markings and the ship markings of the plurality of sea surface navigation video frames are determined, and the same ship between different video frames in the same video is determined as the same ship marking.

[0067] It should be noted that before training the model, it is necessary to collect a certain amount of training data. The present invention collects several videos of different time periods and locations in actual application scenarios, and respectively marks the sea horizon and ships in each video frame image in the video.

[0068] In particular, the same ID number is assigned to the same ship target between different video frames in the same video, which can be used for training appearance feature description.

[0069] The present invention facilitates the identification of appearance feature descriptions during the training process by labeling different targets in a video frame training set.

[0070] Based on any of the above embodiments, the anchor-free paradigm detection model is determined, and the anchor-free paradigm detection model is decomposed into multiple tasks based on the decoupling head to obtain multiple target detection branch models, including:

[0071] Obtaining a feature map of the anchor-free paradigm detection model;

[0072] A number of preset convolution kernels are added to the feature map to obtain a sea level position branch model, a target position branch model and a target appearance feature branch model.

[0073] Specifically, an anchor-free paradigm detection model is adopted. Three 3x3 convolutional kernels are applied to the feature map respectively to obtain three prediction branches, thereby achieving the decoupling of the prediction branches. The three branches are respectively responsible for predicting the sea horizon position, the target position, and the appearance feature description. Based on the decoupled head, the detection model is improved and the appearance feature description is added, such as Figure 3 as shown.

[0074] Here, the appearance feature description branch is responsible for generating an Nx1-dimensional feature vector to characterize the target appearance, where N is a hyperparameter that can be set according to actual needs.

[0075] By improving the traditional object detection model, the present invention improves the detection performance and obtains the appearance feature extraction ability, and eliminates the mutual influence between multi-task branches through the decoupled head.

[0076] Based on any one of the above embodiments, training the multiple target detection branch models based on the labeled video frame training set to obtain the sea vessel target detection model includes:

[0077] Determine the upper left corner coordinates of the sea horizon detection box in the sea horizon position information, as well as the width and height of the sea horizon detection box;

[0078] Obtain the detection results of each vessel target in the labeled video frame training set, and determine the lower right corner coordinates of the vessel detection box for each vessel target detection result;

[0079] Determine a preset protection threshold. If it is judged that the sum of the ordinate of the lower right corner coordinates of the vessel detection box and the preset protection threshold is less than the ordinate of the upper left corner coordinates of the sea horizon detection box, then delete the current vessel target detection result;

[0080] Traverse all the vessel target detection results in the labeled video frame training set to obtain the sea vessel target detection model.

[0081] Specifically, when training the detection model, set the sea horizon extraction threshold thresh_sea to obtain the sea horizon position information (x, y, w, h), where x and y are the upper left corner coordinates of the sea horizon detection box, and w and h are the width and height of the sea horizon detection box respectively.

[0082] Set the lower right corner coordinates of the vessel detection box for each vessel target detection result in the labeled video frame training set as (xr, yr), and set a preset protection threshold thresh_p. If yr + thresh_p < y, it means that the current detection result is on the sea horizon and is regarded as a false detection and is excluded. This operation can prevent vessels that are basically near the sea horizon from being falsely excluded due to detection errors.

[0083] Traverse each ship target detection result, repeat the above operation for all video frame training sets, and the training is completed to obtain the sea surface ship target detection model.

[0084] The present invention trains the model through labeled data. The trained model has the ability to detect ship targets, ship target appearance features and sea horizon. The ship detection results are post-processed through sea horizon position information to eliminate targets above the sea horizon, thereby reducing the false alarm rate and improving the detection performance.

[0085] Based on any of the above embodiments, the post-processing of the sea surface vessel detection result based on the sea level detection result to obtain the sea surface vessel detection and tracking result includes:

[0086] Determine a detection frame area threshold, and divide the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold;

[0087] Tracking and matching the preset large target detection set based on the appearance feature vector, and tracking and matching the preset small target detection set based on the template matching feature, to obtain a current detection box matching tracking object, put the current detection box matching tracking object into a matching success queue, and put an unmatched object into a matching failure queue;

[0088] Based on the intersection-over-union (IOU) ratio, the detection box in the failed matching queue is matched again with the unmatched tracking object. If the match is successful, the tracking object is updated; otherwise, the current tracking result is placed in the matching candidate queue;

[0089] If it is determined that the number of lost frames in the detection results in the matching failure queue continuously exceeds the preset frame threshold, deleting the corresponding tracking object;

[0090] If it is determined that the confidence of the detection result in the matching candidate queue exceeds a preset confidence threshold, and the corresponding number of frames exceeds a preset number of frames continuously, the corresponding tracking object is determined as a new tracking object;

[0091] The sub-images corresponding to each current video frame are stored in all tracked objects to obtain the position information of the next frame.

[0092] The tracking and matching of the preset large target detection set based on the appearance feature vector and the tracking and matching of the preset small target detection set based on the template matching feature to obtain the current detection frame matching tracking object include:

[0093] Based on the appearance feature vector, the detection result of the current frame in the preset large target detection set is compared with all the tracked objects stored in the previous frame one by one to obtain a feature vector comparison result. If the Euclidean distance of the feature vector comparison result is less than a preset distance threshold and is a minimum distance, the current detection frame matches the tracked object;

[0094] Based on the template matching feature, template matching is performed between the image of the area corresponding to the detection result of the current frame in the preset small target detection set and the image of the area corresponding to all the tracked objects stored in the previous frame to obtain the similarity between the images. If the similarity between the images is greater than the preset similarity threshold and is the best match, the current detection frame matches the tracked object.

[0095] Specifically, after obtaining the detection result, the present invention inputs the detection result into the tracking module, completes the tracking object state and adds a new tracking object, which specifically includes:

[0096] Set the detection box area threshold thresh_area, and divide the detection results into a preset large target detection set and a preset small target detection set according to the size of the detection box area. The preset large target detection set with an area larger than thresh_area is called detect_app, and the preset small target detection set with an area smaller than thresh_area is called detect_tem.

[0097] Here, since the detection box in the preset large target detection set detect_app has a large area and contains rich feature information, the present invention uses appearance feature vectors for matching, and compares the detection result with all tracking objects stored in the previous needle one by one according to the appearance feature vectors. If the Euclidean distance between the appearance feature vectors is less than the set threshold and is the minimum distance, the two are matched successfully and the tracking object status is updated. Otherwise, no tracking object matching the current detection box is found, and the current detection result is placed in the matching failure queue detect_lost, which is used for matching in subsequent steps.

[0098] However, the detection box in the preset small target detection set detect_tem has a small area and contains insufficient feature information, which can easily cause mismatching. Therefore, the present invention adopts template matching features for matching, and performs template feature matching on the image of the area corresponding to the detection result and the images corresponding to all tracking objects stored in the previous frame. If the similarity between the two images is greater than the set threshold and it is the best match, the two are matched successfully and the tracking object status is updated. Otherwise, no tracking object matching the current detection box is found, and the current detection result is placed in the matching failure queue detect_lost, which is used for matching in subsequent steps.

[0099] Furthermore, the unmatched detection boxes in the match failure queue detect_lost are matched again with the unmatched tracking objects based on IOU (Intersection over Union). If the match is successful, the tracking object is updated. If the match fails, the current detection result is placed in the matching candidate queue detect_candidate. The state of the unmatched tracking object is defined as lost. If the number of lost frames exceeds the preset frame threshold thresh_buffer continuously, the tracking object is deleted. Here, thresh_buffer is a hyperparameter that can be set by yourself.

[0100] If the confidence of the detection result of the matching candidate queue detect_candidate exceeds the preset confidence threshold thresh_new, the target may be a newly appeared target. A new tracking object is constructed based on the detection result. If the object appears continuously for more than a preset number of frames, such as 3 frames, it is determined to be a new tracking object.

[0101] Finally, all tracked objects are stored in the sub-images corresponding to each sub-frame, and the position information of the next frame is predicted for tracking and matching in the next frame.

[0102] The present invention designs a dual matching strategy for large and small targets, fully utilizes the target appearance features for detection and tracking, matches the detected ship targets iteratively according to the designed tracking strategy, and finally completes the detection and tracking of sea surface ship targets.

[0103] The sea surface vessel detection and tracking system provided by the present invention is described below. The sea surface vessel detection and tracking system described below and the sea surface vessel detection and tracking method described above can be referenced to each other.

[0104] Figure 4 FIG. 1 is a schematic diagram of the structure of the sea surface vessel detection and tracking system provided by the present invention. Figure 4 As shown, it includes: an acquisition module 41, a detection module 42 and a tracking module 43, wherein:

[0105] The acquisition module 41 is used to acquire video frames of sea surface ship navigation; the detection module 42 is used to input the sea surface ship navigation video frames into a pre-trained sea surface ship target detection model to obtain sea surface ship detection results; wherein the sea surface ship target detection model is obtained by training a target detection model improved based on multi-task branches based on a labeled video frame training set; the tracking module 43 is used to post-process the sea surface ship detection results based on the sea horizon detection results to obtain sea surface ship detection and tracking results.

[0106] The present invention improves the detection performance and obtains the appearance feature extraction capability by improving the anchor-free paradigm detection model, and stipulates that the detection model completes the sea horizon detection, and utilizes the double matching strategy of large and small targets to fully utilize the target appearance features for detection and tracking, thereby improving the accuracy of the sea surface ship detection and tracking algorithm.

[0107] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the sea surface ship detection and tracking method, which includes: obtaining a sea surface ship navigation video frame; inputting the sea surface ship navigation video frame into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by training a target detection model improved based on a multi-task branch based on a labeled video frame training set; post-processing the sea surface ship detection result based on the sea level detection result to obtain a sea surface ship detection and tracking result.

[0108] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0109] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the sea surface vessel detection and tracking method provided by the above-mentioned methods, which includes: obtaining a sea surface vessel navigation video frame; inputting the sea surface vessel navigation video frame into a pre-trained sea surface vessel target detection model to obtain a sea surface vessel detection result; wherein the sea surface vessel target detection model is based on a labeled video frame training set, and is obtained by training an improved target detection model based on multi-task branches; the sea surface vessel detection result is post-processed based on the sea horizon detection result to obtain a sea surface vessel detection and tracking result.

[0110] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the surface vessel detection and tracking method provided by the above-mentioned methods, the method comprising: obtaining a surface vessel navigation video frame; inputting the surface vessel navigation video frame into a pre-trained surface vessel target detection model to obtain a surface vessel detection result; wherein the surface vessel target detection model is obtained by training an improved target detection model based on a multi-task branch based on a labeled video frame training set; and post-processing the surface vessel detection result based on the sea horizon detection result to obtain a surface vessel detection and tracking result.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting and tracking sea vessels, characterized in that: include: Get the video frame of the ship sailing on the sea surface; The sea surface ship navigation video frame is input into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by the following steps: obtaining a labeled video frame training set; obtaining a feature map of an anchor-free node paradigm detection model; adding a number of preset convolution kernels to the feature map to obtain a sea horizon position prediction branch model, a target position prediction branch model and a target appearance feature branch model, and obtaining multiple target detection branch models; training the multiple target detection branch models based on the labeled video frame training set to obtain the sea surface ship target detection model; The sea surface vessel detection result is post-processed based on the sea horizon detection result to obtain a sea surface vessel detection and tracking result.

2. The method for detecting and tracking sea vessels according to claim 1, characterized in that: The step of obtaining a labeled video frame training set includes: Acquire multiple sea surface navigation video frames at different time periods and / or different locations; The sea level markings and the ship markings of the plurality of sea surface navigation video frames are determined, and the same ship between different video frames in the same video is determined as the same ship marking.

3. The method for detecting and tracking sea vessels according to claim 1, characterized in that: The multiple target detection branch models are trained based on the labeled video frame training set to obtain the sea surface ship target detection model, including: Determine the coordinates of the upper left corner of the sea horizon detection frame in the sea horizon position information, as well as the width and height of the sea horizon detection frame; Obtain each ship target detection result in the labeled video frame training set, and determine the coordinates of the lower right corner of the ship detection box of each ship target detection result; Determine a preset protection threshold, and if it is determined that the sum of the ordinate of the lower right corner coordinate of the ship detection frame and the preset protection threshold is less than the ordinate of the upper left corner coordinate of the sea level detection frame, delete the current ship target detection result; All the ship target detection results in the labeled video frame training set are traversed to obtain the sea surface ship target detection model.

4. The method for detecting and tracking sea vessels according to claim 1, characterized in that: The post-processing of the sea surface vessel detection result based on the sea level detection result to obtain the sea surface vessel detection and tracking result includes: Determine a detection frame area threshold, and divide the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold; Tracking and matching the preset large target detection set based on the appearance feature vector, and tracking and matching the preset small target detection set based on the template matching feature, to obtain a current detection box matching tracking object, put the current detection box matching tracking object into a matching success queue, and put an unmatched object into a matching failure queue; Based on the intersection-over-union (IOU) ratio, the detection box in the failed matching queue is matched again with the unmatched tracking object. If the match is successful, the tracking object is updated; otherwise, the current tracking result is placed in the matching candidate queue; If it is determined that the number of lost frames in the detection results in the matching failure queue continuously exceeds the preset frame threshold, deleting the corresponding tracking object; If it is determined that the confidence of the detection result in the matching candidate queue exceeds a preset confidence threshold, and the corresponding number of frames exceeds a preset number of frames continuously, the corresponding tracking object is determined as a new tracking object; The sub-images corresponding to each current video frame are stored in all tracked objects to obtain the position information of the next frame.

5. The method for detecting and tracking sea vessels according to claim 4, characterized in that: The tracking and matching of the preset large target detection set based on the appearance feature vector and the tracking and matching of the preset small target detection set based on the template matching feature to obtain the current detection frame matching tracking object include: Based on the appearance feature vector, the detection result of the current frame in the preset large target detection set is compared with all the tracked objects stored in the previous frame one by one to obtain a feature vector comparison result. If the Euclidean distance of the feature vector comparison result is less than a preset distance threshold and is a minimum distance, the current detection frame matches the tracked object; Based on the template matching feature, template matching is performed between the image of the area corresponding to the detection result of the current frame in the preset small target detection set and the image of the area corresponding to all the tracked objects stored in the previous frame to obtain the similarity between the images. If the similarity between the images is greater than the preset similarity threshold and is the best match, the current detection frame matches the tracked object.

6. A sea surface vessel detection and tracking system, characterized in that: include: An acquisition module is used to acquire video frames of ships sailing on the sea surface; The detection module is used to input the sea surface ship navigation video frame into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; wherein the sea surface ship target detection model is obtained by the following steps: obtaining a labeled video frame training set; obtaining a feature map of an anchor-free node paradigm detection model; adding a number of preset convolution kernels to the feature map to obtain a sea horizon position prediction branch model, a target position prediction branch model and a target appearance feature branch model, and obtaining multiple target detection branch models; training the multiple target detection branch models based on the labeled video frame training set to obtain the sea surface ship target detection model; The tracking module is used to post-process the sea surface ship detection result based on the sea level detection result to obtain the sea surface ship detection and tracking result.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the sea surface vessel detection and tracking method as claimed in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for detecting and tracking sea surface vessels as claimed in any one of claims 1 to 5 are implemented.

Images