A ship intelligent navigation observation auxiliary method

Abstract

The present invention discloses a method for assisting intelligent navigation and observation of ships, comprising the following steps: downloading COCO data sets, crawling various types of ship navigation data, and constructing a ship panoramic imaging system; obtaining video streams of multi-directional cameras of ships, using SIFT feature extraction algorithm to extract key points and feature descriptors in images, using fast approximate nearest neighbor FLANN algorithm to roughly match feature points extracted from heterogeneous images, and obtaining all matching key point pairs between two images; using the obtained matching key point pairs to calculate the homography matrix H, remove wrong matching, and obtain a multi-directional panoramic image of ship navigation; introducing YOLOv5 target detection algorithm to detect various types of targets appearing in the process of ship navigation on the panoramic image; passing the detection results obtained by the model to DeepSort multi-target tracking algorithm. The present invention can assist drivers in maritime navigation and comprehensively improve the safety of navigation.

Description

Technical Field

[0001] The present invention relates to the technical field of ship intelligent navigation systems, and in particular to a ship intelligent navigation and observation auxiliary method. Background Art

[0002] At present, the ship intelligent navigation and observation assistance system has attracted widespread attention due to its special needs in the field of ocean navigation. The system has important strategic significance in monitoring specific sea areas, port flow statistics, marine safety management, unknown ship identity recognition and behavior analysis. However, the actual use effect of such systems is often affected by factors such as wave interference, cloud and fog occlusion, and target motion changes. At the same time, low-quality video images will also bring certain challenges to target tracking. Low-resolution video will cause image features to blur and affect target recognition; and low frame rate or frame missing video will cause the tracking target to be lost during the process. Therefore, improving the accuracy of recognition and tracking of ship intelligent navigation and observation assistance systems has always been a focus of attention.

[0003] Previously, it was proposed to use algorithms such as correlation filtering (CF) for ship tracking, but this method only considers local feature information, is easily affected by background interference, and has poor adaptability to target scale changes and partial occlusion. Later, a method based on background difference was proposed. Although the accuracy has been improved, the background model cannot adapt to changes in the environment due to slow updates, and has poor adaptability to sudden scene changes and dynamic backgrounds. In recent years, the residual neural network method has been used for ship target recognition and tracking. This method has good anti-interference ability for high-resolution targets, but its recognition performance will be reduced for low-resolution targets.

[0004] In addition, although RANSAC can be well applied to the feature point screening of ship image matching, the threshold of the traditional RANSAC algorithm is set based on experience when using it for feature point screening. A strict threshold may exclude some correct corresponding points. If the threshold is relaxed, some correct matching points will be accepted as internal points, but the cost is that the matching accuracy decreases. If you want to obtain a suitable threshold, a lot of experiments are needed to adjust it, which will consume a lot of time.

[0005] How to solve the above technical problems is the research direction that technicians in this field are committed to. Summary of the invention

[0006] The purpose of the present invention is to provide a ship intelligent navigation and observation auxiliary method.

[0007] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0008] A method for assisting intelligent navigation and observation of a ship, characterized in that it comprises the following steps:

[0009] S01, download the COCO dataset, and use the you-get library to crawl various ship navigation data from the Internet for model training and testing, and build a ship panoramic imaging system based on OpenCV;

[0010] S02, obtaining the video stream of the multi-directional camera of the ship, using the SIFT feature extraction algorithm to extract the key points and feature descriptors in the image, and using the fast approximate nearest neighbor FLANN algorithm to perform rough matching on the feature points extracted from the heterogeneous images to obtain all matching key point pairs between the two images;

[0011] S03, using the matching key point pairs obtained in step S02 to calculate the homography matrix H, using the improved adaptive threshold RANSAC algorithm to further optimize the feature point matching, remove the wrong matching, and obtain a multi-directional panoramic image of the ship's navigation;

[0012] S04, introduces the YOLOv5 target detection algorithm to detect various targets appearing during the ship's navigation process on the panoramic image;

[0013] S05, passing the detection result obtained in step S04 to the DeepSort multi-target tracking algorithm to achieve tracking of ships and other targets during the navigation process.

[0014] As a specific implementation, in step S02, the key points and feature descriptors in the image are extracted using the SIFT feature extraction algorithm, and the steps are as follows:

[0015] S11, create a Gaussian pyramid, which contains the original image and its Gaussian blurred versions at different scales;

[0016] S12, calculating the difference Gaussian image between adjacent scales in the Gaussian pyramid, namely, the DOG image, and searching for local extreme points in the DOG image as potential key points;

[0017] S13, for each potential key point, accurately locate the position and scale of the key point by interpolation method;

[0018] S14, calculating the gradient magnitude and direction of the neighboring pixels of each key point, generating a direction histogram, and selecting the main peak in the histogram as the main direction of the key point;

[0019] S15, with the key point as the center, constructs a 16×16 neighborhood window on the scale of the key point, divides the window into 4×4 sub-regions, calculates the gradient histogram in 8 directions for each sub-region, and normalizes the 128-dimensional feature vector as the feature descriptor of the key point.

[0020] As a specific implementation, in step S02, the fast approximate nearest neighbor FLANN algorithm is used to perform rough matching on the feature points extracted from the heterogeneous images, and the steps are as follows:

[0021] S21, using the image key points detected by the SIFT feature extraction algorithm to calculate descriptors for each feature point, and storing the feature points and their descriptors in a NumPy array for subsequent processing;

[0022] S22, select the KD tree to construct the FLANN index and use the SIFT feature descriptor extracted from the first image to train the FLANN index;

[0023] S23, using the feature descriptor of the second image as a query input FLANN index to obtain the nearest neighbor matching feature point;

[0024] S24, calculating the distance ratio between the best match and the second best match through the Lowe ratio test, and screening out reliable matches according to the set threshold, and further applying the distance threshold judgment to remove outliers.

[0025] As a specific implementation, in step S03, the adaptive threshold RANSAC algorithm based on F1-Score maximization is performed as follows:

[0026] S31, sorting the coarsely matched feature point pairs from small to large according to the feature distance, selecting the first 80% of the sorted matching point pairs to form a new matching point set, and calculating the homography matrix H;

[0027] S32, calculate the Euclidean distance between all matching points and their transformed model points, denoted as d ( P i , HP ei ),

[0028] ,

[0029] Where: P i Indicates the target image i Matching points; HP ei Represents the first i Matching points;

[0030] S33, calculate the maximum value max_dis and the minimum value min_dis obtained in the previous step,

[0031] max_dis = max [ d (P i ,HP ei )],

[0032] min_dis = min [ d ( P i ,HP ei )],

[0033] In the formula i =1,2,…, m ,in, m Indicates the number of matching points;

[0034] S34, calculate the distance from each threshold to another threshold step , the value of each threshold is:

[0035] ,

[0036] threshold ( i )=( step × i )+ min _ dis ,

[0037] In the formula i =1: step : num ,

[0038] in num is the threshold number, threshold ( i ) represents the desired adaptive threshold;

[0039] S35, traverse possible thresholds within the threshold range, and use the selected threshold to calculate the point P of the perceived image and its corresponding point HP in the transformed model ei If the distance is greater than 2 pixels, it is considered to be a mismatched point, and then the accuracy Precision and recall Recall are calculated, and then the F1-Score is obtained. The calculation formulas of Precision Precision, recall Recall and F1-Score are as follows:

[0040] ;

[0041] ;

[0042] ;

[0043] Where: TP represents the number of matching points that are correctly matched as correct; FP represents the number of matching points that are incorrectly matched as correct; FN represents the number of matching points that are incorrectly matched as incorrect;

[0044] S36, select the one that maximizes F1-Score threshold As a threshold, the RANSAC algorithm is used to screen internal points.

[0045] As a specific implementation, in step S04, various targets appearing during the navigation of the ship are detected on the panoramic image, and the steps include:

[0046] S41, collect and annotate a panoramic image dataset containing ships and other targets, and preprocess the data;

[0047] S42, select the YOLOv5 target detection algorithm as the detection model, and train and adjust the parameters of the model;

[0048] S43, applying the detection model trained in step S42 to the panoramic image, detecting various types of objects in the image, and visualizing the results;

[0049] S44, evaluates the detection performance of the detection model, analyzes the detection effects of different categories of targets, and optimizes them.

[0050] As a specific implementation method, in step S05, the detection results obtained by the detection model are integrated with the DeepSort multi-target tracking algorithm to realize the detection and tracking of various targets during the navigation process of the ship, and the steps are as follows:

[0051] S51, using the trained detection model to perform target detection on the panoramic image, and obtain the bounding box position and category information of various targets, including ships, buoys and obstacles;

[0052] S52, passing the detection result in step S51 to the DeepSort multi-target tracking algorithm, the DeepSort multi-target tracking algorithm assigns a unique ID to each newly detected target and initializes its tracking state. For subsequent image frames, the DeepSort multi-target tracking algorithm uses the detection result of the current frame and the tracking state of the previous frame to predict and update the position and motion state of each target, and combines the appearance features and motion information of the target to perform multi-target tracking;

[0053] S53, uses the DeepSort multi-target tracking algorithm to continuously track each target and maintain its motion trajectory information, including position sequence, speed, and heading;

[0054] S54, plotting the target trajectory information calculated by the DeepSort multi-target tracking algorithm on the panoramic image to intuitively display the movement of each target.

[0055] Compared with the prior art, the present invention is beneficial in that:

[0056] 1) The present invention adopts a variety of advanced computer vision algorithms to achieve comprehensive perception of complex navigation scenes. Specifically, it integrates target detection based on deep learning, homography matrix estimation based on traditional visual algorithms, and multi-target tracking technology based on appearance features and motion information. This algorithm fusion method gives full play to the advantages of different methods, and can effectively deal with complex factors such as wave interference, cloud and fog occlusion, target motion changes, etc., and improves the overall perception accuracy; the system adopts a method that combines advanced deep learning and traditional visual algorithms, which reflects the advantages of technology integration. Deep learning models such as YOLOv5 have excellent target detection performance and can accurately identify various types of navigation targets; while traditional visual algorithms such as SIFT feature matching and RANSAC homography matrix estimation provide robust geometric correction capabilities and solve problems such as perspective changes and partial occlusion. The synergy of the two types of algorithms has greatly improved the system's perception capabilities in complex navigation scenarios;

[0057] 2) In the feature point matching of ship images, manually selecting thresholds will lead to mismatching and missed matching problems. The threshold is determined in an adaptive manner based on F1-Score maximization. After the processing of the adaptive threshold, the ship image matching has been greatly improved in terms of rotation, viewing angle, and brightness. The accuracy is greatly improved compared with before the improvement, which greatly reduces the dependence on manually set thresholds and improves the reliability and practicality of the algorithm;

[0058] 3) The present invention also integrates the DeepSort multi-target tracking algorithm, which can continuously track each navigation target and maintain its motion trajectory information. This can not only provide the driver with real-time dynamic monitoring of the target, but also analyze the target trajectory and identify abnormal behavior, providing an important reference for maritime safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 This is the overall structural diagram of the ship intelligent navigation and observation auxiliary system;

[0061] Figure 2 Design flow chart for the panorama stitching algorithm;

[0062] Figure 3 It is the feature point detected by SIFT algorithm;

[0063] Figure 4 To use the fast approximate nearest neighbor FLANN algorithm for key point matching and the RANSAC algorithm to further optimize the homography matrix estimation to remove the matching results after the wrong matches;

[0064] Figure 5 This is the final panorama after homography matrix perspective transformation stitching and black edge removal processing;

[0065] Figure 6 This is the ship target detection result obtained based on the YOLOv5 detection model;

[0066] Figure 7 This is the result of tracking the ship target using the DeepSort tracking algorithm based on the YOLOv5 detection model;

[0067] Figure 8 This is an overall interface diagram of the ship intelligent navigation and lookout assistance system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention:

[0069] like Figure 1 As shown, a method for assisting intelligent navigation and observation of a ship comprises the following steps:

[0070] S01, download the COCO dataset, and use the you-get library to crawl various ship navigation data from the Internet for model training and testing, and build a ship panoramic imaging system based on OpenCV;

[0071] S02, obtaining the video stream of the ship's multi-directional camera, using the SIFT feature extraction algorithm to extract key points and feature descriptors in the image, and using the fast approximate nearest neighbor (FLANN) algorithm to perform rough matching on the feature points extracted from the heterogeneous images to obtain all matching key point pairs between the two images;

[0072] S03, using the matching key point pairs obtained in step S02 to calculate the homography matrix H, using the improved adaptive threshold RANSAC algorithm to further optimize the feature point matching, remove the wrong matching, and obtain a multi-directional panoramic image of the ship's navigation,

[0073] Here, the improved adaptive threshold RANSAC algorithm is a threshold adaptive RANSAC algorithm based on F1-Score maximization. The homography matrix H is estimated by using the improved adaptive threshold RANSAC algorithm, and the feature point screening matching is further optimized to effectively eliminate the external points in the matching. Then, the estimated homography matrix H is used to perform perspective transformation on the source image to align it with the target image. After image fusion and post-processing, a multi-directional panoramic image of the ship's navigation is obtained.

[0074] S04, introduces the YOLOv5 target detection algorithm to detect various targets appearing during the ship's navigation process on the panoramic image;

[0075] S05, passing the detection result obtained in step S04 to the DeepSort multi-target tracking algorithm to achieve tracking of ships and other targets during the navigation process.

[0076] In step S01, the data collection of the dataset is mainly through the Python library you-get, downloading media content from most popular websites, such as YouTube, Youku, Niconico, etc. Find the first-person video of the ship sailing and download it for the panoramic stitching test.

[0077] like Figure 2 and Figure 3 As shown, in step S02, the main steps of the SIFT feature extraction algorithm include:

[0078] S11, creates a Gaussian pyramid, which includes the original image and its Gaussian blurred versions at different scales;

[0079] S12, calculating the difference of Gaussian (DOG) image between adjacent scales in the pyramid, and finding local extreme points in the DOG image as potential key points;

[0080] S13, for each potential key point, accurately locate the position and scale of the key point by interpolation method;

[0081] S14, for each key point, calculate the gradient magnitude and direction of its neighborhood pixels, generate a direction histogram, and select the main peak in the histogram as the main direction of the key point;

[0082] S15, with the key point as the center, constructs a 16×16 neighborhood window on the scale of the key point, divides the window into 4×4 sub-regions, calculates the gradient histogram in 8 directions for each sub-region, and normalizes the 128-dimensional feature vector as the feature descriptor of the key point.

[0083] like Figure 4 As shown, as a specific implementation, in step S02, the feature points extracted from the heterogeneous images are roughly matched using the fast approximate nearest neighbor (FLANN) algorithm, and the steps are as follows:

[0084] S21, using the image key points detected by the SIFT feature extraction algorithm to calculate descriptors for each feature point, and storing the feature points and their descriptors in a NumPy array for subsequent processing;

[0085] S22, select the KD tree to construct the FLANN index and use the SIFT feature descriptor extracted from the first image to train the FLANN index;

[0086] S23, using the feature descriptor of the second image as a query input FLANN index to obtain the nearest neighbor matching feature point;

[0087] S24, through the Lowe ratio test, calculate the distance ratio between the best match and the second best match, and screen out reliable matches according to the set threshold, and further apply the distance threshold judgment to remove outliers to improve the accuracy and robustness of the matching.

[0088] Here, in step S03, a threshold adaptive RANSAC algorithm based on F1-Score maximization is proposed, which takes the accuracy and recall rate into consideration and automatically finds a threshold that is most suitable for matching the image. The algorithm flow of the adaptive threshold is as follows:

[0089] S31, sorting the coarsely matched feature point pairs from small to large according to the feature distance, selecting the first 80% of the sorted matching point pairs to form a new matching point set, and calculating the homography matrix H;

[0090] S32, calculate the Euclidean distance between all matching points and their transformed model points, denoted as d ( P i , HP ei ),

[0091] ,

[0092] Where: P i Represents the i-th matching point in the target image; HP eiRepresents the i-th matching point after the transformation model;

[0093] S33, calculate the maximum value max_dis and the minimum value min_dis obtained in the previous step,

[0094] max_dis=max[d(P i ,HP ei )],

[0095] min_dis=min[d(P i ,HP ei )],

[0096] Where i = 1, 2, ..., m, where m represents the number of matching points;

[0097] S34, calculate the distance from each threshold to another threshold, that is, the step length step, and the value of each threshold is:

[0098] ,

[0099] threshold ( i )=( step × i )+ min _ dis ,

[0100] In the formula i =1: step : num ,

[0101] in num is the threshold number, threshold ( i ) represents the desired adaptive threshold;

[0102] S35, traverse possible thresholds within the threshold range, and use the selected threshold to calculate the point P of the perceived image and its corresponding point HP in the transformed model ei If the distance is greater than 2 pixels, it is considered to be a mismatched point, and then the precision and recall are calculated to get the F1-Score. The calculation formulas for precision, recall and F1-Score are as follows:

[0103] ;

[0104] ;

[0105] ;

[0106] Where: TPIndicates the number of points that are correctly matched; FP Indicates the number of matching points that were incorrectly matched as correct; FN Indicates the number of matching points that are incorrectly matched;

[0107] S36, selecting the threshold that maximizes the F1-Score as the threshold, and performing inlier screening of the RANSAC algorithm.

[0108] It should be noted that the homography matrix H is a 3×3 non-singular matrix that describes the geometric transformation relationship between two planes. A ( x A , y A ), and feature points B ( x B , y B ) matches, as shown below:

[0109] ,

[0110] like Figure 5 As shown, the relationship between the two images is obtained through the perspective transformation of the homography matrix, and then the images are stitched together through the perspective transformation function in the OpenCV library. Next, the OpenCV library is used to find the contour and the background is set to black because it is looking for white objects from a black background. After threshold processing, all non-background scenes are white. Finally, the largest rectangular area inside the panorama is found and cut out to obtain the final panorama.

[0111] In step S04, the ship dataset is extracted from the COCO dataset, and the images and annotation information are classified. These data are carefully screened, preprocessed, and the original label format is adjusted. Here, a total of about 2,100 pictures are extracted. Considering the wide variety and different models of ship targets on actual water transport roads, about 900 pictures of ships of different types and models are collected from the Internet. Among them, some unlabeled data are manually labeled using the labelme tool. After merging these data with the previous data, there are a total of about 3,000 pictures, including more than 10,000 ship instances. According to the ratio of 7:3, a part of the data is randomly selected as the training set and test set, and about 2,100 training set pictures and 900 test set pictures are obtained.

[0112] like Figure 6As shown, in step S04, the steps of detecting various types of targets appearing during the navigation of a ship on a panoramic image include: data preparation, model selection and training, target detection, result visualization, and performance evaluation. First, it is necessary to collect and annotate a panoramic image data set containing ships and other targets, and preprocess the data. Then select the YOLOv5 target detection algorithm as the detection model, train the model and adjust the parameters. Next, apply the trained model to the panoramic image, detect various types of targets in the image, and visualize the results. Finally, evaluate the detection performance of the model, analyze the detection effects of different categories of targets, and optimize them.

[0113] It should be noted that the loss function of the YOLOv5 network of the present invention is shown as follows:

[0114] Loss = a × lossobj + b × lossrect + c × lossclc,

[0115] in, lossclc is the classification loss, lossrect is the rectangular box loss, lossobj is the confidence loss, a, b, c Represents the corresponding loss weight, through reasonable selection and adjustment a, b, c , which can effectively balance the positioning, confidence, and classification capabilities of YOLOv5 in target detection, thereby improving the overall performance of the model.

[0116] like Figure 7 and Figure 8 As shown, in step S05, the target detection model and the DeepSort multi-target tracking algorithm are integrated to realize the detection and tracking of various targets during the ship's navigation process. The steps are as follows:

[0117] S51, use the trained YOLOv5 model to perform target detection on the panoramic image to obtain the bounding box position and category information of various targets (such as ships, buoys, obstacles, etc.);

[0118] S52, passing these detection results to the DeepSort multi-target tracking algorithm. DeepSort assigns a unique ID to each newly detected target and initializes its tracking state. For subsequent image frames, DeepSort uses the detection results of the current frame and the tracking state of the previous frame to predict and update the position and motion state of each target. It combines the appearance features and motion information of the target to achieve robust multi-target tracking.

[0119] S53, DeepSort will continue to track each target and maintain its motion trajectory information, including position sequence, speed, heading, etc. Even if the target disappears briefly and then reappears, DeepSort can correctly associate and continue tracking.

[0120] S54, plotting the target trajectory information calculated by DeepSort on the panoramic image to intuitively display the movement of each target. Analyzing these trajectory data can provide auxiliary decision support for ship navigation.

[0121] It should be emphasized that the above are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the case.

Claims

1. A ship intelligent navigation and observation auxiliary method, characterized in that: The following steps are involved: S01, download the COCO dataset, and use the you-get library to crawl various ship navigation data from the Internet for model training and testing, and build a ship panoramic imaging system based on OpenCV; S02, obtaining the video stream of the multi-directional camera of the ship, using the SIFT feature extraction algorithm to extract the key points and feature descriptors in the image, and using the fast approximate nearest neighbor FLANN algorithm to perform rough matching on the feature points extracted from the heterogeneous images to obtain all matching key point pairs between the two images; S03, using the matching key point pairs obtained in step S02 to calculate the homography matrix H, using the improved adaptive threshold RANSAC algorithm to further optimize the feature point matching, remove the wrong matching, and obtain a multi-directional panoramic image of the ship's navigation; S04, introduces the YOLOv5 target detection algorithm to detect various targets appearing during the ship's navigation process on the panoramic image; S05, passing the detection result obtained in step S04 to the DeepSort multi-target tracking algorithm to realize the tracking of ships and other targets during the navigation process, the steps are as follows: S51, using the trained detection model to perform target detection on the panoramic image, and obtain the bounding box position and category information of various targets, including ships, buoys and obstacles; S52, passing the detection result in step S51 to the DeepSort multi-target tracking algorithm, the DeepSort multi-target tracking algorithm assigns a unique ID to each newly detected target and initializes its tracking state. For subsequent image frames, the DeepSort multi-target tracking algorithm uses the detection result of the current frame and the tracking state of the previous frame to predict and update the position and motion state of each target, and combines the appearance features and motion information of the target to perform multi-target tracking; S53, uses the DeepSort multi-target tracking algorithm to continuously track each target and maintain its motion trajectory information, including position sequence, speed, and heading; S54, plotting the target trajectory information calculated by the DeepSort multi-target tracking algorithm on the panoramic image to intuitively display the movement of each target.

2. A ship intelligent navigation and observation auxiliary method according to claim 1, characterized in that: In step S02, the SIFT feature extraction algorithm is used to extract key points and feature descriptors in the image. The steps are as follows: S11, create a Gaussian pyramid, which contains the original image and its Gaussian blurred versions at different scales; S12, calculating the difference Gaussian image between adjacent scales in the Gaussian pyramid, namely, the DOG image, and searching for local extreme points in the DOG image as potential key points; S13, for each potential key point, accurately locate the position and scale of the key point by interpolation method; S14, calculating the gradient magnitude and direction of the neighboring pixels of each key point, generating a direction histogram, and selecting the main peak in the histogram as the main direction of the key point; S15, with the key point as the center, constructs a 16×16 neighborhood window on the scale of the key point, divides the window into 4×4 sub-regions, calculates the gradient histogram in 8 directions for each sub-region, and normalizes the 128-dimensional feature vector as the feature descriptor of the key point.

3. The method for assisting ship intelligent navigation and observation according to claim 1, characterized in that: In step S02, the fast approximate nearest neighbor FLANN algorithm is used to perform rough matching on the feature points extracted from the heterogeneous images. The steps are as follows: S21, using the image key points detected by the SIFT feature extraction algorithm to calculate descriptors for each feature point, and storing the feature points and their descriptors in a NumPy array for subsequent processing; S22, select the KD tree to construct the FLANN index and use the SIFT feature descriptor extracted from the first image to train the FLANN index; S23, using the feature descriptor of the second image as a query input FLANN index to obtain the nearest neighbor matching feature point; S24, calculating the distance ratio between the best match and the second best match through the Lowe ratio test, and screening out reliable matches according to the set threshold, and further applying the distance threshold judgment to remove outliers.

4. The method for assisting ship intelligent navigation and observation according to claim 1, characterized in that: In step S03, the adaptive threshold RANSAC algorithm based on F1-Score maximization is used, and the process is as follows: S31, sorting the coarsely matched feature point pairs from small to large according to the feature distance, selecting the first 80% of the sorted matching point pairs to form a new matching point set, and calculating the homography matrix H; S32, calculate the Euclidean distance between all matching points and their transformed model points, denoted as d ( P i , HP ei ), ， Where: P i Indicates the target image i Matching points; HP ei Represents the first i Matching points; S33, calculate the maximum value max_dis and the minimum value min_dis obtained in the previous step, max_dis ＝ max [ d ( P i ,HP ei )]， min_dis ＝ min [ d ( P i ,HP ei )]， In the formula i =1,2,…, m ,in, m Indicates the number of matching points; S34, calculate the distance from each threshold to another threshold step , the value of each threshold is: ， threshold ( i )＝( step × i )+ min _ dis ， In the formula i =1: step : num , in num is the threshold number, threshold ( i ) represents the desired adaptive threshold; S35, traverse possible thresholds within the threshold range, and use the selected threshold to calculate the point P of the perceived image and its corresponding point HP in the transformed model ei If the distance is greater than 2 pixels, it is considered to be a mismatched point, and then the accuracy Precision and recall Recall are calculated, and then the F1-Score is obtained. The calculation formulas of Precision Precision, recall Recall and F1-Score are as follows: ； ； ； Where: TP represents the number of matching points that are correctly matched as correct; FP represents the number of matching points that are incorrectly matched as correct; FN represents the number of matching points that are incorrectly matched as incorrect; S36, select the one that maximizes F1-Score threshold As a threshold, the RANSAC algorithm is used to screen internal points.

5. The method for assisting ship intelligent navigation and observation according to claim 1, characterized in that: In step S04, various targets appearing during the ship's voyage are detected on the panoramic image, and the steps include: S41, collect and annotate a panoramic image dataset containing ships and other targets, and preprocess the data; S42, select the YOLOv5 target detection algorithm as the detection model, and train and adjust the parameters of the model; S43, applying the detection model trained in step S42 to the panoramic image, detecting various objects in the image, and visualizing the results; S44, evaluates the detection performance of the detection model, analyzes the detection effects of different categories of targets, and optimizes them.

Images