A method and system for ship target detection in natural scenes

Abstract

The application provides a natural scene ship target detection method and system, first acquires a ship detection data set, a COCO data set and multiple to-be-detected natural scene pictures, inputs the multiple to-be-detected natural scene pictures into a YOLOv5x network model trained based on the ship detection data set, obtains multiple natural scene pictures containing ships, and then inputs the multiple natural scene pictures into a DINO target detection model trained based on the COCO data set to obtain first target detection results of all ships in each natural scene picture, and then a cutting region containing a ship is cut according to the first target detection results of each ship, then a contour finding function, a rotation matrix function and an affine transformation function are used to calculate the included angle between the connecting line of two contour points and the horizontal direction, and the cutting region is rotated according to the calculated included angle to obtain a cutting region after ship direction correction, and finally the cutting region after ship direction correction is input into a trained YOLOv5nano network model for verification to complete natural scene ship target detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent shipping technology, specifically to a method and system for detecting ship targets in natural scenes. Background Technology

[0002] Ships are important targets on the sea, and ship identification research has always been a hot topic. Ship identification and detection technologies have broad application prospects. Existing ship detection algorithms directly input natural scene images into a single-stage detection algorithm. However, due to the many interference factors in natural scenes, the accuracy of single-stage detection algorithms cannot meet the requirements of high-accuracy ship identification.

[0003] In addition, some detection methods use the Faster-RCNN two-stage algorithm to detect and identify ships. However, since ship images are taken in natural scenes, the accuracy of identification is low due to various interference factors such as size, lighting, distance, and angle affecting distant ships. Furthermore, deep learning methods based on GoogleNet suffer from low detection efficiency, high power consumption, and a lack of targeted identification for distant small boats, leading to potential missed detections.

[0004] To address the above issues, there is an urgent need for a ship detection method that is highly efficient, practical, and capable of accurately identifying small vessels at a distance to meet the challenges of ship tracking and segmentation in current maritime images. Summary of the Invention

[0005] To address the problems of low recognition accuracy, low detection efficiency, and lack of targeted identification of distant small boats in existing ship detection processes, leading to missed detections, this invention provides a method for ship target detection in natural scenes. By utilizing a three-stage deep learning detection algorithm (YOLOv5x network model, DINO object detection model, and YOLOv5nano network model) to sequentially identify ships in the natural scene image under test, the detection rate of ships is significantly improved. Furthermore, for difficult-to-detect ships, the accuracy and precision in identifying distant and blurred ships are greatly enhanced. This invention also relates to a system for ship target detection in natural scenes.

[0006] The technical solution of the present invention is as follows:

[0007] A method for detecting ship targets in natural scenes, characterized by comprising the following steps:

[0008] Image and dataset acquisition steps: Acquire the ship detection dataset, COCO dataset, and video images of the natural scene to be tested, and perform frame segmentation processing on the video images of the natural scene to be tested to obtain multiple images of the natural scene to be tested;

[0009] The first stage involves screening images containing ships: using the ship detection dataset as a training set sample, the YOLOv5x network model and the YOLOv5nano network model are trained respectively to obtain the trained YOLOv5x network model and the YOLOv5nano network model. Multiple natural scene images to be tested are then input into the trained YOLOv5x network model for the first stage of screening and detection to obtain multiple natural scene images containing ships.

[0010] The second stage involves fine-grained ship detection and cropping region acquisition: The COCO dataset is used to train the DINO target detection model to obtain a trained DINO target detection model. Multiple natural scene images containing ships obtained from the first stage of screening and detection are then input into the trained DINO target detection model for the second stage of screening and detection. The first target detection results of all ships in each natural scene image are obtained, and the cropping region containing the ship is cropped based on the first target detection results of each ship.

[0011] Ship orientation correction steps: The ship outline is extracted from the clipping region containing the ship using a contour lookup function. Based on the first target detection result, the points corresponding to the minimum and maximum horizontal coordinates are extracted from the ship outline as the first and second contour points. The angle between the line connecting the two contour points and the horizontal direction is calculated based on the first and second contour points. A rotation matrix is ​​then created using a rotation matrix function. Based on the rotation matrix and an affine transformation function, the ship outline in the clipping region is rotated according to the calculated angle to obtain the clipping region after ship orientation correction.

[0012] The third stage of detection and result verification involves inputting the cropped region after ship orientation correction into a lightweight, pre-trained YOLOv5nano network model for third-stage detection. This yields the second target detection result for the ship within the cropped region after ship orientation correction. The second target detection result output by the pre-trained YOLOv5nano network model is then used to verify whether the first target detection result output by the second stage of the pre-trained DINO target detection model contains false positives or false negatives. If not, the second target detection result is used as the final ship target detection result. If so, the false positives or false negatives are corrected by combining the first and second target detection results to complete the ship target detection in the natural scene.

[0013] Preferably, in the first stage of the ship image screening step, multiple natural scene images to be tested are input into the trained YOLOv5x network model for first-stage screening and detection, resulting in multiple natural scene images containing ships, specifically including:

[0014] Multiple natural scene images to be tested are input into a trained YOLOv5x network model, which outputs the detection results of all ships in each natural scene image. The ship detection results include bounding boxes, class labels, and confidence scores. The confidence scores of all ship detection results for a certain natural scene image are compared with a preset confidence threshold. If the confidence score of at least one ship detection result is greater than or equal to the confidence threshold, the natural scene image is considered to contain ships and is saved and labeled, thus obtaining multiple natural scene images containing ships.

[0015] Preferably, in the third stage of detection and detection result verification step, the correction of false detections or missed detections by combining the first target detection result and the second target detection result includes: if there is a false detection, then according to the first target detection result output by the DINO target detection model, the false detection boxes that do not belong to the ship identified from the second target detection result output by the YOLOv5nano network model are removed from the final detection result; if there is a missed detection, then the missed targets identified by the YOLOv5nano network model are added to the first target detection result output by the DINO model, and the positions of the targets are re-labeled.

[0016] Preferably, in the second stage of ship fine detection and cropping region acquisition, before inputting the multiple natural scene images containing ships obtained from the first stage of screening and detection into the trained DINO object detection model, the multiple natural scene images containing ships selected by YOLOv5x are first resized and standardized to adapt to the input requirements of the trained DINO object detection model.

[0017] Preferably, in the image and dataset acquisition step, the acquired ship detection dataset includes various types of ship images and annotations, and the COCO dataset includes multiple different object categories, including combinations of people, bicycles, cars, motorcycles, airplanes, buses, trains, trucks, ships, traffic lights, fire hydrants, stop signs, parking meters, and benches.

[0018] A natural scene ship target detection system, characterized by comprising, in sequence, an image and dataset acquisition module, a first-stage ship image screening module, a second-stage ship fine detection and cropping region acquisition module, a ship orientation correction module, and a third-stage detection and detection result verification module.

[0019] The image and dataset acquisition module acquires the ship detection dataset, the COCO dataset, and the video images of the natural scene to be tested, and performs frame-segmentation processing on the video images of the natural scene to be tested to obtain multiple images of the natural scene to be tested.

[0020] The first stage includes a ship image screening module. The ship detection dataset is used as a training set sample to train the YOLOv5x network model and the YOLOv5nano network model respectively, resulting in trained YOLOv5x network models and YOLOv5nano network models. Multiple natural scene images to be tested are then input into the trained YOLOv5x network model for the first stage of screening and detection, resulting in multiple natural scene images containing ships.

[0021] The second-stage ship fine detection and cropping region acquisition module uses the COCO dataset to train the DINO target detection model to obtain a trained DINO target detection model. The multiple natural scene images containing ships obtained from the first-stage screening and detection are then input into the trained DINO target detection model for the second-stage screening and detection. The first target detection results of all ships in each natural scene image are obtained, and the cropping region containing the ship is cropped according to the first target detection results of each ship.

[0022] The ship orientation correction module uses a contour lookup function to extract the ship contour from the clipping region containing the ship, and extracts the points corresponding to the minimum and maximum horizontal coordinates from the ship contour according to the first target detection result, as the first contour point and the second contour point. The angle between the line connecting the two contour points and the horizontal direction is calculated based on the first contour point and the second contour point. Then, a rotation matrix function is used to create a rotation matrix. Based on the rotation matrix and an affine transformation function, the ship contour in the clipping region is rotated according to the calculated angle to obtain the clipping region after ship orientation correction.

[0023] The third-stage detection and result verification module inputs the cropped region after ship orientation correction into a lightweight, pre-trained YOLOv5nano network model for third-stage detection, obtaining the second target detection result of the ship within the cropped region after ship orientation correction. It then verifies whether the first target detection result output by the pre-trained DINO target detection model in the second stage of screening and detection contains false positives or false negatives based on the second target detection result output by the pre-trained YOLOv5nano network model. If not, the second target detection result is used as the final ship target detection result; otherwise, the false positives or false negatives are corrected by combining the first and second target detection results to complete the ship target detection in the natural scene.

[0024] Preferably, in the first stage of the ship image screening module, multiple natural scene images to be tested are input into the trained YOLOv5x network model for first-stage screening and detection, resulting in multiple natural scene images containing ships, specifically including:

[0025] Multiple natural scene images to be tested are input into a trained YOLOv5x network model, which outputs the detection results of all ships in each natural scene image. The ship detection results include bounding boxes, class labels, and confidence scores. The confidence scores of all ship detection results for a certain natural scene image are compared with a preset confidence threshold. If the confidence score of at least one ship detection result is greater than or equal to the confidence threshold, the natural scene image is considered to contain ships and is saved and labeled, thus obtaining multiple natural scene images containing ships.

[0026] Preferably, in the third-stage detection and detection result verification module, the correction of false detections or missed detections by combining the first target detection result and the second target detection result includes: if there is a false detection, then according to the first target detection result output by the DINO target detection model, the false detection boxes that do not belong to the ship identified from the second target detection result output by the YOLOv5nano network model are removed from the final detection result; if there is a missed detection, then the missed targets identified by the YOLOv5nano network model are added to the first target detection result output by the DINO model, and the positions of the targets are re-labeled.

[0027] Preferably, in the second-stage ship fine detection and cropping region acquisition module, before inputting the multiple natural scene images containing ships obtained from the first-stage screening and detection into the trained DINO object detection model, the multiple natural scene images containing ships selected by YOLOv5x are first resized and standardized to adapt to the input requirements of the trained DINO object detection model.

[0028] Preferably, the vessel detection dataset includes various types of vessel images and annotations, and the COCO dataset includes multiple different object categories, including combinations of people, bicycles, cars, motorcycles, airplanes, buses, trains, trucks, ships, traffic lights, fire hydrants, stop signs, parking meters, and benches.

[0029] The beneficial effects of this invention are as follows:

[0030] This invention provides a method for ship target detection in natural scenes. First, it acquires a ship detection dataset, a COCO dataset, and video images of the natural scene to be detected. The video images are then segmented into multiple images of the natural scene to be detected. The YOLOv5x, DINO, and YOLOv5nano network models corresponding to the pre-constructed three-stage deep learning detection algorithm are trained on the different datasets. Using the ship detection dataset, YOLOv5x and YOLOv5nano network models of the same series but different scales are trained. These two models have similar architectures but different depths and widths, resulting in a well-trained YOLOv5x / DINO ... The YOLOv5x and YOLOv5nano network models were used. Multiple natural scene images to be tested were input into the trained YOLOv5x network model for the first stage of screening and detection, resulting in multiple natural scene images containing ships. YOLOv5x is one of the largest models in the YOLOv5 series, which has the characteristics of balancing speed and accuracy. It is suitable for large target or complex scene detection tasks on high-performance hardware. This stage quickly filters images containing ships from a large number of images. By using a labeled ship detection dataset containing ship images, the YOLOv5x network model was fine-tuned to ensure that the fine-tuned model can detect ships efficiently and accurately, and can better adapt to specific application scenarios. The DINO object detection model is then trained using the COCO dataset. Multiple natural scene images containing ships, obtained from the first stage of screening, are then input into the trained DINO object detection model for the second stage of screening. This yields the first target detection results for all ships in each natural scene image. Based on these first target detection results, the cropped regions containing the ships are extracted. The YOLOv5x network model is then fine-tuned using the COCO dataset, which contains images of various types of ships, to improve the model's accuracy in recognizing small ships. This stage efficiently detects small boats that are very difficult to detect, precisely identifying all ships and achieving the goal of comprehensive detection.Then, a contour lookup function is used to extract the ship contour from the cropped region containing the ship. Based on the first target detection result, the points corresponding to the minimum and maximum x-coordinate values ​​are extracted from the ship contour as the first contour point and the second contour point. The angle between the line connecting the two contour points and the horizontal direction is calculated. A rotation matrix function is then used to create a rotation matrix. Based on the rotation matrix and an affine transformation function, the ship contour in the cropped region is rotated according to the calculated angle to obtain the cropped region after ship orientation correction. By adjusting the ship to an angle close to parallel to the horizontal line, it helps to reduce the error of the model in detection and classification. Through contour extraction and affine transformation, the ship angle is corrected. By unifying the ship angle, the fluctuation of detection results caused by different ship orientations can be reduced, making the model perform more consistently under various angles and improving the stability and effectiveness of the overall system. Finally, the cropped region after ship orientation correction is input into the trained YOLOv5nano network model for the third stage of detection, obtaining the second target detection result of the ship within the cropped region after ship orientation correction. YOLOv5nano is a lightweight version of the model designed to provide fast target detection capabilities in resource-constrained environments. This model reduces the number of parameters and computational load by reducing the depth and width of the network, significantly reducing latency and required computational resources. The second target detection result output by the YOLOv5nano network model is used to verify whether there are false positives or false negatives in the first target detection result output by the DINO target detection model. If so, the false positives or false negatives are corrected by combining the first and second target detection results to complete the ship target detection in natural scenes. In this way, the YOLOv5nano network model can verify and improve the output of the DINO target detection model, thereby improving the overall accuracy of ship detection.

[0031] This invention utilizes a three-stage deep learning detection algorithm to identify ships, improving both the ship detection rate and accuracy. For difficult-to-detect ships, it significantly enhances the accuracy and precision in identifying distant and blurry vessels. The invention employs a trained YOLOv5x network model to filter multiple images of natural scenes, detecting almost all ships at sea. A trained DINO object detection model is then used for further refined detection, significantly improving the detection rate. Furthermore, after detection by the trained DINO object detection model, the image containing the target is cropped, affine-transformed to correct the target, and then input into a trained YOLOv5nano network model for further detection, greatly improving accuracy.

[0032] This invention also relates to a natural scene ship target detection system, which corresponds to the aforementioned natural scene ship target detection method. This system can be understood as a system that implements the aforementioned natural scene ship target detection method, comprising a sequentially connected image and dataset acquisition module, a first-stage ship image screening module, a second-stage ship fine detection and cropping region acquisition module, a ship orientation correction module, and a third-stage detection and detection result verification module. These modules work collaboratively, using a YOLOv5x network model to perform the first-stage screening and detection on multiple natural scene images, detecting almost all ships at sea. Then, a trained DINO target detection model is used for further second-stage fine detection, significantly improving the detection rate. Furthermore, after the images detected by the trained DINO target detection model are cropped to reveal the target area, subjected to affine transformation to orthogonalize the target, and then input into the YOLOv5nano network model for further third-stage detection, significantly improving accuracy. By utilizing a three-stage deep learning detection algorithm (YOLOv5x network model, DINO object detection model, and YOLOv5nano network model) to sequentially identify ships in the natural scene images under test, the ship detection rate is significantly improved. In addition, for ships that are difficult to detect, the accuracy and precision of identifying distant and blurry ships can be greatly improved. Attached Figure Description

[0033] Figure 1 This is a flowchart of the natural scene ship target detection method of the present invention.

[0034] Figure 2 This is a schematic diagram of the first stage of screening ship-related images using the YOLOv5x network model trained by this invention.

[0035] Figure 3 This is a schematic diagram of the second stage of fine screening of all ships using the DINO target detection model trained by this invention.

[0036] Figure 4 This is a schematic diagram of the ship's orientation correction method according to the present invention.

[0037] Figure 5 This is a schematic diagram of the YOLOv5nano network model trained by this invention performing a further third-stage detection of ships within the clipping region. Detailed Implementation

[0038] The present invention will now be described with reference to the accompanying drawings.

[0039] This invention relates to a method for detecting ship targets in natural scenes, the flowchart of which is shown below. Figure 1As shown, the process involves image → YOLOv5x → DINO → affine transformation → YOLOv5nano. For multiple natural scene images to be tested, a three-stage deep learning detection algorithm is used, such as... Figure 1 The YOLOv5x (i.e., YOLOv5x network model), DINO (i.e., DINO object detection model), and YOLOv5nano (i.e., YOLOv5nano network model) shown are used for phased ship identification. Between the second stage of DINO detection and the third stage of YOLOv5nano detection, contour extraction and affine transformation are used to correct the ship angle. This scheme is particularly effective for difficult-to-detect ships, significantly improving the identification of distant and blurry ships, increasing the ship detection rate, and also improving the accuracy of ship detection. High detection rate: This invention first uses the YOLOv5x network model (or YOLOv5x model algorithm) for the first stage of image screening and detection, setting a low threshold to filter out images containing ships. Then, the DINO network model is used for the second stage of fine detection, filtering out almost all ships at sea, significantly improving the detection rate. High accuracy: After the image is detected by the DINO network model (or DINO model algorithm), the area where the target is located is cropped, affinely transformed to align the target, and then input into the YOLOv5nano network model (or YOLOv5nano algorithm) for further third-stage detection, significantly improving accuracy. Specifically, this method for detecting ship targets in natural scenes includes the following steps:

[0040] I. Image and Dataset Acquisition Steps: Acquire the ship detection dataset, COCO dataset, and video images of the natural scene to be tested, and perform frame segmentation processing on the video images of the natural scene to be tested to obtain multiple images of the natural scene to be tested.

[0041] Specifically, the process begins by acquiring a ship detection dataset and a COCO dataset. The ship detection dataset includes various types of ship images and annotations, while the COCO dataset includes multiple object categories, such as people, bicycles, cars, motorcycles, airplanes, buses, trains, trucks, ships, traffic lights, fire hydrants, stop signs, parking meters, and benches, among other combinations. Then, video cameras are installed near ports of entry or on ships to capture images of nearby vessels, obtaining natural scene video images under different scenarios as test natural scene video images. These test natural scene video images are then segmented into frames to obtain multiple test natural scene images, i.e., each frame of the natural scene image.

[0042] II. First-stage ship image screening steps: Using the ship detection dataset as training set samples, the constructed YOLOv5x network model and YOLOv5nano network model are trained respectively to obtain the trained YOLOv5x network model and YOLOv5nano network model. Multiple natural scene images to be tested are input into the trained YOLOv5x network model for the first-stage screening and detection, resulting in multiple natural scene images containing ships.

[0043] Specifically, such as Figure 2 As shown, the YOLOv5x network model is first trained using a ship detection dataset as the training set. Then, multiple test images of natural scenes are preprocessed, such as resizing and normalizing, to adapt to the input requirements of the trained YOLOv5x network model. These preprocessed images are then input into the trained YOLOv5x network model for the first stage of screening and detection, outputting the ship detection results for each natural scene image. Leveraging the speed and accuracy of YOLOv5x, images with ships are quickly filtered from a massive dataset. Ocean-going vessels sail on the sea, and for the vast majority of the time, the sea is empty, but there may be small boats in the distance. Therefore, a very fast and accurate model is needed to filter out most images without ships, retaining images with ships, or even images containing only small boats in the distance.

[0044] The ship detection results include bounding boxes, category labels, and confidence scores. Then, the confidence scores of all ship detection results for a given natural scene image are compared to a preset confidence threshold (e.g., 0.5). If at least one ship detection result has a confidence score greater than or equal to the confidence threshold (e.g., 0.5), the detected ship result is considered valid, and the natural scene image is marked as containing a ship, i.e., the YOLOv5x-detected image with a ship shown in b). Otherwise, the natural scene image is not marked, i.e., the image without a ship shown in a), ultimately resulting in multiple natural scene images containing ships.

[0045] III. Second-stage fine-grained ship detection and cropping region acquisition steps: The DINO object detection model is trained using the COCO dataset to obtain a trained DINO object detection model. Multiple natural scene images containing ships obtained from the first-stage screening are then input into the trained DINO object detection model for the second-stage screening. The first object detection results of all ships in each natural scene image are obtained, and the cropping region containing the ship is cropped based on the first object detection results of each ship.

[0046] This step involves meticulously inspecting all vessels and cutting out specific areas, such as... Figure 3 As shown in the figure, first use the COCO dataset to train the DINO object detection model (preferably the 5scale-swin-L large model of the DINO object detection model), and obtain the trained DINO object detection model. After fine-tuning the trained DINO object detection model on the COCO dataset, the mAP 50 can reach 63. Among them, mAP (mean Average Precision) is an index in object detection, which is used to measure the detection accuracy of the model. mAP 50 represents the average precision when the IoU (Intersection over Union) is greater than 0.5 in the detection of different categories. The mAP 50 reaching 63 indicates that various scales and small targets can be detected, achieving the purpose of detecting all that should be detected. Then, resize and standardize multiple natural scene images containing ships screened by the YOLOv5x network model to meet the input requirements of the trained DINO model. Then, input multiple natural scene images containing ships into the trained DINO object detection model respectively, obtain the first object detection results of all ships in each natural scene image, and crop the cropping area containing the ship according to the first object detection results of each ship.

[0047] IV. Ship direction correction step: Use the contour search function to extract the ship contour from the cropping area containing the ship, and respectively extract the points corresponding to the minimum abscissa value and the maximum abscissa value from the ship contour as the first contour point and the second contour point. Calculate the angle between the line connecting the two contour points and the horizontal direction according to the first contour point and the second contour point. Then use the rotation matrix function to create a rotation matrix, and based on the rotation matrix, use the affine transformation function to rotate the ship contour in the cropping area according to the calculated angle to obtain the cropping area after ship direction correction.

[0048] Specifically, as Figure 4 shown in the figure, correct the angle of the target ship through contour extraction and affine transformation. First, use the contour search function cv2.findContours to extract the ship contour from the cropping area, and respectively extract the points corresponding to the minimum abscissa value and the maximum abscissa value from the ship contour as the first contour point (xmin, y1) and the second contour point (xmax, y2). Calculate the angle ɑ between the line connecting the two contour points and the horizontal direction according to the first contour point and the second contour point, and calculate according to the following formula:

[0049]

[0050] In the above formula, when y2 > y1, k = 0; when y2 < y1, k = 1.

[0051] Then, a rotation matrix is ​​created using the rotation matrix function cv2.getRotationMatrix2D. Based on the rotation matrix, the ship in the clipping area is rotated according to the calculated angle α using the affine transformation function cv2.warpAffine to correct the ship's orientation. That is, the ship is rotated counterclockwise by the angle α to correct the ship's orientation to the horizontal direction, thus obtaining the clipping area after the ship's orientation is corrected.

[0052] V. Third-stage detection and result verification steps: The cropped area after ship orientation correction is input into the lightweight trained YOLOv5nano network model for third-stage detection. The second target detection result of the ship within the cropped area after ship orientation correction is obtained. The second target detection result output by the trained YOLOv5nano network model is used to verify whether there are false detections or missed detections in the first target detection result output by the second-stage screening detection of the trained DINO target detection model. If there are, the false detections or missed detections are corrected by combining the first target detection result and the second target detection result to complete the ship target detection in the natural scene.

[0053] Specifically, this step involves using a lightweight yolov5nano network model to further detect the clipping region boxes containing the target object output by the DINO network model, ensuring the accuracy of ship detection. For example... Figure 5 As shown, each clipping region is first sized and normalized to fit the input requirements of the YOLOv5nano model. Then, the clipping region after ship orientation correction is input into the YOLOv5nano network model to obtain the second target detection result of the ship within the clipping region after ship orientation correction. The second target detection result output by the YOLOv5nano network model is used to verify whether there are false positives or false negatives in the first target detection result output by the DINO target detection model. If not, the second target detection result is used as the final ship target detection result; if so, the first and second target detection results are combined to correct for false positives or false negatives, thus completing the ship target detection in the natural scene. For example,

[0054] 1) False detection handling

[0055] False positives identified: False positive boxes that do not belong to the ship category were identified from the results of YOLOv5nano.

[0056] False detections are removed: Based on the detection results of the DINO model, these false detection boxes are removed from the final detection results.

[0057] 2) Handling of missed detections

[0058] Identify missed detections: Identify ship targets that were not detected by the DINO model but were detected by YOLOv5nano from the YOLOv5nano results.

[0059] Supplementing missed detections: Add the missed targets identified by YOLOv5nano to the detection results of the DINO model and re-label the locations of these targets.

[0060] 3) Overall Results:

[0061] Merging: By combining the first target detection result output by the DINO target detection model and the second target detection result output by the YOLOv5nano network model, the target detection boxes are updated and corrected to ensure the final result is as accurate as possible. By using the YOLOv5nano network model to improve the detection results of the DINO target detection model, the accuracy of the overall detection system can be effectively enhanced.

[0062] This invention also relates to a ship target detection system in natural scenes. This system corresponds to the aforementioned ship target detection method in natural scenes and can be understood as a system that implements the above method. The system includes, in sequence, an image and dataset acquisition module, a first-stage ship image filtering module, a second-stage ship fine detection and cropping region acquisition module, a ship direction correction module, and a third-stage detection and detection result verification module. Specifically,

[0063] The image and dataset acquisition module acquires the ship detection dataset, the COCO dataset, and the video images of the natural scene to be tested, and performs frame-segmentation processing on the video images of the natural scene to be tested to obtain multiple images of the natural scene to be tested.

[0064] The first stage includes a ship image screening module. The ship detection dataset is used as a training set sample to train the YOLOv5x network model and the YOLOv5nano network model respectively, resulting in trained YOLOv5x network models and YOLOv5nano network models. Multiple natural scene images to be tested are then input into the trained YOLOv5x network model for the first stage of screening and detection, resulting in multiple natural scene images containing ships.

[0065] The second-stage ship fine detection and cropping region acquisition module uses the COCO dataset to train the DINO target detection model to obtain a trained DINO target detection model. The multiple natural scene images containing ships obtained from the first-stage screening and detection are then input into the trained DINO target detection model for the second-stage screening and detection. The first target detection results of all ships in each natural scene image are obtained, and the cropping region containing the ship is cropped according to the first target detection results of each ship.

[0066] The ship orientation correction module uses a contour lookup function to extract the ship contour from the clipping region containing the ship, and extracts the points corresponding to the minimum and maximum horizontal coordinates from the ship contour according to the first target detection result, as the first contour point and the second contour point. The angle between the line connecting the two contour points and the horizontal direction is calculated based on the first contour point and the second contour point. Then, a rotation matrix function is used to create a rotation matrix. Based on the rotation matrix and an affine transformation function, the ship contour in the clipping region is rotated according to the calculated angle to obtain the clipping region after ship orientation correction.

[0067] The third-stage detection and result verification module inputs the cropped region after ship orientation correction into a lightweight, pre-trained YOLOv5nano network model for third-stage detection, obtaining the second target detection result of the ship within the cropped region after ship orientation correction. It then verifies whether the first target detection result output by the pre-trained DINO target detection model in the second stage of screening and detection contains false positives or false negatives based on the second target detection result output by the pre-trained YOLOv5nano network model. If not, the second target detection result is used as the final ship target detection result; otherwise, the false positives or false negatives are corrected by combining the first and second target detection results to complete the ship target detection in the natural scene.

[0068] Preferably, in the first stage of the ship image screening module, multiple natural scene images to be tested are input into the trained YOLOv5x network model for first-stage screening and detection, resulting in multiple natural scene images containing ships, specifically including:

[0069] Multiple natural scene images to be tested are input into a trained YOLOv5x network model, which outputs the detection results of all ships in each natural scene image. The ship detection results include bounding boxes, class labels, and confidence scores. The confidence scores of all ship detection results for a certain natural scene image are compared with a preset confidence threshold. If the confidence score of at least one ship detection result is greater than or equal to the confidence threshold, the natural scene image is considered to contain ships and is saved and labeled, thus obtaining multiple natural scene images containing ships.

[0070] Preferably, in the third-stage detection and detection result verification module, the correction of false detections or missed detections by combining the first target detection results and the second target detection results includes: if there are false detections, then based on the first target detection results output by the DINO target detection model, the false detection boxes that do not belong to the ship identified from the second target detection results output by the YOLOv5nano network model are removed from the final detection results; if there are missed detections, then the missed targets identified by the YOLOv5nano network model are added to the first target detection results output by the DINO model, and the positions of the targets are re-labeled.

[0071] Preferably, in the second-stage ship fine detection and cropping region acquisition module, before inputting the multiple natural scene images containing ships obtained from the first-stage screening and detection into the trained DINO object detection model, the multiple natural scene images containing ships selected by YOLOv5x are first resized and standardized to adapt to the input requirements of the trained DINO object detection model.

[0072] Preferably, the vessel detection dataset includes various types of vessel images and annotations. The COCO dataset includes multiple different object categories, including combinations of people, bicycles, cars, motorcycles, airplanes, buses, trains, trucks, ships, traffic lights, fire hydrants, stop signs, parking meters, and benches.

[0073] This invention provides an objective and scientific method and system for ship target detection in natural scenes. By utilizing a three-stage deep learning detection algorithm (YOLOv5x network model, DINO target detection model, and YOLOv5nano network model) to sequentially identify ships in the natural scene image under test, the ship detection rate is greatly improved. In addition, for ships that are difficult to detect, the accuracy and precision of identifying distant and blurry ships can be greatly improved.

[0074] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail with reference to the accompanying drawings and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention patent.

Claims

1. A natural scene ship target detection method, characterized in that, The method comprises the following steps: An image and dataset acquisition step: acquiring a ship detection dataset, a COCO dataset, and a to-be-tested natural scene video image, and performing frame separation processing on the to-be-tested natural scene video image to obtain a plurality of to-be-tested natural scene images; A first-stage ship image screening step: training a YOLOv5x network model and a YOLOv5nano network model using the ship detection dataset as a training set sample, respectively obtaining a trained YOLOv5x network model and a trained YOLOv5nano network model, and inputting the plurality of to-be-tested natural scene images into the trained YOLOv5x network model for first-stage screening detection to obtain a plurality of natural scene images containing ships; A second-stage ship fine detection and region cropping step: training a DINO target detection model using the COCO dataset to obtain a trained DINO target detection model, and inputting the plurality of natural scene images containing ships obtained in the first-stage screening detection into the trained DINO target detection model for second-stage screening detection to obtain first target detection results of all ships in each natural scene image, and cropping a ship-containing cropped region according to the first target detection results of each ship; A ship direction correction step: extracting a ship contour from the ship-containing cropped region by using a contour finding function, extracting points corresponding to a minimum horizontal coordinate and a maximum horizontal coordinate from the ship contour as first and second contour points according to the first target detection results, calculating an included angle between a line connecting the first and second contour points and a horizontal direction, creating a rotation matrix by using a rotation matrix function, and rotating the ship contour in the cropped region according to the calculated included angle based on the rotation matrix by using an affine transformation function to obtain a ship direction-corrected cropped region; A third-stage detection and detection result verification step: inputting the ship direction-corrected cropped region into a lightweight trained YOLOv5nano network model for third-stage detection to obtain second target detection results of the ships in the ship direction-corrected cropped region, verifying whether the first target detection results output by the trained DINO target detection model in the second-stage screening detection contain false positives or false negatives according to the second target detection results output by the trained YOLOv5nano network model, and if not, taking the second target detection results as final ship target detection results; if so, correcting the false positives or false negatives by combining the first target detection results and the second target detection results to complete ship target detection in a natural scene.

2. The natural scene ship target detection method according to claim 1, characterized in that, In the first-stage ship image screening step, the plurality of to-be-tested natural scene images are inputted into the trained YOLOv5x network model for first-stage screening detection to obtain the plurality of natural scene images containing ships, and the method specifically comprises the following steps: The plurality of to-be-tested natural scene pictures are input into the trained YOLOv5x network model respectively, and all ship detection results in each natural scene picture are output, the ship detection results including a bounding box, a category label and a confidence score, and the confidence scores in all ship detection results of a certain natural scene picture are compared with a preset confidence threshold respectively, if there is at least one ship detection result whose confidence score is greater than or equal to the confidence threshold, it is considered that the natural scene picture contains a ship and is saved and marked, and finally a plurality of natural scene pictures containing a ship are obtained.

3. The natural scene ship target detection method according to claim 1, characterized in that, In the third stage detection and detection result verification step, the false detection or missed detection is corrected by combining the first target detection result and the second target detection result, including: if there is a false detection, the false detection box that does not belong to the ship recognized from the second target detection result output by the YOLOv5nano network model is removed from the final detection result according to the first target detection result output by the DINO target detection model; if there is a missed detection, the missed target recognized by the YOLOv5nano network model is added to the first target detection result output by the DINO model, and the position of the target is relabeled.

4. The natural scene ship target detection method according to one of claims 1 to 3, characterized in that, In the second stage ship fine detection and cropped region acquisition step, before the plurality of natural scene pictures containing ships screened by the first stage are input into the trained DINO target detection model, the plurality of natural scene pictures containing ships screened by the YOLOv5x are first subjected to size adjustment and standardization processing to adapt to the input requirements of the trained DINO target detection model.

5. The natural scene ship target detection method according to claim 1, wherein, In the image and data set acquisition step, the ship detection data set includes a plurality of different types of ship images and labels, the COCO data set includes a plurality of different object categories, and the plurality of different object categories include a person, a bicycle, a car, a motorcycle, an airplane, a bus, a train, a truck, a ship, a traffic light, a fire hydrant, a stop sign, a parking meter, and a bench.

6. A natural scene ship target detection system characterized by, The method comprises the sequentially connected image and data set acquisition module, the first stage ship picture screening module, the second stage ship fine detection and cropped region acquisition module, the ship direction correction module, and the third stage detection and detection result verification module, The image and data set acquisition module acquires a ship detection data set, a COCO data set, and a to-be-tested natural scene video image, and frames the to-be-tested natural scene video image to obtain a plurality of to-be-tested natural scene pictures; The first stage ship picture screening module uses the ship detection data set as a training set sample to train a YOLOv5x network model and a YOLOv5nano network model respectively, obtains a trained YOLOv5x network model and a trained YOLOv5nano network model respectively, and inputs the plurality of to-be-tested natural scene pictures into the trained YOLOv5x network model for first stage screening detection to obtain a plurality of natural scene pictures containing a ship; The first stage ship picture screening module uses the ship detection data set as a training set sample to train a YOLOv5x network model and a YOLOv5nano network model respectively, obtains a trained YOLOv5x network model and a trained YOLOv5nano network model respectively, and inputs the plurality of to-be-tested natural scene pictures into the trained YOLOv5x network model for first stage screening detection to obtain a plurality of natural scene pictures containing a ship; The second-stage ship fine detection and cropped region acquisition module uses the COCO dataset to train the DINO target detection model to obtain a trained DINO target detection model, and inputs a plurality of natural scene pictures containing ships obtained through the first-stage screening detection into the trained DINO target detection model for second-stage screening detection, to obtain first target detection results of all ships in each natural scene picture, and crops a cropped region containing a ship according to the first target detection result of each ship. The ship direction correction module extracts a ship contour from the cropped region containing the ship by using a contour searching function, extracts a first contour point and a second contour point from the ship contour according to the first target detection result, the first contour point and the second contour point being points corresponding to the minimum horizontal coordinate and the maximum horizontal coordinate respectively, calculates an included angle between a line connecting the two contour points and the horizontal direction, creates a rotation matrix by using a rotation matrix function, and rotates the ship contour in the cropped region according to the calculated included angle based on the rotation matrix by using an affine transformation function, to obtain a cropped region after ship direction correction. The third-stage detection and detection result verification module inputs the cropped region after ship direction correction into a lightweight trained YOLOv5nano network model for third-stage detection, to obtain second target detection results of the ship in the cropped region after ship direction correction, and verifies whether the first target detection results output by the second-stage screening detection of the trained DINO target detection model have false detection or missed detection according to the second target detection results output by the trained YOLOv5nano network model, if not, the second target detection results are taken as final ship target detection results, and if yes, the false detection or missed detection is corrected by combining the first target detection results and the second target detection results, to complete ship target detection in a natural scene.

7. The natural scene ship target detection system of claim 6, wherein, In the first-stage ship-containing picture screening module, a plurality of natural scene pictures to be detected are input into the trained YOLOv5x network model for first-stage screening detection, to obtain a plurality of natural scene pictures containing ships, which specifically include: The plurality of natural scene pictures to be detected are input into the trained YOLOv5x network model, to output all ship detection results in each natural scene picture, the ship detection results including a bounding box, a class label and a confidence score, and the confidence scores in all ship detection results of a certain natural scene picture are compared with a preset confidence threshold, if there is at least one ship detection result whose confidence score is greater than or equal to the confidence threshold, it is considered that the natural scene picture contains a ship and is saved and marked, and finally a plurality of natural scene pictures containing ships are obtained.

8. The natural scene ship target detection system of claim 6, wherein, In the third stage detection and detection result verification module, the false detection or missed detection is corrected by combining the first target detection result and the second target detection result, including: if there is a false detection, removing the false detection box that does not belong to a ship from the final detection result according to the first target detection result output by the DINO target detection model; if there is a missed detection, adding the missed detection target recognized by the YOLOv5nano network model to the first target detection result output by the DINO model, and relabeling the position of the target.

9. The natural scene ship target detection system according to one of claims 6 to 8, characterized in that In the second stage ship fine detection and cropped region acquisition module, before the multiple natural scene pictures containing ships screened by the first stage are respectively input into the trained DINO target detection model, the multiple natural scene pictures containing ships screened by the YOLOv5x are subjected to size adjustment and standardization processing to adapt to the input requirements of the trained DINO target detection model.

10. The natural scene ship target detection system of claim 6, wherein, The ship detection dataset includes a plurality of different types of ship images and annotations, and the COCO dataset includes a plurality of different object categories, including people, bicycles, cars, motorcycles, airplanes, buses, trains, trucks, ships, traffic lights, fire hydrants, stop signs, parking meters, and benches.

Images