Ship identity recognition method and device based on unmanned aerial vehicle, and electronic equipment

Abstract

The invention discloses a ship identity recognition method and device based on an unmanned aerial vehicle and electronic equipment, and relates to the technical field of ship recognition, and the ship identity recognition method comprises the steps: employing the unmanned aerial vehicle to collect an image of a target water area, obtaining image data, carrying out the preprocessing of the image, obtaining a target image, and carrying out the recognition of the target image; and processing the target image by using a preset ship detection model to obtain ship information, performing feature extraction on the target image based on the bounding box to obtain a target feature vector, and determining target identity information of the target ship from a preset ship feature database based on the target feature vector. According to the invention, the technical problem of low accuracy of ship identity recognition in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship identification, in particular to a ship identity identification method and device based on a UAV and an electronic device. BACKGROUND

[0002] Water traffic supervision, port scheduling, and fishery management have long relied on accurate and real-time identification and classification of ship targets. With the increasing frequency of shipping activities and the rapid development of the marine economy, traditional ship identification methods have gradually exposed limitations in real-time performance, accuracy, and environmental adaptability. Traditional ship identification includes the following methods:

[0003] (1) Traditional ship identification method. This method mainly relies on manual observation, radar monitoring, and AIS (Automatic Identification System). However, manual observation is inefficient, easily influenced by subjective factors, and difficult to achieve all-weather coverage; radar systems can achieve monitoring within a certain range, but have low resolution for small targets and are easily disturbed by weather and sea conditions. The AIS system achieves identification by actively sending identity information from the ship, but its coverage is limited by the popularity of the equipment, and some small ships (such as fishing boats and yachts) do not have AIS terminals, and there is a risk of signal forgery or closure, resulting in regulatory loopholes. For example, in complex waters or at night, AIS signals may be lost or delayed, making it impossible to meet real-time monitoring requirements.

[0004] (2) Computer vision-based identification technology. With the advancement of image processing technology, traditional computer vision algorithms (such as edge detection and template matching) can be used for ship detection. This method analyzes the shape features of the ship (such as the hull contour, color texture) to achieve preliminary classification, but it is heavily dependent on image quality and has a high false detection rate in complex lighting, wave interference, or foggy weather. In addition, this method is difficult to adapt to multi-scale ship targets (such as small fishing boats in the distance and cargo ships in the near distance), especially for partially occluded or angle-changed ships, and needs to adjust parameters repeatedly for different scenarios, which has poor generalization ability.

[0005] (3) Deep learning in ship identification. With the introduction of deep learning models represented by convolutional neural networks into the field of ship identification, the accuracy can be improved by automatically learning ship features. However, these models are designed only for static images and have poor real-time performance, making them difficult to apply directly to unmanned aerial vehicle (UAV) aerial photography or video stream scenarios. For example, the detection accuracy of small target ships (such as buoys and speedboats) is insufficient, and they are easily affected by water surface reflections and weather changes. In addition, current deep learning models rely heavily on data collected by fixed cameras, while dynamic identification techniques based on mobile platforms such as UAVs are still not mature, especially the images taken by UAVs have problems such as shaking and varying scales, which further increase the difficulty of identification.

[0006] To address the above problems, no effective solutions have been proposed so far. SUMMARY

[0007] Embodiments of the present application provide a ship identity identification method and device based on a UAV, and an electronic device, to at least solve the technical problem of low accuracy in identifying the identity of a ship in related technologies.

[0008] According to an aspect of an embodiment of the present application, a ship identity identification method based on a UAV is provided, comprising: collecting images of a target water area using a UAV to obtain image data, wherein the image data includes: images and image corresponding image space information, and the image space information at least includes: collection time, position information; pre-processing the images to obtain target images; processing the target images using a preset ship detection model to obtain ship information, wherein the ship information at least includes: a bounding box of a target ship, and the bounding box is used to mark the position of the target ship in the target image; based on the bounding box, extracting features from the target image to obtain a target feature vector, and based on the target feature vector, determining target identity information of the target ship from a preset ship feature database, wherein the preset ship feature database includes: a plurality of identity information associated with the feature vector.

[0009] Further, the UAV at least includes a positioning module and an inertial measurement unit, and the UAV is connected with a camera through a three-axis stabilization gimbal. The step of collecting images of a target water area using a UAV to obtain image data includes: controlling the UAV to cruise along a patrol route on the target water area; during the cruising of the UAV, controlling the camera to shoot images according to a preset frame rate and recording the collection time of the images shot by the camera; based on the collection time, determining the position information corresponding to the images and the attitude angle of the UAV when the images are shot, wherein the position information is obtained through the positioning module, and the attitude angle of the UAV is obtained through the inertial measurement unit; based on the images, the collection time corresponding to the images, the position information, and the attitude angle of the UAV, constructing the image data.

[0010] Further, the step of preprocessing the image to obtain the target image comprises: performing noise reduction processing on the image to obtain a noise-reduced image; performing contrast enhancement processing on the noise-reduced image to obtain a first enhanced image; performing image enhancement processing on the first enhanced image to obtain a second enhanced image; performing color space conversion on the second enhanced image, and cropping the converted second enhanced image to obtain the target image.

[0011] Further, before processing the target image using the preset ship detection model to obtain ship information, the method further comprises: collecting historical images, and performing enhancement processing on each historical image, wherein the enhancement processing comprises at least one of the following: flip processing, rotation processing, brightness adjustment processing, contrast adjustment processing, noise addition processing, and weather simulation processing; constructing a training image set based on all the historical images and the historical images after the enhancement processing; labeling each training image in the training image set to obtain labeling information, wherein the labeling information comprises a labeled bounding box of a ship on the training image; constructing an initial ship detection model, wherein the model structure of the initial ship detection model comprises at least: a pyramid network structure, a path aggregation network structure, and a detection structure; the pyramid network structure is used to extract features of training images of different scales to obtain a first multi-layer feature map, and fuse the first multi-layer feature map from top to bottom to obtain a first fused feature map; the path aggregation network structure is used to extract features of training images of different scales to obtain a second multi-layer feature map, and fuse the second multi-layer feature map from bottom to top to obtain a second fused feature map; the detection structure is used to detect the first fused feature map and the second fused feature map to obtain a predicted bounding box of a ship on each training image; training the initial ship detection model using the training image set and the labeling information of each training image until a loss value determined by a loss function is less than a preset loss threshold to obtain the preset ship detection model, wherein the loss value is determined by the loss function based on the labeled bounding box and the predicted bounding box.

[0012] Further, the step of extracting features from the target image based on the bounding box to obtain a target feature vector comprises: extracting a ship target region from the target image based on the bounding box; and extracting features of the ship target region to obtain the target feature vector.

[0013] Further, the step of determining the target identity information of the target ship from the preset ship feature database based on the target feature vector comprises: determining a similarity between the target feature vector and each feature vector in the preset ship feature database; determining whether the maximum similarity exceeds a preset similarity threshold, and in the case that the maximum similarity exceeds the preset similarity threshold, determining the identity information associated with the feature vector indicated by the maximum similarity as the target identity information.

[0014] Furthermore, after determining the target vessel's identity information from a pre-set vessel feature database based on the target feature vector, the process further includes: acquiring vessel information collected by the Automatic Identification System (AIS), wherein the vessel information includes at least: vessel identity information corresponding to the vessel's location and timestamp; performing spatiotemporal registration with the vessel information based on image spatiotemporal information to obtain target vessel identity information matching the collection time and location information; comparing the target vessel identity information with the target identity information to obtain a comparison result; and triggering a warning message if the comparison result indicates that the target vessel identity information and the target identity information are inconsistent.

[0015] According to another aspect of the present invention, a vessel identification device based on an unmanned aerial vehicle (UAV) is also provided, comprising: a data acquisition unit, configured to acquire images of a target water area using an UAV to obtain image data, wherein the image data includes: an image and corresponding spatiotemporal information of the image, the spatiotemporal information of the image including at least: acquisition time and location information; a first processing unit, configured to preprocess the image to obtain a target image; a second processing unit, configured to process the target image using a preset vessel detection model to obtain vessel information, wherein the vessel information includes at least: a bounding box of the target vessel, the bounding box being used to mark the position of the target vessel in the target image; and a determination unit, configured to extract features from the target image based on the bounding box to obtain a target feature vector, and determine the target identity information of the target vessel from a preset vessel feature database based on the target feature vector, wherein the preset vessel feature database includes: multiple identity information associated with the feature vector.

[0016] Furthermore, the UAV includes at least: a positioning module and an inertial measurement unit. The UAV is connected to the camera via a three-axis stabilization gimbal. The acquisition unit includes: a first cruise module for controlling the UAV to cruise along the inspection route over the target water area; a first shooting module for controlling the camera to capture images at a preset frame rate during the UAV's cruise and recording the acquisition time of the captured images; a first determination module for determining the position information and UAV attitude angle corresponding to the captured image based on the acquisition time, wherein the position information is obtained through the positioning module and the UAV attitude angle is obtained through the inertial measurement unit; and a first construction module for constructing image data based on the image, the acquisition time corresponding to the image, the position information, and the UAV attitude angle.

[0017] Further, the first processing unit includes: a first noise reduction module for performing noise reduction processing on the image to obtain a noise-reduced image; a first enhancement module for performing contrast enhancement processing on the noise-reduced image to obtain a first enhanced image; a second enhancement module for performing image enhancement processing on the first enhanced image to obtain a second enhanced image; and a first conversion module for performing color space conversion on the second enhanced image and cropping the converted second enhanced image to obtain a target image.

[0018] Furthermore, the ship identification device also includes: a first acquisition module, used to acquire historical images before processing the target image using a preset ship detection model to obtain ship information, and to perform enhancement processing on each historical image, wherein the enhancement processing includes at least one of the following: flipping processing, rotation processing, brightness adjustment processing, contrast adjustment processing, noise addition processing, and simulated weather processing; a second construction module, used to construct a training image set based on all historical images and the enhanced historical images; a first annotation module, used to annotate each training image in the training image set to obtain annotation information, wherein the annotation information includes: the labeled bounding box of the ship on the training image; and a third construction module, used to construct an initial ship detection model, wherein the model structure of the initial ship detection model includes at least: a pyramid network structure and a path aggregation network. The system comprises a network structure and a detection structure. A pyramid network structure is used to extract features from training images at different scales, obtaining a first multi-layer feature map. These first multi-layer feature maps are then fused from top to bottom to obtain a first fused feature map. A path aggregation network structure is used to extract features from training images at different scales, obtaining a second multi-layer feature map. These second multi-layer feature maps are then fused from bottom to top to obtain a second fused feature map. The detection structure is used to detect the first and second fused feature maps, obtaining the predicted bounding boxes of ships on each training image. A first training module is used to train an initial ship detection model using the training image set and the annotation information of each training image until the loss value determined by the loss function is less than a preset loss threshold, thus obtaining a preset ship detection model. The loss value is determined by the loss function based on the labeled bounding boxes and the predicted bounding boxes.

[0019] Furthermore, the determining unit includes: a first extraction module for extracting the ship target region from the target image based on the bounding box; and a second extraction module for extracting features from the ship target region to obtain a target feature vector.

[0020] Furthermore, the determining unit also includes: a second determining module, used to determine the similarity between the target feature vector and each feature vector in the preset ship feature database; and a first judging module, used to judge whether the maximum similarity exceeds a preset similarity threshold, and if the maximum similarity exceeds the preset similarity threshold, to determine the identity information associated with the feature vector indicated by the maximum similarity as the target identity information.

[0021] Furthermore, the ship identification device also includes: a first acquisition module, used to acquire ship information collected by the automatic ship identification system after determining the target ship's identity information from a preset ship feature database based on the target feature vector, wherein the ship information includes at least: ship identity information corresponding to the ship's position and timestamp; a first registration module, used to perform spatiotemporal registration with the ship information based on image spatiotemporal information to obtain target ship identity information matching the acquisition time and location information; a first comparison module, used to compare the target ship identity information and the target identity information to obtain a comparison result; and a first trigger module, used to trigger a warning message when the comparison result indicates that the target ship identity information and the target identity information are inconsistent.

[0022] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for ship identification based on unmanned aerial vehicles.

[0023] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described unmanned aerial vehicle-based ship identification methods.

[0024] In this invention, an unmanned aerial vehicle (UAV) is used to collect images of the target water area to obtain image data. The images are preprocessed to obtain target images. A preset ship detection model is used to process the target images to obtain ship information. Based on the bounding box, features are extracted from the target images to obtain target feature vectors. Based on the target feature vectors, the target identity information of the target ship is determined from a preset ship feature database, thereby solving the technical problem of low accuracy in identifying ship identity in related technologies.

[0025] In this invention, image data of the target water area is collected by unmanned aerial vehicles (UAVs). By integrating high-precision GPS (Global Positioning System) and IMU (Inertial Measurement Unit), the spatiotemporal information of the images (including acquisition time and location information) is accurately recorded. This achieves the goal of obtaining high-quality ship images with precise spatiotemporal coordinates, thus realizing the technical effect of rapid ship positioning and identity binding based on accurate spatiotemporal information. Specifically, image quality is improved through image preprocessing techniques (such as denoising and enhancement). A ship detection model is used to efficiently analyze the processed target image to obtain ship detection information (at least including bounding boxes), which accurately pinpoint the ship's position in the image. Then, based on these bounding boxes, deep learning feature extraction methods are used to obtain target feature vectors that characterize the "ship's face." These vectors are then matched with feature vectors in a pre-set ship feature database to determine the specific identity information of the target ship (such as ship name and MMSI (Maritime Mobile Service Identity)). This effectively solves the problems of low accuracy and poor robustness of vision-based ship identification in related technologies, achieving accurate ship identification in complex environments. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0027] Figure 1 This is a flowchart of an optional unmanned aerial vehicle (UAV)-based ship identification method according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of an optional ship identification process based on a drone and the YOLOv8 algorithm according to an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of an optional unmanned aerial vehicle (UAV)-based ship identification device according to an embodiment of the present invention;

[0030] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for a ship identification method based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] It should be noted that all related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected and involved in this invention are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and it does not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0034] In this invention, while deep learning-based static image recognition methods offer high detection accuracy, their complex models and high computational demands result in lengthy processing times for single-frame images. In dynamic scenarios such as real-time UAV tracking of ships, the target's state constantly changes due to frequent variations in UAV flight altitude and angle, making it difficult to achieve high-speed processing while maintaining accuracy. This fails to meet the practical needs of real-time monitoring and immediate response to ship dynamics in maritime supervision. Furthermore, while AIS systems can provide near-real-time data, their information update frequency and signal stability are limited by equipment and environment, and they cannot always provide continuous and reliable data streams at critical decision-making moments. This invention provides a ship identification method and system based on a UAV platform. By optimizing the algorithm model structure (e.g., adopting and specifically improving the computationally efficient single-stage detection network YOLOv8 (You Only Look Once version 8, the target detection algorithm)) and data processing flow, the processing latency of image analysis is significantly reduced. This enables rapid detection and identification of ship targets in video streams on a mobile UAV platform, ensuring that the entire process from image acquisition to result output is completed in a very short time, effectively supporting real-time dynamic monitoring and rapid response to moving ships.

[0035] In this invention, the appearance of ships navigating in real marine environments is affected by a variety of complex factors, including sea surface reflection, changes in light intensity, weather conditions (such as fog and rain), wave interference, and potential occlusion of the target itself. Traditional image processing methods suffer from poor robustness and a sharp drop in recognition accuracy when faced with these interferences. Even deep learning-based methods exhibit unstable recognition accuracy when dealing with challenges such as shadows caused by strong light contrast, water surface reflection obscuring target features, blurred long-distance imaging of small targets, and image quality degradation caused by severe weather. This invention introduces targeted image enhancement techniques, improves the model to better learn the stable features of ships, and designs an effective feature fusion mechanism. This enables the recognition system to effectively resist interference from complex environmental factors such as sea surface reflection, changes in light intensity, fog, waves, and varying target scales. It also focuses on improving the detection and recognition accuracy of the algorithm in the case of small targets, blurred targets, and partially occluded targets, ensuring a high and stable recognition accuracy in various complex real-world application scenarios.

[0036] In this invention, many current technical solutions exhibit an "island-like distribution" and fail to effectively integrate. For example, the AIS system provides ship identity and location data but lacks intuitive visual information; drone aerial photography can acquire rich visual data, but traditional recognition methods only output the ship's bounding box and rough category, unable to directly and accurately bind it to the specific ship identity in AIS. Data barriers and process breakpoints exist between the visual recognition system and business systems (such as command and dispatch, risk evidence collection), preventing the automated closed-loop completion of the process from "target detection" to "identity recognition" to "response feedback." This fragmented state makes it difficult for the system to achieve true end-to-end integration. The use of edge intelligent identification and application not only increases the cost of manual intervention but also reduces the overall regulatory efficiency. This invention designs and implements an end-to-end system that integrates UAV perception, intelligent algorithm analysis, multi-source data (such as visual data and AIS data) fusion, and business applications. This system can automatically complete the entire process from UAV image acquisition, ship target detection and positioning, "ship face" feature extraction and identity recognition, to automatic association with AIS information, risk assessment, result visualization, and evidence data generation. By breaking down information silos, it realizes an intelligent closed loop of perception, cognition, decision-making, and execution, ultimately improving the overall efficiency and automation level of maritime supervision.

[0037] The present invention will now be described in detail with reference to various embodiments.

[0038] Example 1

[0039] According to an embodiment of the present invention, an embodiment of a ship identification method based on unmanned aerial vehicles is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0040] Figure 1 This is a flowchart of an optional UAV-based ship identification method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0041] Step S101: Use a drone to collect images of the target water area to obtain image data. The image data includes: the image and the corresponding spatiotemporal information of the image. The spatiotemporal information of the image includes at least: the collection time and location information.

[0042] In this embodiment of the invention, a drone can be used as an image acquisition platform. This drone possesses flight stability and precise positioning capabilities. The drone is equipped with a high-resolution visible light camera, which automatically cruises along a preset route during flight, continuously acquiring video streams or periodically capturing still images. Simultaneously with image acquisition, the drone's built-in high-precision GPS module and inertial measurement unit (IMU) record the acquisition time, location information (latitude, longitude, and altitude), and drone attitude information for each image. This information collectively constitutes the spatiotemporal information of the image, ensuring the spatiotemporal consistency between the image data and the real scene.

[0043] Step S102: Preprocess the image to obtain the target image.

[0044] In this embodiment of the invention, the acquired original image may be affected by noise, uneven lighting, fog, etc. Therefore, non-local means denoising (NL-Means Filtering) or BM3D (Block-Matching and 3D filter algorithm) algorithms can be used to denoise the image first. Then, contrast-limited adaptive histogram equalization (CLAHE) technology is used to enhance the image contrast and improve dark details. For backlit or highly reflective scenes, image enhancement algorithms based on Retinex theory can be applied to restore the true color and texture information of the image. In addition, the image size can be uniformly scaled or cropped to 640x640 pixels to ensure that it meets the input requirements of the YOLOv8 model (preset ship detection model). Furthermore, the color space can be converted to HSV (Hue, Saturation, Value) to better separate color information, thereby obtaining the preprocessed target image.

[0045] Step S103: The target image is processed using a preset ship detection model to obtain ship information, wherein the ship information includes at least the bounding box of the target ship, which is used to mark the position of the target ship in the target image.

[0046] In this embodiment of the invention, a pre-trained YOLOv8 target detection model (i.e., a preset ship detection model) is used to analyze the preprocessed target image. YOLOv8 is a single-stage detection algorithm that can quickly and accurately locate and identify ships in target images, outputting ship information including bounding box coordinates, class confidence, and class label.

[0047] Here, the bounding box is a rectangular box output by the model, used to accurately calibrate the position of the ship in the image.

[0048] Step S104: Based on the bounding box, feature extraction is performed on the target image to obtain the target feature vector. Based on the target feature vector, the target identity information of the target ship is determined from the preset ship feature database. The preset ship feature database includes multiple identity information associated with the feature vector.

[0049] In this embodiment of the invention, for each ship bounding box detected by YOLOv8, features can be extracted using a pre-trained deep convolutional neural network (such as a variant of ResNet or EfficientNet) to obtain a high-dimensional visual feature vector of the ship. These feature vectors can characterize the ship's hull structure, principal scale proportions, color distribution, and other "ship face" features. Then, the target feature vector is compared with feature vectors in a pre-built ship feature database using similarity calculations (such as cosine similarity) to determine the identity information of the target ship. This database stores standardized feature vectors of known ships and their corresponding identity information, such as ship name, MMSI code, and ship type.

[0050] In summary, by using drones to collect image data of the target water area and integrating high-precision GPS and IMU to ensure accurate recording of image spatiotemporal information (including acquisition time and location information), the goal of obtaining high-quality ship images with accurate spatiotemporal coordinates can be achieved. This enables the technical effect of rapid ship positioning and identification based on accurate spatiotemporal information. Specifically, image quality is improved through image preprocessing techniques (such as denoising and enhancement). A ship detection model is then used to efficiently analyze the processed target image to obtain ship detection information (at least including bounding boxes), which accurately pinpoint the ship's position in the image. Then, based on these bounding boxes, deep learning feature extraction methods are used to obtain target feature vectors that can represent the "ship's face." By matching these vectors with feature vectors in a pre-set ship feature database, the specific identity information of the target ship (such as ship name and MMSI code) is determined. This effectively solves the problems of low accuracy and poor robustness of vision-based ship identification in related technologies, achieving accurate ship identification in complex environments.

[0051] Optionally, the UAV includes at least a positioning module and an inertial measurement unit. The UAV is connected to the camera via a three-axis stabilization gimbal. To improve the accuracy of the constructed image data, in the UAV-based ship identification method provided in Embodiment 1 of this application, the UAV is controlled to cruise along the inspection route in the target waters. During the UAV's cruise, the camera is controlled to capture images at a preset frame rate, and the acquisition time of the captured images is recorded. Based on the acquisition time, the position information and UAV attitude angle corresponding to the captured image are determined, wherein the position information is obtained through the positioning module, and the UAV attitude angle is obtained through the inertial measurement unit. Based on the image, image data is constructed using the acquisition time, position information, and UAV attitude angle corresponding to the image.

[0052] In this embodiment of the invention, a drone can be used as the flight platform. This drone possesses excellent flight stability, wind resistance, and long endurance. Its built-in high-precision GPS module and inertial measurement unit (IMU) ensure accurate positioning and attitude control. The drone is equipped with a high-definition visible light camera with optical zoom as its main sensor, with a resolution of no less than 12 megapixels, supporting 4K video streaming. To ensure image quality, the camera sensor needs to have a high dynamic range to cope with complex lighting conditions such as strong water surface reflections. The camera is connected to the drone via a three-axis stabilization gimbal to effectively counteract vibrations during flight, ensuring clear and stable images.

[0053] In this embodiment of the invention, the inspection route of the UAV is pre-set through the route planning of the ground control station, covering key waterways, anchorages, and sensitive areas of the target waterway. The UAV automatically cruises according to the preset route, and the flight altitude can be adjusted within the range of 50-150 meters according to the identification requirements to balance the field of view and image resolution. During the cruise, the camera continuously collects video streams or takes high-definition still images at set frame rates (e.g., 10-30 FPS). To cope with changes in lighting at different times, it supports automatic or manual adjustment of camera parameters (e.g., shutter speed, ISO, white balance).

[0054] In this embodiment of the invention, the drone transmits the acquired image data to a ground data processing center or edge computing node in real time via a 5G network or a dedicated wireless image transmission link. The transmission process must ensure sufficient bandwidth (typically no less than 20Mbps) to guarantee low latency and stability of the video stream. The raw image data and related spatiotemporal information (such as latitude and longitude, altitude, shooting time, and drone attitude angle) are synchronously recorded and stored on a dedicated solid-state drive or cloud storage server for subsequent querying and retrieval.

[0055] Specifically, the UAV is route-planned and automatically controlled via a ground control station. First, the operator inputs the geographic coordinates of the target water area and key monitoring points into the route planning software at the ground control station. Based on this information, the system automatically generates or manually adjusts the UAV's patrol route. This route covers all the water areas requiring monitoring, ensuring comprehensive, blind-spot-free image acquisition. The UAV automatically takes off and cruises according to the preset route, exhibiting a high degree of automation. The UAV's onboard high-definition visible light camera, by calling the camera API (Application Programming Interface) or SDK (Software Development Kit), sets a suitable frame rate (e.g., 25 FPS) and continuously streams video during the UAV's cruise. This frame rate selection comprehensively considers image quality and real-time data transmission, ensuring the continuity and richness of the video stream. Simultaneously, the camera's built-in clock module records the precise acquisition time of each frame, saving it along with the image data for subsequent data processing and spatiotemporal information registration. During drone cruise photography, the positioning module (such as GPS) and inertial measurement unit (IMU) simultaneously record the drone's real-time position information (including latitude, longitude, and altitude) and attitude information (including pitch, roll, and yaw angles). This information is recorded synchronously with image acquisition time, ensuring consistency between the image and its corresponding metadata. Specifically, for each frame captured by the drone, the system simultaneously records the GPS coordinates and the drone's attitude angles measured by the IMU at that moment; these data are integrated into the image's metadata.

[0056] In this embodiment, the UAV can be efficiently controlled to automatically cruise over the target water area, while simultaneously capturing high-quality video stream images at a preset frame rate. The acquisition time, location information, and UAV attitude angle of each frame are accurately recorded, constructing image data containing this metadata. This series of operations provides a solid foundation for subsequent image preprocessing, rapid location and identification of ship targets, and accurate matching of ship identity information.

[0057] To improve the accuracy of identifying the target image, in the UAV-based ship identification method provided in Embodiment 1 of this application, the image is denoised to obtain a denoised image; the denoised image is contrast-enhanced to obtain a first enhanced image; the first enhanced image is image-enhanced to obtain a second enhanced image; the second enhanced image is color-space converted, and the converted second enhanced image is cropped to obtain the target image.

[0058] In this embodiment of the invention, in order to overcome common noise, uneven lighting, and complex environmental interference in drone aerial images, the original images captured by the drone can be processed to improve image quality and provide standardized input for subsequent algorithm analysis, including:

[0059] Image Denoising and Enhancement: Since drone aerial photography is susceptible to atmospheric turbulence and equipment noise, non-local means denoising (NL-Means) or BM3D algorithms are first used to reduce noise, effectively smoothing the noise while preserving ship edge details. To address common issues on water surfaces such as uneven lighting and fog interference, Limit Contrast Adaptive Histogram Equalization (CLAHE) is employed to enhance image contrast and improve shadow details. For backlit or highly reflective scenes, an image enhancement algorithm based on Retinex theory is applied to compress the dynamic range and restore the image's true color and texture information.

[0060] Image size normalization and color space conversion: To meet the input requirements of the YOLOv8 model, all input images are uniformly scaled or cropped to 640x640 pixels. Simultaneously, the images are converted from the original RGB (Red Green Blue) color space to a color space more suitable for computer vision tasks. For example, when performing specific feature analysis, they can be converted to the HSV color space for better separation of color information.

[0061] Specifically, due to the complex weather conditions during drone flights, as well as the thermal noise of electronic components and fluctuations in ambient light, the original images often carry a certain degree of noise, such as Gaussian noise or speckle noise. To reduce the impact of this noise on the subsequent recognition process, the BM3D algorithm can be used for image denoising. Here, BM3D is an efficient, non-linear denoising method that effectively removes noise while preserving image details and structure well through block matching and 3D transform domain filtering techniques. After denoising, the image undergoes contrast enhancement to improve the uneven brightness and loss of detail caused by water reflections, fog, etc. For example, the CLAHE algorithm can be used. Here, CLAHE enhances image contrast through local adaptive histogram equalization, but limits the range of brightness level changes for each pixel to avoid excessive contrast enhancement at any pixel, which would lead to noise amplification. This algorithm is particularly suitable for processing images with uneven lighting, highlighting local details and making ship features clearer when photographed under backlight or low light conditions. The processed image becomes the first enhanced image, with significantly improved contrast and detail, laying a solid foundation for subsequent feature extraction. Building upon contrast enhancement, the first enhanced image is further processed using Retinex theory-based image enhancement techniques. Retinex theory states that the human eye's perception of light is adaptive, capable of extracting the inherent color and shape information of objects from complex lighting conditions. Based on this theory, the Retinex algorithm restores the natural color and contrast of an image by decomposing its illumination and reflection components. In this embodiment, the Retinex algorithm is used to handle water surface reflections, shadows, and uneven brightness, particularly optimizing for details in dark areas during backlighting and overexposure under strong light, ensuring clear visibility of the ship's structure and markings. The processed image, serving as the second enhanced image, exhibits higher color fidelity and illumination uniformity, facilitating accurate feature extraction. The second enhanced image, processed by the Retinex algorithm, is then converted to the HSV color space. This conversion helps separate and process color information in the image because, in the HSV color space, hue, saturation, and value become independent dimensions, enabling a more accurate description of the ship's appearance features, such as paint color and markings—crucial for ship identification. After color space conversion, the image is cropped and adjusted to a uniform input size (e.g., 640x640 pixels) to ensure input standardization for the YOLOv8 model and remove irrelevant background to reduce computational burden. This standardized image is the target image, serving as the direct input for subsequent feature extraction and ship detection.

[0062] In this embodiment, the denoising step reduces random noise in the image, while contrast enhancement and Retinex algorithm-based image enhancement improve contrast and color fidelity, particularly highlighting ship details under complex lighting conditions, thus creating favorable conditions for subsequent feature extraction. Color space conversion to HSV not only aids the model's understanding of ship color information but also makes image processing more intuitive and effective. Finally, image cropping and normalization ensure the efficiency and consistency of the YOLOv8 model during processing, reducing the demand for image computational resources.

[0063] To improve the accuracy of training the ship detection model, in the UAV-based ship identification method provided in Embodiment 1 of this application, before processing the target image using a preset ship detection model to obtain ship information, historical images are collected, and each historical image undergoes enhancement processing. The enhancement processing includes at least one of the following: flipping, rotating, adjusting brightness, adjusting contrast, adding noise, and simulating weather. A training image set is constructed based on all historical images and the enhanced historical images. Each training image in the training image set is labeled to obtain annotation information, which includes the bounding boxes of ships on the training images. An initial ship detection model is constructed, wherein the model structure of the initial ship detection model includes at least: a pyramid network structure and path aggregation. The network structure and detection structure are as follows: A pyramid network structure is used to extract features from training images at different scales to obtain a first multi-layer feature map, and then the first multi-layer feature maps are fused from top to bottom to obtain a first fused feature map; A path aggregation network structure is used to extract features from training images at different scales to obtain a second multi-layer feature map, and then the second multi-layer feature maps are fused from bottom to top to obtain a second fused feature map; The detection structure is used to detect the first and second fused feature maps to obtain the predicted bounding boxes of ships on each training image; The initial ship detection model is trained using the training image set and the annotation information of each training image until the loss value determined by the loss function is less than a preset loss threshold, thus obtaining the preset ship detection model. The loss value is determined by the loss function based on the labeled bounding boxes and the predicted bounding boxes.

[0064] In this embodiment of the invention, to improve the robustness of the trained ship detection model to changes in scale, angle, and illumination, image data is augmented online or offline during the training phase. Augmentation methods include, but are not limited to: random horizontal flipping, random rotation (-10° to +10°), random adjustment of brightness / contrast, addition of random Gaussian noise, and simulation of rain and fog effects. Data augmentation effectively expands the dataset size and enhances the model's generalization ability.

[0065] In this embodiment of the invention, the YOLOv8 object detection algorithm architecture can be adopted, and the model can be trained using a large-scale ship dataset (e.g., a dataset containing 6000 labeled infrared ship images or other visible light ship datasets). The training process is optimized using stochastic gradient descent or an optimizer, the learning rate can be adjusted using a cosine annealing strategy, and the loss function can be a multi-task loss function (including classification loss, localization loss, and confidence loss).

[0066] In this embodiment of the invention, the YOLOv8 model possesses powerful multi-scale feature extraction capabilities. Its pyramid network and path aggregation network structure effectively fuse shallow location information and deep semantic information, thereby achieving accurate detection of ships of different scales (from large cargo ships nearby to small fishing boats in the distance). During inference, the model uses a sliding window approach to predict on feature maps at multiple scales, ensuring a good detection rate for targets of different sizes. Furthermore, the model can output bounding boxes, class confidence scores, and class labels for all detected ship targets in a single forward propagation of the input image. The bounding boxes accurately pinpoint the positions of the ships in the image. In the post-processing stage, a non-maximum suppression algorithm is applied to eliminate redundant detection boxes, retaining the optimal detection result with the highest confidence score. The entire detection process is highly optimized; on edge computing devices equipped with GPUs (Graphics Processing Units), the inference time for a single image can be controlled within 50 milliseconds, meeting real-time requirements.

[0067] Specifically, data can be collected from a large number of historical images accumulated during drone inspection missions, ensuring that the dataset covers various weather conditions (sunny, cloudy, rainy, foggy, etc.), lighting conditions (daytime, nighttime, backlighting, front lighting, etc.), and different types of sea areas (ports, waterways, nearshore, open ocean, etc.). Then, each historical image undergoes a series of enhancement processes to simulate and amplify various changes in the image data, improving the model's generalization ability. These enhancement processes include, but are not limited to: flipping, rotation (angle range adjustable, such as -10° to +10°), adjusting brightness and contrast (randomly increasing or decreasing by a certain percentage), adding noise (such as Gaussian noise), and simulating weather effects (such as simulating raindrops and fog). These enhancement methods allow the model to encounter more diverse image inputs during learning, thus enabling it to better handle the complex and variable situations encountered in real-world scenarios.

[0068] Then, the original historical images are merged with the images generated after the above enhancement process to form a large training image set. This is to expand the scale and diversity of the training data through data augmentation techniques, provide richer and more comprehensive samples for model training, and improve the model's performance on unknown data.

[0069] Next, each training image in the training image set is labeled. The labeling process involves accurately marking the ship targets in each training image. Labeling software is used to draw bounding boxes for each ship and assign corresponding category labels (such as bulk carriers, container ships, tankers, etc.) to the ships within the bounding boxes. The labeling information is not limited to the bounding boxes but also includes additional attributes such as the ship's size, orientation, and attitude, which provides detailed target location and category information for model training.

[0070] Furthermore, an initial ship detection model can be constructed, whose structure includes at least: a pyramid network structure, a path aggregation network structure, and a detection structure. The pyramid network structure is used to extract features from training images at different scales, obtaining a series of feature maps at different levels (the first multi-layer feature map). Each layer of feature map represents a different resolution of the image, thus capturing the fine-grained features of the target to macroscopic structural features. Feature fusion is performed from top to bottom, ultimately obtaining a first fused feature map containing multi-scale information to enhance the model's ability to detect ships of different sizes. The path aggregation network structure is used for feature extraction, but focuses more on acquiring and fusing detailed information from the lower layers. A second multi-layer feature map is obtained through a bottom-up path, and feature fusion is performed to obtain a second fused feature map. This fusion process helps the model retain more positional details during detection, which is especially important for small target ships. The detection structure performs detection based on the first and second fused feature maps, outputting the predicted bounding box of the ship on each training image. The detection structure typically includes two parts: classification and regression. The classification part is used to identify the target category, and the regression part is used to accurately predict the target's location (i.e., the coordinates of the bounding box).

[0071] Next, the initial ship detection model is trained using a set of training images and the annotation information for each training image. The training process involves inputting the training image set and corresponding annotation information (including ship category labels and bounding box locations) into the initial model, and using stochastic gradient descent or an optimizer, along with a multi-task loss function (including classification loss, localization loss, and confidence loss), to iteratively optimize the model parameters. The loss function quantifies the difference between the model's predicted bounding boxes and the ground truth labeled bounding boxes, as well as the degree of mismatch between the predicted and actual categories. During the iterative training process, the model continuously adjusts its parameters to minimize the value of the loss function until the loss value falls below a preset threshold. This typically indicates that the model has converged to a relatively ideal performance state. Once this state is reached, the model is considered to have completed training and becomes the preset ship detection model, which can be used in practical applications for rapid ship identification and localization.

[0072] In this embodiment, through the construction of a large-scale dataset and targeted data augmentation strategies, the model has been exposed to and adapted to ship images under various lighting, weather, and angle changes during the training phase. This means that in actual deployment, the model can more stably handle dynamic image sequences captured by drones. The combination of the pyramid network structure and the path aggregation network structure not only enables the model to detect ships of different scales but also ensures the accurate capture of positional details.

[0073] To improve the accuracy of determining the target feature vector, in the UAV-based ship identification method provided in Embodiment 1 of this application, the ship target region is extracted from the target image based on the bounding box; the ship target region is then subjected to feature extraction to obtain the target feature vector.

[0074] In this embodiment of the invention, for each ship target region (i.e., the image within the bounding box) detected by the model, high-dimensional visual features are extracted using a pre-trained deep convolutional neural network. These features can capture unique and relatively stable "ship face" features such as the ship's hull structure (e.g., freeboard height, superstructure layout), principal scale proportions, hull color distribution, funnel markings, waterline features, and the ship's name / port of registry text area. Here, the feature extraction network (deep convolutional neural network) can be fine-tuned on a large maritime image dataset to make its feature representation more suitable for the ship identification task.

[0075] Specifically, after the UAV images are preprocessed and input into the model for detection, the model outputs the detection results in the form of a series of bounding boxes. Each bounding box contains the precise location information of the ship target in the image, namely the coordinates of its top-left and bottom-right corners, as well as the target's classification information and confidence score. The bounding box extraction is based on the output of the target detection, namely the bounding box coordinates predicted by the model. In this way, the ship portion in the target image can be separated from the non-ship background, ensuring that the subsequent feature extraction process focuses on the ship's details without being interfered with by irrelevant image information. For the image region within each ship bounding box, feature extraction is performed to capture and encode the ship's visual features. For example, a pre-trained deep convolutional neural network is used as the feature extractor. This network has been trained on a large-scale image dataset and has mastered rich image feature representation capabilities. The feature extractor converts each ship target region image into a fixed vector form, namely the target feature vector. This vector contains a high-dimensional feature representation of the ship, covering hull structure, superstructure layout, main scale proportions, color distribution, chimney markings, etc.

[0076] In this embodiment, the image regions of ships can be accurately located and extracted from images captured by drones. These image regions are then converted into high-dimensional target feature vectors, providing a solid foundation for accurate ship identification and authentication. This series of processes not only improves the accuracy of ship identification but also provides standardized feature inputs for feature matching and AIS data fusion in subsequent steps, thereby enhancing the overall performance and reliability of the entire ship identification system.

[0077] To improve the accuracy of determining the target identity information of a target vessel, the UAV-based vessel identity recognition method provided in Embodiment 1 of this application determines the similarity between the target feature vector and each feature vector in the preset vessel feature database; it determines whether the maximum similarity exceeds a preset similarity threshold, and if the maximum similarity exceeds the preset similarity threshold, the identity information associated with the feature vector indicated by the maximum similarity is determined as the target identity information.

[0078] In this embodiment of the invention, the extracted feature vector of the ship to be identified can be compared with a pre-built ship feature database (containing standardized feature vectors of known ships and their corresponding identity information, such as ship name, MMSI code, ship type, etc.) using similarity calculations, for example, cosine similarity or Euclidean distance. By setting a similarity threshold (e.g., 0.8), when the feature similarity between the ship to be identified and a record in the database exceeds this threshold, it is identified as the ship, and its identity information is output. For ships with clear textual information such as ship names, an optical character recognition (OCR) module can be integrated for direct ship name recognition, which is then cross-verified with the feature matching results to improve recognition accuracy.

[0079] Specifically, after the feature extraction module extracts the target feature vector from the detected ship target region (image within the bounding box), it can compare this feature vector with each feature vector in a pre-set ship feature database to determine the identity of the target ship. This comparison process involves calculating the similarity between two feature vectors, typically using cosine similarity or Euclidean distance as the metric. Cosine similarity measures the degree of similarity between two vectors by calculating the cosine of the angle between them, without considering their magnitude or amplitude, which is very useful when comparing directional features. Euclidean distance measures the straight-line distance between two vectors in multidimensional space, suitable for evaluating the precise matching degree of features. Therefore, the similarity metric can be selected based on the properties of the feature vectors and the specific application requirements.

[0080] Here, the pre-defined ship feature database contains standardized feature vectors of known ships and their corresponding identity information, including but not limited to ship name, MMSI code, and ship type. During database construction, each ship is photographed from multiple angles and perspectives using high-precision photography or drones, from which unique "ship face" feature vectors are extracted and associated with the ship's identity information. These feature vectors form the search benchmark in the database for subsequent feature matching.

[0081] After calculating similarity with all feature vectors in the database, a series of similarity values ​​are obtained. Then, the maximum similarity value is found, and it is determined whether this value exceeds a preset similarity threshold. This preset similarity threshold is a threshold set based on experimental debugging and experience, used to filter out candidate vectors with low similarity that may be incorrectly matched. If the maximum similarity value exceeds this threshold, then a feature vector in the database that matches the target feature vector has been found. The ship identification information associated with this feature vector (such as ship name, MMSI code, etc.) is then identified as the current target ship's identification information. This threshold setting mechanism effectively avoids misidentification and improves the accuracy and reliability of the identification.

[0082] In this embodiment, the detected ship targets in images captured by the drone can be effectively matched with their true identities. This process not only utilizes the powerful feature capture capabilities of deep learning models but also combines the authority of a pre-set ship feature database. Through scientific similarity measurement and threshold judgment mechanisms, accurate confirmation of ship identities is achieved.

[0083] To further improve the accuracy of identity recognition, in the UAV-based ship identity recognition method provided in Embodiment 1 of this application, after determining the target ship's identity information from a preset ship feature database based on the target feature vector, ship information collected by the automatic ship identification system is obtained. The ship information includes at least: ship identity information corresponding to the ship's location and timestamp; spatiotemporal registration is performed between the image spatiotemporal information and the ship information to obtain target ship identity information matching the collection time and location information; the target ship identity information and the target identity information are compared to obtain a comparison result; and if the comparison result indicates that the target ship identity information and the target identity information are inconsistent, a warning message is triggered.

[0084] In this embodiment of the invention, to improve system reliability, visual recognition results can be fused with Automatic Identification System (AIS) information. Through spatiotemporal registration (converting the pixel coordinates of ships in the image to their approximate geographic coordinates using a georeferenced method), the visually identified candidate ships are compared with AIS message information from the same time and location. If the visual recognition result matches the identity information reported by the AIS, the recognition confidence is significantly improved; if an anomaly is detected, such as missing AIS signals or discrepancies between the visually recognized identity and the AIS report, a subsequent early warning mechanism is triggered.

[0085] In this embodiment of the invention, AIS is a navigation aid system based on VHF (Very High Frequency) wireless communication technology, consisting of a shore-based base station and shipboard equipment. Ships automatically broadcast their static information (such as ship name, MMSI code, ship type, and size) and dynamic information (such as latitude and longitude, speed, and heading) on ​​a specific VHF channel via their shipboard AIS terminal, according to the SOTDMA (Self-Organizing Time Division Multiple Access) protocol. The update frequency can be adjusted according to changes in the ship's status (e.g., up to once every 2 seconds at high speeds). After receiving this information, shore-based base stations or other ships can visualize the dynamics of surrounding vessels on electronic charts or display terminals, enabling collision avoidance and traffic management. Here, the AIS system mainly includes Category A equipment (compliant with IMO (International Maritime Organization) mandatory standards, used for vessels over 300 gross tons) and Category B equipment (lower cost, used for small and medium-sized vessels). Its communication mechanism is based on TDMA (Time Division Multiple Access), dividing channel time into time slots. Vessels occupy time slots according to rules to send data packets, avoiding collisions. The system can access network AIS services or parse data through a local receiver and integrate with a video surveillance system, attempting to project the vessel's latitude and longitude information parsed by AIS onto the video feed via coordinate mapping to achieve information overlay.

[0086] Specifically, vessel information can be obtained first from the Automatic Identification System (AIS). AIS is a system for automatically exchanging information between vessels at sea and between vessels and shore-based facilities, primarily used for vessel tracking, identification, and maritime traffic management. AIS information contains rich vessel status data, including but not limited to the vessel's MMSI code, latitude and longitude position, heading, and speed. This information is continuously broadcast in a standardized format via VHF (Very High Frequency) radio channels, and shore-based receiving stations or shipborne equipment can receive and decode this information. In this embodiment, AIS information is obtained in real time from vessel AIS reports within the UAV's patrol area through a connection with the UAV data processing center.

[0087] Then, each image captured by the drone is accompanied by precise spatiotemporal information, including the capture timestamp and the drone's precise location (latitude and longitude) at the time of capture. To associate AIS information with drone image information, spatiotemporal registration is required. This involves parsing the capture timestamp and location coordinates of the drone image and comparing them with the timestamp and latitude / longitude coordinates in the AIS information to determine which ships reported by the AIS system at a specific time and location match targets detected in the drone image. Based on preset matching criteria (such as tolerance ranges for time and location), the AIS reports most likely corresponding to the same ship can be identified, thus obtaining the target ship's identity information. The comparison process involves comparing the target identity information obtained through feature extraction and matching (i.e., the ship's identity identified from the image) with the target ship identity information obtained through spatiotemporal registration (i.e., the ship's identity in the AIS report). This comparison can be done through simple string matching (when the ship name matches) or more complex feature vector similarity calculation (when there is no direct ship name, feature vector comparison is used to identify whether it is the same ship). The comparison results indicate whether the two sets of identity information are consistent, providing a basis for subsequent early warning decisions.

[0088] If a discrepancy is found between the target vessel's identity information and the target vessel's identity information during the above comparison process, an early warning mechanism will be immediately triggered. The generation of early warning information allows for immediate response to potential maritime security threats, including but not limited to unidentified vessels (not registered in the AIS system or with their AIS signals disabled) and vessels with inconsistent identity information (potentially AIS signal forgery). Furthermore, the target can be automatically highlighted on a visualization platform, and monitoring personnel will be notified via sound alerts, pop-up notifications, etc. If necessary, emergency notifications can also be sent to relevant maritime management departments via SMS or email. The triggering conditions and response procedures of the early warning mechanism can be configured according to specific application scenarios and security requirements, ensuring the system's flexibility and response speed.

[0089] In some optional embodiments, real-time results display and vessel information display can be achieved by constructing a web-based geographic information system visualization platform. The platform interface uses an electronic nautical chart as its base map, displaying the UAV's location, track, and the location icons of all identified vessels in real time. Clicking on a vessel icon will bring up a detailed information window, including: a real-time snapshot of the vessel, the identified vessel name / MMSI code, vessel type, nationality, size, speed, and heading (calculated based on consecutive frames). Identification results and historical tracks can be stored synchronously, supporting rapid retrieval and playback by time, region, and vessel attributes.

[0090] In some alternative embodiments, abnormal vessel warnings can be implemented: the system has a built-in warning rule engine that can automatically detect abnormal behavior and generate warning information based on preset rules. Warning rules include: identifying "unfamiliar" vessels not entered into the database (which may be vessels without AIS or those that have not reported); vessels entering prohibited or restricted waters; abnormal vessel tracks (such as loitering, rapid changes of direction, etc.); and discrepancies between visual identification and AIS reported identity (suspected AIS spoofing). Once a warning is triggered, the system will highlight it on the platform interface (e.g., with a red highlight or flashing icon), issue an audible alarm, and push the warning information to relevant management personnel via SMS, email, etc.

[0091] In other alternative embodiments, trajectory tracking and analysis can be performed: for key monitored vessels, the system supports manual or automatic lock-on tracking. The drone can adjust its course or gimbal angle according to instructions to maintain continuous monitoring of the vessel and record its complete trajectory. Combined with algorithms such as Kalman filtering, short-term predictions of the vessel's future position can be made, providing more comprehensive data support for regulatory decisions. The system can also perform statistical analysis on long-term vessel trajectory data, generating traffic flow heat maps, navigation habit analysis reports, etc., serving macro-level maritime management.

[0092] This embodiment not only improves the accuracy and robustness of vessel identification but also enhances the system's safety early warning function, ensuring the timeliness and effectiveness of maritime supervision. Specifically, the real-time collection and matching of AIS information, combined with image recognition results, achieves dual verification of vessel identity, significantly reducing the false alarm rate of single identification methods. The immediate triggering of the early warning mechanism upon detecting potential anomalies ensures that regulatory personnel receive alerts immediately and take necessary safety measures, providing strong technical support for preventing maritime accidents and combating risky behaviors.

[0093] Figure 2 This is a schematic diagram of an optional ship identification process based on a drone and the YOLOv8 algorithm according to an embodiment of the present invention, as shown below. Figure 2As shown, the identification process is as follows: The acquired AIS data (MMSI, latitude and longitude, ship name) is parsed, and the visual detection result (image pixel coordinate bounding box) is obtained through the YOLOv8 algorithm. Then, coordinate transformation is performed based on the two information, that is, the AIS latitude and longitude are projected onto the image pixels. After that, the association matching judgment is performed. If the identity matching is successful, the AIS identity information is output. If the matching fails, the retrieval based on visual features is triggered, and the final ship identity is output.

[0094] In this embodiment of the invention, a drone and its high-precision stabilization gimbal are used to ensure the clarity and stability of image acquisition, reducing image quality degradation caused by motion blur at the source and providing a high-quality data foundation for subsequent accurate identification. Furthermore, through denoising and enhancement techniques, as well as diverse data augmentation strategies employed during the training phase, the model's adaptability to complex environments such as changes in lighting, water surface reflections, and minor fog is significantly improved, reducing false positives and false negatives. In addition, the effective fusion of visual features and AIS information completes the crucial leap from "detecting a ship" to "identifying the specific ship's identity," improving the accuracy and reliability of identification.

[0095] The following is a detailed description with reference to another embodiment.

[0096] Example 2

[0097] The ship identification device based on unmanned aerial vehicles provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in Embodiment 1 above.

[0098] Figure 3 This is a schematic diagram of an optional unmanned aerial vehicle (UAV)-based ship identification device according to an embodiment of the present invention, such as... Figure 3 As shown, the ship identification device may include: a data acquisition unit 30, a first processing unit 31, a second processing unit 32, and a determination unit 33.

[0099] The acquisition unit 30 is used to acquire images of the target water area using a drone to obtain image data. The image data includes: the image and the corresponding spatiotemporal information of the image. The spatiotemporal information of the image includes at least: acquisition time and location information.

[0100] The first processing unit 31 is used to preprocess the image to obtain the target image;

[0101] The second processing unit 32 is used to process the target image using a preset ship detection model to obtain ship information, wherein the ship information includes at least: the bounding box of the target ship, the bounding box being used to mark the position of the target ship in the target image;

[0102] The determining unit 33 is used to extract features from the target image based on the bounding box to obtain the target feature vector, and to determine the target identity information of the target ship from the preset ship feature database based on the target feature vector. The preset ship feature database includes multiple identity information associated with the feature vector.

[0103] The aforementioned ship identification device utilizes drones to collect image data of the target waters. By integrating high-precision GPS and IMU, it ensures accurate recording of image spatiotemporal information (including acquisition time and location information), achieving the goal of acquiring high-quality ship images with precise spatiotemporal coordinates. This enables rapid ship positioning and identification based on accurate spatiotemporal information. Specifically, image quality is improved through image preprocessing techniques (such as denoising and enhancement). A ship detection model is then used to efficiently analyze the processed target image, obtaining ship detection information (at least including bounding boxes). These bounding boxes accurately pinpoint the ship's position in the image. Based on these bounding boxes, deep learning feature extraction methods are used to obtain target feature vectors that characterize the "ship's face." These vectors are then matched with feature vectors in a pre-set ship feature database to determine the specific identity information of the target ship (such as ship name and MMSI code). This effectively solves the problems of low accuracy and poor robustness in vision-based ship identification technologies, achieving accurate ship identification in complex environments.

[0104] Optionally, the UAV includes at least: a positioning module and an inertial measurement unit. The UAV is connected to the camera via a three-axis stabilization gimbal. The acquisition unit includes: a first cruise module for controlling the UAV to cruise along the inspection route over the target water area; a first shooting module for controlling the camera to capture images at a preset frame rate during the UAV's cruise and recording the acquisition time of the captured images; a first determination module for determining the position information and UAV attitude angle corresponding to the captured image based on the acquisition time, wherein the position information is obtained through the positioning module and the UAV attitude angle is obtained through the inertial measurement unit; and a first construction module for constructing image data based on the image, the acquisition time corresponding to the image, the position information, and the UAV attitude angle.

[0105] Optionally, the first processing unit includes: a first noise reduction module for performing noise reduction processing on the image to obtain a noise-reduced image; a first enhancement module for performing contrast enhancement processing on the noise-reduced image to obtain a first enhanced image; a second enhancement module for performing image enhancement processing on the first enhanced image to obtain a second enhanced image; and a first conversion module for performing color space conversion on the second enhanced image and cropping the converted second enhanced image to obtain a target image.

[0106] Optionally, the ship identification device further includes: a first acquisition module, used to acquire historical images before processing the target image using a preset ship detection model to obtain ship information, and to perform enhancement processing on each historical image, wherein the enhancement processing includes at least one of the following: flipping processing, rotation processing, brightness adjustment processing, contrast adjustment processing, noise addition processing, and simulated weather processing; a second construction module, used to construct a training image set based on all historical images and the enhanced historical images; a first annotation module, used to annotate each training image in the training image set to obtain annotation information, wherein the annotation information includes: the labeled bounding box of the ship on the training image; and a third construction module, used to construct an initial ship detection model, wherein the model structure of the initial ship detection model includes at least: a pyramid network structure and a path aggregation network. The system comprises the following structures: a pyramid network structure for extracting features from training images at different scales to obtain a first multi-layer feature map, which is then fused from top to bottom to obtain a first fused feature map; a path aggregation network structure for extracting features from training images at different scales to obtain a second multi-layer feature map, which is then fused from bottom to top to obtain a second fused feature map; a detection structure for detecting the first and second fused feature maps to obtain predicted bounding boxes for ships on each training image; and a first training module for training an initial ship detection model using the training image set and the annotation information of each training image until the loss value determined by the loss function is less than a preset loss threshold, thus obtaining a preset ship detection model. The loss value is determined by the loss function based on the labeled bounding boxes and the predicted bounding boxes.

[0107] Optionally, the determining unit includes: a first extraction module for extracting the ship target region from the target image based on the bounding box; and a second extraction module for extracting features from the ship target region to obtain a target feature vector.

[0108] Optionally, the determining unit further includes: a second determining module, used to determine the similarity between the target feature vector and each feature vector in the preset ship feature database; and a first judging module, used to judge whether the maximum similarity exceeds a preset similarity threshold, and if the maximum similarity exceeds the preset similarity threshold, to determine the identity information associated with the feature vector indicated by the maximum similarity as the target identity information.

[0109] Optionally, the ship identification device further includes: a first acquisition module, used to acquire ship information collected by the automatic ship identification system after determining the target ship's identity information from a preset ship feature database based on the target feature vector, wherein the ship information includes at least: ship identity information corresponding to the ship's position and timestamp; a first registration module, used to perform spatiotemporal registration with the ship information based on image spatiotemporal information to obtain target ship identity information matching the acquisition time and location information; a first comparison module, used to compare the target ship identity information and the target identity information to obtain a comparison result; and a first trigger module, used to trigger a warning message when the comparison result indicates that the target ship identity information and the target identity information are inconsistent.

[0110] The aforementioned ship identification device may also include a processor and a memory. The aforementioned acquisition unit 30, first processing unit 31, second processing unit 32, determination unit 33, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0111] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured. By adjusting kernel parameters, features are extracted from the target image based on bounding boxes to obtain target feature vectors. Based on these target feature vectors, the target vessel's identity information is determined from a pre-defined vessel feature database.

[0112] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0113] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: using a drone to collect images of a target water area to obtain image data, preprocessing the images to obtain a target image, processing the target image using a preset ship detection model to obtain ship information, extracting features from the target image based on bounding boxes to obtain a target feature vector, and determining the target identity information of the target ship from a preset ship feature database based on the target feature vector.

[0114] According to another aspect of the present invention, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for ship identification based on unmanned aerial vehicles.

[0115] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the above-described UAV-based ship identification method.

[0116] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) for a ship identification method based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more processors (e.g., Figure 4 The processors 402a, 402b, ..., 402n, etc., may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), and a memory 404 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0117] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0118] The embodiments or examples disclosed herein are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0119] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] In the several embodiments provided by this invention, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection can be through some interfaces; the indirect coupling or communication connection of units or modules can be electrical or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0124] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for ship identification based on unmanned aerial vehicles (UAVs), characterized in that, include: Images of the target water area are collected using a drone to obtain image data, wherein the image data includes: the image and the corresponding spatiotemporal information of the image, and the spatiotemporal information of the image includes at least: collection time and location information; The image is preprocessed to obtain the target image; The target image is processed using a preset ship detection model to obtain ship information, wherein the ship information includes at least: the bounding box of the target ship, the bounding box being used to mark the position of the target ship in the target image; Based on the bounding box, feature extraction is performed on the target image to obtain a target feature vector. Based on the target feature vector, the target identity information of the target vessel is determined from a preset vessel feature database. The preset vessel feature database includes multiple identity information associated with the feature vector.

2. The ship identification method according to claim 1, characterized in that, The drone includes at least a positioning module and an inertial measurement unit. The drone is connected to a camera via a three-axis stabilization gimbal. The steps for acquiring image data by using the drone to collect images of the target water area include: Control the drone to cruise along the inspection route over the target waters; During the drone's cruise, the camera is controlled to capture images at a preset frame rate, and the acquisition time of the images captured by the camera is recorded. Based on the acquisition time, the location information and UAV attitude angle corresponding to the image when the image was captured are determined, wherein the location information is obtained through the positioning module and the UAV attitude angle is obtained through the inertial measurement unit; Based on the image, the image data is constructed using the acquisition time, location information, and UAV attitude angle corresponding to the image.

3. The ship identification method according to claim 1, characterized in that, The step of preprocessing the image to obtain the target image includes: The image is then subjected to noise reduction processing to obtain a denoised image; The denoised image is subjected to contrast enhancement processing to obtain a first enhanced image; The first enhanced image is subjected to image enhancement processing to obtain the second enhanced image; The second enhanced image is subjected to color space conversion, and the converted second enhanced image is cropped to obtain the target image.

4. The ship identification method according to claim 1, characterized in that, Before processing the target image using a preset ship detection model to obtain ship information, the process also includes: Historical images are acquired, and each of the historical images is enhanced, wherein the enhancement processing includes at least one of the following: flipping processing, rotation processing, brightness adjustment processing, contrast adjustment processing, noise addition processing, and weather simulation processing; A training image set is constructed based on all the historical images and the enhanced historical images; Each training image in the training image set is labeled to obtain labeling information, wherein the labeling information includes: the labeled bounding box of the ship on the training image; An initial ship detection model is constructed, wherein the model structure of the initial ship detection model includes at least: a pyramid network structure, a path aggregation network structure, and a detection structure; the pyramid network structure is used to extract features from the training images at different scales to obtain a first multi-layer feature map, and then fuses the first multi-layer feature map from top to bottom to obtain a first fused feature map; the path aggregation network structure is used to extract features from the training images at different scales to obtain a second multi-layer feature map, and then fuses the second multi-layer feature map from bottom to top to obtain a second fused feature map; the detection structure is used to detect the first fused feature map and the second fused feature map to obtain the predicted bounding box of the ship on each training image; The initial ship detection model is trained using the training image set and the annotation information of each training image until the loss value determined by the loss function is less than a preset loss threshold, thus obtaining the preset ship detection model. The loss value is determined by the loss function based on the labeled bounding box and the predicted bounding box.

5. The ship identification method according to claim 1, characterized in that, The step of extracting features from the target image based on the bounding box to obtain the target feature vector includes: Based on the bounding box, extract the ship target region from the target image; Feature extraction is performed on the target region of the ship to obtain the target feature vector.

6. The ship identification method according to claim 1, characterized in that, The step of determining the target identity information of the target vessel from a preset vessel feature database based on the target feature vector includes: Determine the similarity between the target feature vector and each feature vector in the preset ship feature database; Determine whether the maximum similarity exceeds a preset similarity threshold, and if the maximum similarity exceeds the preset similarity threshold, determine the identity information associated with the feature vector indicated by the maximum similarity as the target identity information.

7. The ship identification method according to claim 1, characterized in that, After determining the target identity information of the target vessel from a preset vessel feature database based on the target feature vector, the process further includes: The vessel information collected by the Automatic Identification System (AIS) includes at least: vessel identity information corresponding to the vessel's location and timestamp; Based on the spatiotemporal information of the image, spatiotemporal registration is performed with the ship information to obtain the target ship identity information that matches the acquisition time and the location information; The target vessel identity information and the target identity information are compared to obtain a comparison result; If the comparison result indicates that the target vessel's identity information is inconsistent with the target's identity information, an early warning message is triggered.

8. A ship identification device based on unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition unit is used to acquire images of a target water area using a drone to obtain image data. The image data includes the image and the corresponding spatiotemporal information of the image. The spatiotemporal information of the image includes at least the acquisition time and location information. The first processing unit is used to preprocess the image to obtain the target image; The second processing unit is used to process the target image using a preset ship detection model to obtain ship information, wherein the ship information includes at least: the bounding box of the target ship, the bounding box being used to mark the position of the target ship in the target image; The determining unit is configured to extract features from the target image based on the bounding box to obtain a target feature vector, and determine the target identity information of the target vessel from a preset vessel feature database based on the target feature vector, wherein the preset vessel feature database includes multiple identity information associated with the feature vector.

9. A computer program product, characterized in that, The method includes a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the unmanned aerial vehicle-based ship identification method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the UAV-based ship identification method according to any one of claims 1 to 7.

Images