Ship target detection method and system based on unmanned aerial vehicle low-altitude remote sensing image

Abstract

The application provides a ship target detection method and system based on unmanned aerial vehicle low-altitude remote sensing images, wherein the method comprises: acquiring a to-be-detected ship image of a target marine area; inputting the to-be-detected ship image into a ship target detection model to perform ship target detection and obtain a ship detection result image of the target marine area; wherein the ship target detection model is obtained by improving an RT-DETR detection model as a baseline network; the improvement at least comprises: setting a multi-scale bar convolution module in a backbone network of the baseline network and setting a state space evolution module in a neck network of the baseline network; and the ship detection result image is marked with a plurality of ship target position identifiers. Thus, by applying the technical solution of the application, the detection efficiency and accuracy of ship targets in complex sea conditions can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of geographic information technology, and in particular to a method and system for detecting ship targets based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of the global marine economy, drone aerial photography has become an important auxiliary means for the automated detection of ship targets and is widely used in maritime supervision and waterway safety detection. However, the marine environment is complex, often with strong interference backgrounds such as waves, reflections, and sea fog, which severely obscure the visual features of ship targets. At the same time, ship targets in aerial images vary greatly in scale and often exhibit significant aspect ratios, posing a serious challenge to feature extraction and accurate localization in the target detection process.

[0003] Currently, deep learning-based target detection models (such as RT-DETR) perform well in general scenarios, but they still have significant limitations when facing the complex sea conditions mentioned above. On the one hand, the standard convolutional kernels used in their backbone networks are difficult to effectively capture the slender geometric features of ship targets, have insufficient representation capabilities for irregularly shaped targets, and are easily affected by cluttered sea surface ripples. On the other hand, in the feature fusion stage, the model has limited global contextual awareness of distant small targets or occluded targets, making it difficult to accurately distinguish small ships from similar noise (such as waves) in complex backgrounds, leading to frequent missed detections and false detections.

[0004] Therefore, the existing technical solutions cannot meet the actual application requirements in terms of detection accuracy and robustness when conducting actual ship target detection under complex sea conditions. There is an urgent need for a ship target detection method that can be used in complex marine scenarios to effectively improve the detection efficiency and accuracy of ship targets. Summary of the Invention

[0005] Based on this, the purpose of this application is to provide a method, system, readable storage medium, and computer equipment for ship target detection based on UAV low-altitude remote sensing imagery, which can effectively improve the detection efficiency and accuracy of ship targets under complex sea conditions.

[0006] The objective of this application can be achieved through the following technical solutions:

[0007] A ship target detection method based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs) includes the following steps: acquiring images of ships to be detected in a target ocean area; inputting the images of ships to be detected into a ship target detection model to perform ship target detection and obtain a ship detection result image of the target ocean area; wherein, the ship target detection model is obtained by improving the RT-DETR detection model as a baseline network; the improvement includes at least: setting a multi-scale strip convolution module in the backbone network of the baseline network and setting a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship target; the state space evolution module is used to provide global context awareness of the ship targets in the target ocean area; the ship detection result image is marked with several ship target location identifiers.

[0008] A ship target detection system based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs) includes: an image acquisition unit for acquiring images of ships to be detected in a target ocean area; and a detection execution unit for inputting the images of the ships to be detected into a ship target detection model to perform ship target detection and obtain a ship detection result image of the target ocean area. The ship target detection model is an improved version of the RT-DETR detection model as a baseline network. The improvement includes at least: setting a multi-scale strip convolution module in the backbone network of the baseline network and a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship targets; the state space evolution module is used to provide global context awareness of the ship targets in the target ocean area; and the ship detection result image is marked with several ship target location identifiers.

[0009] A computer device includes a processor and a memory; the memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, it implements the steps in the above-described method for ship target detection based on UAV low-altitude remote sensing imagery.

[0010] A computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the above-described method for ship target detection based on UAV low-altitude remote sensing imagery.

[0011] Compared to existing technologies, the method described in this application incorporates a multi-scale strip convolution module in the backbone network of the ship target detection model. This module, through a parallel structure including strip convolution branches, enhances the feature extraction capability for ship targets with large aspect ratios and irregular geometric shapes, effectively suppressing interference from sea surface background noise. Simultaneously, a state space evolution module is incorporated into the feature pyramid of the neck network. This module, through discretized calculation of the state space equation, strengthens the global context awareness and long-distance dependency modeling capability for ship targets (especially small and disturbed targets) in complex sea conditions. Therefore, the method described in this application, through the aforementioned targeted structural improvements at the two key levels of feature extraction and feature fusion, enables the ship target detection model to more accurately identify and locate ship targets from aerial images with complex backgrounds, thereby significantly improving the overall accuracy and robustness of ship target detection in complex marine scenarios.

[0012] To better understand and implement this application, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0013] Figure 1 A flowchart of a ship target detection method based on UAV low-altitude remote sensing imagery provided for this application; Figure 2 A flowchart illustrating the steps for acquiring images of the ship to be detected in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 3 A schematic diagram of the overall network structure of the ship target detection model in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 4 A schematic diagram of the network structure of the multi-scale strip convolutional module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 5 A flowchart illustrating the working steps of the multi-scale strip convolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 6 A schematic diagram of the network structure of the state space evolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 7 A flowchart illustrating the working steps of the state space evolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application; Figure 8 A structural principle diagram of a ship target detection system based on low-altitude remote sensing imagery from an unmanned aerial vehicle (UAV) provided in this application; Figure 9A schematic diagram of a computer device for implementing the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application. Detailed Implementation

[0014] This application provides a method and system for ship target detection based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs). To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0015] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0016] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0017] The invention will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0018] Example 1 Please refer to Figure 1 , Figure 1 A flowchart illustrating a ship target detection method based on low-altitude remote sensing imagery from an unmanned aerial vehicle (UAV) is provided for this application. The method includes the following steps: S10: Acquire images of the vessels to be detected in the target ocean area; S20: Input the image of the ship to be detected into the ship target detection model to perform ship target detection and obtain the ship detection result image of the target ocean area.

[0019] Compared to existing technologies, the technical solution of this application addresses the shortcomings of the RT-DETR model in extracting sufficient geometric features of ship targets and its insufficient ability to perceive background interference and small target context when applied to UAV low-altitude remote sensing images in complex marine scenes. Two core and synergistic improvements are made at the model architecture level of the ship target detection model. Specifically, in the backbone network, this application introduces a multi-scale strip convolution module, which segments the input features and performs 3×3 standard convolution, 1×11 horizontal strip convolution, and 11×1 vertical strip convolution through multiple parallel branches. The identity mapping process effectively enhances the ability to extract features from the slender geometric shapes of ships, such as those with large aspect ratios and directional characteristics, while suppressing interference from irrelevant sea surface textures (ripples). In the neck network, this application embeds a state space evolution module in the path of the feature pyramid processing the highest resolution feature map. This module recursively calculates the corresponding feature sequences through discretized state space equations, achieving efficient modeling of global context information. This significantly strengthens the ship target detection model's ability to identify and associate small, blurred, or partially occluded ship targets under complex sea conditions such as waves and reflections.

[0020] Therefore, the technical solution of this application, through the progressive model architecture optimization of enhancing geometric perception capability in the feature extraction stage and enhancing global context modeling capability in the feature fusion stage, enables the improved ship target detection model to deeply adapt to the unique challenges of the marine scene, thereby significantly improving the detection accuracy and robustness of multi-scale and multi-form ship targets in UAV low-altitude remote sensing images, and effectively overcoming the problem of limited performance of existing general detection models in the specific scenario of complex ocean.

[0021] For step S10: Obtain images of the ships to be detected in the target ocean area.

[0022] The target ocean area is a specific sea area where ship target monitoring and identification are required; the ship image to be detected is a digital image acquired by an image acquisition device that contains potential ship targets within the target ocean area.

[0023] In one embodiment, step S10 can be performed by a drone platform equipped with an image acquisition device flying over the target ocean area to carry out aerial photography to collect raw image data, and then obtaining images of the ship to be detected that conform to the input specifications of the ship target detection model through a series of preset image preprocessing operations.

[0024] Please refer to Figure 2 , Figure 2 A flowchart illustrating the steps involved in acquiring an image of the ship to be detected in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application.

[0025] In one embodiment, step S10 includes: S101: Acquire aerial video streams or aerial image sequences of the target ocean area using image sensors mounted on the UAV.

[0026] The image sensor carried by the UAV is a charge-coupled device or complementary metal-oxide-semiconductor camera used to capture images in the visible light or infrared bands; the aerial video stream is dynamic image data composed of frames continuously acquired by the image sensor and arranged in time sequence; the aerial image sequence is a collection of multiple static images continuously acquired by the image sensor within a specific time interval.

[0027] In an optional embodiment, the image sensor is a visible light camera with high dynamic range imaging capabilities to adapt to high-contrast lighting environments where strong reflections and shadows coexist on the sea surface.

[0028] S102: Extract the original aerial images from the aerial video stream or the aerial image sequence, and perform dehazing, deblurring, and dynamic range enhancement processing on the original aerial images to obtain a quality-enhanced image.

[0029] In one embodiment, the original aerial image is a single frame extracted from the aerial video stream at a preset frame rate, or a single image selected from the aerial image sequence; the dehazing process uses an algorithm based on dark channel prior or a preset physical model to process the original aerial image to reduce the attenuation effect of sea fog and haze on image visibility; the deblurring process uses an algorithm based on Wiener filtering or blind deconvolution to process the original aerial image to compensate for image blur caused by camera shake or target movement; the dynamic range enhancement process uses an algorithm based on Retinex theory or histogram equalization to process the original aerial image to simultaneously suppress overexposure in the highlight areas of the sea surface and brighten the details of ships in the shadows; the quality-enhanced image is an intermediate image whose visual quality is improved after the original aerial image has undergone the above series of image restoration and enhancement operations.

[0030] In an optional embodiment, based on the inventive concept of this application, the defogging process may employ a dark channel prior algorithm based on an atmospheric scattering physics model, and the dynamic range enhancement process may employ a multi-scale Retinex algorithm.

[0031] S103: According to the input requirements of the ship target detection model, the quality enhancement image is scaled and pixel value normalized to obtain the ship image to be detected.

[0032] In one embodiment, the input requirements for the ship target detection model are a predefined fixed spatial size (e.g., 640x640 pixels) and a pixel value range (e.g., [0,1] interval); the scaling transformation is performed by using bilinear interpolation or bicubic interpolation algorithms to scale or crop the quality-enhanced image to conform to the fixed spatial size; the pixel value normalization process is performed by linearly mapping the pixel values ​​of the quality-enhanced image from the original range (e.g., 0-255) to the target value range; the ship image to be detected is the final preprocessed image after the quality-enhanced image has been standardized in terms of scale and value range, which can be directly input into the ship target detection model for inference.

[0033] In an optional embodiment, based on the inventive concept of this application, the scaling transformation may be to uniformly scale the quality-enhanced image to 640x640 pixels using a bilinear interpolation algorithm, and the pixel value normalization processing may be to map the corresponding pixel value from the [0,255] interval to the [0,1] interval.

[0034] In one specific embodiment, step S10 is implemented according to the following process: First, the Zenmuse H20T visible light camera mounted on the DJI Matrice 300 RTK drone flies to the target sea area and acquires an aerial video stream at a rate of 30 frames per second; then, the original aerial image is extracted from the video stream at a rate of 1 frame per second; next, the extracted image is sequentially subjected to dehazing processing based on dark channel prior, deblurring processing based on Wiener filtering, and dynamic range enhancement processing based on multi-scale Retinex to obtain a quality-enhanced image; finally, the quality-enhanced image is scaled to 640x640 pixels using bilinear interpolation, and the pixel values ​​are normalized to the [0,1] interval to obtain the image of the ship to be detected.

[0035] For step S20: Input the image of the ship to be detected into the ship target detection model to perform ship target detection and obtain the ship detection result image of the target ocean area.

[0036] The ship target detection model is a neural network model based on a deep learning architecture designed to identify and locate ship targets from complex marine scene images. The model is an improvement upon the RT-DETR detection model as a baseline network. The improvement includes at least: setting a multi-scale strip convolution module in the backbone network of the baseline network and a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship targets (specifically, it can adaptively extract the geometric shape features of ship targets with different aspect ratios and orientations from the input image features); the state space evolution module is used to provide global context awareness of ship targets in the target marine area (i.e., by modeling long-distance dependencies in the input feature map, it provides stronger global context awareness for ship targets (especially small targets and partially occluded targets) in the target marine area); the ship detection result image is marked with several ship target location identifiers.

[0037] In one embodiment, the ship detection result image is a visual output image that clearly marks the position, category, and confidence level of each ship target identified by the ship target detection model on the original image of the ship to be detected, using rectangular boxes, category labels, and confidence scores.

[0038] In one embodiment, step S20 involves inputting the preprocessed image of the ship to be detected into the ship target detection model for inference, and then overlaying the detection results (such as target location and category) output by the ship target detection model onto the image of the ship to be detected in a visually recognizable graphic annotation form (such as a rectangle) to generate the ship detection result image. Therefore, in step S20, the multi-scale strip convolution module in the ship target detection model enhances the robust extraction capability of ship geometric features; simultaneously, the state space evolution module effectively suppresses background interference from complex sea conditions. These two elements work synergistically to achieve more accurate and robust automated detection of ship targets in UAV low-altitude remote sensing imagery.

[0039] In one embodiment, please refer to Figure 3 , Figure 3 This is a schematic diagram of the overall network structure of the ship target detection model in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application. The ship target detection model includes a backbone network, a neck network, and a detection decoding head connected in sequence.

[0040] Based on this, in one specific embodiment, the backbone network is responsible for performing hierarchical feature extraction on the input image of the ship to be detected. It includes several basic convolutional modules and at least one pooling layer set in the front stage of the backbone network for performing preliminary feature mapping and spatial downsampling; and several multi-scale strip convolutional modules (InceptionNeXT Block) set in the back stage of the backbone network for specifically enhancing the feature extraction capability for the slender and irregular geometric shape features of the ship target at a deeper network level.

[0041] The neck network is an encoder with a feature pyramid structure. Its input is connected to the output of the backbone network, and it is used to fuse and enhance the multi-scale features output by the backbone network to construct a feature representation rich in semantic and detailed information. The neck network includes: several 1×1 convolutional layers (which can be used to adjust and compress the feature channel dimension), several upsampling layers (which can be used to improve the spatial resolution of deep semantic features), several reparameterized convolutional modules (which can be used to enhance the representational ability of features without increasing inference time), and the state space evolution module (SS). M); the convolutional kernel is a 1×1 convolutional layer, an upsampling layer, and a reparameterized convolutional module connected together to form several fusion paths for cross-scale feature transfer and fusion; the fusion paths are fused through splicing operations to achieve information complementarity between high and low level features; the state space evolution module is set on the fusion path for receiving and processing the highest resolution feature map from the backbone network (corresponding to the shallow high-resolution features in the feature pyramid, i.e., the P2 layer), and performs global context modeling on the features on the fusion path to improve the perception ability of small targets in complex scenes.

[0042] In one embodiment, the 1×1 convolutional kernel, the upsampling layer, and the reparameterized convolutional module are interconnected by a connection method that includes top-down and bottom-up pathways (i.e., the structure of the corresponding feature pyramid FPN / PANet).

[0043] The input end of the detection decoding head is connected to the output end of the neck network to receive fused and enhanced multi-scale features and output the corresponding ship detection result image. In some embodiments, the detection decoding head includes a Transformer-based decoder structure, which corresponds to the encoder structure of the neck network. It interacts with the encoded features through learnable query vectors to decode the position offset and category probability distribution of each potential ship target, and finally aggregates and outputs the corresponding ship detection result image.

[0044] Please refer to this simultaneously. Figure 4 and Figure 5 ,Figure 4 This is a schematic diagram of the network structure of the multi-scale strip convolutional module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application. Figure 5 The flowchart illustrates the working steps of the multi-scale strip convolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application.

[0045] In one embodiment, the multi-scale strip convolution module employs a structure containing four parallel branches; the multi-scale strip convolution module processes the input first feature map in the following manner. X Processing is performed to output a second feature map. Y : S201: Transfer the first feature map X The feature set is divided into four feature subsets along the channel dimension, denoted as the first feature subset. X 1. Second feature subset X 2. Third feature subset X 3 and the fourth feature subset X 4.

[0046] In one specific embodiment, the segmentation can be an equal segmentation.

[0047] S202: For the first feature subset X 1. Perform a depthwise convolution transformation with a 3×3 kernel to obtain the first intermediate feature. X 1 ’ : .

[0048] S203: For the second feature subset X 2. Perform a depthwise convolution transformation with a kernel of 1×11 to obtain the second intermediate feature. X 2 ’ : .

[0049] S204: For the third feature subset X 3. Perform a depthwise convolution transformation with an 11×1 kernel to obtain the third intermediate feature. X 3 ’ : .

[0050] S205: For the fourth feature subset X 4. Perform identity mapping to obtain the fourth intermediate feature. X 4 ’ : .

[0051] S206: Transfer the first intermediate feature X 1 ’ The second intermediate feature X 2 ’ The third intermediate feature X 3 ’ and the fourth intermediate feature X 4 ’ By concatenating the data along the channel dimension, the corresponding preliminary fusion features are obtained. X c : .

[0052] S207: Regarding the preliminary fusion features X c Perform layer normalization to obtain the corresponding normalized features. X norm : ; Among them, LayerNorm() is a preset layer normalization function.

[0053] S208: Normalize the features X norm The input is fed into a preset multilayer perceptron for cross-channel information interaction and nonlinear transformation to obtain information interaction features. X mid : ; Wherein, MLP(·) is the mapping processing function corresponding to the preset multilayer perceptron, which is used to realize cross-channel information interaction and nonlinear transformation.

[0054] In one specific embodiment, the multilayer perceptron is a feedforward neural network module for feature transformation and cross-channel information interaction. It receives an input feature tensor and processes it through at least one learnable linear transformation layer (also known as a fully connected layer) and a nonlinear activation function, ultimately outputting a feature tensor. Specifically, in an optional embodiment, the structure of the multilayer perceptron includes a first linear layer, a first nonlinear activation function, and a second linear layer, which can be represented as follows: , in, W 1 and b 1 represents the weights and bias parameters of the first linear layer, and σ(·) represents the corresponding nonlinear activation function (such as GELU, SiLU, or ReLU function). W 2 and b 2 represents the weights and bias parameters of the second linear layer.

[0055] S209: The information interaction features are... X mid With the first feature map X The second feature map is obtained by adding elements one by one. Y : .

[0056] Please refer to this simultaneously. Figure 6 and Figure 7 , Figure 6 This is a schematic diagram of the network structure of the state-space evolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application. Figure 7 A flowchart illustrating the working steps of the state-space evolution module in the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application.

[0057] In one embodiment, the state space evolution module processes the input third feature map in the following manner. X r Processing is performed to output the fourth feature map. Y r : S210: Input the third feature map X r The same left and right features are replicated to form two parallel branches.

[0058] S211: For the left-side features, perform a preset linear mapping, a 3×3 depthwise convolution transformation, and a preset activation function processing sequentially to generate left-side gated features. X left : .

[0059] S212: For the right-side features, perform a preset linear mapping, a 3×3 depthwise convolution transformation, and a preset activation function processing sequentially to generate right-side gated features. X right : .

[0060] S213: The left-side gating feature... X left Expanding in the spatial dimension yields the corresponding feature sequence { x 1, x 2, ..., x L}, where L is the total number of feature vectors in the feature sequence.

[0061] S214: Input the feature sequence into a preset state space model, and perform recursive calculations based on the discretized state space equations corresponding to the state space model to obtain the output feature sequence { y 1, y 2, ..., y L}: ; in, x t For the first in the feature sequence t Feature vectors at each position and This is the system matrix obtained by discretizing the preset continuous system parameters. C and D For a preset, learnable projection matrix, h t For the corresponding to the first t The hidden state vector at each position, y t For the corresponding to the first t Output feature vectors at each position.

[0062] S215: Based on the left-side gate control feature X left The spatial dimensions are determined by recombining the output feature sequences to obtain state space features with the same spatial dimensions. X ssm .

[0063] S216: Transfer the state space features X ssm With the right-side gating feature X right The splicing is performed along the channel dimension to obtain the splicing features. X concat : .

[0064] In one embodiment, the splicing can be element-by-element splicing.

[0065] S217: The splicing feature is... X concat The fourth feature map is obtained by projecting the output linear layer through a preset method to transform its channel count to a preset expected output channel count. Y r : .

[0066] In one embodiment, step S20 includes: S21: The ship target detection model processes the image of the ship to be detected and outputs prediction information corresponding to potential ship targets in the image of the ship to be detected.

[0067] The prediction information constitutes a set of structured data, where each data point corresponds to a potential ship target identified by the ship target detection model; the prediction information includes the position coordinates and category confidence of each potential ship target.

[0068] In one embodiment, the position coordinates are used to define the spatial location and extent of the potential ship target in the image of the ship to be detected, and are typically expressed as the corner coordinates of the bounding box (such as the x, y coordinates of the top left and bottom right corners) or the center point coordinates combined with the width and height (x, y). center y center (width, height); The category confidence score is the probability score by which the ship target detection model judges that the potential ship target belongs to the specific category of "ship". Its value is between 0 and 1. The higher the value, the higher the confidence of the ship target detection model in the judgment.

[0069] S22: Based on a preset confidence threshold, perform threshold filtering on the prediction information to remove prediction information whose category confidence is lower than the confidence threshold.

[0070] In one embodiment, the preset confidence threshold is a scalar parameter configurable by those skilled in the art, used to control the stringency of ship detection results, and its value range can be set between 0.5 and 0.9; the threshold filtering is a post-processing operation that compares the category confidence of each prediction with the preset confidence threshold, and retains only predictions with a category confidence not lower than the confidence threshold, thereby initially filtering out low-quality and unreliable false detection results.

[0071] S23: Determine the corresponding candidate prediction box based on the position coordinates of the filtered prediction information.

[0072] The candidate prediction box is a rectangular region on the two-dimensional plane of the image of the ship to be detected, uniquely determined by the position coordinates.

[0073] S24: Using a non-maximum suppression algorithm, calculate the overlap of each candidate prediction box, and eliminate redundant candidate prediction boxes based on the overlap to obtain the ship target detection box.

[0074] In one embodiment, the non-maximum suppression algorithm is a standard post-processing algorithm in the field of computer vision used to eliminate multiple duplicate detection boxes around the same target; the overlap can be measured by the intersection-union ratio, which is calculated as the ratio of the intersection area to the union area of ​​two candidate prediction boxes; the redundancy elimination operation is as follows: retain the candidate prediction box with the highest class confidence and calculate its overlap with all remaining candidate prediction boxes, suppress (i.e. delete) other candidate prediction boxes whose overlap with the candidate prediction box with the highest class confidence exceeds a preset overlap threshold, and iterate this process until all candidate prediction boxes are processed; the ship target detection box is the final set of prediction boxes that are retained after processing by this algorithm and have no high overlap with each other, and each prediction box corresponds to a uniquely confirmed ship target.

[0075] In an optional embodiment, the preset overlap threshold can be set to 0.5.

[0076] S25: The ship target detection box is superimposed on the image of the ship to be detected to obtain the ship detection result image marked with the ship target location identifier.

[0077] The overlay rendering is an image compositing operation that overlays the geometry, preset color and line width, and optional category labels and confidence score text of the ship target detection bounding box onto the corresponding positions of the original image of the ship to be detected through graphic rendering. The ship target location marker refers to the ship target detection bounding box drawn on the original image of the ship to be detected. The ship detection result image is the final output image containing all the ship target location markers, which can be directly viewed and analyzed by the user.

[0078] In one specific embodiment, step S20 can be implemented as follows: First, the ship target detection model performs inference on a 640x640 pixel image of the ship to be detected, and outputs a set of prediction information containing 50 potential ship targets. Each prediction information includes bounding box coordinates (x, y, y). center y centerThe image is first defined as follows: The bounding box coordinates (width, height) and a class confidence score (e.g., 0.92) are used. Then, a preset confidence threshold of 0.6 is set, and 28 predictions with a class confidence score higher than 0.6 are retained after filtering. Next, based on the bounding box coordinates in these 28 predictions, 28 candidate rectangular prediction boxes are generated on the image of the ship to be detected. Then, a non-maximum suppression algorithm is used, with an overlap threshold of 0.5, to process these candidate prediction boxes, eliminating duplicate boxes for the same ship, resulting in 12 independent ship target detection boxes. Finally, using OpenCV library drawing functions, these 12 ship target detection boxes and their class confidence scores are drawn onto the original image of the ship to be detected in the form of green rectangles and white text, and saved as the final ship detection result image.

[0079] In addition, this application also provides some ship target detection steps that can be used in the ship target detection method based on UAV low-altitude remote sensing imagery described in this application, for performance verification of the ship target detection model. Specifically, before inputting the image of the ship to be detected into the ship target detection model, a step of performance testing of the ship target detection model is included, the performance testing step including: S11: Calculate the performance evaluation parameters of the ship target detection model using a preset set of ship target test images.

[0080] The ship target test set images are a collection of images with labeled real ship locations, independent of the ship target detection model's training and validation sets. These images are used to objectively evaluate the ship target detection model's final generalization performance on unseen data. The performance evaluation parameters are indicators used to quantify the ship target detection model's detection capability, including at least the model's precision, recall, and mean precision. Precision measures the proportion of ships correctly predicted by the model; recall measures the proportion of all real ship targets correctly predicted by the model; and mean precision is an indicator that comprehensively considers the detection accuracy of the ship target detection model under different intersection-union thresholds, reflecting the model's overall detection performance.

[0081] S12: Compare each of the calculated performance evaluation parameters with the corresponding preset performance thresholds. When each of the performance evaluation parameters reaches or exceeds its corresponding preset performance threshold, it is determined that the ship target detection model has passed the performance detection.

[0082] The preset performance thresholds are the pass / fail lines for each indicator pre-set based on the actual application scenario's requirements for detection accuracy, coverage, and reliability. For example, an accuracy threshold P can be set. th Recall threshold R thand mean accuracy threshold mAP th .

[0083] In one specific embodiment, the performance testing step is implemented according to the following process: First, the weight file of the trained and saved ship target detection model is loaded; then, the images in the ship target test set are sequentially input into the ship target detection model to obtain the prediction results for each image, including the bounding box coordinates, class confidence, and class label; next, using a standard evaluation tool (e.g., the pycocotools library based on the COCO evaluation format), all prediction results are compared with the ground truth annotation files corresponding to the ship target test set images, and the precision, recall, and mean precision (e.g., mAP50 and mAP50-95) are automatically calculated. Specific numerical values; in step S12, the preset performance thresholds can be set according to historical performance benchmarks or actual business needs. For example, the precision threshold can be set to be no less than 0.90, the recall threshold to be no less than 0.85, and the mAP50-95 threshold to be no less than 0.80. Meanwhile, in step S12, if the calculated precision is ≥0.90, the recall is ≥0.85, and the mAP50-95 is ≥0.80, then the ship target detection model is determined to have passed the performance test, its performance meets the deployment requirements, and it can be used for subsequent actual ship target detection tasks. If any indicator fails to reach the corresponding threshold, then the model is determined to have failed the test, and it is necessary to return to check the quality of the training data, adjust the model hyperparameters, or retrain the model.

[0084] It should be noted that the above embodiments and accompanying drawings are only for more clearly illustrating the specific implementation methods of this application, and are not intended to limit the scope of protection of this application. Those skilled in the art should understand that the core inventive point of the ship target detection method based on UAV low-altitude remote sensing imagery provided in this application lies in two key improvements to the baseline RT-DETR model: introducing a multi-scale strip convolution module into the backbone network to enhance the targeted extraction capability for ship targets with large aspect ratios and irregular geometric shapes; and integrating a state space evolution module into the feature pyramid of the neck network to strengthen the global context awareness and long-distance dependency modeling capability for ship targets (especially small targets and disturbed targets) under complex sea conditions. Through the synergistic effect of these two modules, optimization is achieved at both the feature extraction and feature fusion levels, thereby effectively solving the technical problem of insufficient accuracy and robustness in ship target detection under complex marine scenarios.

[0085] Although the above embodiments are described using ship detection under complex sea conditions as a specific application scenario, the technical solution of this application, after adaptive modifications (e.g., adjusting model input size, re-labeling training data, fine-tuning network hyperparameters, etc.), can also be applied to other visual detection tasks with similar technical challenges (i.e., targets are directional, have significant aspect ratios, and are in complex backgrounds with interference from small targets). For example, this solution can be transferred to ship monitoring in inland waterways, identification of logistics vehicles in port areas, detection of linear features (such as roads and aircraft) in remote sensing images, and defect localization of specific long and narrow parts in industrial vision. Any application that improves the model based on the technical concepts disclosed in this application to solve the robust detection problem of directional, multi-scale targets in other similar scenarios should fall within the protection scope of this application.

[0086] Example 2 Please refer to Figure 8 This application also provides a ship target detection system based on UAV low-altitude remote sensing imagery to implement the steps of the ship target detection method based on UAV low-altitude remote sensing imagery described in the above embodiments. The ship target detection system based on UAV low-altitude remote sensing imagery includes: an image acquisition unit 1001 and a detection execution unit 1002.

[0087] The image acquisition unit 1001 is used to acquire images of the ships to be detected in the target ocean area; The detection execution unit 1002 is used to input the image of the ship to be detected into the ship target detection model, perform ship target detection, and obtain the ship detection result image of the target ocean area. The ship target detection model is an improved version of the RT-DETR detection model as a baseline network. The improvement includes at least the following: setting a multi-scale strip convolution module in the backbone network of the baseline network and setting a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship target; the state space evolution module is used to provide global context awareness of the ship target in the target ocean area; and the ship detection result image is marked with several ship target location identifiers.

[0088] It should be noted that the above embodiment of the ship target detection system based on UAV low-altitude remote sensing imagery is only illustrated by the above-described division of functional modules when implementing a ship target detection method based on UAV low-altitude remote sensing imagery. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the equipment can be divided into different functional modules to complete all or part of the functions described above.

[0089] Furthermore, the ship target detection system based on UAV low-altitude remote sensing imagery provided in the above embodiments and the ship target detection method based on UAV low-altitude remote sensing imagery in Embodiment 1 belong to the same concept. The implementation process is detailed in the method embodiment, namely Embodiment 1, and will not be repeated here.

[0090] Example 3 This application provides a computer device, such as Figure 9 As shown, the computer device 21 may include: a processor 210, a memory 211, and a computer program 212 stored in the memory 211 and capable of running on the processor 210, such as a ship target detection program; when the processor 210 executes the computer program 212, it implements the steps in the above embodiment 1.

[0091] The processor 210 may include one or more processing cores. The processor 210 connects to various parts within the computer device 21 using various interfaces and lines. It executes various functions of the computer device 21 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 211, and by accessing data in the memory 211. Optionally, the processor 210 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 210 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 210 and may be implemented as a separate chip.

[0092] The memory 211 may include random access memory (RAM) or read-only memory. Optionally, the memory 211 may include a non-transitory computer-readable storage medium. The memory 211 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 211 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 211 may also be at least one storage device located remotely from the aforementioned processor 210.

[0093] Example 4 This application also provides a computer storage medium that can store multiple instructions applicable to the steps of the ship target detection method based on UAV low-altitude remote sensing imagery described in Embodiment 1 above, which are loaded and executed by a processor. The specific execution process can be found in the detailed description of the above embodiments, and will not be repeated here.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0101] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and this application also intends to include these modifications and variations.

Claims

1. A method for ship target detection based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Acquire images of the vessels to be detected in the target ocean area; The image of the ship to be detected is input into the ship target detection model to perform ship target detection and obtain the ship detection result image of the target ocean area; The ship target detection model is an improved version of the RT-DETR detection model as a baseline network. The improvement includes at least the following: setting a multi-scale strip convolution module in the backbone network of the baseline network and setting a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship target; the state space evolution module is used to provide global context awareness of the ship target in the target ocean area; and the ship detection result image is marked with several ship target location identifiers.

2. The ship target detection method based on UAV low-altitude remote sensing imagery according to claim 1, characterized in that, The steps for acquiring images of the target ocean area of ​​the vessel to be detected include: The drone uses its onboard image sensor to capture aerial video streams or sequences of aerial images of the target ocean area. The original aerial images are extracted from the aerial video stream or the aerial image sequence, and the original aerial images are subjected to dehazing, deblurring and dynamic range enhancement processing to obtain quality-enhanced images. Based on the input requirements of the ship target detection model, the quality enhancement image is scaled and pixel value normalized to obtain the ship image to be detected.

3. The ship target detection method based on UAV low-altitude remote sensing imagery according to claim 1, characterized in that, The ship target detection model includes a backbone network, a neck network, and a detection decoding head connected in sequence. The backbone network includes several basic convolutional modules and at least one pooling layer disposed in the front stage of the backbone network, and several multi-scale strip convolutional modules disposed in the rear stage of the backbone network. The neck network is an encoder with a feature pyramid structure, whose input is connected to the output of the backbone network, and is used to fuse and enhance the multi-scale features output by the backbone network. The neck network includes: several convolutional layers with 1×1 kernels, several upsampling layers, several reparameterized convolutional modules, and the state space evolution module; the 1×1 convolutional layers, upsampling layers, and reparameterized convolutional modules are interconnected to form several fusion paths for cross-scale feature transfer and fusion; the fusion paths are fused through a splicing operation; the state space evolution module is located on the fusion path for receiving and processing the highest resolution feature map from the backbone network. The input end of the detection decoding head is connected to the output end of the neck network to receive fused and enhanced multi-scale features and output the corresponding ship detection result image.

4. The ship target detection method based on UAV low-altitude remote sensing imagery according to any one of claims 1 to 3, characterized in that, The multi-scale strip convolution module adopts a structure containing four parallel branches; the multi-scale strip convolution module processes the input first feature map in the following manner. X Processing is performed to output a second feature map. Y : The first feature map X The feature set is divided into four feature subsets along the channel dimension, denoted as the first feature subset. X 1. Second feature subset X 2. Third feature subset X 3 and the fourth feature subset X 4; For the first feature subset X 1. Perform a depthwise convolution transformation with a 3×3 kernel to obtain the first intermediate feature. X 1 ’ : ； For the second feature subset X 2. Perform a depthwise convolution transformation with a kernel of 1×11 to obtain the second intermediate feature. X 2 ’ : ； For the third feature subset X 3. Perform a depthwise convolution transformation with an 11×1 kernel to obtain the third intermediate feature. X 3 ’ : ； For the fourth feature subset X 4. Perform identity mapping to obtain the fourth intermediate feature. X 4 ’ : ； The first intermediate feature X 1 ’ The second intermediate feature X 2 ’ The third intermediate feature X 3 ’ and the fourth intermediate feature X 4 ’ By concatenating the data along the channel dimension, the corresponding preliminary fusion features are obtained. X c : ； Regarding the preliminary fusion features X c Perform layer normalization to obtain the corresponding normalized features. X norm : ； Where LayerNorm() is a preset layer normalization function; The normalized features X norm The input is fed into a preset multilayer perceptron for cross-channel information interaction and nonlinear transformation to obtain information interaction features. X mid : ； Wherein, MLP(·) is the mapping processing function corresponding to the preset multilayer perceptron, used to realize cross-channel information interaction and nonlinear transformation; The information interaction features X mid With the first feature map X The second feature map is obtained by adding elements one by one. Y : 。 5. The ship target detection method based on UAV low-altitude remote sensing imagery according to any one of claims 1 to 3, characterized in that, The state-space evolution module processes the input third feature map in the following manner. X r Processing is performed to output the fourth feature map. Y r : The input third feature map X r The same left-path and right-path features are copied to form two parallel branches; The left-side features are sequentially processed by a preset linear mapping, a 3×3 depthwise convolution transformation, and a preset activation function to generate left-side gated features. X left : ； The right-side features are sequentially subjected to a preset linear mapping, a 3×3 depthwise convolution transformation, and a preset activation function to generate right-side gated features. X right : ； The left-side gated feature X left Expanding in the spatial dimension yields the corresponding feature sequence { x 1, x 2, ..., x L }, where L is the total number of feature vectors in the feature sequence; The feature sequence is input into a preset state space model, and recursive calculations are performed based on the discretized state space equations corresponding to the state space model to obtain the output feature sequence { y 1, y 2, ..., y L }: ； in, x t For the first in the feature sequence t Feature vectors at each position and This is the system matrix obtained by discretizing the preset continuous system parameters. C and D For a preset, learnable projection matrix, h t For the corresponding to the first t The hidden state vector at each position, y t For the corresponding to the first t Output feature vectors at each position; Based on the left-side gate control features X left The spatial dimensions are determined by recombining the output feature sequences to obtain state space features with the same spatial dimensions. X ssm ; The state space features X ssm With the right-side gating feature X right The splicing is performed along the channel dimension to obtain the spliced ​​features. X concat : ； The splicing feature X concat The fourth feature map is obtained by projecting the output linear layer through a preset method to transform its channel number to a preset expected output channel number. Y r : 。 6. The ship target detection method based on UAV low-altitude remote sensing imagery according to claim 1, characterized in that, The step of inputting the image of the ship to be detected into the ship target detection model to perform ship target detection and obtain the ship detection result image of the target ocean area includes: The ship target detection model processes the image of the ship to be detected and outputs prediction information corresponding to potential ship targets in the image of the ship to be detected. The prediction information includes the position coordinates and category confidence of each potential ship target. Based on a preset confidence threshold, the predicted information is subjected to threshold filtering to remove predicted information whose category confidence is lower than the confidence threshold; Based on the location coordinates of the filtered prediction information, the corresponding candidate prediction boxes are determined; The non-maximum suppression algorithm is used to calculate the overlap of each candidate prediction box, and redundant candidate prediction boxes are eliminated according to the overlap to obtain the ship target detection box. The ship target detection box is superimposed on the image of the ship to be detected to obtain the ship detection result image marked with the ship target location identifier.

7. The ship target detection method based on UAV low-altitude remote sensing imagery according to claim 1, characterized in that, Before inputting the image of the ship to be detected into the ship target detection model, the method further includes a performance testing step for the ship target detection model, the performance testing step including: Using a pre-set set of ship target test images, the performance evaluation parameters of the ship target detection model are calculated, wherein the performance evaluation parameters include at least the precision, recall, and mean precision of the ship target detection model. Each of the calculated performance evaluation parameters is compared with its corresponding preset performance threshold. When each of the performance evaluation parameters reaches or exceeds its corresponding preset performance threshold, the ship target detection model is judged to have passed the performance detection.

8. A ship target detection system based on low-altitude remote sensing imagery from unmanned aerial vehicles (UAVs), characterized in that, include: The image acquisition unit is used to acquire images of the ships to be detected in the target ocean area; The detection execution unit is used to input the image of the ship to be detected into the ship target detection model, perform ship target detection, and obtain the ship detection result image of the target ocean area; The ship target detection model is an improved version of the RT-DETR detection model as a baseline network. The improvement includes at least the following: setting a multi-scale strip convolution module in the backbone network of the baseline network and setting a state space evolution module in the neck network of the baseline network; the multi-scale strip convolution module is used to extract the geometric shape features of the ship target; the state space evolution module is used to provide global context awareness of the ship target in the target ocean area; and the ship detection result image is marked with several ship target location identifiers.

9. A computer device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; when the processor executes the computer-readable program, it implements the steps in the ship target detection method based on UAV low-altitude remote sensing imagery as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the ship target detection method based on UAV low-altitude remote sensing imagery as described in any one of claims 1 to 7.

Images