Lightweight real-time target detection method based on marine environment and related equipment

Abstract

The invention discloses a marine environment-based lightweight real-time target detection method and related equipment, and relates to the field of image recognition. By implementing the method, the improved detection model is fused with a MobileNetV3-small network and a multi-scale feature fusion detection head, and the key pain points of large environmental interference and unclear seeing and limited computing power and incapability of running in traditional ocean target detection are accurately solved. In the scenes such as marine scientific investigation, underwater search and rescue and the like with extremely high requirements on real-time performance, the method can quickly output a target category and a bounding box, so that the timeliness of detection is ensured, the comprehensiveness of feature extraction is also considered, and a lightweight model still keeps stable detection performance in a complex marine environment.

Description

Technical Field

[0001] This application relates to the field of image recognition, and in particular to a lightweight real-time target detection method and related equipment based on a marine environment. Background Technology

[0002] With the rapid development of the marine economy and the deepening of marine development, marine environmental target detection technology has become particularly important. In tasks such as marine scientific research, underwater search and rescue, subsea pipeline inspection, and marine biodiversity surveys, the use of underwater robots or unmanned vessels equipped with vision systems to collect images and identify and locate specific targets (such as fish, divers, and seabed obstacles) in the images has become a research hotspot in the field of marine engineering.

[0003] Currently, marine environmental target detection methods mainly rely on deep learning techniques, especially general target detection algorithms based on convolutional neural networks (CNNs). In practical applications, complex large-scale network models are typically used, such as Faster R-CNN, YOLOv4, or detection architectures based on deep backbone networks like ResNet-50 and ResNet-101. By stacking a large number of convolutional layers and feature extraction modules, rich semantic information and detailed features in images are obtained, thereby enabling the identification of various targets in complex marine backgrounds.

[0004] However, due to the large variations in marine ambient light and severe interference from sea surface reflections, and the fact that devices equipped with vision systems are often limited by computing resources and power consumption, the above methods face problems of high computational load and slow detection speed in practical applications. Summary of the Invention

[0005] This application provides a lightweight real-time target detection method and related equipment based on the marine environment, which can quickly and accurately capture targets while reducing computing power.

[0006] In a first aspect, this application provides a lightweight real-time target detection method based on the marine environment, applied to a lightweight real-time target detection device. The method includes: acquiring a target image of the marine environment; inputting the target image into an improved detection model to obtain the target category and bounding box in the target image; the detection model includes a MobileNetV3-small network and a multi-scale feature fusion detection head; the MobileNetV3-small network extracts multi-scale feature images based on the target image; and the multi-scale feature fusion detection head processes the multi-scale feature images through a multi-branch parallel structure to generate target detection feature images.

[0007] By adopting the above technical solutions, the improved detection model integrates the MobileNetV3-small network with a multi-scale feature fusion detection head, accurately addressing the key pain points of traditional marine target detection: "large environmental interference, unclear visibility" and "limited computing power, unable to run smoothly." The MobileNetV3-small network, as a lightweight backbone network, features small parameter size and high computational efficiency, enabling efficient operation on devices with limited computing resources, such as underwater robots and unmanned vessels. This reduces the detection model's dependence on hardware computing power and solves the problem of traditional deep networks (such as ResNet-50 and Faster R-CNN) being unable to respond in real-time in marine scenes due to high computational demands. Simultaneously, the MobileNetV3-small network extracts multi-scale feature images tailored to the characteristics of marine images. Combined with a multi-branch parallel structure multi-scale feature fusion detection head, it achieves comprehensive coverage of targets of different sizes (such as small fish, large obstacles, and divers), avoiding target misses or false detections caused by single-scale feature extraction. In scenarios with extremely high real-time requirements, such as marine scientific research and underwater search and rescue, this method can quickly output target categories and bounding boxes, ensuring both the timeliness of detection and the comprehensiveness of feature extraction, so that the lightweight model can maintain stable detection performance in complex marine environments.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the MobileNetV3-small network includes an h-swish activation function and a Squeeze-and-Excitation module. The h-swish activation function performs a nonlinear mapping operation on the image to be detected based on a piecewise linear function. The Squeeze-and-Excitation module performs a global pooling operation on the image to be detected to obtain global spatial information, and generates weight coefficients for each channel through a fully connected layer, and multiplies the weight coefficients by the image to be detected channel by channel.

[0009] By adopting the above technical solutions, the h-swish activation function and the Squeeze-and-Excitation (SE) module are introduced into the MobileNetV3-small network, further enhancing the feature extraction capability and detection accuracy of the detection model in complex marine environments, and making up for the shortcomings of the basic lightweight network in feature representation. The h-swish activation function performs nonlinear mapping operations based on piecewise linear functions. Compared with traditional activation functions (such as ReLU), it can better fit the nonlinear features of complex scenarios such as large changes in illumination and severe reflection interference in marine images while maintaining computational lightweightness. It avoids the gradient vanishing problem caused by activation function saturation, enabling the MobileNetV3-small network to effectively transmit feature information in deep computation and improving the ability to identify targets in low-light and highly reflective areas. The SE module obtains global spatial information of the image to be detected through global pooling, and then generates weight coefficients for each channel through a fully connected layer and weights them channel by channel. This adaptively highlights the features of the target region and suppresses redundant information in the ocean background (such as water flow disturbance and reflection from suspended particles), allowing the detection model to focus more on valuable target features and reducing the impact of background interference on the detection results. The combination of these two technologies enables the MobileNetV3-small network to achieve a synergy between "efficient computation" and "accurate feature extraction" while maintaining its lightweight advantages. This solves the problem of insufficient detection accuracy in complex ocean environments caused by the weak feature representation capabilities of traditional lightweight models, ensuring that the detection model can accurately identify various targets while running in real time, thus improving the reliability of the detection results.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the multi-scale feature fusion detection head processes multi-scale feature images through a multi-branch parallel structure to generate target detection feature images. Specifically, this includes: using a 1×1 convolutional layer to perform channel dimension mapping transformation on the multi-scale feature images to obtain intermediate feature images; inputting the intermediate feature images into a first processing path and a second processing path respectively; in the first processing path, using a standard convolutional layer to calculate and extract local detail features from the intermediate feature images to obtain a first path output; in the second processing path, using a dilated reparameterization unit including multiple convolutional branches with different dilation rates to extract multi-scale spatial features to obtain a second path output; concatenating the first path output and the second path output in the channel dimension to obtain concatenated features; and using a 1×1 convolutional layer to fuse and compress the concatenated features to output the target detection feature image.

[0011] By employing the aforementioned technical solution, a 1×1 convolutional layer is used to map and transform the channel dimensions of multi-scale feature images. This achieves reasonable adjustment of feature dimensions without increasing computational load, laying the foundation for subsequent branch processing. A dual-path parallel processing mode is then adopted: the first path focuses on local detail features through a standard convolutional layer, accurately capturing subtle information such as the edges and textures of the target, avoiding the loss of target details due to marine environmental interference; the second path uses dilated reparameterized units with multiple convolutional branches of different dilation rates to simultaneously extract multi-scale spatial features under different receptive fields. This effectively adapts to the characteristics of marine scenes with large differences in target size (such as tiny marine organisms and large underwater equipment) and complex spatial distribution, solving the problem that a single receptive field cannot comprehensively cover various targets. Finally, channel-dimensional concatenation and 1×1 convolutional fusion compression organically integrate local detail features and multi-scale spatial features, preserving feature richness while reducing subsequent computational costs through compression, achieving a balance between "feature comprehensiveness" and "computational efficiency." This method enables the multi-scale feature fusion detection head to analyze target features from different dimensions and scales, effectively addressing the challenges of varied target morphologies and complex backgrounds in marine environments. It significantly reduces the probability of missed detections and false detections, especially in scenarios where targets are partially occluded or have variable sizes, demonstrating stronger robustness and further enhancing the practical value of lightweight models in marine environments.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the detection model further includes an attention-based intra-scale feature interaction module, which performs interactive enhancement on the multi-scale image features before the multi-scale feature fusion detection head processes the multi-scale feature image through a multi-branch parallel structure to generate the target detection feature image.

[0013] By adopting the above technical solution, the detection model also includes an attention-based intra-scale feature interaction module. This module enhances the interaction of multi-scale image features before the multi-scale feature fusion detection head processes the data, effectively solving the problems of "isolated features between scales and low information utilization" in traditional multi-scale feature extraction. This significantly improves the detection model's adaptability to complex marine environments. In marine environments, target features often exhibit inconsistent scales and fragmented characteristics due to changes in illumination and water flow disturbances. Single-scale features are often insufficient to comprehensively represent target attributes, and the lack of effective interaction between features at different scales can lead to feature redundancy or the omission of key information. The attention-based intra-scale feature interaction module can focus on the effective information in features at each scale. Through cross-scale feature interaction and enhancement, it strengthens the correlation and complementarity between different scales, allowing the detection model to more accurately capture the core features of the target at different scales, while suppressing irrelevant background interference information (such as suspended particles and sea surface reflections) at each scale. This preprocessing operation not only provides a higher quality and more relevant feature base for subsequent multi-branch parallel processing, but also further improves the accuracy and robustness of target detection without significantly increasing the amount of computation. This enables the detection model to still output reliable detection results stably when facing scenarios with varying target scales and blurred features in the marine environment. It is especially suitable for tasks such as marine scientific research and subsea pipeline inspection that require high detection accuracy.

[0014] In some embodiments, in conjunction with the first aspect, the method further includes: calculating the feature variance value of the background region in the multi-scale feature image; generating a sea state complexity coefficient based on the feature variance value; and dynamically adjusting the confidence threshold of the decoder when performing non-maximum suppression processing based on the sea state complexity coefficient.

[0015] By adopting the above technical solution, the confidence threshold is dynamically adjusted based on sea state complexity, achieving adaptive matching between the detection model and changes in the marine environment. This perfectly solves the pain point of traditional fixed threshold strategies struggling to balance "missed detections" and "false detections" under complex sea conditions. The complexity of the marine environment (such as wave size, current speed, and light intensity) directly affects the stability of the image background, leading to significant differences in the variance of background region features—the more complex the sea state, the larger the variance of background features, and the more difficult it is to distinguish between the target and the background; the more stable the sea state, the smaller the variance of background features, and the more prominent the target features. This method generates a sea state complexity coefficient by calculating the feature variance values ​​of the background region in multi-scale feature images. It can quantify the interference of the current sea state on detection in real time and dynamically adjust the confidence threshold of the decoder's non-maximum suppression (NMS): appropriately lowering the threshold under complex sea conditions to avoid the target features being misjudged as noise and missed due to background interference; and appropriately raising the threshold under stable sea conditions to filter out weak false features and reduce false detections. This dynamic adjustment mechanism allows the detection model to optimize its detection strategy based on actual sea conditions without human intervention, ensuring both the integrity of target detection under complex sea conditions and the accuracy of detection results under stable sea conditions.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after inputting the image to be detected into the improved detection model to obtain the target category and bounding box in the image to be detected, the method further includes: determining the target tracking trajectory based on the historical detected targets and historical bounding boxes of historical frames; predicting the predicted bounding box of the current frame based on the target tracking trajectory; matching the bounding box and the predicted bounding box when the historical detected targets of historical frames are the same as the target category in the image to be detected; if the bounding boxes do not match successfully, initializing the trajectory to be confirmed; if the trajectory to be confirmed matches successfully in the next preset number of consecutive frames, adding the trajectory to be confirmed to the target tracking trajectory; if the trajectory to be confirmed is lost or does not match successfully in the next preset number of consecutive frames, determining the trajectory to be confirmed as a false trajectory.

[0017] By adopting the above technical solution, a target tracking and trajectory verification mechanism is added to the target detection, effectively solving problems such as "trajectory breakage" and "false trajectory misjudgment" caused by target occlusion, feature blurring, or background interference in the marine environment, significantly improving the continuity and reliability of detection results. Traditional target detection methods only process single-frame images independently and cannot utilize inter-frame correlation information. In marine scenes, they are prone to missed detections due to temporary target occlusion, rapid movement, or background interference, or generate incorrect detection results due to false features. This method determines the tracking trajectory by using the detected targets and bounding boxes in historical frames and predicts the predicted bounding box in the current frame, realizing the correlation and tracing of targets between frames. For unmatched bounding boxes, the initialization of "trajectory to be confirmed" and subsequent multi-frame verification mechanism effectively filters false trajectories—only trajectories that are continuously matched in a preset number of future frames are included in the formal tracking trajectory; otherwise, they are judged as false trajectories. This design can not only quickly recover the tracking trajectory of occluded targets and ensure the continuity of target detection, but also accurately eliminate temporary false detection results caused by background interference, avoiding the impact of erroneous trajectories on subsequent analysis.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, when the historical detection target in the historical frame is of the same category as the target in the image to be detected, the bounding box and the predicted bounding box are matched. Specifically, this includes: when the historical detection target in the historical frame is of the same category as the target in the image to be detected, calculating the positional deviation metric and the shape overlap metric of the bounding box and the predicted bounding box, wherein the positional deviation metric is calculated based on the Euclidean distance between the center point of the bounding box and the center point of the predicted bounding box, and the shape overlap metric is calculated based on the intersection-union ratio between the bounding box and the predicted bounding box; and determining the matching status of the bounding box and the predicted bounding box based on the positional deviation metric and the shape overlap metric.

[0019] By adopting the above technical solution, a dual metric for bounding box matching (positional deviation metric and shape overlap metric) is defined, providing a precise and objective basis for judging inter-frame target association. This effectively solves the problems of "inaccurate matching" and "poor robustness of association" in marine scenes caused by traditional single matching criteria (such as those based solely on intersection-over-union ratio). In the marine environment, targets may exhibit various forms of motion, such as translation, rotation, and scaling. Relying solely on a single-dimensional matching metric is insufficient to comprehensively measure the consistency of targets between frames—relying only on position may lead to misjudgment of different targets due to slight target shifts, and relying solely on shape may result in missed associations due to changes in target attitude. The positional deviation metric, calculated based on the Euclidean distance between the center points of the bounding boxes, can accurately quantify the spatial positional association of targets between frames, avoiding matching failures caused by small-scale target movements. The shape overlap metric, calculated based on the intersection-over-union ratio (IoU) of the bounding boxes, can effectively characterize the morphological consistency of targets between frames, filtering out false associations where targets are close in position but have significant shape differences. The combination of dual measurement standards comprehensively judges the target matching status from two core dimensions: spatial location and morphological characteristics. This makes the matching results more objective and robust, and can adapt to the characteristics of diverse target movement forms and possible morphological changes in marine scenarios.

[0020] In a second aspect, embodiments of this application provide a lightweight real-time target detection device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, which includes computer instructions, and the one or more processors call the computer instructions to cause the lightweight real-time target detection device to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, embodiments of this application provide a computer program product containing instructions that, when the computer program product is run on a lightweight real-time target detection device, cause the lightweight real-time target detection device to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a lightweight real-time target detection device, cause the lightweight real-time target detection device to perform the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the lightweight real-time target detection device provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By adopting the above technical solutions, the improved detection model integrates the MobileNetV3-small network with a multi-scale feature fusion detection head, accurately addressing the key pain points of traditional marine target detection: "large environmental interference, unclear visibility" and "limited computing power, unable to run smoothly." The MobileNetV3-small network, as a lightweight backbone network, features small parameter size and high computational efficiency, enabling efficient operation on devices with limited computing resources, such as underwater robots and unmanned vessels. This reduces the detection model's dependence on hardware computing power and solves the problem of traditional deep networks (such as ResNet-50 and Faster R-CNN) being unable to respond in real-time in marine scenes due to high computational demands. Simultaneously, the MobileNetV3-small network extracts multi-scale feature images tailored to the characteristics of marine images. Combined with a multi-branch parallel structure multi-scale feature fusion detection head, it achieves comprehensive coverage of targets of different sizes (such as small fish, large obstacles, and divers), avoiding target misses or false detections caused by single-scale feature extraction. In scenarios with extremely high real-time requirements, such as marine scientific research and underwater search and rescue, this method can quickly output target categories and bounding boxes, ensuring both the timeliness of detection and the comprehensiveness of feature extraction, so that the lightweight model can maintain stable detection performance in complex marine environments.

[0025] 2. By adopting the above technical solution, the h-swish activation function and the Squeeze-and-Excitation (SE) module are introduced into the MobileNetV3-small network, further enhancing the feature extraction capability and detection accuracy of the detection model in complex marine environments, and making up for the shortcomings of the basic lightweight network in feature representation. The h-swish activation function performs nonlinear mapping operations based on piecewise linear functions. Compared with traditional activation functions (such as ReLU), it can better fit the nonlinear features of complex scenes such as large changes in illumination and severe reflection interference in marine images while maintaining computational lightweightness. It avoids the gradient vanishing problem caused by activation function saturation, enabling the MobileNetV3-small network to effectively transmit feature information in deep computation and improving the ability to identify targets in low-light and highly reflective areas. The SE module obtains global spatial information of the image to be detected through global pooling, and then generates weight coefficients for each channel through a fully connected layer and weights them channel by channel. This adaptively highlights the features of the target region and suppresses redundant information in the ocean background (such as water flow disturbance and reflection from suspended particles), allowing the detection model to focus more on valuable target features and reducing the impact of background interference on the detection results. The combination of these two technologies enables the MobileNetV3-small network to achieve a synergy between "efficient computation" and "accurate feature extraction" while maintaining its lightweight advantages. This solves the problem of insufficient detection accuracy in complex ocean environments caused by the weak feature representation capabilities of traditional lightweight models, ensuring that the detection model can accurately identify various targets while running in real time, thus improving the reliability of the detection results.

[0026] 3. By adopting the above technical solution, a 1×1 convolutional layer is used to map and transform the channel dimensions of multi-scale feature images. This achieves reasonable adjustment of feature dimensions without increasing computational cost, laying the foundation for subsequent branch processing. A dual-path parallel processing mode is then employed: the first path focuses on local detail features through a standard convolutional layer, accurately capturing subtle information such as the edges and textures of the target, avoiding the loss of target details due to marine environmental interference; the second path uses dilated reparameterized units with multiple convolutional branches of different dilation rates to simultaneously extract multi-scale spatial features under different receptive fields. This effectively adapts to the characteristics of marine scenes with large differences in target size (such as tiny marine organisms and large underwater equipment) and complex spatial distribution, solving the problem that a single receptive field cannot comprehensively cover various targets. Finally, channel-dimensional concatenation and 1×1 convolutional fusion compression organically integrate local detail features and multi-scale spatial features, preserving feature richness while reducing subsequent computational costs through compression, achieving a balance between "feature comprehensiveness" and "computational efficiency." This method enables the multi-scale feature fusion detection head to analyze target features from different dimensions and scales, effectively addressing the challenges of varied target morphologies and complex backgrounds in marine environments. It significantly reduces the probability of missed detections and false detections, especially in scenarios where targets are partially occluded or have variable sizes, demonstrating stronger robustness and further enhancing the practical value of lightweight models in marine environments. Attached Figure Description

[0027] Figure 1 This is a detection model architecture diagram of a lightweight real-time target detection method based on a marine environment in the embodiments of this application; Figure 2 This is a basic architecture diagram of MobileNetV3-small in the embodiments of this application; Figure 3 This is a basic architecture diagram of the DRBC3 module in the embodiments of this application; Figure 4 This is a schematic diagram of the Dilated Reparam Block in the DRBC3 module in an embodiment of this application; Figure 5 This is a schematic diagram of the reparameterized network structure in an embodiment of this application; Figure 6 This is a flowchart illustrating a lightweight real-time target detection method based on a marine environment, as described in this application. Figure 7 This is another flowchart illustrating the lightweight real-time target detection method based on the marine environment in this application embodiment; Figure 8 This is a schematic diagram of the physical structure of a lightweight real-time target detection device in the embodiments of this application. Detailed Implementation

[0028] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0029] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0030] To facilitate understanding, the application scenarios of the embodiments of this application are described below.

[0031] Target detection in marine environments presents unique challenges not found in land-based scenarios. Existing general-purpose technologies are either too cumbersome to run smoothly (high-performance models (such as the Transformer-based DETR) require too much computation and have too many parameters, while marine equipment (drones, unmanned vessels, edge computing boxes) has limited computing power and cannot support these large models, making real-time detection impossible), or their visual capabilities are insufficient (violent wave movements can deform or obscure targets (such as small boats or people in the water); sea surface reflections, sea fog, and backlighting can blur images, etc.).

[0032] To resolve the aforementioned contradictions, this method is an improvement upon RT-DETR (a real-time detection model), resulting in the MRL-DETR model. For example... Figure 1 As shown, this embodiment of the invention provides a detection model architecture diagram for a lightweight real-time target detection method based on the marine environment. The MRL-DETR model utilizes a lightweight backbone network (MobileNetV3-small) (see [link to relevant documentation]). Figure 2 ) and the multi-scale feature fusion detection head (DRBC3 module) (see Figure 3 This enables lightweight, real-time target detection.

[0033] 1. Lightweight backbone network (MobileNetV3-small) Technical action: The bulky ResNet backbone network in the original model was replaced with the extremely lightweight MobileNetV3-small; principle: Using depthwise separable convolution: breaking down regular convolution into smaller parts significantly reduces the number of parameters; Introducing the Squeeze-and-Excitation (SE) module: This is an attention mechanism that can dynamically adjust channel weights, allowing the model to automatically ignore wave noise and focus on target features; Explanation: The original backbone network was like a heavy-duty truck engine; although powerful, it consumed a lot of fuel and couldn't power drones. Now it has been replaced with MobileNetV3-small (a high-performance motorcycle engine), which is small in size and fast. The SE module is like adding an intelligent control system to the engine. When there is only ocean noise (background noise), it can automatically amplify the "human voice" (target signal) and reduce the "wind noise" (ocean noise).

[0034] Optionally, in general, the MobileNetV3-small network includes an h-swish activation function and a Squeeze-and-Excitation module. The h-swish activation function performs a nonlinear mapping operation on the image to be detected based on a piecewise linear function. The Squeeze-and-Excitation module performs a global pooling operation on the image to be detected to obtain global spatial information, and generates weight coefficients for each channel through a fully connected layer, multiplying the weight coefficients by the image to be detected channel by channel.

[0035] Among them, MobileNetV3-small network refers to a lightweight convolutional neural network backbone structure designed specifically for resource-constrained devices. It represents a feature extraction network with depthwise separable convolution as its core, small parameter size, and high computational efficiency, adapting to the computing power requirements of marine inspection equipment such as underwater robots and unmanned vessels. h-swish activation function refers to a nonlinear activation function based on piecewise linear functions, used to represent a function that can effectively fit nonlinear features of an image while maintaining computational lightweightness. Piecewise linear functions are functions composed of multiple linear segments, used to represent mathematical tools that replace traditional complex nonlinear operations through piecewise linear fitting. Nonlinear mapping operation refers to the computational process of mapping image pixel features from the original space to a nonlinear feature space, used to represent the core operation that enhances feature representation capabilities. Squeeze-and-Excitati The on module (SE module) is a network module that enhances effective features through a channel attention mechanism. It represents a feature enhancement unit that can adaptively adjust the feature weights of each channel. Global pooling refers to the pooling operation performed on the entire feature map, representing the calculation method for obtaining global spatial information of the image. Global spatial information refers to the feature distribution information covering the entire image region, representing the spatial correlation features between the target and the background within the entire image. A fully connected layer is a network layer in which all neurons in the neural network are connected to all neurons in the previous layer, representing the network structure that realizes feature dimension transformation and weight generation. Weight coefficients are numerical values ​​used to characterize the importance of features in each channel, representing the basis for weighting features in different channels. Channel-wise multiplication refers to the operation of multiplying the feature value of each channel by its corresponding weight coefficient one by one, representing the operation of enhancing or suppressing features in different channels.

[0036] For example, in ocean low-light image detection, the h-swish activation function performs non-linear mapping on the dark features of the image through a piecewise linear function, avoiding the gradient vanishing caused by the traditional ReLU function, and allowing weak target features to be preserved; the SE module performs global pooling on images containing seabed pipes to obtain the spatial distribution information of the pipes in the whole image, and then generates high-weight coefficients for the channels where the pipe features are located through a fully connected layer, and multiplies them channel by channel to enhance the pipe features and suppress the water flow background features.

[0037] 2. Multi-scale feature fusion detection head (DRBC3 module) Technical action: A DRBC3 (Dilated Reparameter Block) module was introduced into the detection head section; principle: Dilated convolution: without increasing computational cost, it expands the receptive field (i.e., sees more), can capture long-distance spatial dependencies, and can handle target deformation; Structural reparameterization: During training, a multi-branch structure is used to extract rich features (both details and outlines are examined), and during inference (use), these branches are merged into one path, resulting in extremely fast speed; Explanation: Ships at sea appear distorted due to the rolling waves. Ordinary convolutional kernels are like looking at a leopard through a tube, only seeing a part and unable to piece together the ship's shape. The DRBC3 module is like giving the model a pair of wide-angle dual-focal glasses, allowing it to see both close-up details (local features) and distant outlines (global features) clearly.

[0038] Optionally, in general, the multi-scale feature fusion detection head processes multi-scale feature images through a multi-branch parallel structure to generate target detection feature images. This can be achieved in the following ways, without limitation: A 1×1 convolutional layer is used to perform channel-dimensional mapping transformation on the multi-scale feature images to obtain intermediate feature images; the intermediate feature images are input into the first processing path and the second processing path respectively; in the first processing path, a standard convolutional layer is used to calculate the intermediate feature images to extract local detail features, resulting in the first path output; in the second processing path, a dilated reparameterization unit including multiple convolutional branches with different dilation rates is used to extract multi-scale spatial features, resulting in the second path output; the first path output and the second path output are concatenated along the channel dimension to obtain the concatenated features; a 1×1 convolutional layer is used to fuse and compress the concatenated features to output the target detection feature image.

[0039] In this context, a 1×1 convolutional layer refers to a convolutional operation layer with a kernel size of 1 pixel × 1 pixel. It represents a network component that performs channel dimension transformation, feature fusion, or dimensionality compression without changing the spatial dimensions (height and width) of the feature map. It has low computational cost, high efficiency, and is suitable for lightweight detection requirements. Channel dimension mapping transformation refers to the process of adjusting (upgrading or downgrading) the number of channels in multi-scale feature images and mapping the feature distribution. It represents the operation of uniformly adapting the channel dimensions of features at different scales to subsequent branch processing. Intermediate feature images refer to the transitional feature data obtained after the 1×1 convolutional layer mapping transformation, representing channel dimension standardization and initial optimization of feature representation. The transformed feature map provides a unified input for dual-path processing; the first processing path and the second processing path refer to two independent and synchronously running feature processing channels in a multi-branch parallel structure, used to focus on feature extraction tasks of different dimensions respectively; the standard convolutional layer refers to a network layer that uses conventional convolutional kernels (such as 3×3, 5×5) and operates according to standard convolution rules, used to represent the core components for accurately extracting local detail features; local detail features refer to the subtle feature information such as the edges, textures, and corners of the target, used to represent feature data that can distinguish the local morphology of the target; the first path output refers to the feature result obtained after the first processing path completes the extraction of local detail features; the dilated reparameterized unit refers to a unit containing multiple different The dilation rate convolution branch, which simplifies the training and inference process through reparameterization, represents a module that expands the receptive field and extracts multi-scale spatial features without significantly increasing computation. Different dilation rate convolutions refer to convolution operations with varying kernel spacing, representing convolution methods that obtain different receptive fields by adjusting the dilation rate (e.g., 1, 2, 3), thereby covering targets of different sizes. Multi-scale spatial features refer to the overall shape and spatial distribution features of targets in different spatial ranges (near-ground, mid-ground, far-ground) and at different sizes, representing global feature data that can adapt to differences in target size. The second path output refers to the second processing path completing multi-scale... The feature results obtained after feature extraction in the degree space; concatenation in the channel dimension refers to the operation of superimposing and combining the output features of two paths in the channel dimension (e.g., merging C1 channel and C2 channel into C1+C2 channel), which is used to represent the fusion method that preserves the integrity of the two-path features; concatenated features refer to feature data containing both local details and multi-scale spatial information after channel concatenation; fusion compression refers to the process of integrating the concatenated high-dimensional features and reducing the channel dimension, which is used to represent reducing computational complexity while retaining key features; target detection feature image refers to the final output feature data that integrates multi-dimensional effective features, which is used to directly support subsequent target classification and localization calculations.

[0040] For example, in the scenario of inspecting subsea pipelines, multi-scale feature images include the edge texture of the pipeline (small-scale features), the overall direction of the pipeline (medium-scale features), and the spatial relationship between the pipeline and its surrounding environment (large-scale features). A 1×1 convolutional layer is used to uniformly adjust the number of channels for features at different scales to 64, resulting in an intermediate feature image. The first processing path uses a 3×3 standard convolutional layer to extract local detail features such as weld texture and surface scratches, forming the first path output. The second processing path uses three convolutional branches with dilation rates of 1, 2, and 3 to extract multi-scale spatial features such as the overall contour of the pipeline under different receptive fields and its spatial distribution with the seabed topography, forming the second path output. The two outputs (each with 64 channels) are concatenated in the channel dimension to obtain a 128-channel concatenated feature image. Finally, a 1×1 convolutional layer compresses the 128 channels into 64 channels, resulting in a target detection feature image containing both local pipeline details and global spatial information.

[0041] The execution timing of this method occurs after the MobileNetV3-small network extracts multi-scale feature images and before the detection model performs target classification and bounding box regression. It is the core operation process of the multi-scale feature fusion detection head and is applicable to all marine environmental target detection scenarios. It is especially suitable for marine scenarios with large differences in target size (such as small marine organisms and large underwater equipment), varied target shapes (such as swimming fish and fixed obstacles), and complex background interference (such as water flow disturbance and suspended particles). It can effectively improve the comprehensiveness and accuracy of feature extraction.

[0042] Specifically, this implementation method is carried out step by step in the following steps: Channel Dimension Mapping Transformation: First, a 1×1 convolutional layer is used to process the multi-scale feature images output by the MobileNetV3-small network. Since the number of channels may differ between feature images of different scales (e.g., 32 channels for shallow small-scale features and 128 channels for deep large-scale features), the 1×1 convolutional layer performs channel dimension mapping transformation on features at each scale by setting an appropriate number of output channels (e.g., uniformly 64 channels). Simultaneously, convolutional operations are used to initially optimize the feature distribution, achieving channel dimension standardization without changing the feature map space dimensions (height and width), resulting in intermediate feature images. The core function of this step is to provide a unified dimensional input for subsequent dual-path parallel processing, avoiding feature fusion conflicts caused by inconsistent channel numbers. Furthermore, the 1×1 convolutional layer has low computational cost and does not increase the computational burden on the lightweight model.

[0043] Dual-path parallel feature extraction: The intermediate feature image is simultaneously input into the first and second processing paths, with parallel computation on both paths to improve processing efficiency. In the first processing path, standard convolutional layers (typically 3×3 or 5×5 kernels) are used for feature calculation. The small receptive field of standard convolutional layers allows for precise focusing on local details of the target, such as the scale texture of marine life, details of a diver's equipment, and the edge contours of obstacles, effectively capturing subtle features easily lost due to marine environmental interference. The final output of the first path contains local details. In the second processing path, the core is the dilated reparameterization unit, which contains multiple convolutional branches with different dilation rates (e.g., dilation rates of 1, 2, and 3). (Multiple branches) Convolutional kernels with different dilation rates correspond to different receptive fields—a convolutional kernel with a dilation rate of 1 has a small receptive field, focusing on near-field features of the target; a convolutional kernel with a dilation rate of 2 has a medium receptive field, capturing mid-field features of the target; a convolutional kernel with a dilation rate of 3 has a large receptive field, covering far-field features of the target and its spatial relationship with the background. Multi-scale spatial features are extracted in parallel through multiple branches, and then the multi-branch structure is equivalent to a single convolutional layer in the inference stage through reparameterization technology, reducing computational complexity without losing feature extraction capabilities, and finally outputting a second path output containing multi-scale spatial information.

[0044] Feature stitching and fusion compression: First, the outputs of the first path and the second path are stitched together along the channel dimension. For example, if the first path output is a feature map of H×W×64 (height×width×number of channels) and the second path output is a feature map of H×W×64, the stitched feature map is obtained as H×W×128. This step can completely preserve local detail features and multi-scale spatial features, avoiding information loss during feature fusion. Then, a 1×1 convolutional layer is used to fuse and compress the stitched features. By setting an appropriate number of output channels (such as 64 channels), the high-dimensional features after stitching are integrated in the convolution operation to select the core features most valuable for object detection. At the same time, the feature dimension is reduced to reduce the computational load of subsequent classification and regression tasks. Finally, an object detection feature image of H×W×64 is output.

[0045] like Figure 4The diagram illustrates the internal structure and parameter configuration of the extended reparameterized block of a 5x5 convolutional kernel. DRepBlock utilizes an evolutionary convolution that enhances the equivalent 5x5 performance with a 3x3 kernel. The extended reparameterized block enhances standard large kernel convolutions by stacking multiple small-kernel extended convolutional layers. From a parametric structure perspective, these dilated convolutions are mathematically equivalent to a larger sparse convolutional kernel without dilation. Through reparameterization, the entire block can be merged into a single standard large-kernel convolutional layer. In the example shown, the effective kernel size is K=5: more dilated layers can be stacked accordingly to obtain a larger receptive field. This setup allows the block to capture fine-grained contextual information with a relatively small kernel while maintaining a large effective receptive field. During inference, these parallel branches are merged into an equivalent large kernel through structural reparameterization, significantly reducing computational overhead. Compared to the original RepC3 module, DRBC3 offers better feature representation and spatial modeling capabilities, making it particularly effective for detecting multi-scale and morphologically diverse marine objects.

[0046] like Figure 5 As shown, this is a schematic diagram of a reparameterized network structure, illustrating how a network structure with multiple parallel branches (left) can be reparameterized into a single convolutional layer (right).

[0047] The following describes the process of the method provided in this implementation, based on the above scenario. Please refer to [link / reference]. Figure 6 This is a flowchart illustrating a lightweight real-time target detection method based on a marine environment in an embodiment of this application.

[0048] S601. Acquire the image of the marine environment to be detected; Among them, the marine environment refers to the area of ​​the Earth's surface covered by seawater and the surrounding related waters, including different geographical ranges such as nearshore, offshore, deep sea, bay, and strait, and the related environment after natural or artificial intervention with typical characteristics such as variable lighting, water flow disturbance, suspended particulate matter, and sea surface reflection; the image to be detected refers to the visual data of the marine scene that needs to be identified and located, used to represent image information including potential detection objects (such as marine life, divers, seabed obstacles, seabed pipelines, ship wrecks, illegal fishing equipment, etc.), which can be acquired in the marine environment through various image acquisition devices.

[0049] Specifically, the method for acquiring images of the marine environment to be detected can be determined according to the actual application scenario: If the lightweight real-time target detection device is an integrated detection device mounted on a mobile carrier such as an underwater robot or unmanned vessel, it can actively acquire images of the marine environment to be detected directly through the integrated image acquisition module (such as an underwater pressure-resistant camera, an infrared imager, or a high-definition industrial camera). The image acquisition parameters are dynamically adjusted according to the task requirements. For example, in a low-light deep-sea environment, the infrared supplementary lighting function is automatically turned on, the acquisition resolution is set to 1920×1080 to ensure feature clarity, and the acquisition frame rate is controlled at 3-5 frames / second to balance data volume and real-time performance. If the lightweight real-time target detection device is a fixed detection device based on land or on a ship, it can receive images of the marine environment to be detected uploaded by front-end mobile acquisition devices (such as underwater detectors or drones) through wired (such as fiber optics) or wireless (such as satellite communication, underwater acoustic communication, or 5G) transmission links.

[0050] The lightweight real-time target detection equipment performs integrity verification and format parsing on the data to ensure that the image data is not lost or damaged. It can also perform preprocessing operations on the images to be detected, including image format standardization (converting to JPG or PNG format), pixel value normalization (adjusting the pixel range to the [0, 1] interval to adapt to the model input), and removal of image noise (such as blurry noise caused by water flow vibration and bad pixels caused by sensor failure).

[0051] S602. Input the image to be detected into the improved detection model to obtain the target category and bounding box in the image to be detected. The detection model includes the MobileNetV3-small network and the multi-scale feature fusion detection head. The MobileNetV3-small network extracts multi-scale feature images based on the image to be detected. The multi-scale feature fusion detection head processes the multi-scale feature images through a multi-branch parallel structure to generate target detection feature images.

[0052] The improved detection model refers to a detection algorithm framework optimized for marine environmental characteristics and lightweight, real-time requirements. It represents a comprehensive detection model integrating the MobileNetV3-small network and a multi-scale feature fusion detection head. Compared to traditional general detection models, it significantly improves in parameter size, computational efficiency, and adaptability to marine scenes. The target category refers to the specific attribute type of the target in the image to be detected, identified by the improved detection model. It represents the target's classification label, such as fish, whales, divers, rocks, metal obstacles, underwater pipelines, and illegal fishing nets. The bounding box is a rectangular region used to accurately select the target's spatial location in the image to be detected. It represents the target's specific range in the image's pixel coordinate system, typically defined by the upper left pixel coordinates (x1, y1) and lower right pixel coordinates (x2, y2), intuitively reflecting the target's position and approximate size. The MobileNetV3-small network is a lightweight convolutional neural network backbone structure used to extract basic features from the image to be detected. Through depthwise separable convolution, bottleneck structures, and dynamic activation functions, it significantly improves performance in terms of parameter size, computational efficiency, and real-time adaptability. While reducing model parameters and computational load, it retains efficient feature representation capabilities and is compatible with resource-constrained marine detection equipment. The multi-scale feature fusion detection head is the core module responsible for processing multi-dimensional features and achieving accurate target localization and classification. It receives multi-scale features output by the MobileNetV3-small network and generates the final detection result through feature fusion and parsing. The multi-scale feature image refers to the feature data of different levels extracted layer by layer from the image to be detected by the MobileNetV3-small network. It is used to represent the feature information of the image under different receptive fields, including shallow edge and texture details (small-scale features), mid-level local morphological features (medium-scale features), and deep semantic category features (large-scale features). The multi-branch parallel structure refers to the multiple independent and synchronously running feature processing paths contained within the multi-scale feature fusion detection head. It is used to extract feature information of different dimensions and different receptive fields respectively, and to achieve parallel computing to improve processing efficiency. The target detection feature image refers to the final feature data that integrates multi-dimensional effective features after being processed by the multi-scale feature fusion detection head. It is used to directly support the judgment of target category and the regression calculation of bounding boxes. For example, in underwater search and rescue scenarios, the "diver" output by the detection model is the target category, and the coordinates (210, 150, 380, 460) of the area where the diver is located are the bounding box; the edge texture features (small scale), body contour features (medium scale), and human semantic features (large scale) extracted by the MobileNetV3-small network together constitute a multi-scale feature image.

[0053] Specifically, firstly, the lightweight real-time target detection device converts the preprocessed image to be detected into a tensor format recognizable by the improved detection model. For example, it converts the image from an H×W×C (height×width×channel) pixel matrix into a C×H×W tensor and normalizes the tensor data to ensure that the data range meets the model's input requirements. Then, the tensor data is input into the improved detection model through the computing unit (such as GPU or FPGA) of the lightweight real-time target detection device. The model first uses the MobileNetV3-small network to extract features from the image: the network replaces traditional convolutional layers with depthwise separable convolutional layers, reducing computation while retaining feature extraction capabilities. Combined with the h-swish activation function and the Squeeze-and-Excitation module, it enhances the target features in the ocean image, outputting multi-scale feature images containing information at different scales layer by layer. Next, the multi-scale... The feature image is transmitted to a multi-scale feature fusion detection head, which processes features synchronously through a multi-branch parallel structure: one branch focuses on local detail features through a standard convolutional layer to accurately capture key information such as the edge and texture of the target; the other branch extracts multi-scale spatial features through dilated convolutions with different dilation rates to adapt to the large size differences of targets in marine scenes. The outputs of the two branches are then concatenated along the channel dimension, and feature fusion and dimensionality compression are performed through a 1×1 convolutional layer to generate a target detection feature image that integrates effective multi-dimensional features. Finally, based on the target detection feature image, the detection model calculates the class probability of each target through a classification branch (such as a fully connected layer + softmax function) to determine the target class, predicts the precise coordinates of the target through a regression branch (such as a bounding box regressor), generates a bounding box, and outputs the target class and bounding box information to the display unit or storage unit of the device to complete one target detection operation. Throughout the process, thanks to the lightweight design of the MobileNetV3-small network and the efficient computation of its multi-branch parallel structure, the device is able to keep the detection time of a single frame image within 100 milliseconds while ensuring detection accuracy. This meets the real-time detection requirements of marine scenarios. For example, in unmanned vessel maritime security monitoring, it can quickly identify vessels that have entered restricted areas and output vessel type and location information in real time.

[0054] Optionally, in general, the detection model also includes an attention-based intra-scale feature interaction module. Before the multi-scale feature fusion detection head processes the multi-scale feature image through a multi-branch parallel structure to generate the target detection feature image, the attention-based intra-scale feature interaction module performs interactive enhancement on the multi-scale image features.

[0055] Among them, the attention-based intra-scale feature interaction module refers to a network component that integrates attention mechanisms and cross-scale feature interaction logic. It is used to represent a preprocessing module that can adaptively focus on key features and promote information flow and enhancement between features of different scales. Its core is to achieve selective enhancement and interaction of features through attention weight allocation. Multi-scale image features refer to different levels of feature data extracted layer by layer from the ocean image to be detected by the MobileNetV3-small network. It is used to represent a multi-dimensional feature set containing shallow detail features (such as target edges and textures), mid-level morphological features (such as target local contours), and deep semantic features (such as target category association information). Interaction enhancement refers to the process by which the module allows feature data of different scales to pass information and complement each other through specific operations, while strengthening key features and suppressing redundant information. It is used to represent operations that improve feature correlation and effectiveness.

[0056] For example, in marine life monitoring scenarios, multi-scale image features include the scale texture of small fish (small-scale features), the overall swimming posture of fish (medium-scale features), and the spatial distribution of fish and coral reefs (large-scale features). The attention-based intra-scale feature interaction module identifies the key feature of "fish posture" through an attention mechanism, strengthens the correlation between small-scale texture features and meso-scale posture features, and suppresses redundant information of coral reef background that is unrelated to fish in large-scale features, so that subsequent detection heads can more accurately capture fish target features.

[0057] Optionally, the feature variance value of the background region in the multi-scale feature image can be calculated; the sea state complexity coefficient can be generated based on the feature variance value; and the confidence threshold of the decoder can be dynamically adjusted based on the sea state complexity coefficient when performing non-maximum suppression processing.

[0058] By adopting the above technical solutions, the improved detection model integrates the MobileNetV3-small network with a multi-scale feature fusion detection head, accurately addressing the key pain points of traditional marine target detection: "large environmental interference, unclear visibility" and "limited computing power, unable to run smoothly." The MobileNetV3-small network, as a lightweight backbone network, features small parameter size and high computational efficiency, enabling efficient operation on devices with limited computing resources, such as underwater robots and unmanned vessels. This reduces the detection model's dependence on hardware computing power and solves the problem of traditional deep networks (such as ResNet-50 and Faster R-CNN) being unable to respond in real-time in marine scenes due to high computational demands. Simultaneously, the MobileNetV3-small network extracts multi-scale feature images tailored to the characteristics of marine images. Combined with a multi-branch parallel structure multi-scale feature fusion detection head, it achieves comprehensive coverage of targets of different sizes (such as small fish, large obstacles, and divers), avoiding target misses or false detections caused by single-scale feature extraction. In scenarios with extremely high real-time requirements, such as marine scientific research and underwater search and rescue, this method can quickly output target categories and bounding boxes, ensuring both the timeliness of detection and the comprehensiveness of feature extraction, so that the lightweight model can maintain stable detection performance in complex marine environments.

[0059] The following provides a more detailed description of the process of the method provided in this implementation. Please refer to [link / reference]. Figure 7 This is another flowchart illustrating the lightweight real-time target detection method based on the marine environment in this application embodiment.

[0060] After inputting the image to be detected into the improved detection model to obtain the target category and bounding box in the image, the following steps can be performed, or not, and are not limited here: S701. Determine the target tracking trajectory based on the historical detected targets and historical bounding boxes in historical frames.

[0061] Here, a historical frame refers to a continuous sequence of images that have been detected before the current image to be detected, used to represent historical image data recording the motion state of the target; a historical detected target refers to a target instance in a historical frame that has been successfully identified by the detection model, used to represent a historical target with a defined category label; a historical bounding box refers to the position coordinate information of the target in a historical frame, used to represent the target's spatial position data recorded in pixel coordinates; and a target tracking trajectory refers to the motion trajectory obtained by analyzing the position changes of the historically detected target in consecutive historical frames, used to represent a data sequence describing the motion pattern of the target in the time dimension.

[0062] Specifically, the lightweight real-time target detection device first reads a preset number of historical frames (e.g., the first 30 frames) and extracts the historical detected targets and their corresponding historical bounding boxes from each frame. For each historical detected target, the device analyzes its bounding box coordinate sequence in consecutive historical frames and determines the target's motion direction and velocity by calculating the displacement vector of the center point of the bounding boxes between adjacent frames. Based on these motion characteristics, the device constructs a target tracking trajectory and records the target's motion state information, providing a basis for subsequent trajectory prediction and matching. For example, for a ship sailing on the sea, by analyzing its position changes in historical frames, a tracking trajectory representing the ship's sailing path can be obtained.

[0063] S702. Based on the target tracking trajectory, predict the predicted bounding box of the current frame.

[0064] The predicted bounding box refers to the region in the current frame where the target may appear, calculated by extrapolating the target tracking trajectory. It represents the model's prediction of the target's current position. The current frame refers to the image currently being detected, representing the latest image data that needs to be associated with historical trajectories.

[0065] Specifically, lightweight real-time target detection equipment uses motion prediction algorithms (such as Kalman filters) to predict the target's position in the current frame based on an established target tracking trajectory. The prediction process considers the target's motion characteristics exhibited in its historical trajectory, such as velocity, acceleration, and direction of motion. A mathematical model is used to calculate the most likely location region of the target, generating a predicted bounding box. The prediction result includes the coordinates of the bounding box's center point and the box's size, providing prior knowledge for subsequent trajectory matching. For example, for a submersible moving at a constant speed, its position in the current frame can be accurately predicted based on its historical trajectory.

[0066] S703. When the target detected in the historical frame is of the same category as the target in the image to be detected, the bounding box and the predicted bounding box are matched.

[0067] Among them, bounding box matching refers to the process of spatially associating the actual bounding box detected by the detection model in the current frame with the predicted bounding box, which is used to represent the operation of verifying the accuracy of the prediction result; same category means that the historical detection target and the current detection target belong to the same category label, which is used to represent the condition of ensuring that the target type is consistent across frames.

[0068] Specifically, the lightweight real-time object detection device first verifies category consistency, ensuring that historically detected targets belong to the same category as those detected in the current frame. After achieving category consistency, the device calculates a positional deviation metric (based on the Euclidean distance between the center point) and a shape overlap metric (based on the intersection-union ratio of the boxes) between the predicted bounding box and the actual bounding box. These two metrics are then used to determine if the bounding boxes match. When both metrics meet preset thresholds, a successful match is considered achieved, confirming a correlation between the currently detected target and the historical trajectory.

[0069] S704. If the bounding box fails to match, initialize the trajectory to be confirmed.

[0070] Among them, the trajectory to be confirmed refers to the newly detected target motion trajectory whose reliability has not yet been verified. It is used to represent temporary trajectory data that needs to be verified through multiple subsequent frames to determine whether it is a real trajectory. Initialization refers to the process of creating a new trajectory record and setting the initial state. It is used to represent the operation of starting to track a newly detected target.

[0071] Specifically, when the lightweight real-time target detection device detects a target in the current frame that cannot be matched with any existing tracking trajectory, it indicates that a new target may have appeared. At this time, the lightweight real-time target detection device creates a new trajectory to be confirmed for that target, records its initial position, category, and other information, and marks its status as "pending confirmation." This trajectory to be confirmed will undergo reliability verification in subsequent frames. For example, when a new floating object suddenly appears on the sea surface, the lightweight real-time target detection device will initialize a trajectory to be confirmed for it.

[0072] S705. If the trajectory to be confirmed is successfully matched in a preset number of consecutive frames in the future, the trajectory to be confirmed will be added to the target tracking trajectory.

[0073] Among them, the preset number of frames refers to the number of consecutive image frames used to verify the reliability of the trajectory to be confirmed, and is used to represent the length of the time window for judging the authenticity of the trajectory; a successful match means that a detection target that matches the trajectory prediction can be found in each frame, and is used to represent the state in which the trajectory prediction and the actual detection result are continuously consistent.

[0074] Specifically, the lightweight real-time target detection device tracks the trajectory to be confirmed in a preset number of images (e.g., 5 consecutive frames). If a target matching the predicted trajectory location can be detected in each of these frames, and the target category remains consistent, the trajectory is considered reliable. At this point, the lightweight real-time target detection device updates the status of this trajectory to "confirmed" and officially adds it to the target tracking trajectory set, starting regular tracking.

[0075] S706. If the trajectory to be confirmed is lost or fails to match within a preset number of consecutive frames in the future, the trajectory to be confirmed is determined to be a false trajectory.

[0076] Trajectory loss refers to the situation where no relevant target can be detected in a certain frame, which indicates that the target has disappeared from the image; false trajectory refers to the unreal target trajectory generated due to detection errors, environmental interference, etc., which indicates that erroneous trajectory data needs to be filtered out; unmatched means that the detection result is seriously inconsistent with the trajectory prediction, which indicates that the trajectory prediction and the actual detection result cannot establish a reliable association.

[0077] Specifically, when the lightweight real-time target detection device is tracking a trajectory to be confirmed, if any of the following situations occur: the target is not detected at all in a certain frame (trajectory loss), or the detected target position deviates significantly from the trajectory prediction (match unsuccessful), and this state persists for a preset number of frames, the lightweight real-time target detection device determines this to be a false trajectory. Such false trajectories may be caused by false detections due to environmental interference such as wave reflections or plankton. The lightweight real-time target detection device will mark this trajectory as "false" and delete it from the tracking system to prevent it from affecting subsequent target tracking.

[0078] Optionally, under normal circumstances, when the historical detected target in the historical frame is of the same category as the target in the image to be detected, the matching of the bounding box and the predicted bounding box can be achieved in the following way, without limitation: When the historical detected target in the historical frame is of the same category as the target in the image to be detected, calculate the positional deviation metric and the shape overlap metric of the bounding box and the predicted bounding box. The positional deviation metric is calculated based on the Euclidean distance between the center point of the bounding box and the center point of the predicted bounding box, and the shape overlap metric is calculated based on the intersection-union ratio between the bounding box and the predicted bounding box; based on the positional deviation metric and the shape overlap metric, determine the matching status of the bounding box and the predicted bounding box.

[0079] The lightweight real-time target detection device in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference]. Figure 8 This is a schematic diagram of the physical structure of a lightweight real-time target detection device in the embodiments of this application.

[0080] It should be noted that, Figure 8 The structure of the lightweight real-time target detection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0081] like Figure 8As shown, the lightweight real-time target detection device includes a CPU 801, which can perform various appropriate actions and processes according to a program stored in the read-only memory ROM 802 or a program loaded from the storage section 808 into the random access memory RAM 803, such as performing the methods described in the above embodiments. The RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus 804. An I / O interface 805 is also connected to the bus 804.

[0082] The following components are connected to I / O interface 805: input section 806 including audio input devices, push-button switches, etc.; output section 807 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 808 including a hard disk, etc.; and communication section 809 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 809 performs communication processing via a network such as the Internet. Drive 810 is also connected to I / O interface 805 as needed. Removable media 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 810 as needed so that computer programs read from them can be installed into storage section 808 as needed.

[0083] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by CPU 801, it performs the various functions defined in the present invention.

[0084] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0086] Specifically, the lightweight real-time target detection device in this embodiment includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, it implements the lightweight real-time target detection method based on the marine environment provided in the above embodiment.

[0087] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the lightweight real-time target detection device described in the above embodiments; or it may exist independently and not assembled into the lightweight real-time target detection device. The storage medium carries one or more computer programs, which, when executed by a processor of the lightweight real-time target detection device, cause the lightweight real-time target detection device to implement the lightweight real-time target detection method based on the marine environment provided in the above embodiments.

[0088] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0089] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0090] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A lightweight real-time target detection method based on marine environment, characterized in that, The method, applied to a lightweight real-time target detection device, includes: Acquire images of the marine environment to be inspected; The image to be detected is input into the improved detection model to obtain the target category and bounding box in the image to be detected. The detection model includes a MobileNetV3-small network and a multi-scale feature fusion detection head. The MobileNetV3-small network extracts multi-scale feature images based on the image to be detected. The multi-scale feature fusion detection head processes the multi-scale feature images through a multi-branch parallel structure to generate target detection feature images.

2. The method according to claim 1, characterized in that, The MobileNetV3-small network includes an h-swish activation function and a Squeeze-and-Excitation module. The h-swish activation function performs a nonlinear mapping operation on the image to be detected based on a piecewise linear function. The Squeeze-and-Excitation module performs a global pooling operation on the image to be detected to obtain global spatial information, and generates weight coefficients for each channel through a fully connected layer. The weight coefficients are then multiplied by the image to be detected channel by channel.

3. The method according to claim 1, characterized in that, The multi-scale feature fusion detection head processes the multi-scale feature image through a multi-branch parallel structure to generate a target detection feature image, specifically including: The multi-scale feature image is transformed by mapping the channel dimension using a 1×1 convolutional layer to obtain an intermediate feature image; The intermediate feature images are input into the first processing path and the second processing path, respectively. In the first processing path, a standard convolutional layer is used to calculate the intermediate feature image to extract local detail features, and the first path output is obtained. In the second processing path, multi-scale spatial features are extracted using a dilated reparameterization unit that includes multiple convolutional branches with different dilation rates, and the output of the second path is obtained. The output of the first path and the output of the second path are concatenated along the channel dimension to obtain the concatenated features; The concatenated features are fused and compressed using a 1×1 convolutional layer to output the target detection feature image.

4. The method according to claim 1, characterized in that, The detection model also includes an attention-based intra-scale feature interaction module. Before the multi-scale feature fusion detection head processes the multi-scale feature image through a multi-branch parallel structure to generate the target detection feature image, the attention-based intra-scale feature interaction module performs interactive enhancement on the multi-scale image features.

5. The method according to claim 1, characterized in that, The method further includes: Calculate the feature variance value of the background region in the multi-scale feature image; Based on the aforementioned feature variance values, a sea state complexity coefficient is generated; Based on the sea state complexity coefficient, the confidence threshold of the decoder is dynamically adjusted when performing non-maximum suppression processing.

6. The method according to claim 1, characterized in that, After the step of inputting the image to be detected into the improved detection model to obtain the target category and bounding box in the image to be detected, the method further includes: Based on the historical detected targets and historical bounding boxes in historical frames, the target tracking trajectory is determined; Based on the target tracking trajectory, predict the bounding box of the current frame; When the historical detection target in the historical frame is the same as the target category in the image to be detected, the bounding box and the predicted bounding box are matched; If the bounding box fails to match, initialize the trajectory to be confirmed. If the trajectory to be confirmed is successfully matched in a preset number of consecutive frames in the future, then the trajectory to be confirmed will be added to the target tracking trajectory; If the trajectory to be confirmed is lost or fails to match within a predetermined number of consecutive frames, then the trajectory to be confirmed is determined to be a false trajectory.

7. The method according to claim 6, characterized in that, When the historical detected target in the historical frame has the same target category as the target in the image to be detected, matching the bounding box and the predicted bounding box specifically includes: When the historical detected target in the historical frame is of the same category as the target in the image to be detected, the positional deviation metric and the shape overlap metric of the bounding box and the predicted bounding box are calculated. The positional deviation metric is calculated based on the Euclidean distance between the center point of the bounding box and the center point of the predicted bounding box, and the shape overlap metric is calculated based on the intersection-union ratio between the bounding box and the predicted bounding box. Based on the positional deviation metric and the shape overlap metric, the matching status between the bounding box and the predicted bounding box is determined.

8. A lightweight real-time target detection device, characterized in that, The lightweight real-time target detection device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the lightweight real-time target detection device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on a lightweight real-time target detection device, the lightweight real-time target detection device performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on a lightweight real-time target detection device, the lightweight real-time target detection device performs the method as described in any one of claims 1-7.

Images