Multi-target tracking method for seaborne rain and fog and jittering environment

Abstract

The invention discloses a multi-target tracking method for an offshore rain, fog and jitter environment, and the method comprises the steps: obtaining multi-target image data in the offshore rain, fog and jitter environment, and carrying out the target cutting and fusion of the multi-target image data based on a target enhancement segmentation strategy, and obtaining an enhanced sample image; mapping the enhanced sample image and target features in the multi-target image data into a candidate frame set of a pixel scale, and obtaining a sample data set containing adaptive Anchors according to the candidate frame set based on a clustering algorithm; performing model training on the multi-target detection network through the sample data set to obtain an optimal detection model so as to realize target detection; and defining a target state vector and a target observation vector according to a detection result, and realizing multi-target tracking under the marine rain and fog and jitter environment based on an improved Kalman filtering algorithm. The problems that in the prior art, the precision of multi-target detection under the marine rain and fog and jittering environment is insufficient, and a systematic solution for multi-target tracking under the marine rain and fog environment and the jittering scene is lacked are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection and tracking, and in particular to a multi-target tracking method for rain and fog and shaking environments at sea. BACKGROUND

[0002] The present application provides an innovative solution to the technical bottleneck of multi-target tracking in rain and fog and shaking conditions at sea, which is prone to discontinuous track recognition, ID hopping and high latency. Currently, traditional methods are limited by single-scale feature extraction and fixed noise models. On the one hand, remote targets at sea are often extremely small in scale, and are affected by waves, reflections, rain and fog, and other disturbances to the experimental platform. Traditional detection models have problems such as insufficient recall rate and decreased precision in such scenarios. On the other hand, due to the shaking of the shooting platform and the vertical fluctuation of the target itself, traditional Kalman filters based on fixed noise assumptions are difficult to adapt to drastic changes, often leading to target drift, loss and ID identity hopping. On a real sea state unmanned boat platform, the shaking of the shooting platform and the irregular motion of the target will cause the video tracking to be prone to drift, loss and identity switching. At the same time, under multi-target conditions, "occlusion and reappearance" and data association of dense targets occur simultaneously, and once the track is unstable, it will contaminate the downstream in the opposite direction, forming a cascade error of "perception - decision - control". Therefore, although existing research has improved in the aspects of lightweight detection or data association, it generally lacks a systematic solution to multi-target tracking in rain and fog environments and shaking scenarios at sea. SUMMARY

[0003] The present application provides a multi-target tracking method for rain and fog and shaking environments at sea to overcome the above technical problems.

[0004] To achieve the above purpose, the technical solution of the present application is: A multi-target tracking method for rain and fog and shaking environments at sea, specifically comprising the steps of: S1: obtaining multi-target image data in rain and fog and shaking environments at sea; and the multi-target image data includes frame images and corresponding YOLO labels; the YOLO labels at least include target type and anchor frame; S2: performing target cropping and fusion processing on the multi-target image data based on a target enhancement segmentation strategy to obtain enhanced sample images; S3: mapping the enhanced sample images and target features in the multi-target image data to a candidate frame set in pixel scale, and based on The clustering algorithm obtains a sample data set containing adaptive Anchors from the candidate frame set; S4: Construct a multi-target detection network YOLOv7Tiny-Lites, which includes an input module, a lightweight backbone network, a multi-scale fusion neck network, a detection network, and an output module; the input module is used to input sample images in a sample data set to the lightweight backbone network; the lightweight backbone network is used to extract target pixel features in the sample images and obtain multi-scale pixel feature maps; the multi-scale fusion neck network is used to fuse the multi-scale pixel feature maps and obtain multi-scale fusion feature maps; the detection network is used to obtain detection results according to the multi-scale fusion feature maps and output the detection results through the output module; S5: Model training is performed on the multi-target detection network YOLOv7Tiny-Lites through a sample data set to obtain an optimal detection model, and target detection in a sea rain and fog and shaking environment is realized according to the optimal detection model; S6: Target state vectors and target observation vectors are defined according to the detection results, and multi-target tracking in a sea rain and fog and shaking environment is realized based on an improved Kalman filtering algorithm.

[0005] Further, the target enhancement segmentation strategy in S2 specifically includes the following steps: S21: Based on a target screening rule, target sample feature maps that meet a scale measurement in the multi-target image data are screened; and the scale measurement is a normalized minimum side of a target width and height; And the expression of the target screening rule is

[0006] In the formula: indicates screening of the scale measurement; indicates the width and height of the target, respectively; S22: Based on a preset pixel coordinate system, a target image of the target sample feature map is cropped to obtain target silhouette data; And the expression for obtaining the target silhouette data is

[0007] In the formula: , indicates the pixel coordinates of the target center; , indicates the width and height of the original image pixel frame; , , respectively, , indicate the diagonal endpoint coordinates of the cropped pixel frame; S23: Linear enhancement processing is performed on the silhouette pixels in the target silhouette data to obtain a linear enhancement image, and the expression for performing linear enhancement processing on the silhouette pixels in the target silhouette data is

[0008] In the formula: represents a silhouette pixel in the target silhouette data; represents a pixel after linear enhancement processing; S24: At the center of the pixel of the linear enhancement image, a scale perturbation that maintains the aspect ratio of the image is applied to obtain a perturbed image; and a feathering fusion operation is performed on the perturbed image to obtain a fused image; and the expression of performing the feathering fusion operation on the perturbed image is

[0009]

[0010]

[0011] In the formula: represents a feathering weight; represents a distance field of a linear enhancement image pixel to a set boundary calculated on a preset silhouette mask domain; represents a binary mask domain of a silhouette, i.e., a pixel set; represents a maximum value of the distance field in the binary mask domain; represents a background pixel of the linear enhancement image; represents a fusion result; S25: After the silhouette pasting on the fused image, a YOLO normalized label is obtained, and the formula for obtaining the YOLO normalized label is

[0012]

[0013] In the formula: , respectively represent the pixel width and height before the silhouette pasting; represents a pasting scale; , respectively represent the pixel width and height after the silhouette pasting; , represents the pixel center coordinate of the silhouette pasting; , represents the center coordinate of the YOLO normalized frame; , represents the width and height of the YOLO normalized frame; The fused image and the corresponding YOLO normalized label thereof are taken as an enhanced sample image.

[0014] Furthermore, S3 specifically includes the following steps: S31: Based on the preset expected input size, map the target features in the enhanced sample image and multi-target image data into a set of candidate boxes at the pixel scale; S32: Using the preset lower limit of the pixel box min_box as the filtering index, filter out abnormal candidate boxes in the candidate box set that are smaller than the filtering index, and obtain the set of labeled boxes for clustering. S33: Given the number of anchor frames k, with Perform similarity metric on the set of bounding boxes Clustering operation to obtain the anchor box collection class; And the expression for retrieving the anchor box collection class is:

[0015]

[0016] In the formula: Represents the set of anchor frames; This represents the total number of bounding boxes participating in the clustering. This represents the width and height pair of the bounding box corresponding to the nth sample image in the pixel domain or uniform scale domain. , Indicates correspondence Width and height; express A specific anchor box within the collection, , Indicates the width and height of the anchor frame; This represents the intersection-union ratio of the annotation box b and the anchor box a when they are aligned with the same central axis, i.e., the ratio of the intersection area to the union area; Based on the anchor box set class, different scales of anchors are set to obtain the sample dataset for enhanced sample images and multi-target image data.

[0017] Furthermore, the lightweight backbone network described in S4 includes a first network connected in sequence. Layer, First C3Ghost Layer, First Layer, second C3Ghost layer, second Layer, third C3Ghost layer, second Layer 4, the fourth C3Ghost layer, and the SPPF module; The first The layer is used to perform depthwise convolution operations on sample images in the sample dataset; The first C3Ghost layer is used for the first The layer's output performs pixel feature extraction; the first Used to perform a convolution operation on the output of the first C3Ghost layer to obtain a pixel feature map at the first scale; The second C3Ghost layer is used to perform pixel feature extraction operations on the first-scale feature map; the second The layer is used to perform a convolution operation on the output of the second C3Ghost layer to obtain the second-scale pixel feature map; The third C3Ghost layer is used to perform pixel feature extraction operations on the second-scale feature map. The fourth C3Ghost layer is used to perform a depthwise convolution operation on the output of the third C3Ghost layer. The output of the layer performs pixel feature extraction; the SPPF module is used to perform pooling operation on the output of the fourth C3Ghost layer to obtain the third-scale pixel feature map.

[0018] Furthermore, the multi-scale fusion neck network described in S4 includes a third network connected in sequence. Layer, Fourth Layer, Fifth Layer, First Upsampling Layer, First Stitching Layer, Fifth C3Ghost Layer, Second Upsampling Layer, Second Stitching Layer, Sixth C3Ghost Layer, Sixth Layer, third splicing layer, seventh C3Ghost layer, seventh The fourth splicing layer and the eighth C3Ghost layer; The third The layer is used to perform convolution operations on the third-scale pixel feature map; The fourth The layer is used for the third The output of the layer and the second The output of the fifth layer performs a convolution operation; The layer is used for the fourth The output of the layer and the first The layer's output performs a convolution operation; The first upsampling layer is used for the third The output of the fifth layer The output of the first upsampling layer is upsampled; the first concatenation layer is used to concatenate the output of the first upsampling layer; the fifth C3Ghost layer is used to perform a depthwise convolution operation on the output of the first concatenation layer; the second upsampling layer is used to upsample the output of the fifth C3Ghost layer; the second concatenation layer is used to concatenate the output of the second upsampling layer with the output of the fourth C3Ghost layer. The output of the layer is used to perform a concatenation operation; the sixth C3Ghost layer is used to perform a depthwise convolution operation on the second concatenation layer to obtain a first-scale fused feature map; The sixth The layer is used for the third The output of the third layer is convolved with the feature map fused with the first scale; the third concatenation layer is used to process the sixth layer. The output of the third layer is used to perform a concatenation operation; the seventh C3Ghost layer is used to perform a depthwise convolution operation on the output of the third concatenation layer to obtain the second-scale fused feature map; The seventh The fourth concatenation layer is used to perform convolution operations on the second-scale fused feature map; the fifth concatenation layer is used to perform convolution operations on the seventh-scale fused feature map. The output of the fourth concatenation layer is concatenated with the output of the fifth C3Ghost layer; the eighth C3Ghost layer is used to perform a depthwise convolution operation on the output of the fourth concatenation layer to obtain a third-scale fused feature map.

[0019] Furthermore, the method for obtaining the optimal detection model in S5 specifically includes the following steps: S51: Randomly divide the sample dataset into training and validation sets according to a preset ratio; S52: Train the YOLOv7Tiny-Lites multi-object detection network using the training set to obtain the trained YOLOv7Tiny-Lites multi-object detection network. S53: Based on the loss function, the trained multi-object detection network YOLOv7Tiny-Lites is validated using the validation set; and the loss function includes, but is not limited to, any one of the cross-entropy loss function and the mean squared error function; That is, to determine whether the output of the trained multi-object detection network YOLOv7Tiny-Lites has converged; If the output of the trained multi-object detection network YOLOv7Tiny-Lites converges, then the trained multi-object detection network YOLOv7Tiny-Lites is confirmed to be the optimal detection model. Otherwise, the parameters of the trained multi-object detection network YOLOv7Tiny-Lites are adaptively adjusted based on the backpropagation algorithm, and step S52 is repeated until it is confirmed that the parameters of the trained multi-object detection network YOLOv7Tiny-Lites with converged output are the optimal weight parameters, and the multi-object detection network YOLOv7Tiny-Lites is reconstructed to obtain the optimal detection model.

[0020] Furthermore, S6 specifically includes the following steps: S61: Define the detection results obtained by the detection network as and =( , , , , ,class), where , Indicates the coordinates of the target center; , Indicates the width and height of the detection anchor frame; The score represents the detection confidence level, i.e., the score of the detected anchor box; class represents the category of the detected target. S62: Define the target state vector based on the detection results. and With target observation vector and ;in Represents the coordinates of the target center , The result after normalization; The planar velocity representing the center position of the target; This indicates the rate of change of the width and height of the detected anchor frame; Indicates transpose; S63: Based on the target state vector Define the state prediction equation; And the state prediction equation: ; in, Represents the target state vector Prior state estimation; Represents the state transition matrix; Represents the target state vector Posterior state estimation; And obtain the residual equation based on the state prediction equation: ; in, The residual represents the difference between the actual measurement and the predicted measurement. express The target observation vector at time; S64: Obtain the target state estimation equation and initial covariance matrix based on the state prediction equation and residual equation. Their expressions are as follows:

[0021]

[0022]

[0023]

[0024] In the formula: express Posterior state estimation at time t; Indicates Kalman gain; Represents the prior covariance; Represents the process noise covariance; Indicates the posterior covariance; Represents the observation matrix; The covariance represents the uncertainty of the expected residuals; Indicates the measurement noise covariance; Indicates transpose; The output of the target state estimation equation is used as the state estimate of the target motion in the current iteration. S65: Obtain the updated covariance matrix based on the initial covariance matrix;

[0025] In the formula: This represents the updated covariance matrix; Represents the identity matrix; S66: Construct a Kalman filter update strategy, which includes a measurement noise covariance update mechanism and a process noise covariance update mechanism for target state estimation at the next time step. And the expression for the measurement noise covariance update mechanism is:

[0026] In the formula: This indicates the detection confidence level and the score ∈ [0,1]. k s Represents the fractional sensitivity coefficient; The expression for the process noise covariance update mechanism is as follows:

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] In the formula: Indicates phase conflict indication quantity; A general symbol representing indicator functions; express The change in; express -1 is the longitudinal velocity state component of the target; This indicates that a threshold value has been set. express Detection confidence at time -1; Indicates the velocity retention factor; express The minimum value; express base value and ≤ ≤1; This represents the measurement noise covariance corresponding to the longitudinal position dimension of the target; Indicates the phase lift factor; This represents the covariance of the corresponding vertical position dimension of the target; This represents the observation matrix corresponding to the longitudinal position dimension of the target; This indicates the longitudinal process covariance sub-block used for prediction in the next frame; Indicates the standard deviation multiple threshold; Residual The longitudinal component; Indicates the noise amplification factor; This represents the longitudinal process covariance sub-block used for prediction in the current frame; Indicates the magnification factor and ∈[1.5,3]; Indicates the fallback factor and ∈[0.8,0.95]; S67: Based on the updated covariance matrix combined with the Kalman filter update strategy, obtain the target state estimate for the next time step; S68: The target state estimate and updated covariance matrix at the next moment are used as inputs to the preset Kalman filter module again, and steps S63 to S67 are repeated to achieve multi-target tracking in marine rain, fog and shaking environments.

[0033] Beneficial effects: This invention provides a multi-target tracking method for marine rain, fog and shaking environments. By addressing the core pain point of "multi-target tracking under marine rain, fog and shaking conditions", it achieves higher recall target detection and stable multi-target tracking with anti-oscillation by optimizing data distribution, network structure and Kalman filtering, which significantly improves the perception performance of unmanned ships under marine rain, fog and shaking conditions. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1This is a flowchart of the multi-target tracking method for marine rain, fog, and shaking environments according to the present invention; Figure 2 This is a schematic diagram of the structure of the YOLOv7Tiny-Lites multi-target detection network constructed in this embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This embodiment provides a multi-target tracking method for marine rain, fog, and shaking environments, such as... Figure 1 As shown, the specific steps include: S1: Acquire multi-target image data under marine rain, fog, and shaking conditions; and the multi-target image data includes frame images and corresponding YOLO tags; the YOLO tags at least include target type and anchor box; S2: Based on the target enhancement segmentation strategy, target cropping and fusion processing is performed on the multi-target image data to obtain enhanced sample images; In this embodiment, the target enhancement segmentation strategy takes the original image and its YOLO label as input. First, the label is parsed image by image and the target image is automatically selected according to the scale threshold based on the target selection rules. The selected target image is then precisely cropped using its position in the pixel coordinate system to construct a "target silhouette library". Then, a closed-loop process of "visibility enhancement, flexible feathering pasting, label recalculation and batch output" is executed. This systematically improves the positive sample density and scale coverage of the P2 high-resolution branch while maintaining the consistency of background lighting and tone. The target enhancement segmentation strategy is specifically as follows: S21: Based on the target selection rules, filter the target sample feature maps in the multi-target image data that satisfy the scale metric; and the scale metric is the normalized minimum side of the target's width and height. And the expression of the target filtering rule is: (1) In the formula: Indicates the filter range and in (The range can be set by the user) The target will be automatically selected within the range; These represent the width and height of the target, respectively. S22: Based on a preset pixel coordinate system, crop the target image of the target sample feature map to obtain target silhouette data; And the expression for obtaining the target silhouette data is: (2) In the formula: , Represents the pixel coordinates of the target center; , This represents the width and height of the original image's pixel frame; , ),( , ) represent the coordinates of the diagonal endpoints of the cropped pixel box; S23: In this embodiment, the foreground silhouette obtained by cropping often has insufficient contrast under imaging conditions such as weak texture and hazy diffusion. Therefore, linear enhancement processing is performed on the silhouette pixels in the target silhouette data to obtain a linearly enhanced image to improve visibility. The expression for performing linear enhancement processing on the silhouette pixels in the target silhouette data is as follows: (3) In the formula: This represents the silhouette pixels in the target silhouette data; This represents the pixel after linear enhancement processing; and Indicate design parameters and , ; S24: At the pixel center of the linearly enhanced image, apply a scale perturbation that maintains the image aspect ratio to obtain a perturbed image; and perform a feathering fusion operation on the perturbed image to obtain a fused image; In this embodiment, the enhanced silhouette samples are included in the "reusable target library" for use in the post-compositing stage. During compositing, the system performs joint sampling of the background image and silhouette samples, using a preset "lower half of the image priority (probability 0.8) + global random" point distribution method, and applies a scale perturbation that maintains the aspect ratio to each silhouette. Scale perturbation is first performed at the pixel center (u,v). To avoid jagged edges and color block artifacts caused by hard pasting, this embodiment adopts a soft paste mechanism: first, the distance field from the pixel to the boundary is calculated on the silhouette mask domain. Then, based on this, a feathering weight that decreases with the edge is generated; The expression for performing feathering and blending operations on the perturbed image is: (4) (5) (6) In the formula: Indicates feathering weight; This represents the distance field from the linearly enhanced image pixels to a set boundary calculated over a preset silhouette mask domain; This represents the maximum value of the range field within the binary mask domain; The background pixel represents the linearly enhanced image pixel; Indicates the fusion result; The binary mask domain representing the silhouette is the set of pixels, i.e., mask={(x,y)|M(x,y)=1}). In this embodiment, continuous feathering is used to form a natural brightness and texture transition at the pasted edge, which can significantly reduce compositing artifacts. Each background image has 1-3 targets pasted by default (adjustable parameter), which significantly improves the spatial distribution density and scale diversity of small targets. S25: After pasting the silhouette of the fused image, obtain the YOLO normalized label; In this embodiment, at the label level, in order to keep the subsequent YOLO training and evaluation process unchanged, the YOLO normalized label of each pasted target, i.e., the fused image, is recalculated. And the formula for obtaining YOLO normalized labels is: (7) (8) In the formula: , These represent the pixel width and height before the silhouette is pasted; Indicates the pasting dimensions; , These represent the pixel width and height of the pasted silhouette; , This indicates the pixel center coordinates of the silhouette pasted; , This represents the center coordinates of the YOLO normalized bounding box; , This represents the width and height of the YOLO normalized bounding box; The fused image and its corresponding YOLO normalized label are used as augmentation sample images.

[0038] In this embodiment, all fused images and their YOLO normalized annotations are output to output / images and output / labels in a one-to-one correspondence, and the total number of copies (default 1000), the single image pasting limit (per-image, default 3), and the filtering threshold (min) are preset. / --max The method adjusts operating parameters to adapt to different resolutions and sea state transparency. Compared with conventional enhancements such as geometric transformation and random pruning, this embodiment uses "target-level screening + continuous feathering fusion + native label docking" as the core path. While maintaining the consistency of real background lighting and color tone on the sea surface, it significantly expands the effective positive sample size of targets and is directly compatible with the YOLO training framework. It has been verified that it can improve recall and overall mAP in small-scale target detection tasks such as distant ships, buoys and floating objects, providing reliable data layer support for the detection subsystem, and can be used to complement the backend adaptive Kalman tracking to build an integrated solution for intelligent maritime perception. Compared with conventional geometric transformations and random cropping enhancement methods, the target copy-paste method in this embodiment has three significant advantages: First, through target-level screening and feathering fusion mechanisms of specific sizes, it can maintain the consistency of lighting and tone of the real sea surface background, avoiding color block artifacts caused by simple superposition; Second, it automatically expands the spatial distribution density of the target, enabling the detection model to multiply the number of positive samples in the P2 high-resolution branch; Third, the synthesis process maintains the original label format unchanged, directly compatible with the YOLO series training framework, without the need for additional conversion.

[0039] S3: Map the target features in the enhanced sample image and multi-object image data to a set of candidate boxes at the pixel scale, and based on... Clustering algorithms obtain a sample dataset containing adaptive anchors based on the candidate box set, specifically including the following steps: S31: Based on the preset expected input size, map the target features in the enhanced sample image and multi-target image data into a set of candidate boxes at the pixel scale; And the expression for mapping the target feature to a set of candidate boxes at the pixel scale is: (9) In the formula: This represents the input size received by the network, used to map the normalized width and height (w,h) to pixel scale (W,H); S32: Using the preset lower limit of the pixel box min_box as the filtering index, filter out abnormal candidate boxes in the candidate box set that are smaller than the filtering index, and obtain the set of labeled boxes for clustering. S33: Given the number of anchor frames k, with Perform similarity metric on the set of bounding boxes Clustering operation maximizes the best match of all bounding boxes to a given anchor box. The average value of the dataset is used to obtain the anchor box collection class; And the expression for retrieving the anchor box collection class is: (10) (11) In the formula: Represents the set of anchor frames; This represents the total number of bounding boxes participating in the clustering. This represents the width and height pair of the bounding box corresponding to the nth sample image in the pixel domain or uniform scale domain. , Indicates correspondence Width and height; express A specific anchor box within the collection, , Indicates the width and height of the anchor frame; This represents the intersection-union ratio (IUU) of the annotation box b and the anchor box a when they are aligned with the same central axis, i.e., the ratio of the intersection area to the union area. In this embodiment, after obtaining the anchor box set class, the obtained cluster centers are subjected to random scale perturbation (0.8–1.2 times, adjustable parameter) and the process is repeated. The clustering process further improves the best match of all bounding boxes to a given anchor box. The average value of the dataset is used to improve the clustering accuracy of anchor box set classes; Based on the anchor box set class, different scales of anchors are set to obtain the sample dataset for enhanced sample images and multi-target image data.

[0040] Specifically, in this embodiment, the final calculation is based on the anchor frame area. After sorting the anchor box set in ascending order and rearranging it into a 3×3 three-layer, three-group format, it is mapped to the stride of the detection heads P2 / P3 / P4. During the export phase, the Anchors array is directly printed according to the YAML syntax required by the training framework. During inference, the scale is calculated by the detection head according to the stride S, and its expression is:

[0041] This matches the sampling density of the three-scale feature map, ensuring that the target preferentially falls on the high-resolution branch. The above process and parameters (-- --k, --min_box, --layers, --per_layer) are all explicitly implemented and validated in the pre-defined program: default input scale =1280, total number of anchor boxes k=9, number of layers layers=3 and per_layer=3 for each layer, and force constraint k=layers×per_layer k to ensure that the exported structure can be directly used by the three-head detector, pixel lower limit min_box=2 to remove noise annotations. The whole set of pre-set programs can make the anchor boxes fit the true scale distribution of the current dataset in the prior without changing the detection network, significantly improving the positive sample matching rate and training convergence speed of the target under rain and fog conditions; In this embodiment, normalized width and height (w, h) are extracted from the YOLO format labels in the training set, and combined with the expected input size. Mapping to a pixel-scale candidate set and setting a minimum pixel threshold to remove abnormally small bounding boxes. Then, using... K-means++ clustering for similarity learns anchor box centers, resulting in more stable initialization and faster convergence. The clustering objective is to maximize the average value of all samples to their "best-fitting" anchor boxes. This process allows the prior data to better reflect the true scale distribution of the data. After obtaining the initial anchor boxes, a light random perturbation (such as scaling by 0.8–1.2 times, which can be set by the user) is used for global improvement, and the best ones are retained. Finally, the anchor boxes are rearranged into three layers and three groups (corresponding to P2 (first scale fusion feature map) / P3 (second scale fusion feature map) / P4 (third scale fusion feature map)) in ascending order of area. During export, the sampling density of the feature maps is aligned with the step size S of the detection head. Through this process, the anchor box density corresponding to the high-resolution branch (P2) is significantly improved, which simultaneously enhances the positive sample matching rate and recall ability of the target boxes during the training phase, laying a data layer foundation for online detection performance.

[0042] S4: Construct the multi-object detection network YOLOv7Tiny-Lites, such as Figure 2 As shown, it includes an input module, a lightweight backbone network, a multi-scale fusion neck network, a detection network, and an output module. The input module is used to input sample images from the sample dataset into the lightweight backbone network. The lightweight backbone network is used to extract target pixel features from the sample images to obtain multi-scale pixel feature maps. The multi-scale fusion neck network is used to fuse multi-scale pixel feature maps to obtain multi-scale fusion feature maps. Specifically, the lightweight backbone network includes a first... connected in sequence Layer 64x3x1, First C3Ghost layer 64x1, First Layer 128x3x2, Second C3Ghost layer 128x2, Second Layer 256x3x2, third C3Ghost layer 256x1, second Layer 512x3x2, fourth C3Ghost layer 512x1, and SPPF module 128x1x1; The first The layer is used to perform depthwise convolution operations on sample images in the sample dataset; The first C3Ghost layer is used for the first The layer's output performs pixel feature extraction; the first Used to perform a convolution operation on the output of the first C3Ghost layer to obtain a pixel feature map at the first scale; The second C3Ghost layer is used to perform pixel feature extraction operations on the first-scale feature map; the second The layer is used to perform a convolution operation on the output of the second C3Ghost layer to obtain the second-scale pixel feature map; The third C3Ghost layer is used to perform pixel feature extraction operations on the second-scale feature map. The fourth C3Ghost layer is used to perform a depthwise convolution operation on the output of the third C3Ghost layer. The output of the layer performs pixel feature extraction; the SPPF module is used to perform pooling operation on the output of the fourth C3Ghost layer to obtain the third-scale pixel feature map. The multi-scale fusion neck network includes a third network connected in sequence. Layer 64x3x1, Fourth Layer 256x1x1, Fifth Layer 64x3x1, First Upsampling Layer, First Stitching Layer 1x1, Fifth C3Ghost Layer 256x1, Second Upsampling Layer, Second Stitching Layer 1x1, Sixth C3Ghost Layer 128x1, Sixth Layer 512x1, Third splicing layer 1x1, Seventh C3Ghost layer 512x3x2, Seventh Layer 256x1, fourth splicing layer 1x1, and eighth C3Ghost layer 256x3x2; The third The layer is used to perform convolution operations on the third-scale pixel feature map; The fourth The layer is used for the third The output of the layer and the second The output of the fifth layer performs a convolution operation; The layer is used for the fourth The output of the layer and the first The layer's output performs a convolution operation; The first upsampling layer is used for the third The output of the fifth layer The output of the first upsampling layer is upsampled; the first concatenation layer is used to concatenate the output of the first upsampling layer; the fifth C3Ghost layer is used to perform a depthwise convolution operation on the output of the first concatenation layer; the second upsampling layer is used to upsample the output of the fifth C3Ghost layer; the second concatenation layer is used to concatenate the output of the second upsampling layer with the output of the fourth C3Ghost layer. The output of the layer is used to perform a concatenation operation; the sixth C3Ghost layer is used to perform a depthwise convolution operation on the second concatenation layer to obtain a first-scale fused feature map; The sixth The layer is used for the third The output of the third layer is convolved with the feature map fused with the first scale; the third concatenation layer is used to process the sixth layer. The output of the third layer is used to perform a concatenation operation; the seventh C3Ghost layer is used to perform a depthwise convolution operation on the output of the third concatenation layer to obtain the second-scale fused feature map; The seventh The fourth concatenation layer is used to perform convolution operations on the second-scale fused feature map; the fifth concatenation layer is used to perform convolution operations on the seventh-scale fused feature map. The output of the fourth concatenation layer is concatenated with the output of the fifth C3Ghost layer; the eighth C3Ghost layer is used to perform a depthwise convolution operation on the output of the fourth concatenation layer to obtain a third-scale fused feature map. The multi-object detection network YOLOv7Tiny-Lites in this embodiment uses YOLOv7-tiny as its backbone and adds a high-resolution branch P2 (stride=4) in addition to the conventional P3 (stride=8) and P4 (stride=16) branches. This allows for the effective representation of fine-grained textures and edge cues of targets under rain and fog conditions at a higher spatial sampling density. The YOLOv7Tiny-Lites multi-object detection network operates in a closed-loop sequence: input preprocessing, lightweight backbone, multi-scale fusion neck, three-scale detection head, decoding and candidate set generation, thresholding and NMS, and structured output. The input end performs size alignment and normalization on a single frame image before sending it to the backbone. The backbone consists of... C3Ghost and The lightweight residual bottleneck, composed of alternating stacks, significantly reduces the number of parameters and FLOPs while maintaining the feature quantity of the backbone channels. Furthermore, an SPPF_Lite module is used at the end of the lightweight backbone to expand the receptive field and aggregate multi-scale context. Its output (third-scale feature map) and the mid-shallow features (first-scale and second-scale feature maps) are fed into the multi-scale fusion neck network. The neck performs upsampling operations progressively from P4 to P3 to P2, and then through multiple splicing layers... With horizontal The "top-down + bottom-up" bidirectional fusion ensures that the high-resolution P2 simultaneously inherits category discrimination clues from deep layers and fine location information from shallow layers.

[0043] The detection network is used to obtain detection results based on multi-scale fused feature maps and output them through the output module. Specifically, in this embodiment, the Detect module at the head end, i.e., the detection network, takes three-scale features (P2, P3, P4, and P2, P3, P4) as input and outputs bounding boxes, targetability, and category distribution in parallel, forming a collaborative detection pattern of "high-resolution small targets and medium-scale large targets". In the decoding and candidate set generation stage of the detection network, the three-scale feature maps are gridded to obtain grid feature maps. The network output of each grid and its anchor box on the grid feature map is then decoded into grid-anchor boxes to directly recover normalized bounding boxes and scores, which are then merged into a candidate set. For example, in this embodiment, the input size is assumed to be (W, H), and the scale step size is... (Corresponding to P2 / P3 / P4), the feature map grid coordinates are (i,j), and the anchor box is ( , ), the regressor is ( , , , The target score is The formulas for obtaining the center, width, height, and target probability are as follows: (12) In the formula: express function, This represents an exponential mapping; in this embodiment, after merging the three-scale candidate sets, a "threshold + non-maximum suppression" mechanism is used to complete the screening and output of the detection results, the expression of which is: (13) In the formula: Denotes the candidate set from the range s∈{P2,P3,P4} (each element is ( , (,w,h,score,class); express The inhibition threshold; This represents the score threshold. Based on NMS (Non-Maximum Suppression), this embodiment uses a "threshold + non-maximum suppression" mechanism to filter duplicates in object detection: when multiple boxes appear near the same object, only the detection box with the highest score is retained. In this embodiment, the output detection results will be... As a unified ( , , , , The structured result (class) is compared with the backend settings. The tracker is directly connected. The score is used in the backend for scheduling of score-adaptive measurement noise and velocity leakage. (w,h) is used as a size prior to participate in the scaling of process noise, so that detection-tracking can form an adaptive closed loop under wind and wave jitter and low visibility conditions and maintain trajectory continuity and stable identity ID. S5: Train the YOLOv7Tiny-Lites multi-target detection network using a sample dataset to obtain the optimal detection model, and use the optimal detection model to achieve target detection in marine rain, fog and shaking environments. Specifically, the method for obtaining the optimal detection model is as follows: S51: Randomly divide the sample dataset into training and validation sets according to a preset ratio; S52: Train the YOLOv7Tiny-Lites multi-object detection network using the training set to obtain the trained YOLOv7Tiny-Lites multi-object detection network. S53: Based on the loss function, the trained multi-object detection network YOLOv7Tiny-Lites is validated using the validation set; and the loss function includes, but is not limited to, any one of the cross-entropy loss function and the mean squared error function; That is, to determine whether the output of the trained multi-object detection network YOLOv7Tiny-Lites has converged; If the output of the trained multi-object detection network YOLOv7Tiny-Lites converges, then the trained multi-object detection network YOLOv7Tiny-Lites is confirmed to be the optimal detection model. Otherwise, the parameters of the trained multi-object detection network YOLOv7Tiny-Lites are adaptively adjusted based on the backpropagation algorithm, and step S52 is repeated until it is confirmed that the parameters of the trained multi-object detection network YOLOv7Tiny-Lites with converged output are the optimal weight parameters, and the multi-object detection network YOLOv7Tiny-Lites is reconstructed to obtain the optimal detection model. S6: Define the target state vector and target observation vector based on the detection results, and based on... An improved Kalman filter algorithm for the tracker enables multi-target tracking in rainy, foggy, and turbulent marine environments. Specifically, the following steps are included: S61: Define the detection results obtained by the detection network as and =( , , , , ,class), where , Indicates the coordinates of the target center; , Indicates the width and height of the detection anchor frame; The score represents the detection confidence level, i.e., the score of the detected anchor box; class represents the category of the detected target. S62: Define the target state vector based on the detection results. and With target observation vector and ;in Represents the coordinates of the target center , The result after normalization; The planar velocity representing the center position of the target; This indicates the rate of change of the width and height of the detected anchor frame; Indicates transpose; S63: Based on the target state vector Define the state prediction equation; And the state prediction equation: (14); in, Represents the target state vector Prior state estimation; Represents the state transition matrix; Represents the target state vector Posterior state estimation; And obtain the residual equation based on the state prediction equation: (15); in, The residual represents the difference between the actual measurement and the predicted measurement, and is used to correct for the state. express The target observation vector at each time step maps the state to the measurement space; S64: Obtain the target state estimation equation and initial covariance matrix based on the state prediction equation and residual equation. Their expressions are as follows: (16) (17) (18) (19) In the formula: express Posterior state estimation at time t; This represents the Kalman gain, used for adaptively balancing the confidence of "prediction" and "measurement"; Represents the prior covariance; Represents the process noise covariance; Indicates the posterior covariance; Represents the observation matrix; The covariance represents the uncertainty of the expected residuals and is used for gating and distance metrics; Indicates the measurement noise covariance; Indicates transpose; The output of the target state estimation equation is used as the state estimate of the target motion in the current iteration. S65: Obtain the updated covariance matrix based on the initial covariance matrix; (20) In the formula: This represents the updated covariance matrix; Represents the identity matrix; S66: Construct a Kalman filter update strategy, which includes a measurement noise covariance update mechanism and a process noise covariance update mechanism for target state estimation at the next time step. And the expression for the measurement noise covariance update mechanism is: (twenty one) In the formula: This indicates the detection confidence level and the score ∈ [0,1]. k s Represents the fractional sensitivity coefficient, and k s >0 (calibrated in the range of 0.5-3, adjustable based on expert experience), where the coefficient approaches 1 when the score is high, and the algorithm degenerates into standard measurement noise; when the score is low, the overall noise is amplified. R Then through and This affects the gain of the current frame, making the update more "conservative"; This embodiment maps the detection confidence score ∈ [0,1] to an overall scaling factor for the measurement noise to reduce the traction of low-confidence observations on the updated trajectory. This scaling is applied during the execution of... The calculation was completed in the previous step; The expression for the process noise covariance update mechanism is as follows: (twenty two) (twenty three) (twenty four) (25) (26) (27) In the formula: Indicates phase conflict indication quantity; The general symbol for an indicator function is 1 if the condition is true, and 0 otherwise. express The change in; express -1 is the longitudinal velocity state component of the target; This indicates that a threshold value has been set. express Detection confidence at time -1; Indicates the velocity retention factor; express The minimum value; express base value and ≤ ≤1; This represents the measurement noise covariance corresponding to the longitudinal position dimension of the target; Indicates the phase lift factor; This represents the covariance of the corresponding vertical position dimension of the target; This represents the observation matrix corresponding to the longitudinal position dimension of the target; This indicates the longitudinal process covariance sub-block used for prediction in the next frame; Indicates the standard deviation multiple threshold; Residual The longitudinal component; Indicates the noise amplification factor; This represents the longitudinal process covariance sub-block used for prediction in the current frame; Indicates the magnification factor and ∈[1.5,3]; Indicates the fallback factor and ∈[0.8,0.95]; In this embodiment, a phase consistency judgment is introduced in the vertical direction. R y The lift occurs when the longitudinal measurement noise covariance of the current frame is out of phase with the existing longitudinal velocity and the phase amplitude exceeds a threshold. ηAt that time, only the longitudinal component of the measurement noise is amplified to suppress the "reverse phase pull" caused by the fluctuation of wind and waves. Its discrimination and rise are represented by the minimum symbol as shown in formula (22) and formula (25). In the process of updating the prediction, this embodiment will trigger a light ACI (Adaptive Covariance Inflation) based on the significance of the innovation amount, and only the longitudinal process noise is reversibly amplified, so that the state can quickly follow when the sea state suddenly disturbs and automatically fall back to a low noise model when it is stable. The triggering criterion and amplification-fallback are shown in formula (26) and formula (27). The result only applies to the prediction of the next frame: In this embodiment, to suppress the slow drift induced by low-confidence observations over a long period, fractional-aware leakage of the longitudinal velocity is implemented before entering the next frame prediction. That is, the lower the score of the previous frame, the smaller the velocity retention coefficient, so as to achieve automatic convergence velocity estimation in the data sparse and poor visibility section, as shown in formulas (23) and (24), after leakage Then it enters the state prediction equation (1). From this point on, the image in this frame completes the closed-loop process of "fraction and phase, prediction, update, longitudinal process adaptation, and velocity leakage setting", and will ( ,w,h,score The detection input is mapped to a more stable trajectory output. (w,h) and measurable uncertainty. The above mechanism corresponds to "fractional adaptive" R "Illustration" "Phase consistency determination and" R y "Raise" "Lightweight ACI trigger and" "Magnification" and "Fraction Perception" The leakage strategy consists of four parts, all of which use constant hyperparameters ( k s , , , , , , , This embodiment, by decoupling from the sampling period, can be directly deployed on different frame rate and computing power platforms. By suppressing "instantaneous pull of erroneous observations" on the measurement side with fractional and phase lift, and alleviating "chronic vertical drift" on the process side with longitudinal ACI and velocity leakage, this program significantly improves the continuity of the trajectory and the accuracy of the identity ID in complex sea conditions such as wind, waves, fog, rain and low visibility. It also complements the high-resolution P2 branch on the detection side, thereby achieving a comprehensive effect of high precision, strong robustness and low latency in the end-to-end detection-tracking integrated process. S67: Based on the updated covariance matrix combined with the Kalman filter update strategy, obtain the target state estimate for the next time step; S68: The target state estimate and updated covariance matrix at the next moment are used as inputs to the preset Kalman filter module again, and steps S63 to S67 are repeated to achieve multi-target tracking in marine rain, fog and shaking environments.

[0044] The beneficial effects of the method described in this embodiment are: Compared with existing technologies, the method described in this embodiment addresses the actual production conditions of "low visibility due to rain and fog at sea, and significant platform vibration." It employs an integrated and collaborative design encompassing four key aspects: data, prior knowledge, detection, and tracking. This approach maintains real-time performance at the deployment end while systematically improving target recall, trajectory continuity of multi-target tracking, and ID stability under rain and fog conditions at sea. Furthermore, it reduces the occurrence of track breakage due to platform vibration and longitudinal drift.

[0045] First, in the data and prior layer, a data augmentation module using "target copying—feathering pasting—label rewriting" automatically selects target silhouettes with normalized sizes within the fuzzy range from the annotation set. After enhancing contrast and brightness compensation, these silhouettes are pasted onto the real background using distance field feathering, and YOLO labels are rewritten simultaneously. Unlike existing general augmentations that rely solely on geometric transformations or random cropping, this method significantly increases the positive sample density of low-resolution targets while maintaining consistency in sea surface texture and lighting, avoiding artifacts caused by harsh boundaries. This results in better stability of detection scores and localization under rain and fog conditions. Combined with an offline adaptive anchor box learning process, the real-size distribution of the dataset is written into the anchor box prior, rearranged in ascending order of area, and mapped to the P2 / P3 / P4 three-scale detection heads. This prioritizes targets in the high-resolution branches, resulting in a higher positive sample matching rate, faster convergence speed, and lower matching noise, reducing the misleading effect of "negative samples" from the source. Secondly, this embodiment optimizes the lightweight architecture of YOLOv7-tiny to form YOLOv7Tiny-Lites-Lites: retaining the P2 (stride=4) detector head to carry fine-grained textures and edge cues, and the backbone network adopts... C3Ghost and A lightweight residual bottleneck, i.e., a lightweight backbone network, is constructed, and the receptive field is expanded using SPPF_Lite. Bidirectional fusion is employed in the scale fusion neck network to ensure the collaborative expression of shallow geometry and deep semantics. The three-scale detection heads and adaptive anchor boxes correspond one-to-one. Compared with existing lightweight detectors that only use P3 / P4 dual scales, this implementation significantly reduces the false negative rate and positioning jitter of maritime targets in rainy and foggy environments without significantly increasing computation and GPU memory. Simultaneously, because the anchor boxes are strictly aligned with the input scale, decoding formula, and derived configuration, the deployment end (ONNX / Tensor RT / edge GPU) can maintain consistent score and box distribution with the training end, avoiding accuracy regression caused by prior mismatch. With only lightweighting at the detection end, this embodiment demonstrates outstanding detection advantages in the "multi-target tracking method for maritime rainy, foggy, and jittery environments," particularly under real-world sea conditions where rain and fog cause low contrast, low resolution, and significant compression artifacts. Employing a shallow backbone and lightweight operators, coupled with small-target-friendly branches and multi-scale fusion, this approach preserves early high-resolution details and enhances the visibility and separability of targets with weak textures and edges. Even in low signal-to-noise ratio frames such as those with fog or raindrops, it consistently produces detection boxes and confidence scores, ensuring target recall and accuracy while reducing false positives, missed positives, and misdetections caused by deep oversmoothing and overfitting of low-quality frames. On the deployment side, the lightweight design significantly reduces computational power and memory requirements, enabling high frame rates, low latency, and low power consumption on shipboard platforms, and minimizing detection breakpoints. This lightweight model demonstrates usability and real-time performance in low-visibility conditions such as rain and fog, and the system achieves higher overall effectiveness and adaptability under marine rain, fog, and low-resolution conditions.

[0046] Furthermore, the method described in this embodiment proposes... The tracker, starting with a Kalman kernel based on standard 8-dimensional states and 4-dimensional observations, introduces an adaptive mechanism driven by fraction, phase, and innovation quantities, as well as velocity suppression: First, the fraction-adaptive measurement noise is dynamically scaled R based on the detected fraction, and low-fraction frames are automatically updated "conservatively" to suppress the instantaneous drag of low-quality observations on the trajectory; Second, phase consistency discrimination and R... y When the current longitudinal increment is in the opposite direction to the existing longitudinal velocity and exceeds the threshold, the lift only increases the longitudinal measurement variance, significantly reducing erroneous updates caused by wind and wave inversion. Furthermore, the lightweight ACI uses the longitudinal innovation amount and its standard deviation to gate and trigger the next frame. It amplifies and rapidly suppresses abnormal motion, then exponentially decreases when the motion stabilizes, balancing both following performance and smoothness; finally, it incorporates score perception. This embodiment converges the longitudinal velocity based on the score of the previous frame before entering prediction, thus suppressing chronic drift caused by the accumulation of low-confidence observations. Compared with the existing "fixed-noise Kalman + Hungarian matching" scheme, the method described in this embodiment can significantly reduce vertical drift variance, reduce ID jumps and trajectory discontinuities under the conditions of rain and fog obstruction and platform jitter, and has faster trajectory recovery capability after the target is briefly obscured. In this embodiment, without changing the size and rate of the detection end, customized constraints and compensations are applied to the core instability factors of rain and fog and camera jitter at sea through mechanisms such as "score-adaptive measurement confidence", "phase consistency-driven longitudinal jitter suppression" and "lightweight fast rebound (ACI)". During the update process, the system dynamically adjusts the measurement noise based on the observation confidence, so that the pull of weak evidence frames such as low visibility and raindrop attachment on the state is intentionally "downweighted", avoiding weak observations from pulling the trajectory away from the real motion. At the same time, short-window detrending and dominant frequency and phase estimation are used to determine whether the longitudinal channel is "out of phase" with the current motion. Once confirmed, noise is only directionally raised on the longitudinal measurement side, causing jitter to be absorbed rather than transformed into a jagged trajectory. When encountering large residuals caused by wave obstruction, the ACI mechanism will briefly relax the vertical process constraints to quickly realign, and then gradually fall back according to the set pattern after the anomaly recedes, balancing response speed and steady-state accuracy. Under long-term weak texture and background disturbance conditions, the tracker applies gentle leakage to the longitudinal velocity component and outputs consistency cues such as position, longitudinal velocity, aspect ratio, and area for gating matching, thereby suppressing chronic longitudinal drift and erroneous associations. As a result, the system can significantly reduce identity ID jumps and trajectory breakage, and improve trajectory continuity and identity reliability in harsh sea conditions.

[0047] Finally, regarding the integrated detection-tracking collaboration, this embodiment forms an "adaptive" closed loop guided by detection scores: the detection score and target scale directly drive the tracker's R, Ry, and ... and Joint adaptation, in turn, uses stable trajectory output to adjust the threshold. With NMS In this way, the system can simultaneously improve accuracy (mAP / Recall), stability (MOTA / IDF1 / HOTA / IDs / Frag), and real-time performance (FPS), and all modules are lightweight, making them easy to deploy on computing-constrained platforms. When lightweight detection and... When trackers are activated in tandem, this embodiment exhibits a systematic advantage in handling complex conditions such as rain, fog, and jitter at sea. The high frame rate enables dense measurements, allowing the filter to estimate the main frequency, phase, detection score, and innovation quantity of platform jitter faster and more stably. Furthermore, the filter's adaptive strategy softens and selectively blunts occasional detection frame offsets and detection score fluctuations during update steps, preventing the filter from being "hard-pulled" by low scores or longitudinal jitter, thus transforming a "fast but potentially noisy" detection flow into a "fast and stable" trajectory flow. During target tracking, this embodiment immediately increases the observation noise sensed by the detection score when visibility decreases due to rain or fog to reduce the weight of low-value measurements. Longitudinal adaptive noise and longitudinal velocity leakage isolate pseudo-motion within the filter during platform fluctuations. Process noise adaptation also completes a "relaxation-recovery" scheduling within the jitter burst and decay loop, avoiding excessive rigidity or excessive following of background drift. For the deployment end, this combination maintains accuracy under the same computing power, and brings the continuity of the trajectory and the stability of the identity ID to a level close to that of "heavy model + heavy post-processing", achieving the triple goal of "low power consumption, long online time and usability in complex sea conditions" in the combined scenarios of rain, fog and shaking at sea.

[0048] In summary, the method described in this embodiment addresses the core pain point of "multi-target tracking under rain, fog, and shaking conditions at sea." Through data distribution, network structure, and filter optimization, it achieves target detection with higher recall and stable multi-target tracking with anti-oscillation capabilities, significantly improving the perception performance of unmanned vessels under rain, fog, and shaking conditions at sea.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-target tracking method for marine rain, fog, and shaking environments, characterized in that, Specifically, the following steps are included: S1: Acquire multi-target image data under marine rain, fog, and shaking conditions; and the multi-target image data includes frame images and corresponding YOLO tags; the YOLO tags at least include target type and anchor box; S2: Based on the target enhancement segmentation strategy, target cropping and fusion processing is performed on the multi-target image data to obtain enhanced sample images; S3: Map the target features in the enhanced sample image and multi-object image data to a set of candidate boxes at the pixel scale, and based on... Clustering algorithms obtain a sample dataset containing adaptive anchors based on the candidate box set; S4: Construct a multi-object detection network YOLOv7Tiny-Lites, which includes an input module, a lightweight backbone network, a multi-scale fusion neck network, a detection network, and an output module. The input module is used to input sample images from the sample dataset into the lightweight backbone network. The lightweight backbone network is used to extract target pixel features from the sample images to obtain multi-scale pixel feature maps. The multi-scale fusion neck network is used to fuse the multi-scale pixel feature maps to obtain a multi-scale fused feature map. The detection network is used to obtain detection results based on the multi-scale fused feature map and output them through the output module. S5: Train the YOLOv7Tiny-Lites multi-target detection network using a sample dataset to obtain the optimal detection model, and use the optimal detection model to achieve target detection in marine rain, fog and shaking environments. S6: Define the target state vector and target observation vector based on the detection results, and realize multi-target tracking in marine rain, fog and shaking environments based on the improved Kalman filter algorithm.

2. The multi-target tracking method for marine rain, fog, and shaking environments according to claim 1, characterized in that, The target enhancement segmentation strategy described in S2 specifically includes the following steps: S21: Based on the target selection rules, filter the target sample feature maps in the multi-target image data that satisfy the scale metric; and the scale metric is the normalized minimum side of the target's width and height. And the expression of the target filtering rule is: In the formula: Indicates the screening scale metric; These represent the width and height of the target, respectively. S22: Based on a preset pixel coordinate system, crop the target image of the target sample feature map to obtain target silhouette data; And the expression for obtaining the target silhouette data is: In the formula: , Represents the pixel coordinates of the target center; , This represents the width and height of the original image's pixel frame; , ),( , ) represent the coordinates of the diagonal endpoints of the cropped pixel box; S23: Perform linear enhancement processing on the silhouette pixels in the target silhouette data to obtain a linearly enhanced image, and the expression for performing linear enhancement processing on the silhouette pixels in the target silhouette data is as follows: In the formula: This represents the silhouette pixels in the target silhouette data; This represents the pixel after linear enhancement processing; S24: At the pixel center of the linearly enhanced image, apply a scale perturbation that maintains the image aspect ratio to obtain a perturbed image; and perform a feathering fusion operation on the perturbed image to obtain a fused image; The expression for performing feathering and blending operations on the perturbed image is: In the formula: Indicates feathering weight; This indicates the calculation of the distance field from the linearly enhanced image pixels to a set boundary over a preset silhouette mask domain; The binary mask domain representing the silhouette is the set of pixels; This represents the maximum value of the range field within the binary mask domain; Represents the background pixels of a linearly enhanced image; Indicates the fusion result; S25: After pasting the silhouette into the fused image, obtain the YOLO normalized label, and the formula for obtaining the YOLO normalized label is as follows: In the formula: , These represent the pixel width and height before the silhouette is pasted; Indicates the pasting dimensions; , These represent the pixel width and height after the silhouette is pasted; , This indicates the pixel center coordinates of the silhouette pasted; , This represents the center coordinates of the YOLO normalized bounding box; , This represents the width and height of the YOLO normalized bounding box; The fused image and its corresponding YOLO normalized label are used as augmentation sample images.

3. The multi-target tracking method for marine rain, fog, and shaking environments according to claim 2, characterized in that, S3 specifically includes the following steps: S31: Based on the preset expected input size, map the target features in the enhanced sample image and multi-target image data into a set of candidate boxes at the pixel scale; S32: Using the preset lower limit of the pixel box min_box as the filtering index, filter out abnormal candidate boxes in the candidate box set that are smaller than the filtering index, and obtain the set of labeled boxes for clustering. S33: Given the number of anchor frames k, with Perform similarity metric on the set of bounding boxes Clustering operation to obtain the anchor box collection class; And the expression for retrieving the anchor box collection class is: In the formula: Represents the set of anchor frames; This represents the total number of bounding boxes participating in the clustering. This represents the width and height pair of the bounding box corresponding to the nth sample image in the pixel domain or uniform scale domain. , Indicates correspondence Width and height; express A specific anchor box within the collection, , Indicates the width and height of the anchor frame; This represents the intersection-union ratio of the annotation box b and the anchor box a when they are aligned with the same central axis, i.e., the ratio of the intersection area to the union area; Based on the anchor box set class, different scales of anchors are set to obtain the sample dataset for enhanced sample images and multi-target image data.

4. The multi-target tracking method for marine rain, fog, and shaking environments according to claim 3, characterized in that, The lightweight backbone network described in S4 includes a first network connected in sequence. Layer, First C3Ghost Layer, First Layer, second C3Ghost layer, second Layer, third C3Ghost layer, second Layer 4, the fourth C3Ghost layer, and the SPPF module; The first The layer is used to perform depthwise convolution operations on sample images in the sample dataset; The first C3Ghost layer is used for the first... The layer's output performs pixel feature extraction. First Used to perform a convolution operation on the output of the first C3Ghost layer to obtain a pixel feature map at the first scale; The second C3Ghost layer is used to perform pixel feature extraction operations on the first-scale feature map; second The layer is used to perform a convolution operation on the output of the second C3Ghost layer to obtain the second-scale pixel feature map; The third C3Ghost layer is used to perform pixel feature extraction operations on the second-scale feature map. The fourth C3Ghost layer is used to perform a depthwise convolution operation on the output of the third C3Ghost layer. The output of the layer performs pixel feature extraction; the SPPF module is used to perform pooling operation on the output of the fourth C3Ghost layer to obtain the third-scale pixel feature map.

5. A multi-target tracking method for marine rain, fog, and shaking environments according to claim 4, characterized in that, The multi-scale fusion neck network described in S4 includes a third network connected sequentially. Layer, Fourth Layer, Fifth Layer, First Upsampling Layer, First Stitching Layer, Fifth C3Ghost Layer, Second Upsampling Layer, Second Stitching Layer, Sixth C3Ghost Layer, Sixth Layer, third splicing layer, seventh C3Ghost layer, seventh The fourth splicing layer and the eighth C3Ghost layer; The third The layer is used to perform convolution operations on the third-scale pixel feature map; The fourth The layer is used for the third The output of the layer and the second The output of the fifth layer performs a convolution operation; The layer is used for the fourth The output of the layer and the first The layer's output performs a convolution operation; The first upsampling layer is used for the third The output of the fifth layer The output of the first upsampling layer is upsampled; the first concatenation layer is used to concatenate the output of the first upsampling layer; the fifth C3Ghost layer is used to perform a depthwise convolution operation on the output of the first concatenation layer; the second upsampling layer is used to upsample the output of the fifth C3Ghost layer; the second concatenation layer is used to concatenate the output of the second upsampling layer with the output of the fourth C3Ghost layer. The output of the layer is used to perform a concatenation operation; the sixth C3Ghost layer is used to perform a depthwise convolution operation on the second concatenation layer to obtain a first-scale fused feature map; The sixth The layer is used for the third The output of the third layer is convolved with the feature map fused with the first scale; the third concatenation layer is used to process the sixth layer. The layer's output is spliced. The seventh C3Ghost layer is used to perform a depthwise convolution operation on the output of the third concatenation layer to obtain the second-scale fused feature map; The seventh The fourth concatenation layer is used to perform convolution operations on the second-scale fused feature map; the fifth concatenation layer is used to perform convolution operations on the seventh-scale fused feature map. The output of the fourth concatenation layer is concatenated with the output of the fifth C3Ghost layer; the eighth C3Ghost layer is used to perform a depthwise convolution operation on the output of the fourth concatenation layer to obtain a third-scale fused feature map.

6. A multi-target tracking method for marine rain, fog, and shaking environments according to claim 5, characterized in that, The method for obtaining the optimal detection model in S5 includes the following steps: S51: Randomly divide the sample dataset into training and validation sets according to a preset ratio; S52: Train the YOLOv7Tiny-Lites multi-object detection network using the training set to obtain the trained YOLOv7Tiny-Lites multi-object detection network. S53: Based on the loss function, the trained multi-object detection network YOLOv7Tiny-Lites is validated using the validation set; and the loss function includes, but is not limited to, any one of the cross-entropy loss function and the mean squared error function; That is, to determine whether the output of the trained multi-object detection network YOLOv7Tiny-Lites has converged; If the output of the trained multi-object detection network YOLOv7Tiny-Lites converges, then the trained multi-object detection network YOLOv7Tiny-Lites is confirmed to be the optimal detection model. Otherwise, the parameters of the trained multi-object detection network YOLOv7Tiny-Lites are adaptively adjusted based on the backpropagation algorithm, and step S52 is repeated until it is confirmed that the parameters of the trained multi-object detection network YOLOv7Tiny-Lites with converged output are the optimal weight parameters, and the multi-object detection network YOLOv7Tiny-Lites is reconstructed to obtain the optimal detection model.

7. A multi-target tracking method for marine rain, fog, and shaking environments according to claim 5, characterized in that, S6 specifically includes the following steps: S61: Define the detection results obtained by the detection network as and =( , , , , ,class), where , Indicates the coordinates of the target center; , Indicates the width and height of the detection anchor frame; The score represents the detection confidence level, i.e., the score of the detected anchor box; class represents the category of the detected target. S62: Define the target state vector based on the detection results. and With target observation vector and ;in Represents the coordinates of the target center , The result after normalization; The planar velocity representing the center position of the target; This indicates the rate of change of the width and height of the detected anchor frame; Indicates transpose; S63: Based on the target state vector Define the state prediction equation; And the state prediction equation: ; in, Represents the target state vector Prior state estimation; Represents the state transition matrix; Represents the target state vector Posterior state estimation; And obtain the residual equation based on the state prediction equation: ; in, The residual represents the difference between the actual measurement and the predicted measurement. express The target observation vector at time; S64: Obtain the target state estimation equation and initial covariance matrix based on the state prediction equation and residual equation. Their expressions are as follows: In the formula: express Posterior state estimation at time t; Indicates Kalman gain; Represents the prior covariance; Represents the process noise covariance; Indicates the posterior covariance; Represents the observation matrix; The covariance represents the uncertainty of the expected residuals; Indicates the measurement noise covariance; Indicates transpose; The output of the target state estimation equation is used as the state estimate of the target motion in the current iteration. S65: Obtain the updated covariance matrix based on the initial covariance matrix; In the formula: This represents the updated covariance matrix; Represents the identity matrix; S66: Construct a Kalman filter update strategy, which includes a measurement noise covariance update mechanism and a process noise covariance update mechanism for target state estimation at the next time step. And the expression for the measurement noise covariance update mechanism is: In the formula: This indicates the detection confidence level and the score ∈ [0,1]. k s Represents the fractional sensitivity coefficient; The expression for the process noise covariance update mechanism is as follows: In the formula: Indicates phase conflict indication quantity; A general symbol representing indicator functions; express The change in; express -1 is the longitudinal velocity state component of the target; This indicates that a threshold value has been set. express Detection confidence at time -1; Indicates the velocity retention factor; express The minimum value; express base value and ≤ ≤1; This represents the measurement noise covariance corresponding to the longitudinal position dimension of the target; Indicates the phase lift factor; This represents the covariance of the corresponding vertical position dimension of the target; This represents the observation matrix corresponding to the longitudinal position dimension of the target; This indicates the longitudinal process covariance sub-block used for prediction in the next frame; Indicates the standard deviation multiple threshold; Residual The longitudinal component; Indicates the noise amplification factor; This represents the longitudinal process covariance sub-block used for prediction in the current frame; Indicates the magnification factor and ∈[1.5,3]; Indicates the fallback factor and ∈[0.8,0.95]; S67: Based on the updated covariance matrix combined with the Kalman filter update strategy, obtain the target state estimate for the next time step; S68: The target state estimate and updated covariance matrix at the next moment are used as inputs to the preset Kalman filter module again, and steps S63 to S67 are repeated to achieve multi-target tracking in marine rain, fog and shaking environments.

Images