A method and system for visual tracking and positioning of marine cage railings

Abstract

The present invention discloses a method and system for visual tracking and positioning of marine cage railings. The method comprises the following steps: preprocessing images collected by a monocular RGB camera; identifying marine cage railings in the preprocessed images, using a deep learning-based target detection algorithm to preliminarily detect the position of the marine cage railings, and further confirming the position of the marine cage railings in combination with a feature matching method; continuously tracking the identified marine cage railings, using Kalman filtering to predict the motion state of the railings, and combining meanshift target tracking, time series information fusion, and motion compensation technology to achieve stable locking of the railing target; based on the identified and tracked marine cage railings, using monocular vision and scale information to estimate the distance between the unmanned ship and the marine cage. The present invention is based on an economical and practical monocular camera to achieve accurate identification, stable tracking, and distance estimation of marine cage railings.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned ship visual navigation, and in particular to a method and system for visual tracking and positioning of marine cage railings. Background Art

[0002] In recent years, with the rapid development of unmanned ship technology, its application in the field of marine fisheries has become increasingly widespread. Unmanned ships can replace manual labor to perform tasks such as marine inspections, marine cage maintenance, and water quality monitoring, effectively reducing labor costs and safety risks and improving production efficiency.

[0003] The maintenance of marine cages is an important application scenario of unmanned boats in marine fisheries. Marine cages usually float on the sea surface and farm fish inside, which require regular replenishment, cleaning, harvesting and other operations. The traditional operation method requires manual operation of the boat to approach the marine cages and use a mechanical arm to grab the railings of the marine cages for fixing. This has problems such as high difficulty in operation, low efficiency and high safety risks.

[0004] In order to solve the above problems, the use of unmanned boats to replace manual maintenance of marine cages has become a new development trend. Unmanned boats need to have the ability to autonomously approach marine cages and grab railings, which puts higher requirements on the perception and positioning technology of unmanned boats.

[0005] The existing patent document (CN109444911A) discloses a method for detecting, identifying and locating surface targets of unmanned boats by fusing monocular camera and laser radar information. By combining the data of monocular camera and laser radar, the detection, identification and positioning capabilities of unmanned boats for surface targets are significantly improved. This technology uses convolutional neural networks to process images, and obtains three-dimensional information of the target through laser radar, thereby achieving matching and positioning of the target. In addition, the system designs a threshold method for matching the center circle of the point cloud cluster and the image detection frame, which enhances the robustness to uncertainty factors.

[0006] However, this technology also has some shortcomings. First, the performance of LiDAR may be affected by fluctuations in the water surface environment and changes in illumination, which may reduce the accuracy of target detection. Secondly, the real-time performance of the technology faces challenges, because the fusion of convolutional neural networks and point cloud processing requires higher computing resources, which places higher requirements on the hardware configuration of unmanned boats. In addition, the complexity of data fusion may increase the difficulty of debugging the system, and there may be limitations in the rapid adaptability to dynamic targets. Finally, high-precision LiDAR and advanced image processing systems may increase the cost of unmanned boats, which to some extent limits the popularization and application of the technology. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for visual tracking and positioning of marine cage railings to solve the problems of high cost, susceptibility to interference, insufficient accuracy, etc. in the prior art, to achieve accurate identification and stable tracking of marine cage railings by unmanned ships under complex sea conditions, and real-time estimation of the distance between the unmanned ship and the marine cage, providing a reliable visual basis for the unmanned ship to autonomously approach the marine cage and grab the railings.

[0008] The purpose of the present invention is achieved by at least one of the following technical solutions.

[0009] A method for visual tracking and positioning of marine cage railings, comprising the following steps:

[0010] S1. Image preprocessing: Preprocess the images collected by the monocular RGB camera to improve image quality and provide clear image data for subsequent algorithm processing;

[0011] S2. Railing recognition: Identify the railings of the marine cages in the preprocessed image, use the deep learning-based target detection algorithm to preliminarily detect the position of the marine cage railings, and then combine the feature matching method to further confirm the position of the marine cage railings to improve the recognition accuracy and robustness;

[0012] S3, Railing Tracking: Figure 4 , continuously track the identified marine cage railings, use Kalman filtering to predict the motion state of the railings, and combine meanshift target tracking, time series information fusion, and motion compensation technology to achieve stable locking of the railing targets;

[0013] S4. Distance estimation: Based on the identified and tracked ocean cage railings, the distance between the unmanned boat and the ocean cage is estimated using monocular vision and scale information, providing a reference for the motion control of the unmanned boat.

[0014] Further, in step S1, the image captured by the monocular RGB camera is preprocessed, including contrast enhancement, defogging / deblurring, and fluctuation compensation;

[0015] The contrast enhancement is to enhance the image contrast and improve the image clarity by using the histogram equalization algorithm, as follows:

[0016] set up The image captured by the monocular RGB camera is used as the input image. for The pixel value in The probability density function of the contrast-enhanced image is It is expressed as:

[0017]

[0018] in, Indicates that the calculation result in the brackets is rounded to ensure that the final grayscale value is an integer; is the gray level, =256; Calculated by the following formula:

[0019]

[0020] in, is the pixel value The number of times it appears in the input image; and are the width and height of the input image respectively;

[0021] The defogging / deblurring method uses a dark channel priori algorithm to remove image blur caused by sea fog or wave disturbance, and specifically includes the following steps:

[0022] (1) Calculate the dark channel image: For the contrast-enhanced image , which is in pixel coordinates The dark channel value at Defined as:

[0023]

[0024] in, Represented in pixel coordinates The dark channel value at ; Represents the pixel coordinates in the image and can be expressed as ,in represents the horizontal coordinate, Represents the vertical coordinate; Indicates the color channel, the value range is , , and Represent the red, green and blue channels respectively; In pixels The local window is the center, and the window size can be set according to the actual situation; Represents a local window Pixel coordinates within ; Representing images In pixel coordinates Color channels at The pixel value of

[0025] (2) Estimation of atmospheric light value : Select dark channel image The top a% (preferably 0.1%) of pixels with the highest brightness in the image after contrast enhancement The pixel with the highest brightness among the pixels at the corresponding position in is selected as the estimated atmospheric light value ;

[0026] (3) Estimated transmittance : According to the dark channel prior, the transmittance Estimated to be:

[0027]

[0028] For each color channel }, find the pixel The local window at the center All pixels in the channel The pixel value and atmospheric illumination value The minimum value of the ratio: ; Take the minimum value among the minimum values ​​of the three color channels ; Minimum value Multiplying the parameters , and then subtract the result from 1 to get the final transmittance estimate ; To retain the set fog parameters; Indicates that atmospheric light is in the color channel Atmospheric illumination refers to the scattered light in the air, which is enhanced by contrast. The brightest area in the image is estimated.

[0029] (4) Image restoration: Based on the atmospheric scattering model, the dehazed image can be obtained. :

[0030]

[0031] in, is the minimum transmittance set to avoid division by zero error;

[0032] The wave compensation is to estimate the motion of the unmanned ship and the marine cage by detecting the motion vector between adjacent frames, and to perform motion compensation on the image to reduce the motion blur caused by waves, which specifically includes the following steps:

[0033] (1) Feature point detection: Use the FAST corner detection algorithm to detect feature points in two adjacent frames;

[0034] (2) Feature matching: Use the KLT optical flow algorithm to match the detected feature points and obtain the motion vector of the feature points;

[0035] (3) Motion estimation: Use the RANSAC algorithm to remove abnormal feature points matched in feature matching and estimate the global motion model;

[0036] (4) Motion compensation: Based on the estimated global motion model, the current frame image is inversely transformed to compensate for the motion caused by fluctuations.

[0037] Furthermore, the RANSAC algorithm is used to remove the feature points matched in the abnormal feature matching and estimate the global motion model, which specifically includes the following steps:

[0038] (3.1) Random sampling: A subset is randomly selected from all matched feature points as a hypothesis set; the matched feature points in the hypothesis set are used to fit the affine transformation model;

[0039] (3.2) Affine transformation model fitting: Use the matched feature points in the hypothesis set to calculate a model parameter set using the least squares method to obtain a fitted affine transformation model;

[0040] (3.3) Calculate the error: Substitute all matched feature points into the fitted affine transformation model, calculate the error of each matched feature point, set a threshold, and distinguish between inliers and outliers; the error is defined as the distance between the actual position of the matched feature point and the position predicted by the affine transformation model; points with errors less than the preset threshold are considered inliers, that is, they conform to the fitted affine transformation model; points with errors greater than the preset threshold are considered outliers, that is, abnormal matched feature points;

[0041] (3.4) Repeating process: Repeat iterative steps (3.1) to (3.3) for a set number of times, and randomly select a different hypothesis set each time; for each hypothesis set, the calculated model will produce a different set of internal points, and obtain an affine transformation model with a different set of model parameters;

[0042] (3.5) Model selection: The affine transformation model with the most inliers is selected as the final result and the abnormal matched feature points in the selected affine transformation model are eliminated. The final affine transformation model obtained is the estimated global motion model.

[0043] Further, step S2 includes the following steps:

[0044] S2.1. Use the YOLOv10 network to detect the position of the marine cage railings. The specific process is as follows:

[0045] First, data preparation and preprocessing were performed, including collecting a set number of marine cage railing images, annotating them with labelImg software, performing data enhancement, and adjusting the marine cage railing images after data enhancement to the size and normalization required by the YOLOv10 network to construct a training set.

[0046] Construct a YOLOv10 network, use CSPDarknet53 as the backbone network, combine the SPP module and PANet for feature extraction and fusion, and set detection heads at three scales; the three scales include small-scale feature maps, medium-scale feature maps, and large-scale feature maps;

[0047] Small-scale feature maps: extracted from deeper layers of the backbone network and used to detect smaller objects. The size of the small-scale feature map is 1 / 32 of the original input image size; Medium-scale feature maps: extracted from intermediate layers and used to detect medium-sized objects. The size of the medium-scale feature map is 1 / 16 of the original input image size; Large-scale feature maps: extracted from shallower layers and used to detect larger objects. The size of the large-scale feature map is 1 / 8 of the original input image size;

[0048] Constructing the total loss function : CIoU loss is used to construct bounding box regression, BCE loss is used to construct target confidence, and Focal loss is used to construct category classification. Finally, the three are weighted and summed to obtain the total loss function , as follows:

[0049] Use CIoU (Complete IoU) loss to build bounding box regression loss :

[0050]

[0051]

[0052] in, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; is the diagonal length of the minimum enclosing rectangle that encloses the predicted box and the true box; is the orthogonality coefficient; is the aspect ratio consistency measure; is the predicted bounding box; is the true bounding box;

[0053] Use binary cross entropy (BCE) loss to construct target confidence loss :

[0054]

[0055] in, is the true label, is 0 or 1; is the confidence score of the prediction;

[0056] Use Focal loss to construct category classification loss :

[0057]

[0058] in, Yes Category , i.e., the predicted probability of the ocean cage railing; is the category balancing factor; is the focusing parameter, set to 2;

[0059] Total loss function It is the weighted sum of the three loss values:

[0060]

[0061] in, , , is the weight coefficient of each part of the loss;

[0062] The YOLOv10 network is trained using the training set. During the training process, a transfer learning strategy is adopted, and the YOLOv10 network is initialized using pre-trained weights. The training adopts a phased strategy, first fixing the backbone network and only training the detection head. At the same time, Mosaic data enhancement and adaptive image scaling technology are used, and cosine annealing is used to adjust the learning rate.

[0063] In the inference phase, the output results of the trained YOLOv10 network, including bounding box coordinates and target confidence scores, are processed by NMS and confidence threshold filtering respectively, and the bounding box coordinates after NMS processing are mapped back to the original image scale; finally, the model is quantized and accelerated through TensorRT, and cross-platform deployment is achieved in ONNX format to meet actual application needs;

[0064] Through this series of optimization settings, YOLOv10 can quickly and accurately detect the position of marine cage railings in complex and changeable sea conditions, providing a reliable basis for subsequent target tracking and distance estimation;

[0065] S2.2. Feature-based fine recognition: After the initial detection of the area of ​​the marine cage railing, the color, texture, and shape features of the area are extracted and matched with the pre-established marine cage railing feature template library to further confirm whether it is a genuine marine cage railing and avoid misidentification due to background interference.

[0066] Further, step S3 includes the following steps:

[0067] S3.1, construct a Kalman filter prediction model, and use the Kalman filter prediction model for prediction;

[0068] S3.2, meanshift target tracking: combined with the meanshift target tracking algorithm, by modeling the color histogram of the target area, find the area most similar to the target in the new image to achieve continuous locking of the railing target;

[0069] S3.3, Time series information fusion: The weighted average fusion of the marine cage railing positions detected in multiple consecutive frames is performed to obtain the final fusion position to improve the tracking stability and anti-interference ability; the position closer to the current frame is given a greater weight, and the position farther away from the current frame is given a smaller weight; the weighted average fusion mathematical expression is as follows:

[0070]

[0071] in Refers to the final fusion position; It is The position of the marine cage railing detected by the frame, It corresponds to The weight of the frame, The exponential decay form is used for calculation. The specific formula is:

[0072]

[0073] in, is the index of the frame, =1~ , is the index of the current frame (the latest frame), is the decay rate, ;

[0074] S3.4, Motion compensation: During tracking, according to the motion status of the unmanned vessel and the marine cage, the existing motion compensation standard algorithm, including the motion compensation algorithm based on optical flow, is used to compensate the tracking area to ensure that the railing target can be continuously locked.

[0075] Furthermore, in step S3.1, a Kalman filter prediction model is constructed and the Kalman filter prediction model is used for prediction:

[0076] First, define the center coordinates of the marine cage railing ( , ) and speed( , )'s state vector , and Marine cage railings and Speed ​​in direction;

[0077] Then, based on the assumption of uniform motion in a short period of time, the specific process of constructing the state transfer matrix and observation matrix is ​​as follows:

[0078] The state transfer matrix F can be expressed as:

[0079]

[0080] in is the time interval between two adjacent frames;

[0081] Constructing the observation matrix :

[0082] ( , ) is the center coordinate of the ocean cage railing, and the speed needs to be estimated by the position change; therefore, the observation matrix It can be expressed as:

[0083] ;

[0084] Next, the covariance matrix of process noise and observation noise is set according to the actual situation. The specific process is as follows:

[0085] Process noise covariance matrix Reflects the uncertainty of state prediction; considering the complexity of the ocean environment, a continuous-time white noise acceleration model can be used to set :

[0086]

[0087] in, is the process noise intensity, adjusted experimentally to obtain the best performance; typically, The value of can be set between 0.01 and 1, depending on the dynamic characteristics of the system;

[0088] Finally, the constructed Kalman filter model is used to predict the motion state of the marine cage railing, as follows:

[0089] State prediction and covariance prediction:

[0090]

[0091]

[0092] in, for The system state vector at time ; is the state transfer matrix, describing Time has come Linear relationship between states at different moments; for The system state vector at time ; is the control input matrix, which describes how the control vector affects the state; for The control vector at the moment; is process noise, with mean 0 and covariance matrix Gaussian distribution of for The observation vector at time instant; is the observation matrix, which describes the linear relationship between the state vector and the observation vector; is the observation noise, which has a mean of 0 and a covariance matrix of Gaussian distribution of

[0093] Calculate the Kalman gain, perform state update and error covariance update to obtain the optimal state estimate:

[0094] Calculate the Kalman gain:

[0095]

[0096] in, is the Kalman gain, which determines how much the observation should be trusted; is the prior error covariance obtained in the prediction step; is the observation matrix, which maps the state space to the observation space; is the observation noise covariance matrix; for Transpose of the observation matrix;

[0097] Perform state update and covariance update:

[0098]

[0099] in, is the updated state estimate; is the prior state estimate obtained in the prediction step; is the actual observed value; It is called innovation or residual and represents the difference between the actual observation and the predicted observation. This formula combines the prior estimate with the weighted innovation to obtain the updated state estimate.

[0100]

[0101] in, is the updated error covariance matrix; is the identity matrix;

[0102] The Kalman filter prediction model can effectively predict the motion state of the marine cage railings, providing reliable prior information for target tracking. The Kalman filter algorithm is used to predict the motion state of the railings in the current frame and the next few frames based on the motion trajectory of the railings in the previous few frames, thereby assisting tracking.

[0103] Furthermore, in step S3.2, combined with the meanshift target tracking algorithm, by modeling the color histogram of the target area, the area most similar to the target is found in the new image to achieve continuous locking of the railing target, as follows:

[0104] S3.2.1. Target model building: In the initial frame, a color histogram model is established for the target to be tracked, i.e., the area where the marine cage railing is located, and a weighted color histogram is calculated for the initial target area:

[0105]

[0106] in, Indicates the color histogram The value of the color range; is a normalization constant that ensures that the sum of all color range values ​​is 1; Indicates the sum of all pixels in the target area; is a kernel function used to weight pixels; is the position vector of the pixel relative to the center of the object; is the modulus of the pixel's position vector relative to the target center; the kernel function is usually chosen so that the closer the pixel is to the center, the greater the weight; is the Kronecker delta function; Represents the position vector of the pixel relative to the center of the object The value of the color range to which it belongs, when hour, ; hour, ;

[0107] S3.2.2. Candidate region modeling: In subsequent frames, a color histogram model is established for the candidate region that may include the target, and a weighted color histogram is calculated for the candidate region:

[0108]

[0109] in, is the center of the candidate region, and the candidate region refers to the image region in the current frame image that may include the target, i.e., the railing of the marine cage; is the scale parameter;

[0110] S3.2.3. Similarity calculation: using Bhattacharyya coefficient , calculate the similarity between the target model and the candidate region model. In the meanshift target tracking algorithm, the Bhattacharyya coefficient between the target model and different candidate region models in the image is calculated, and the one that is most similar to the target model is selected. The candidate area with the largest value is used as the new target location:

[0111]

[0112] in, Indicates the candidate region of the target model in the color histogram values ​​in the color range, Indicates the color histogram model of the candidate region model in the first values ​​over a color range;

[0113] S3.2.4. Position update: Calculate the mean shift vector , the candidate regional center Plus The new target center position is obtained:

[0114]

[0115]

[0116] in, is the kernel function The derivative of is the mean shift vector weight, The value of the pixel to the target center The closer the distance, the greater the weight; is the bandwidth parameter, which controls the range of the kernel function;

[0117] The calculation formula is:

[0118]

[0119]

[0120] S3.2.5, Iteration: Repeat steps S3.2.2 to S3.2.4 until convergence or the maximum number of iterations is reached;

[0121] set up is the mean shift vector The displacement threshold, = 0.1 pixels, if , then converge, for The module outputs the final candidate region center As the tracking result of this frame; otherwise, enter the next iteration, reach the maximum number of iterations, and output the final candidate area center As the tracking result of this frame;

[0122] By continuously iterating, the mean offset vector is calculated and the target position is updated. , the meanshift algorithm can effectively track the railings of marine cages.

[0123] Further, step S4 includes the following steps:

[0124] S4.1. Size-distance polynomial regression model building: by setting a certain amount of measured data, the size model of the marine cage railing at different distances is established, that is, the mapping relationship between the railing size and the distance;

[0125] S4.2, Scale-invariant feature matching: Using SIFT scale-invariant features, the scale information of the railing is extracted from the image and matched with the size-distance polynomial regression model to estimate the current distance; the scale space in the SIFT algorithm The Gaussian kernel and the input marine cage railing image Convolution gives:

[0126]

[0127] Among them, the Gaussian kernel function for:

[0128]

[0129] The SIFT feature descriptor is obtained by calculating the gradient direction histogram around the key point. is the scale parameter of the Gaussian kernel; , is the position coordinate of the pixel;

[0130] S4.3, time series filtering: perform time series filtering on the distance estimation results of multiple consecutive frames to reduce the distance jitter caused by factors such as water surface fluctuations, and finally obtain the distance between the unmanned boat and the marine cage.

[0131] Furthermore, in step S4.1, by setting a certain amount of measured data, a size model of the marine cage railing at different distances is established, that is, a mapping relationship between the railing size and the distance. The specific steps are as follows:

[0132] S4.1.1. Data collection: Collect images of the marine cage railings at different distances and record the actual distances;

[0133] S4.1.2, Dimension measurement: Process the collected images to extract the pixel dimensions of the railing, including length and width;

[0134] S4.1.3, Data normalization: Normalize the measured pixel size to the range [0,1]:

[0135]

[0136] in, is the normalized pixel size value; Indicates the minimum value of all measured dimensions; Indicates the maximum value of all measured dimensions; is the pixel size value to be normalized;

[0137] S4.1.4. Model fitting: Use the size-distance polynomial regression model to fit the relationship between size and distance:

[0138]

[0139] in, is the polynomial order, usually 2-4; use the least squares method to solve the coefficients , , …, ; for distance;

[0140] S4.1.5. Model Validation: Use the cross-validation method to evaluate the accuracy and generalization ability of the size-distance polynomial regression model and calculate the root mean square error value. and the coefficient of determination ), set the threshold of the root mean square error value , set the threshold of the coefficient of determination ,if and , it indicates that the model fitting effect is good and meets the requirements, and proceed to step S4.1.6. If it does not meet the requirements, return to step S4.1.4 to adjust the polynomial order, or return to step S4.1.1 to set other nonlinear models (such as exponential model and logarithmic model);

[0141] S4.1.6. Model storage: Save the finalized model parameters for subsequent use.

[0142] A marine cage railing visual tracking and positioning system, comprising

[0143] Image preprocessing module: preprocesses the images collected by the monocular RGB camera to improve image quality and provide clear image data for subsequent algorithm processing;

[0144] Railing recognition module: Identify the railings of the marine cages in the preprocessed image, use the deep learning-based target detection algorithm to preliminarily detect the location of the marine cage railings, and then combine the feature matching method to further confirm the location of the marine cage railings to improve the recognition accuracy and robustness;

[0145] Railing tracking module: Continuously track the identified marine cage railings, use Kalman filtering to predict the motion state of the railings, and combine meanshift target tracking, time series information fusion, and motion compensation technology to achieve stable locking of the railing targets;

[0146] Distance estimation module: Based on the identified and tracked marine cage railings, monocular vision and scale information are used to estimate the distance between the unmanned ship and the marine cage, providing a reference for the motion control of the unmanned ship.

[0147] Compared with the prior art, the present invention has the following advantages and technical effects:

[0148] The present invention combines Kalman filtering, meanshift target tracking, time series information fusion and other algorithms to achieve stable tracking of marine cage railings and adapt to dynamic changes in the marine environment.

[0149] The present invention uses monocular vision and scale information to establish a mapping model between railing size and distance, which can estimate the distance between the unmanned ship and the marine cage in real time and provide a reference for the motion control of the unmanned ship. The monocular RGB camera used in the present invention is low-priced and easy to deploy, reducing the overall cost of the system.

[0150] In summary, the present invention can realize accurate recognition, stable tracking and distance estimation of marine cage railings based on an economical and practical monocular camera in a complex and changeable marine environment, and provide reliable visual support for unmanned boats to autonomously approach and grasp the railings, which has important theoretical significance and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0151] Figure 1 It is a structural schematic diagram of a marine cage railing visual tracking and positioning system based on a monocular RGB camera in an embodiment;

[0152] Figure 2 Schematic diagram of the process of the image preprocessing module in an embodiment of the present invention;

[0153] Figure 3 Schematic diagram of the process of the railing recognition module in an embodiment of the present invention;

[0154] Figure 4 It is a schematic diagram of the process of the railing tracking module in an embodiment of the present invention;

[0155] Figure 5Schematic diagram of the flow of the distance estimation module in an embodiment of the present invention. DETAILED DESCRIPTION

[0156] The specific implementation of the present invention is described below in conjunction with the accompanying drawings and embodiments, but the implementation and protection of the present invention are not limited thereto.

[0157] In this embodiment, a marine cage railing visual tracking and positioning system is provided. Figure 1 As shown, including:

[0158] Image preprocessing module: preprocesses the images collected by the monocular RGB camera to improve image quality and provide clear image data for subsequent algorithm processing;

[0159] Railing recognition module: Identify the railings of the marine cages in the preprocessed image, use the deep learning-based target detection algorithm to preliminarily detect the location of the marine cage railings, and then combine the feature matching method to further confirm the location of the marine cage railings to improve the recognition accuracy and robustness;

[0160] Railing tracking module: Continuously track the identified marine cage railings, use Kalman filtering to predict the motion state of the railings, and combine meanshift target tracking, time series information fusion, and motion compensation technology to achieve stable locking of the railing targets;

[0161] Distance estimation module: Based on the identified and tracked marine cage railings, monocular vision and scale information are used to estimate the distance between the unmanned ship and the marine cage, providing a reference for the motion control of the unmanned ship.

[0162] A method for visually tracking and positioning a marine cage railing in this embodiment includes the following steps:

[0163] S1. Image preprocessing: Preprocess the images collected by the monocular RGB camera to improve image quality and provide clear image data for subsequent algorithm processing;

[0164] like Figure 2 , the image captured by the monocular RGB camera is preprocessed, including contrast enhancement, defogging / deblurring and fluctuation compensation;

[0165] The contrast enhancement is to enhance the image contrast and improve the image clarity by using the histogram equalization algorithm, as follows:

[0166] set up The image captured by the monocular RGB camera is used as the input image. for The pixel value in The probability density function of the contrast-enhanced image is It is expressed as:

[0167]

[0168] in, Indicates that the calculation result in the brackets is rounded to ensure that the final grayscale value is an integer; is the gray level, =256; Calculated by the following formula:

[0169]

[0170] in, is the pixel value The number of times it appears in the input image; and are the width and height of the input image respectively;

[0171] The defogging / deblurring method uses a dark channel priori algorithm to remove image blur caused by sea fog or wave disturbance, and specifically includes the following steps:

[0172] (1) Calculate the dark channel image: For the contrast-enhanced image , which is in pixel coordinates The dark channel value at Defined as:

[0173]

[0174] in, Represented in pixel coordinates The dark channel value at ; Represents the pixel coordinates in the image and can be expressed as ,in represents the horizontal coordinate, Represents the vertical coordinate; Indicates the color channel, the value range is , , and Represent the red, green and blue channels respectively; In pixels The local window is the center. The window size can be set according to the actual situation. Here it is set to a 3x3 window. Represents a local window Pixel coordinates within ; Representing images In pixel coordinates Color channels at The pixel value of

[0175] (2) Estimation of atmospheric light value : Select dark channel image The top 0.1% of pixels with the highest brightness in the image after contrast enhancement The pixel with the highest brightness among the pixels at the corresponding position in is selected as the estimated atmospheric light value ;

[0176] (3) Estimated transmittance : According to the dark channel prior, the transmittance Estimated to be:

[0177]

[0178] For each color channel }, find the pixel The local window at the center All pixels in the channel The pixel value and atmospheric illumination value The minimum value of the ratio: ; Take the minimum value among the minimum values ​​of the three color channels ; Minimum value Multiplying the parameters , and then subtract the result from 1 to get the final transmittance estimate ; In order to retain the set fog parameters, in one embodiment, 0.95 is taken; Indicates that atmospheric light is in the color channel Atmospheric illumination refers to the scattered light in the air, which is enhanced by contrast. The brightest area in the image is estimated.

[0179] (4) Image restoration: Based on the atmospheric scattering model, the dehazed image can be obtained. :

[0180]

[0181] in, is the minimum transmittance set to avoid division by zero error;

[0182] The wave compensation is to estimate the motion of the unmanned ship and the marine cage by detecting the motion vector between adjacent frames, and to perform motion compensation on the image to reduce the motion blur caused by waves, which specifically includes the following steps:

[0183] (1) Feature point detection: Use the FAST corner detection algorithm to detect feature points in two adjacent frames;

[0184] (2) Feature matching: Use the KLT optical flow algorithm to match the detected feature points and obtain the motion vector of the feature points;

[0185] (3) Motion estimation: Use the RANSAC algorithm to remove feature points matched in abnormal feature matching and estimate the global motion model, which specifically includes the following steps:

[0186] (3.1) Random sampling: A subset is randomly selected from all matched feature points as a hypothesis set; the matched feature points in the hypothesis set are used to fit the affine transformation model;

[0187] (3.2) Affine transformation model fitting: using the matched feature points in the hypothesis set, a model parameter set is calculated using the least squares method to obtain a fitted affine transformation model; in one embodiment, three matching points are selected, and a 3x3 affine transformation matrix is ​​calculated using the least squares method. The matrix is ​​the model parameter set, and the fitted affine transformation model is obtained;

[0188] (3.3) Calculate the error: Substitute all matched feature points into the fitted affine transformation model, calculate the error of each matched feature point, set a threshold, and distinguish between inliers and outliers; the error is defined as the distance between the actual position of the matched feature point and the position predicted by the affine transformation model; points with errors less than the preset threshold are considered inliers, that is, they conform to the fitted affine transformation model; points with errors greater than the preset threshold are considered outliers, that is, abnormal matched feature points;

[0189] (3.4) Repeating process: In one embodiment, steps (3.1) to (3.3) are repeated 10 times, and a different hypothesis set is randomly selected each time; for each hypothesis set, the calculated model will generate a different set of interior points, and an affine transformation model with a different set of model parameters is obtained;

[0190] (3.5) Model selection: The affine transformation model with the most inliers is selected as the final result and the abnormal matched feature points in the selected affine transformation model are eliminated. The final affine transformation model obtained is the estimated global motion model.

[0191] (4) Motion compensation: Based on the estimated global motion model, the current frame image is inversely transformed to compensate for the motion caused by fluctuations.

[0192] S2. Railing recognition: Identify the railings of the marine cages in the preprocessed image, use the deep learning-based target detection algorithm to preliminarily detect the location of the marine cage railings, and then combine the feature matching method to further confirm the location of the marine cage railings to improve the recognition accuracy and robustness, such as Figure 3 , including the following steps:

[0193] S2.1. Use the YOLOv10 network to detect the position of the marine cage railings. The specific process is as follows:

[0194] First, data preparation and preprocessing were performed, including collecting a set number of marine cage railing images, annotating them with labelImg software, performing data enhancement, and adjusting the marine cage railing images after data enhancement to the size and normalization required by the YOLOv10 network to construct a training set.

[0195] Construct a YOLOv10 network, use CSPDarknet53 as the backbone network, combine the SPP module and PANet for feature extraction and fusion, and set detection heads at three scales; the three scales include small-scale feature maps, medium-scale feature maps, and large-scale feature maps;

[0196] Small-scale feature maps: extracted from deeper layers of the backbone network and used to detect smaller objects. The size of the small-scale feature map is 1 / 32 of the original input image size; Medium-scale feature maps: extracted from intermediate layers and used to detect medium-sized objects. The size of the medium-scale feature map is 1 / 16 of the original input image size; Large-scale feature maps: extracted from shallower layers and used to detect larger objects. The size of the large-scale feature map is 1 / 8 of the original input image size;

[0197] Constructing the total loss function : CIoU loss is used to construct bounding box regression, BCE loss is used to construct target confidence, and Focal loss is used to construct category classification. Finally, the three are weighted and summed to obtain the total loss function , as follows:

[0198] Use CIoU (Complete IoU) loss to build bounding box regression loss :

[0199]

[0200]

[0201] in, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; is the diagonal length of the minimum enclosing rectangle that encloses the predicted box and the true box; is the orthogonality coefficient; is the aspect ratio consistency measure; is the predicted bounding box; is the true bounding box;

[0202] Use binary cross entropy (BCE) loss to construct target confidence loss :

[0203]

[0204] in, is the true label, is 0 or 1; is the confidence score of the prediction;

[0205] Use Focal loss to construct category classification loss :

[0206]

[0207] in, Yes Category , i.e., the predicted probability of the ocean cage railing; is the category balancing factor; is the focusing parameter, set to 2;

[0208] Total loss function It is the weighted sum of the three loss values:

[0209]

[0210] in, , , is the weight coefficient of each part of the loss;

[0211] The YOLOv10 network is trained using the training set. During the training process, a transfer learning strategy is adopted, and the YOLOv10 network is initialized using pre-trained weights. The training adopts a phased strategy, first fixing the backbone network and only training the detection head. At the same time, Mosaic data enhancement and adaptive image scaling technology are used, and cosine annealing is used to adjust the learning rate.

[0212] In the inference phase, the output results of the trained YOLOv10 network, including bounding box coordinates and target confidence scores, are processed by NMS and confidence threshold filtering respectively, and the bounding box coordinates after NMS processing are mapped back to the original image scale; finally, the model is quantized and accelerated through TensorRT, and cross-platform deployment is achieved in ONNX format to meet actual application needs;

[0213] Through this series of optimization settings, YOLOv10 can quickly and accurately detect the position of marine cage railings in complex and changeable sea conditions, providing a reliable basis for subsequent target tracking and distance estimation;

[0214] S2.2. Feature-based fine recognition: After the initial detection of the area of ​​the marine cage railing, the color, texture, and shape features of the area are extracted and matched with the pre-established marine cage railing feature template library to further confirm whether it is a genuine marine cage railing and avoid misidentification due to background interference.

[0215] S3, Railing Tracking: Continuously track the identified marine cage railings, use Kalman filtering to predict the motion state of the railings, and combine meanshift target tracking, time series information fusion, and motion compensation technology to achieve stable locking of the railing target, including the following steps:

[0216] S3.1. Construct a Kalman filter prediction model and use the Kalman filter prediction model for prediction:

[0217] First, define the center coordinates of the marine cage railing ( , ) and speed( , )'s state vector , and Marine cage railings and Speed ​​in direction;

[0218] Then, based on the assumption of uniform motion in a short period of time, the specific process of constructing the state transfer matrix and observation matrix is ​​as follows:

[0219] State transition matrix It can be expressed as:

[0220]

[0221] in is the time interval between two adjacent frames;

[0222] Constructing the observation matrix :

[0223] ( , ) is the center coordinate of the ocean cage railing, and the speed needs to be estimated by the position change; therefore, the observation matrix It can be expressed as:

[0224] ;

[0225] Next, the covariance matrix of process noise and observation noise is set according to the actual situation. The specific process is as follows:

[0226] Process noise covariance matrix Reflects the uncertainty of state prediction; considering the complexity of the ocean environment, a continuous-time white noise acceleration model can be used to set :

[0227]

[0228] in, is the process noise intensity, adjusted experimentally to obtain the best performance; typically, The value of can be set between 0.01 and 1, depending on the dynamic characteristics of the system;

[0229] Finally, the constructed Kalman filter model is used to predict the motion state of the marine cage railing, as follows:

[0230] State prediction and covariance prediction:

[0231]

[0232]

[0233] in, for The system state vector at time ; is the state transfer matrix, describing Time has come Linear relationship between states at different moments; for The system state vector at time ; is the control input matrix, which describes how the control vector affects the state; for The control vector at the moment; is process noise, with mean 0 and covariance matrix Gaussian distribution of for The observation vector at time instant; is the observation matrix, which describes the linear relationship between the state vector and the observation vector; is the observation noise, which has a mean of 0 and a covariance matrix of Gaussian distribution of

[0234] Calculate the Kalman gain, perform state update and error covariance update to obtain the optimal state estimate:

[0235] Calculate the Kalman gain:

[0236]

[0237] in, is the Kalman gain, which determines how much the observation should be trusted; is the prior error covariance obtained in the prediction step; is the observation matrix, which maps the state space to the observation space; is the observation noise covariance matrix; for Transpose of the observation matrix;

[0238] Perform state update and covariance update:

[0239]

[0240] in, is the updated state estimate; is the prior state estimate obtained in the prediction step; is the actual observed value; It is called innovation or residual and represents the difference between the actual observation and the predicted observation. This formula combines the prior estimate with the weighted innovation to obtain the updated state estimate.

[0241]

[0242] in, is the updated error covariance matrix; is the identity matrix;

[0243] The Kalman filter prediction model can effectively predict the motion state of the marine cage railings, providing reliable prior information for target tracking. The Kalman filter algorithm is used to predict the motion state of the railings in the current frame and the next few frames based on the motion trajectory of the railings in the previous few frames, thereby assisting tracking.

[0244] S3.2, meanshift target tracking: Combined with the meanshift target tracking algorithm, by modeling the color histogram of the target area, find the area most similar to the target in the new image to achieve continuous locking of the railing target, as follows:

[0245] S3.2.1. Target model building: In the initial frame, a color histogram model is established for the target to be tracked, i.e., the area where the marine cage railing is located, and a weighted color histogram is calculated for the initial target area:

[0246]

[0247] in, Indicates the color histogram The value of the color range; is a normalization constant that ensures that the sum of all color range values ​​is 1; Indicates the sum of all pixels in the target area; is a kernel function used to weight pixels; is the position vector of the pixel relative to the center of the object; is the modulus of the pixel's position vector relative to the target center; the kernel function is usually chosen so that the closer the pixel is to the center, the greater the weight; is the Kronecker delta function; Represents the position vector of the pixel relative to the center of the object The value of the color range to which it belongs, when hour, ; hour, ;

[0248] S3.2.2. Candidate region modeling: In subsequent frames, a color histogram model is established for the candidate region that may include the target, and a weighted color histogram is calculated for the candidate region:

[0249]

[0250] in, is the center of the candidate region, and the 'candidate region' refers to the image region in the current frame image that may include the target, i.e., the railing of the marine cage; is the scale parameter;

[0251] S3.2.3. Similarity calculation: using Bhattacharyya coefficient , calculate the similarity between the target model and the candidate region model. In the meanshift target tracking algorithm, the Bhattacharyya coefficient between the target model and different candidate region models in the image is calculated, and the one that is most similar to the target model is selected. The candidate area with the largest value is used as the new target location:

[0252]

[0253] in, Indicates the candidate region of the target model in the color histogram values ​​in the color range, Indicates the color histogram model of the candidate region model in the first values ​​over a color range;

[0254] S3.2.4. Position update: Calculate the mean shift vector , the candidate regional center Plus The new target center position is obtained:

[0255]

[0256]

[0257] in, is the kernel function The derivative of is the mean shift vector weight, The value of the pixel to the target center The closer the distance, the greater the weight; is the bandwidth parameter, which controls the range of the kernel function;

[0258] The calculation formula is:

[0259]

[0260]

[0261] S3.2.5, Iteration: Repeat steps S3.2.2 to S3.2.4 until convergence or the maximum number of iterations is reached;

[0262] set up is the mean shift vector The displacement threshold, = 0.1 pixels, if , then converge, for The module outputs the final candidate region center As the tracking result of this frame; otherwise, enter the next iteration. In one embodiment, the maximum number of iterations is set to 20, and when the maximum number of iterations is reached, the final candidate region center is output. As the tracking result of this frame;

[0263] By continuously iterating, the mean offset vector is calculated and the target position is updated. , the meanshift algorithm can effectively track the railings of marine cages.

[0264] S3.3, Time series information fusion: The weighted average fusion of the marine cage railing positions detected in multiple consecutive frames is performed to obtain the final fusion position to improve the tracking stability and anti-interference ability; the position closer to the current frame is given a greater weight, and the position farther away from the current frame is given a smaller weight; the weighted average fusion mathematical expression is as follows:

[0265]

[0266] in Refers to the final fusion position; It is The position of the marine cage railing detected by the frame, It corresponds to The weight of the frame, The exponential decay form is used for calculation. The specific formula is:

[0267]

[0268] in, is the index of the frame, =1~ , is the index of the current frame (the latest frame), is the decay rate, ;

[0269] S3.4, Motion compensation: During tracking, according to the motion status of the unmanned vessel and the marine cage, the existing motion compensation standard algorithm, including the motion compensation algorithm based on optical flow, is used to compensate the tracking area to ensure that the railing target can be continuously locked.

[0270] S4, distance estimation: Based on the identified and tracked marine cage railings, the distance between the unmanned ship and the marine cage is estimated using monocular vision and scale information to provide a reference for the motion control of the unmanned ship, such as Figure 5 , including the following steps:

[0271] S4.1. Size-distance polynomial regression model building: By setting a certain amount of measured data, the size model of the marine cage railing at different distances is established, that is, the mapping relationship between the railing size and the distance. The specific steps are as follows:

[0272] S4.1.1. Data collection: Collect images of the marine cage railings at different distances and record the actual distances;

[0273] S4.1.2, Dimension measurement: Process the collected images to extract the pixel dimensions of the railing, including length and width;

[0274] S4.1.3, Data normalization: Normalize the measured pixel size to the range [0,1]:

[0275]

[0276] in, is the normalized pixel size value; Indicates the minimum value of all measured dimensions; Indicates the maximum value of all measured dimensions; is the pixel size value to be normalized;

[0277] S4.1.4. Model fitting: Use the size-distance polynomial regression model to fit the relationship between size and distance:

[0278]

[0279] in, is the polynomial order, usually 2-4; use the least squares method to solve the coefficients , , …, ; for distance;

[0280] S4.1.5. Model Validation: Use the cross-validation method to evaluate the accuracy and generalization ability of the size-distance polynomial regression model and calculate the root mean square error value. and the coefficient of determination ( ), in one embodiment, a threshold of the root mean square error value is set , set the threshold of the coefficient of determination ,if and , it indicates that the model fitting effect is good and meets the requirements, and proceed to step S4.1.6. If it does not meet the requirements, return to step S4.1.4 to adjust the polynomial order, or return to step S4.1.1 to set other nonlinear models (such as exponential model and logarithmic model);

[0281] S4.1.6. Model storage: Save the finalized model parameters for subsequent use.

[0282] S4.2, Scale-invariant feature matching: Using SIFT scale-invariant features, the scale information of the railing is extracted from the image and matched with the size-distance polynomial regression model to estimate the current distance; the scale space in the SIFT algorithm The Gaussian kernel and the input marine cage railing image Convolution gives:

[0283]

[0284] Among them, the Gaussian kernel function for:

[0285]

[0286] The SIFT feature descriptor is obtained by calculating the gradient direction histogram around the key point. In one embodiment, it is usually a 128-dimensional vector. is the scale parameter of the Gaussian kernel; , is the position coordinate of the pixel;

[0287] S4.3, time series filtering: perform time series filtering on the distance estimation results of multiple consecutive frames to reduce the distance jitter caused by factors such as water surface fluctuations, and finally obtain the distance between the unmanned boat and the marine cage.

Claims

1. A method for visual tracking and positioning of marine cage railings, characterized in that: The following steps are involved: S1, preprocessing the images collected by the monocular RGB camera; S2, identifying the railings of the marine cages in the preprocessed image, using a deep learning-based target detection algorithm to preliminarily detect the location of the marine cage railings, and then combining a feature matching method to further confirm the location of the marine cage railings; S3, continuously track the identified marine cage railings, use Kalman filtering to predict the motion state of the railings, and combine meanshift target tracking, time series information fusion, and motion compensation technology to achieve locking of the railing target; specifically, the following steps are included: S3.

1. Construct a Kalman filter prediction model and use the Kalman filter prediction model for prediction; specifically including: First, define the center coordinates of the marine cage railing ( , ) and speed( , )'s state vector , and Marine cage railings and Speed ​​in direction; Then, based on the assumption of uniform motion in a short period of time, the specific process of constructing the state transfer matrix and observation matrix is ​​as follows: State transition matrix It is expressed as: in is the time interval between two adjacent frames; Observation Matrix It is expressed as: ； Next, the covariance matrix of process noise and observation noise is set according to the actual situation. The specific process is as follows: Process noise covariance matrix Reflects the uncertainty of state prediction; uses a continuous-time white noise acceleration model to set : in, is the process noise intensity; Finally, the constructed Kalman filter model is used to predict the motion state of the marine cage railing, including: State prediction and covariance prediction: in, for The system state vector at time ; is the state transfer matrix, describing Time has come Linear relationship between states at different moments; for The system state vector at time ; is the control input matrix, describing how the control vector affects the state; for The control vector at the moment; is process noise, with mean 0 and covariance matrix Gaussian distribution of for The observation vector at time instant; is the observation matrix, describing the linear relationship between the state vector and the observation vector; is the observation noise, which has a mean of 0 and a covariance matrix of Gaussian distribution of Calculate the Kalman gain, perform state update and error covariance update to obtain the optimal state estimate: Calculate the Kalman gain: in, is the Kalman gain, which determines how much the observation should be trusted; is the prior error covariance obtained in the prediction step; is the observation matrix, which maps the state space to the observation space; is the observation noise covariance matrix; for Transpose of the observation matrix; Perform state update and covariance update: in, is the updated state estimate; is the prior state estimate obtained in the prediction step; is the actual observed value; It is called innovation or residual and represents the difference between the actual observation and the predicted observation. This formula combines the prior estimate with the weighted innovation to obtain the updated state estimate. in, is the updated error covariance matrix; is the identity matrix; S3.2, combined with the meanshift target tracking algorithm, by modeling the color histogram of the target area, find the area most similar to the target in the new image, and achieve continuous locking of the railing target; S3.

3. Perform weighted average fusion on the positions of the marine cage railings detected in multiple consecutive frames to obtain the final fused position, thereby improving the tracking stability and anti-interference ability. The positions closer to the current frame are given a greater weight, and the positions farther away from the current frame are given a smaller weight. The mathematical expression of weighted average fusion is as follows: in Refers to the final fusion position; It is The position of the marine cage railing detected by the frame, It corresponds to The weight of the frame, The exponential decay form is used for calculation. The specific formula is: in, is the index of the frame, =1~ , is the index of the current frame, is the decay rate; S3.

4. During tracking, according to the motion state of the unmanned ship and the marine cage, the existing standard motion compensation algorithm, including the motion compensation algorithm based on optical flow, is used to compensate the tracking area to ensure that the railing target can be continuously locked; S4. Based on the identified and tracked marine cage railings, estimate the distance between the unmanned boat and the marine cage using monocular vision and scale information; specifically including: S4.

1. Size-distance polynomial regression model building: by setting a certain amount of measured data, the size model of the marine cage railing at different distances is established, that is, the mapping relationship between the railing size and the distance; S4.2, Scale-invariant feature matching: Using SIFT scale-invariant features, the scale information of the railing is extracted from the image and matched with the size-distance polynomial regression model to estimate the current distance; the scale space in the SIFT algorithm The Gaussian kernel and the input marine cage railing image Convolution gives: Among them, the Gaussian kernel function for: The SIFT feature descriptor is obtained by calculating the gradient direction histogram around the key point. is the scale parameter of the Gaussian kernel; , is the position coordinate of the pixel; S4.3, time series filtering: perform time series filtering on the distance estimation results of multiple consecutive frames to reduce the distance jitter caused by water surface fluctuations, and finally obtain the distance between the unmanned boat and the marine cage.

2. A method for visual tracking and positioning of marine cage railings according to claim 1, characterized in that: In step S1, the image captured by the monocular RGB camera is preprocessed, including contrast enhancement, defogging / deblurring, and fluctuation compensation; The contrast enhancement is to enhance the image contrast using a histogram equalization algorithm, as follows: set up The image captured by the monocular RGB camera is used as the input image. for The pixel value in The probability density function of the contrast-enhanced image is It is expressed as: in, Indicates that the calculation result in the brackets is rounded to ensure that the final grayscale value is an integer; is the gray level, =256; Calculated by the following formula: in, is the pixel value The number of times it appears in the input image; and are the width and height of the input image respectively; The defogging / deblurring method uses a dark channel prior algorithm to remove image blur caused by sea fog or wave disturbance, and specifically includes the following steps: (1) Calculate the dark channel image: For the contrast-enhanced image , which is in pixel coordinates The dark channel value at Defined as: in, Represented in pixel coordinates The dark channel value at ; Represents the pixel coordinates in the image, expressed as ,in represents the horizontal coordinate, Represents the vertical coordinate; Indicates the color channel, the value range is , , and Represents the red, green and blue channels respectively. Among the minimum values ​​of the three color channels, take the minimum value ; In pixel coordinates A local window centered on the Represents a local window The pixel coordinates within the range are minimized. ; Representing images In pixel coordinates Color channels at The pixel value of (2) Estimation of atmospheric light value : Select dark channel image The first a% pixels with the highest brightness in the image, a is the set value, in the image after contrast enhancement The pixel with the highest brightness among the pixels at the corresponding position in is selected as the estimated atmospheric light value ; (3) Estimated transmittance : According to the dark channel prior, the transmittance Estimated to be: For each color channel }, find the pixel coordinates The local window at the center All pixels in the channel The pixel value and atmospheric illumination value The minimum value of the ratio: ; Take the minimum value among the minimum values ​​of the three color channels ; Minimum value Multiplying the parameters , and then subtract the result from 1 to get the final transmittance estimate ; To retain the set fog parameters; Indicates that atmospheric light is in the color channel Atmospheric illumination refers to the scattered light in the air, which is enhanced by contrast. The brightest area in the image is estimated; (4) Image restoration: Obtain the dehazed image based on the atmospheric scattering model : in, is the set minimum transmittance; The wave compensation is to estimate the motion of the unmanned ship and the marine cage by detecting the motion vector between adjacent frames, and to perform motion compensation on the image to reduce the motion blur caused by waves, which specifically includes the following steps: (1) Use the FAST corner detection algorithm to detect feature points in two adjacent frames; (2) Use the KLT optical flow algorithm to match the detected feature points and obtain the motion vector of the feature points; (3) Use the RANSAC algorithm to remove the feature points matched in the abnormal feature matching and estimate the global motion model; (4) Based on the estimated global motion model, the current frame image is inversely transformed to compensate for the motion caused by fluctuations.

3. A method for visual tracking and positioning of marine cage railings according to claim 2, characterized in that: The RANSAC algorithm is used to remove the feature points matched in the abnormal feature matching and estimate the global motion model, which specifically includes the following steps: (3.1) Random sampling: Randomly select a subset from all matched feature points as the hypothesis set; (3.2) Affine transformation model fitting: Use the matched feature points in the hypothesis set to calculate a model parameter set using the least squares method to obtain a fitted affine transformation model; (3.3) Calculate the error: Substitute all matched feature points into the fitted affine transformation model, calculate the error of each matched feature point, set a threshold, and distinguish between inliers and outliers; the error is defined as the distance between the actual position of the matched feature point and the position predicted by the affine transformation model; points with errors less than the preset threshold are considered inliers and conform to the fitted affine transformation model; points with errors greater than the preset threshold are considered outliers and belong to matched abnormal feature points; (3.4) Repeating process: Repeat iterative steps (3.1) to (3.3) for a set number of times, and randomly select a different hypothesis set each time; for each hypothesis set, the calculated model will produce a different set of internal points, and obtain an affine transformation model with a different set of model parameters; (3.5) Model selection: The affine transformation model with the most inliers is selected as the final result and the matched abnormal feature points in the selected affine transformation model are eliminated. The final affine transformation model obtained is the estimated global motion model.

4. A method for visual tracking and positioning of marine cage railings according to claim 1, characterized in that: Step S2 specifically includes: S2.

1. Use the YOLOv10 network to detect the position of the marine cage railings. The specific process is as follows: First, data preparation and preprocessing were performed, including collecting a set number of marine cage railing images, annotating them with labelImg software, performing data enhancement, and adjusting the marine cage railing images after data enhancement to the size and normalization required by the YOLOv10 network to construct a training set. Construct a YOLOv10 network, use CSPDarknet53 as the backbone network, combine SPP module and PANet for feature extraction and fusion, and set detection heads at three scales; the three scales include small-scale feature maps, medium-scale feature maps, and large-scale feature maps, corresponding to detection targets of different sizes respectively; Constructing the total loss function : CIoU loss is used to construct bounding box regression, BCE loss is used to construct target confidence, and Focal loss is used to construct category classification. Finally, the three are weighted and summed to obtain the total loss function , as follows: Use CIoU (Complete IoU) loss to build bounding box regression loss : in, It is the intersection of the predicted box and the true box; It is the Euclidean distance between the center point of the predicted box and the true box; is the diagonal length of the minimum enclosing rectangle that encloses the predicted box and the true box; is the orthogonality coefficient; is the aspect ratio consistency measure; is the predicted bounding box; is the true bounding box; Use binary cross entropy (BCE) loss to construct target confidence loss : in, is the true label, is 0 or 1; is the confidence score of the prediction; Use Focal loss to construct category classification loss : in, Yes Category , i.e., the predicted probability of the ocean cage railing; is the category balancing factor; is the focus parameter; Total loss function It is the weighted sum of the three loss values: in, , , is the weight coefficient of each part of the loss; The YOLOv10 network is trained using the training set. During the training process, a transfer learning strategy is adopted, and the YOLOv10 network is initialized using pre-trained weights. The training adopts a phased strategy, first fixing the backbone network and only training the detection head. At the same time, Mosaic data enhancement and adaptive image scaling are used, and cosine annealing is used to adjust the learning rate. In the inference phase, the output results of the trained YOLOv10 network, including bounding box coordinates and target confidence scores, are processed by NMS and confidence threshold filtering respectively, and the bounding box coordinates after NMS processing are mapped back to the original image scale; finally, the model is quantized and accelerated through TensorRT, and cross-platform deployment is achieved in ONNX format; S2.

2. Feature-based fine recognition: After the initial detection of the area of ​​the marine cage railing, the color, texture, and shape features of the area are extracted and matched with the pre-established marine cage railing feature template library to further confirm whether it is a genuine marine cage railing.

5. The method for visual tracking and positioning of marine cage railings according to claim 1, characterized in that: In step S3.2, combined with the meanshift target tracking algorithm, by modeling the color histogram of the target area, the area most similar to the target is found in the new image to achieve continuous locking of the railing target, as follows: S3.2.

1. Target model building: In the initial frame, a color histogram model is established for the target to be tracked, i.e., the area where the marine cage railing is located, and a weighted color histogram is calculated for the initial target area: in, Indicates the color histogram The value of the color range; is a normalization constant that ensures that the sum of all color range values ​​is 1; Indicates the sum of all pixels in the target area; is a kernel function used to weight pixels; is the position vector of the pixel relative to the center of the object; is the magnitude of the pixel's position vector relative to the target center; is the Kronecker delta function; Represents the position vector of the pixel relative to the center of the object The value of the color range to which it belongs, when hour, ; hour, ; S3.2.

2. Candidate region modeling: In subsequent frames, a color histogram model is established for the candidate region that may include the target, and a weighted color histogram is calculated for the candidate region: in, is the center of the candidate region, and the candidate region refers to the image region in the current frame image that may include the target, i.e., the railing of the marine cage; is the scale parameter; S3.2.

3. Similarity calculation: using Bhattacharyya coefficient , calculate the similarity between the target model and the candidate region model. In the meanshift target tracking algorithm, the Bhattacharyya coefficient between the target model and different candidate region models in the image is calculated, and the one that is most similar to the target model is selected. The candidate area with the largest value is used as the new target location: in, Indicates the candidate region of the target model in the first values ​​in the color range, Indicates the color histogram model of the candidate region model in the first values ​​over a color range; S3.2.

4. Position update: Calculate the mean shift vector , the candidate regional center Plus The new target center position is obtained: in, is the kernel function The derivative of is the mean shift vector weight; is the bandwidth parameter, which controls the range of the kernel function; The calculation formula is: S3.2.5, Iteration: Repeat steps S3.2.2 to S3.2.4 until convergence or the maximum number of iterations is reached; set up is the mean shift vector If the displacement threshold , then converge, for The module outputs the final candidate region center As the tracking result of this frame; otherwise, enter the next iteration, reach the maximum number of iterations, and output the final candidate area center As the tracking result of this frame.

6. A method for visual tracking and positioning of marine cage railings according to claim 1, characterized in that: In step S4.1, by setting a certain amount of measured data, a size model of the marine cage railing at different distances is established, that is, a mapping relationship between the railing size and the distance. The specific steps are as follows: S4.1.

1. Data collection: Collect images of the marine cage railings at different distances and record the actual distances; S4.1.2, Dimension measurement: Process the collected images to extract the pixel dimensions of the railing, including length and width; S4.1.3, Data normalization: Normalize the measured pixel size to the range [0,1]: in, is the normalized pixel size value; Indicates the minimum value of all measured dimensions; Indicates the maximum value of all measured dimensions; is the pixel size value to be normalized; S4.1.

4. Model fitting: Use the size-distance polynomial regression model to fit the relationship between size and distance: in, is the polynomial order; use the least squares method to solve for the coefficients , , …, ; for distance; S4.1.

5. Model Validation: Use the cross-validation method to evaluate the accuracy and generalization ability of the size-distance polynomial regression model and calculate the root mean square error value. and the coefficient of determination , set the threshold of the root mean square error value , set the threshold of the coefficient of determination ,if and , it indicates that the model fitting effect is good and meets the requirements, and proceed to step S4.1.

6. If it does not meet the requirements, return to step S4.1.4 to adjust the polynomial order, or return to step S4.1.1 to set other nonlinear models; S4.1.

6. Model storage: Save the finalized model parameters.

7. A system for implementing the method for visual tracking and positioning of marine cage railings according to any one of claims 1 to 6, characterized in that: The system includes: Image preprocessing module: preprocess the images collected by the monocular RGB camera; Railing recognition module: Identify the railings of the marine cages in the preprocessed image, use the deep learning-based target detection algorithm to preliminarily detect the location of the marine cage railings, and then combine the feature matching method to further confirm the location of the marine cage railings; Railing tracking module: continuously tracks the identified marine cage railings, uses Kalman filtering to predict the motion state of the railings, and combines meanshift target tracking, time series information fusion, and motion compensation technology to lock the railing targets; Distance estimation module: Based on the identified and tracked marine cage railings, the distance between the unmanned ship and the marine cage is estimated using monocular vision and scale information, providing a reference for the motion control of the unmanned ship.

Images