Ship overwater multi-target detection method

Abstract

The invention relates to the technical field of ship navigation, and discloses a ship overwater multi-target detection method comprising the following steps: S1, obtaining a real-time original image of a water area where a ship is located; s2, improving the resolution of the original image by using a super-resolution reconstruction algorithm, and generating a high-definition image; s3, inputting the high-definition image into a pre-trained Faster R-CNN model, and carrying out target detection; s4, outputting a position coordinate, a category label and a confidence score of the target through the Faster R-CNN model; s5, starting a multi-target tracking algorithm, and continuously monitoring the motion trail; s6, calculating MCPA and MTCPA according to the motion trail, and calculating a collision risk through a collision risk assessment algorithm; and S7, when the MCPA and the MTCPA of any target are lower than a safety threshold, sending out an early warning signal to indicate to take a collision avoidance measure. According to the method, the high-definition image is generated through the super-resolution reconstruction algorithm, the Faster R-CNN model is utilized, the water target is rapidly and accurately detected, the detection precision and generalization ability are remarkably improved, and navigation safety is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of ship navigation technology, and in particular to a method for detecting multiple targets on a ship. Background Art

[0002] With the increasing traffic on the water, ship safety has become a global concern. Traditional target detection methods rely on low-resolution images and rule-based algorithms, which often cannot meet the requirements of high precision and real-time performance in complex and changing marine environments. In addition, due to the large morphological changes of marine targets such as other ships, buoys, and drifting objects under different light and weather conditions, traditional image processing techniques are difficult to effectively distinguish and track these targets, especially at long distances. The low image resolution leads to the loss of target features, which seriously affects the accuracy and reliability of target detection. In recent years, the development of deep learning technology has provided a new way to solve the above problems. Super-resolution reconstruction technology can restore high-resolution images from low-resolution images, significantly improving image details, while deep learning models such as FasterR-CNN can effectively identify and locate multiple types of targets. However, how to effectively integrate these two technologies and apply them to actual multi-target detection on ships still faces a series of challenges, including the generalization ability of the model, computational efficiency, and how to continuously track and evaluate the collision risk of targets in a dynamically changing marine environment. Summary of the invention

[0003] In order to overcome or alleviate one or more of the above technical problems, the purpose of the present invention is to provide a method for detecting multiple targets on water of ships. By generating high-definition images through a super-resolution reconstruction algorithm and using the Faster R-CNN model, the present invention can quickly and accurately detect and classify various types of water targets, including ships, buoys and drifting objects, etc., significantly improve the detection accuracy and generalization ability, improve the target recognition accuracy and collision warning ability of ships in complex sea conditions, and ensure navigation safety.

[0004] The present invention provides the following technical solutions:

[0005] A method for detecting multiple targets on a ship, comprising the following steps:

[0006] S1: Obtain the real-time original image of the waters where the ship is located;

[0007] S2: using a super-resolution reconstruction algorithm to improve the resolution of the original image to generate a high-definition image;

[0008] S3: Inputting the high-definition image generated in step S2 into a pre-trained Faster R-CNN model to perform target detection, wherein the Faster R-CNN model is trained and optimized for targets in an ocean environment, and the targets include ships, buoys, and drifting objects;

[0009] S4: Output the location coordinates, category label and confidence score of the target through the Faster R-CNN model;

[0010] S5: for each target detected in step S3, start a multi-target tracking algorithm to continuously monitor its motion trajectory, wherein the tracking algorithm at least includes Kalman filter prediction and Hungarian algorithm matching;

[0011] S6: According to the motion trajectory obtained in step S5, the relative motion state between the target and the ship is evaluated, the minimum encounter distance MCPA and the minimum encounter time MTCPA are calculated, and the collision risk is calculated by a collision risk assessment algorithm;

[0012] S7: When the MCPA and MTCPA of any target are lower than the preset safety threshold, an early warning signal is sent to the crew or the ship control system to instruct them to take collision avoidance measures.

[0013] According to some embodiments, the super-resolution reconstruction algorithm in step S2 is based on a deep learning model to enhance image resolution, which includes fine-tuning the hyperparameters of the model and dynamic parameter adjustment, wherein the hyperparameters include a learning rate, a batch size, and a number of iterations, and includes the following steps:

[0014] S21: Learning rate tuning, determine the baseline learning rate:

[0015] First, we select 0.001 as the baseline learning rate for the model. We use the Learning Rate Finder algorithm to gradually increase the learning rate, starting from 1e-5, until the training loss of the model begins to rise sharply. By plotting the relationship between learning rate and loss, we find the range of learning rates before the loss begins to increase significantly, which is about 0.0001 to 0.01.

[0016] Secondly, the learning rate scheduling adopts the Cosine Annealing strategy based on the found learning rate range, starting from 0.001. As the training cycle progresses, the learning rate changes periodically until it reaches the minimum value of 0.0001, and then rises back to 0.001 to form a complete cycle.

[0017] S22: Batch size tuning:

[0018] First, resource assessment, evaluate the hardware resources of the training platform;

[0019] Secondly, gradient accumulation. If the memory of a single GPU is limited, a gradient accumulation strategy is used;

[0020] S23: Iteration number tuning:

[0021] Set up an early stopping mechanism. If the loss on the validation set does not improve significantly within 10 consecutive epochs, stop training. Combine the Cosine Annealing learning rate scheduling strategy. Perform at least 3 complete learning rate cycles. Each cycle takes about 200 epochs. Therefore, the total number of iterations is set to 600 epochs.

[0022] S24: Dynamic parameter adjustment:

[0023] The light intensity and weather condition data are obtained in real time by using the light sensors and weather stations installed on the ship. The light intensity and weather condition data are used to dynamically adjust the attention weights in the model so that the model can maintain high detection performance under different light and weather conditions. An adaptive activation function, such as Swish or Mish, is introduced into the model, which is dynamically adjusted according to the distribution of the input data so that the model can process image features under different light and weather conditions.

[0024] According to some embodiments, the Faster R-CNN model training process of step S3 is an integrated attention mechanism, comprising the following steps:

[0025] S31: Data preprocessing and enhancement: Use cleaning software to clean and annotate ocean images containing ships, buoys, and drifting objects to ensure that each target is accurately labeled. During the training process, random cropping, horizontal flipping, color jittering, and random noise addition data enhancement techniques are used to improve the generalization ability of the model;

[0026] S32: Integrated attention mechanism: In the feature extraction network of Faster R-CNN, the SENet module is embedded to learn the importance of each channel through global average pooling and fully connected layers to adaptively adjust the weight of the feature map;

[0027] S33: Spatial Attention: A spatial attention mechanism is added to the region proposal network (RPN) and the detection head. By calculating the weight of each position in the feature map, the model can focus more on the key areas in the image, improving the detection speed and accuracy.

[0028] S34: Training strategy and evaluation: First, train the RPN network alone until it converges; then freeze the RPN part and only train the detection head; finally, unfreeze the entire network for joint training to optimize the overall performance; during the training process, dynamically adjust the NMS threshold according to the model's prediction confidence to optimize the detection results, use the average precision, recall rate and accuracy indicators to evaluate the model performance, and continuously monitor these indicators during the training process to adjust the training strategy and model parameters in time.

[0029] According to some embodiments, in step S5, the multi-target tracking algorithm further includes assigning a unique identifier to the target, and using a Kalman filter for prediction and a Hungarian algorithm for matching the target with the detection result to achieve accurate trajectory tracking, including the following steps:

[0030] S51: For a newly detected target, first record its initial position and timestamp as the tracking starting point; initialize the state variables of the Kalman filter, including position, velocity, acceleration, and covariance matrix; assign a unique tracking ID to each new target for subsequent tracking and data association;

[0031] S52: Kalman filter prediction, using the prediction step of the Kalman filter, based on the state estimation and control input at the previous moment, predict the target state at the current moment, and the prediction formula is as follows:

[0032] X(k|k-1) = F(k)*X(k-1|k-1)+B(k)U(k) (1)

[0033] Where X(k|k-1) represents the predicted state, F(k) is the state transfer matrix, B(k) is the control input matrix, and U(k) is the control input vector;

[0034] At the same time, the covariance matrix of the predicted state estimate reflects the uncertainty of the state estimate, and the prediction formula is:

[0035] P(k|k-1)=F(k)P(k-1|k-1)*F(k)+Q(k) (2)

[0036] Where P(k|k-1) is the prediction covariance and Q(k) is the process noise covariance matrix;

[0037] S53: Hungarian algorithm matching, comparing the position predicted by the Kalman filter with the target position detected in the current frame, constructing a cost matrix, in which each element represents the distance between the predicted target and the detection result; the rows of the cost matrix correspond to the predicted targets of the Kalman filter, and the columns correspond to the detection results of the current frame; applying the Hungarian algorithm to solve the optimal matching of the cost matrix, the goal is to minimize the total cost; the Hungarian algorithm ensures that each predicted target is matched with only one detection result, and vice versa, avoiding multiple matches or no matches;

[0038] S54: Tracking status update:

[0039] For the predicted target that matches successfully, the Kalman filter update step is used to update the state estimate in combination with the detection results. The update formula is:

[0040] X(k|k)=X(k|k-1)+K(k)*(Z(k)-H(k)*X(k|k-1)) (3)

[0041] Where X(k|k) is the updated state estimate; Z(k) is the measurement value; H(k) is the measurement matrix; K(k) is the Kalman gain;

[0042] For unmatched predicted targets, only the prediction step is used to update the state estimate, indicating that the target may not be detected temporarily. For unmatched detection results, it may be a newly appeared target, which is initialized and tracked;

[0043] S55: Based on the target's motion trajectory, the relative speed and direction of the target relative to the ship are calculated. The current position and speed of the target and the speed information of the ship are used to obtain the relative speed and direction through vector calculation. Minimum approach distance (MCPA) and minimum approach time (MTCPA) calculation: According to the relative speed and direction, the relative distance (MCPA) and time (MTCPA) at the closest moment between the target and the ship are calculated. The MTCPA calculation formula is:

[0044]

[0045] Among them, r is the relative position vector of the target and the ship, v is the relative velocity vector, υ ⊥ is the perpendicular vector to v.

[0046] According to some embodiments, in step S6, the collision risk assessment algorithm dynamically adjusts the safety thresholds of MCPA and MTCPA based on the motion trajectory and speed of the target.

[0047] According to some embodiments, step S7 comprises the following steps:

[0048] S71: Performing automated early warning and obstacle avoidance. When it is detected that the MCPA and MTCPA are lower than the preset safety threshold, an early warning signal is immediately triggered; the early warning signal is sent to the navigation cabin through the ship's automated driving assistance system, and the automatic obstacle avoidance mechanism is activated at the same time;

[0049] S72: According to the specific values ​​of MCPA and MTCPA, the warning signals are divided into different levels of urgency to guide the crew or the automated system to take appropriate collision avoidance measures.

[0050] According to some embodiments, step S7 further comprises:

[0051] S73: Data recording and strategy optimization, used to store target detection results, motion trajectories, collision risk assessments and early warning responses, so as to facilitate post-analysis and optimization of the ship's multi-target detection strategy on the water.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] This paper proposes an innovative method for multi-target detection of ships on water. By integrating SRR technology and deep learning models, it can efficiently and accurately detect and evaluate the collision risks of multiple targets in complex marine environments.

[0054] The quality of detected images has been significantly improved. The use of SRR technology has effectively improved the resolution of the original images of the waters where the ship is located, making the target details clearer and providing high-quality image input for subsequent target detection.

[0055] The target detection accuracy is high. Based on the pre-trained Faster R-CNN model, the present invention can quickly and accurately detect and classify various types of water targets, including ships, buoys and drifting objects, which significantly improves the detection accuracy and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of a method for detecting multiple targets on a ship in water provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention is described in detail below in conjunction with the embodiments and drawings, but it should be understood that the embodiments and drawings are only used to exemplify the present invention and do not constitute any limitation on the protection scope of the present invention. All reasonable changes and combinations within the scope of the inventive concept of the present invention fall within the protection scope of the present invention.

[0058] The present invention will be further described below in conjunction with the accompanying drawings.

[0059] Before implementation, multiple high-definition cameras were installed at key locations on the ship, including the bow, stern, and port and starboard sides, to ensure full coverage without blind spots. The cameras are waterproof, dustproof, and vibration-resistant to adapt to the complex marine environment. In addition, infrared cameras are used at night or in low light conditions to ensure image quality.

[0060] According to Figure 1 In the process shown in the figure, the multi-target detection of ships on the water is first performed. The camera captures real-time video streams at a fixed frequency (such as 25 frames per second), which are transmitted to the central processing unit through a high-speed network. The collected raw images first go through a preprocessing stage, including color correction, contrast enhancement, denoising, etc., to improve the image quality. During preprocessing, the particularity of the marine environment, such as the influence of factors such as fog, salt, and waves, needs to be taken into account.

[0061] In step S2, a super-resolution reconstruction model based on deep learning, such as SRCNN, is selected. This model needs to be pre-trained on a large number of low-resolution and high-resolution image pairs to learn the ability to recover high-resolution images from low-resolution images. The training dataset should contain a wealth of marine environment images, covering scenes under different weather, lighting and sea conditions. To ensure the performance of the model in practical applications, the model's hyperparameters need to be fine-tuned, including learning rate, batch size, number of iterations, etc. In addition, considering the variability of the marine environment, the model can adjust its parameter settings according to real-time lighting, weather and other conditions.

[0062] In step S3, a Faster R-CNN target detection model, such as SSD, is used to train on a large-scale dataset containing targets such as ships, buoys, and drifting objects. During the training process, an attention mechanism can be added to enable the model to focus more on key areas in the image, thereby improving detection speed and accuracy. The key point of this step is to fine-tune the model hyperparameters and dynamically adjust the parameters.

[0063] Example 1

[0064] This embodiment provides a super-resolution reconstruction algorithm based on the Faster R-CNN architecture, and also integrates an attention mechanism to improve detection accuracy. The goal of the super-resolution reconstruction algorithm is to ensure that the model maintains high detection performance in the complex and changing environment of the sea by fine-tuning the model's hyperparameters (learning rate, batch size, number of iterations) and dynamic parameter adjustment strategies.

[0065] S21: Learning rate tuning, determine the baseline learning rate:

[0066] First, based on literature research and preliminary experiments, 0.001 was selected as the baseline learning rate of the model.

[0067] Learning Rate Finder: Using the Learning Rate Finder algorithm, gradually increase the learning rate, starting from 1e-5, until the model's training loss begins to rise sharply. By plotting the relationship between learning rate and loss, we find the range of learning rates before the loss starts to increase significantly, which is about 0.0001 to 0.01.

[0068] Learning rate scheduling: Based on the found learning rate range, the Cosine Annealing strategy is adopted, starting from 0.001. As the training cycle progresses, the learning rate changes periodically until it reaches the minimum value of 0.0001, and then rises back to 0.001 to form a complete cycle.

[0069] S22: Batch size tuning:

[0070] Resource assessment: Considering that the training platform is equipped with an NVIDIA RTX 2080Ti GPU, the available memory of a single graphics card is 11GB. After preliminary testing, it was found that the model can run smoothly without exceeding the memory limit when using a batch size of 8.

[0071] Gradient accumulation: Due to the memory limitation of a single GPU, a gradient accumulation strategy is adopted, that is, accumulating the gradients of 4 batches in each update step, which is equivalent to using a logical batch size of 32 for training.

[0072] S23: Iteration number tuning:

[0073] An early stopping mechanism is set to stop training if the loss on the validation set does not improve significantly within 10 consecutive epochs. Combined with the Cosine Annealing learning rate scheduling strategy, we plan to perform at least 3 full learning rate cycles, each cycle takes about 200 epochs, so the total number of iterations is set to 600 epochs.

[0074] S24: Dynamic parameter adjustment:

[0075] The light intensity and weather condition data are obtained in real time using the light sensors and weather stations installed on the ship. These data are used to dynamically adjust the attention weights in the model so that the model can maintain high detection performance under different light and weather conditions. Adaptive activation functions such as Swish or Mish are introduced into the model, which can be dynamically adjusted according to the distribution of the input data. This helps the model better handle image features under different light and weather conditions.

[0076] In step S3, Faster R-CNN with integrated attention mechanism is trained. The Faster R-CNN model has been pre-trained on a large-scale marine environment dataset, including instance data of target objects such as ships, buoys, and drifting objects, to ensure the generalization ability and accuracy of detection. The Faster R-CNN with integrated attention mechanism training includes the following steps:

[0077] S31: Data preprocessing and enhancement: Use professional software to clean and annotate ocean images containing ships, buoys, and drifting objects to ensure that each target is accurately labeled. During the training process, random cropping, horizontal flipping, color jittering, and random noise addition data enhancement techniques are used to improve the generalization ability of the model;

[0078] S32: Integrated attention mechanism: In the feature extraction network of Faster R-CNN, the SENet module is embedded to learn the importance of each channel through global average pooling and fully connected layers to adaptively adjust the weight of the feature map;

[0079] S33: Spatial Attention: A spatial attention mechanism is added to the region proposal network (RPN) and the detection head. By calculating the weight of each position in the feature map, the model can focus more on the key areas in the image, improving the detection speed and accuracy.

[0080] S34: Training strategy and evaluation: First, train the RPN network alone until it converges, then freeze the RPN part and train only the detection head. Finally, unfreeze the entire network for joint training to optimize the overall performance. During the training process, dynamically adjust the threshold of NMS (Non-Maximum Suppression) according to the prediction confidence of the model to optimize the detection results. Use indicators such as mean average precision (mAP), recall rate, and precision to evaluate model performance, and continuously monitor these indicators during the training process to adjust the training strategy and model parameters in a timely manner.

[0081] Through the implementation of the above-mentioned S21~S24 and S31~S34 solutions, not only the hyperparameters of the model are fine-tuned, but also the performance of Faster R-CNN in the field of marine ship detection is significantly improved by integrating the attention mechanism, ensuring the high detection accuracy and robustness of the model in complex marine environments. The attention mechanism prioritizes focusing on key areas in the image that may contain targets, thereby accelerating the detection process and reducing the consumption of computing resources.

[0082] In order to better adapt to the characteristics of the marine environment, the model needs to undergo additional optimization. The optimization step is performed before step S7 and before model training and application to improve the model's ability to detect small targets and the recognition rate of targets under low contrast or low light conditions. Specifically, it is performed during the training of the super-resolution reconstruction model and the Faster R-CNN target detection model mentioned in steps S2 and S3 to increase the detection capability for small targets and improve the recognition rate of targets under low contrast or low light conditions.

[0083] Other additional optimization methods include:

[0084] 1. Optimize the network structure and add an attention mechanism for the recognition of small targets of ships on the water based on the Faster R-CNN network. The specific implementation steps of the attention mechanism are as follows:

[0085] Step 1: Select ResNet-50 as the feature extraction network of Faster R-CNN. Insert the SENet module before the last layer of ResNet-50.

[0086] Step 2: The feature maps output from the last convolutional layer of ResNet-50 are subjected to global average pooling to obtain a fixed-length vector.

[0087] Step 3: Create two fully connected layers. The output dimension of the first fully connected layer is half the number of channels of the feature map, and the output dimension of the second fully connected layer is the number of channels of the feature map. The ReLU activation function is used after the first fully connected layer, and the sigmoid activation function is used after the second fully connected layer. The output weight vector is the same as the number of channels of the feature map.

[0088] Step 4: Multiply the weight vector output by the sigmoid activation function with the original feature map. Through the dot multiplication operation, the weight of each channel is applied to the corresponding feature map channel to obtain the weighted feature map.

[0089] 2. Preprocess the input image to enhance the contrast of the image and make the ship target features more obvious so that the network can better capture the ship target features and improve the network's recognition rate of the ship.

[0090] The image preprocessing method includes the following steps:

[0091] Step 1: Read the image: Read the image using an image processing library such as OpenCV.

[0092] Step 2: Convert to Grayscale Image: If it is a color image, convert it to grayscale image first, which helps to reduce the computational complexity and simplify the features.

[0093] Step 3: Histogram Equalization: Apply global histogram equalization or local contrast enhancement (such as CLAHE) to increase image contrast.

[0094] Step 4: Gaussian filtering: Use a Gaussian filter to smooth the image to reduce the noise.

[0095] Step 5: Canny Edge Detection: Use Canny edge detection to highlight edge features in the image, which helps to enhance the outline of the target.

[0096] Step 6: Merge edge information: Merge the denoised image with the edge information to maintain the main information while enhancing the edge features.

[0097] Step 7: Save the preprocessed images: Save the processed images as part of the training set.

[0098] In step S5, a combination of Kalman filter and Hungarian algorithm is used to track multiple targets. The Kalman filter is used to predict the future position of the target, and the Hungarian algorithm is used to find the best target-detection result match in each frame. For the newly detected target, its tracking state is initialized and a unique ID is assigned. As the video stream progresses, the target's position information and tracking state are continuously updated based on the prediction results of the Kalman filter and the matching results of the Hungarian algorithm.

[0099] In step S6, the collision risk assessment algorithm calculates the relative speed and direction of the target relative to the ship based on the target's motion trajectory record, and then infers the minimum distance of encounter (MCPA) and the minimum time of encounter (MTCPA). Using the pre-trained deep learning model, real-time target detection is performed on each frame of video to identify targets such as ships, buoys, and drifting objects. The output of the deep learning model includes the target category, bounding box coordinates, and confidence score.

[0100] Therefore, the specific steps of S5 are as follows:

[0101] S51: For a newly detected target, the system first records its initial position and timestamp as the tracking starting point. Initialize the state variables of the Kalman filter, including position, velocity, acceleration, etc., as well as the covariance matrix. Assign a unique tracking ID to each new target for subsequent tracking and data association.

[0102] S52: Kalman filter prediction, using the prediction step of the Kalman filter, based on the state estimation and control input (such as acceleration) at the previous moment, predict the target state at the current moment, and the prediction formula is as follows:

[0103] X(k|k-1) = F(k)*X(k-1|k-1)+B(k)U(k) (1)

[0104] Where X(k|k-1) represents the predicted state, F(k) is the state transfer matrix, B(k) is the control input matrix, and U(k) is the control input vector;

[0105] At the same time, the covariance matrix of the predicted state estimate is used to reflect the uncertainty of the state estimate;

[0106] The prediction formula is:

[0107] P(k|k-1)=F(k)P(k-1|k-1)*F(k)+Q(k) (2)

[0108] Where P(k|k-1) is the prediction covariance and Q(k) is the process noise covariance matrix;

[0109] S53: Hungarian algorithm matching

[0110] Compare the position predicted by the Kalman filter with the position of the target detected in the current frame, and construct a cost matrix, in which each element represents the distance between the predicted target and the detection result; the rows of the cost matrix correspond to the predicted targets of the Kalman filter, and the columns correspond to the detection results of the current frame. Apply the Hungarian algorithm to solve the optimal match of the cost matrix, with the goal of minimizing the total cost (i.e., minimizing the total distance); the Hungarian algorithm ensures that each predicted target is matched with only one detection result, and vice versa, avoiding multiple matches or no matches;

[0111] S54: Tracking status update:

[0112] For the predicted target that matches successfully, the Kalman filter update step is used to update the state estimate in combination with the detection results. The update formula is:

[0113] X(k|k)=X(k|k-1)+K(k)*(Z(k)-H(k)*X(k|k-1)) (3)

[0114] Where X(k|k) is the updated state estimate; Z(k) is the measurement value; H(k) is the measurement matrix; K(k) is the Kalman gain;

[0115] For unmatched predicted targets, only the prediction step is used to update the state estimate, indicating that the target may not be detected temporarily. For unmatched detection results, which may be new targets, they are initialized and tracked.

[0116] S55: Based on the target's motion trajectory, calculate the target's relative speed and direction relative to the ship, and use the target's current position, speed and the ship's speed information to obtain the relative speed and direction through vector calculation; Minimum approach distance (MCPA) and minimum approach time (MTCPA) calculation: Based on the relative speed and direction, calculate the relative distance (MCPA) and time (MTCPA) at the moment of closest approach between the target and the ship. The MTCPA calculation formula is:

[0117]

[0118] Among them, r is the relative position vector of the target and the ship, v is the relative velocity vector, υ ⊥ is the perpendicular vector to v.

[0119] The modules of target detection, Kalman filter prediction, Hungarian algorithm matching, tracking status update and collision risk assessment are integrated into a complete multi-target tracking and collision risk assessment system. The system receives video stream input and outputs the tracking status (position, velocity, acceleration) and collision risk assessment results of the target. For the target detection model, the detection accuracy and speed are improved by means of data enhancement and model fine-tuning. For the Kalman filter, the state transfer matrix, measurement matrix and noise covariance matrix are reasonably set to improve the accuracy of prediction and update.

[0120] Through the detailed implementation steps described above, the multi-target tracking and collision risk assessment system can track targets such as ships, buoys, drifting objects on the sea surface in real time and accurately, and assess the collision risks between these targets and the ship itself, providing important decision-making support for safe navigation at sea.

[0121] In step S6, the safety thresholds of MCPA and MTCPA are dynamically adjusted according to factors such as the current sea conditions, ship speed, and target type to ensure the timeliness and accuracy of the warning signal to adapt to different navigation conditions and environmental factors.

[0122] Step S7 includes the following steps:

[0123] S71: Perform automatic early warning and obstacle avoidance. When it is detected that MCPA and MTCPA are lower than the preset safety threshold, the early warning signal is immediately triggered; the early warning signal is sent to the navigation cabin through the ship's automated driving assistance system, and the automatic obstacle avoidance mechanism is activated at the same time, so as to reduce manual operation delays and improve collision avoidance efficiency;

[0124] S72: According to the specific values ​​of MCPA and MTCPA, the warning signals are classified into different emergency levels to guide the crew or the automated system to take corresponding collision avoidance measures;

[0125] S73: Data recording and strategy optimization. All detection results, target motion trajectories, collision risk assessments and warning responses obtained in steps S4 to S71 are recorded and stored for subsequent analysis. The data analysis results can be used to optimize detection algorithms, warning level assessments, adjust warning strategies, and improve automated driving assistance systems, thereby continuously improving the overall performance of multi-target detection on ships.

[0126] Through the above detailed implementation steps, this embodiment can ensure the efficient detection, accurate tracking and safety warning of various targets by ships in complex waters during implementation, providing solid technical support for maritime navigation safety.

[0127] The above embodiments are only preferred implementations of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting multiple targets on a ship on water, characterized in that: The following steps are involved: S1: Obtain the real-time original image of the waters where the ship is located; S2: using a super-resolution reconstruction algorithm to improve the resolution of the original image and generate a high-definition image; S3: Inputting the high-definition image generated in step S2 into a pre-trained Faster R-CNN model to perform target detection, wherein the Faster R-CNN model is trained and optimized for targets in an ocean environment, and the targets include ships, buoys, and drifting objects; S4: Output the location coordinates, category label and confidence score of the target through the Faster R-CNN model; S5: for each target detected in step S3, start a multi-target tracking algorithm to continuously monitor its motion trajectory, wherein the tracking algorithm at least includes Kalman filter prediction and Hungarian algorithm matching; S6: According to the motion trajectory obtained in step S5, the relative motion state between the target and the ship is evaluated, the minimum encounter distance MCPA and the minimum encounter time MTCPA are calculated, and the collision risk is calculated by a collision risk assessment algorithm; S7: When the MCPA and MTCPA of any target are lower than the preset safety threshold, an early warning signal is sent to the crew or the ship control system to instruct them to take collision avoidance measures.

2. The method for detecting multiple targets on a ship on water according to claim 1, characterized in that: The super-resolution reconstruction algorithm in step S2 is based on a deep learning model to enhance image resolution, which includes fine-tuning the hyperparameters of the model and dynamic parameter adjustment, wherein the hyperparameters include learning rate, batch size and number of iterations, and includes the following steps: S21: Learning rate tuning, determine the baseline learning rate: First, we select 0.001 as the baseline learning rate for the model. We use the Learning Rate Finder algorithm to gradually increase the learning rate, starting from 1e-5, until the training loss of the model begins to rise sharply. By plotting the relationship between learning rate and loss, we find the range of learning rates before the loss starts to increase significantly, which is about 0.0001 to 0.

01. Secondly, the learning rate scheduling adopts the Cosine Annealing strategy based on the found learning rate range, starting from 0.

001. As the training cycle progresses, the learning rate changes periodically until it reaches the minimum value of 0.0001, and then rises back to 0.001 to form a complete cycle. S22: Batch size tuning: First, resource assessment, evaluate the hardware resources of the training platform; Secondly, gradient accumulation. If the memory of a single GPU is limited, a gradient accumulation strategy is used; S23: Iteration number tuning: Set up an early stopping mechanism. If the loss on the validation set does not improve significantly within 10 consecutive epochs, stop training. Combine the Cosine Annealing learning rate scheduling strategy. Perform at least 3 complete learning rate cycles. Each cycle takes about 200 epochs. Therefore, the total number of iterations is set to 600 epochs. S24: Dynamic parameter adjustment: The light intensity and weather condition data are obtained in real time by using the light sensors and weather stations installed on the ship. The light intensity and weather condition data are used to dynamically adjust the attention weights in the model so that the model can maintain high detection performance under different light and weather conditions. An adaptive activation function, such as Swish or Mish, is introduced into the model, which is dynamically adjusted according to the distribution of the input data so that the model can process image features under different light and weather conditions.

3. The method for detecting multiple targets on a ship on water according to claim 1, characterized in that: The FasterR-CNN model training process of step S3 integrates the attention mechanism and includes the following steps: S31: Data preprocessing and enhancement: Use cleaning software to clean and annotate ocean images containing ships, buoys, and drifting objects to ensure that each target is accurately labeled. During the training process, random cropping, horizontal flipping, color jittering, and random noise addition data enhancement techniques are used to improve the generalization ability of the model; S32: Integrated attention mechanism: In the feature extraction network of Faster R-CNN, the SENet module is embedded to learn the importance of each channel through global average pooling and fully connected layers to adaptively adjust the weight of the feature map; S33: Spatial Attention: A spatial attention mechanism is added to the region proposal network (RPN) and the detection head. By calculating the weight of each position in the feature map, the model can focus more on the key areas in the image, improving the detection speed and accuracy. S34: Training strategy and evaluation: First, train the RPN network alone until it converges; then freeze the RPN part and only train the detection head; finally, unfreeze the entire network for joint training to optimize the overall performance; during the training process, dynamically adjust the NMS threshold according to the model's prediction confidence to optimize the detection results, use the average precision, recall rate and accuracy indicators to evaluate the model performance, and continuously monitor these indicators during the training process to adjust the training strategy and model parameters in time.

4. The method for detecting multiple targets on water of a ship according to claim 1, characterized in that: In step S5, the multi-target tracking algorithm further includes assigning a unique identifier to the target, and using a Kalman filter for prediction and a Hungarian algorithm for matching the target with the detection result to achieve accurate trajectory tracking, including the following steps: S51: For a newly detected target, first record its initial position and timestamp as the tracking starting point; initialize the state variables of the Kalman filter, including position, velocity, acceleration, and covariance matrix; assign a unique tracking ID to each new target for subsequent tracking and data association; S52: Kalman filter prediction, using the prediction step of the Kalman filter, based on the state estimation and control input at the previous moment, predict the target state at the current moment, and the prediction formula is as follows: X(k|k-1)=F(k)*X(k-1|k-1)+B(k)U(k) (1) Where X(k|k-1) represents the predicted state, F(k) is the state transfer matrix, B(k) is the control input matrix, and U(k) is the control input vector; At the same time, the covariance matrix of the predicted state estimate reflects the uncertainty of the state estimate, and the prediction formula is: P(k|k-1)=F(k)P(k-1|k-1)*F(k)+Q(k) (2) Where P(k|k-1) is the prediction covariance and Q(k) is the process noise covariance matrix; S53: Hungarian algorithm matching, comparing the position predicted by the Kalman filter with the target position detected in the current frame, constructing a cost matrix, in which each element represents the distance between the predicted target and the detection result; the rows of the cost matrix correspond to the predicted targets of the Kalman filter, and the columns correspond to the detection results of the current frame; applying the Hungarian algorithm to solve the optimal matching of the cost matrix, the goal is to minimize the total cost; the Hungarian algorithm ensures that each predicted target is matched with only one detection result, and vice versa, avoiding multiple matches or no matches; S54: Tracking status update: For the predicted target that matches successfully, the Kalman filter update step is used to update the state estimate in combination with the detection results. The update formula is: X(k|k)=X(k|k-1)+K(k)*(Z(k)-H(k)*X(k|k-1)) (3) Where X(k|k) is the updated state estimate; Z(k) is the measurement value; H(k) is the measurement matrix; K(k) is the Kalman gain; For unmatched predicted targets, only the prediction step is used to update the state estimate, indicating that the target may not be detected temporarily. For unmatched detection results, it may be a newly appeared target, which is initialized and tracked; S55: Based on the target's motion trajectory, the relative speed and direction of the target relative to the ship are calculated. The current position and speed of the target and the speed information of the ship are used to obtain the relative speed and direction through vector calculation. Minimum approach distance (MCPA) and minimum approach time (MTCPA) calculation: According to the relative speed and direction, the relative distance (MCPA) and time (MTCPA) at the closest moment between the target and the ship are calculated. The MTCPA calculation formula is: Among them, r is the relative position vector of the target and the ship, v is the relative velocity vector, υ ⊥ is the perpendicular vector to v.

5. The method for detecting multiple targets on a ship on water according to claim 1, characterized in that: In step S6, the collision risk assessment algorithm dynamically adjusts the safety thresholds of MCPA and MTCPA based on the motion trajectory and speed of the target.

6. The method for detecting multiple targets on a ship on water according to any one of claims 1 to 5, characterized in that: Step S7 includes the following steps: S71: Performing automated early warning and obstacle avoidance. When it is detected that the MCPA and MTCPA are lower than the preset safety threshold, an early warning signal is immediately triggered; the early warning signal is sent to the navigation cabin through the ship's automated driving assistance system, and the automatic obstacle avoidance mechanism is activated at the same time; S72: According to the specific values ​​of MCPA and MTCPA, the warning signals are divided into different levels of urgency to guide the crew or the automated system to take appropriate collision avoidance measures.

7. The method for detecting multiple targets on water of a ship according to claim 6, characterized in that: Step S7 also includes: S73: Data recording and strategy optimization, used to store target detection results, motion trajectories, collision risk assessments and early warning responses, so as to facilitate post-analysis and optimization of the ship's multi-target detection strategy on the water.

Images