Marine target detection and three-dimensional point cloud perception method based on infrared binocular vision

Abstract

The application discloses a kind of based on infrared binocular vision's offshore target detection and three-dimensional point cloud perception method, the method includes by constructing based on EMA attention module and DCNv2 module improvement YOLOv8 network, the YOLOv8-IRECG target detection model obtained;Through sample data set, the optimal target detection model is obtained by model training to YOLOv8-IRECG target detection model, to realize the category detection of offshore target and output two-dimensional detection frame;Based on improved SGBM stereo matching algorithm, the depth feature map is obtained by stereo matching to preprocessed left image and preprocessed right image;Based on the camera internal parameter matrix of prelabeling, according to the depth feature map and the two-dimensional detection frame output by optimal target detection model, the offshore target detection and three-dimensional point cloud perception based on infrared binocular vision are realized.The application solves the problem that existing method cannot realize high-precision target detection and three-dimensional positioning in low-visibility offshore environment, and outputs the comprehensive solution of rich three-dimensional point cloud data.

Description

Technical Field

[0001] This invention relates to the field of marine target detection and 3D point cloud perception technology, and in particular to a method for marine target detection and 3D point cloud perception based on infrared binocular vision. Background Technology

[0002] In fields such as marine engineering, maritime search and rescue, and ship navigation, efficient and accurate target detection and spatial positioning technologies are crucial for ensuring operational safety and improving response efficiency. Currently, maritime target perception mainly relies on marine radar, lidar, and visible light cameras, but these technologies all have significant shortcomings. While marine radar can detect targets at relatively long distances, it is susceptible to sea clutter and electromagnetic interference in complex sea conditions, has limited ability to identify small targets, and cannot provide visual morphological information about the target. Lidar acquires high-precision three-dimensional point cloud data by emitting laser beams, but in harsh marine environments such as rain, fog, and salt spray, the laser signal undergoes severe attenuation and scattering, leading to a sharp reduction in effective detection range and a significant decrease in reliability. Visible light cameras provide intuitive imaging and high resolution, but their performance is highly dependent on ambient light. In low-visibility conditions such as nighttime, dense fog, and backlighting, the imaging quality deteriorates severely, or even renders the system inoperable, making it difficult to meet the needs of all-weather monitoring.

[0003] Infrared imaging technology, by passively receiving the thermal radiation of objects, possesses unique advantages such as being unaffected by visible light, penetrating a certain degree of smoke, and operating day and night, providing an ideal solution for all-weather maritime target perception. However, combining infrared imaging with visual technology for maritime target localization still faces a series of key technical challenges. First, infrared images typically have low contrast, weak texture information, blurred edges, and a "thermal halo" effect, making traditional feature extraction and matching algorithms based on visible light images perform poorly on infrared images. Second, the maritime environment is complex, with issues such as wave reflection, cloud cover, variable target size, and uncertain target attitude, requiring detection algorithms to have extremely high robustness and adaptability. However, existing general-purpose detection models lack effective geometric modeling capabilities for common deformations and partial occlusions of maritime targets, resulting in low detection accuracy for small targets and a high risk of missed detections. Furthermore, the weak texture of infrared images makes it difficult to find accurate corresponding points during stereo matching, especially at target edges and in areas with weak texture. This easily leads to mismatches and parallax holes. Traditional algorithms such as SGBM perform well in scenes with rich texture, but are prone to mismatches in large areas with weak texture, such as the sea surface and sky. Moreover, single-resolution matching strategies cannot balance detail and robustness, resulting in depth maps with a large amount of noise and discontinuities, which seriously affects the accuracy of 3D positioning. At the same time, existing infrared binocular vision solutions are mostly limited to outputting the 3D coordinates of the target and lack the ability to fully reconstruct the 3D structure of the scene, failing to generate scene point cloud data. This limits their value in advanced applications such as 3D environment modeling, fine obstacle recognition, and target geometry measurement.

[0004] In summary, there is an urgent need for a comprehensive solution that can achieve high-precision target detection and 3D positioning in low-visibility marine environments, and output rich 3D point cloud data. Summary of the Invention

[0005] This invention provides a method for marine target detection and three-dimensional point cloud perception based on infrared binocular vision, in order to overcome the above-mentioned technical problems.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A method for marine target detection and 3D point cloud perception based on infrared binocular vision includes the following steps: S1: Simultaneously acquire left and right views of the maritime target scene using an infrared binocular camera to obtain a pair of original infrared images with timestamp alignment; S2: For the left and right views in the original infrared image pair, histogram equalization and gamma correction are performed sequentially to obtain preprocessed left and right images with enhanced contrast and texture features; and target categories and bounding boxes are labeled on the preprocessed left image using a labeling tool to obtain a sample dataset. S3: Improve the YOLOv8 network based on the EMA attention module and the DCNv2 module to obtain the YOLOv8-IRECG target detection model; the model includes an input layer, a backbone feature extraction network, a neck feature fusion network, and a head detection network; The input layer is used to input sample images of the sample dataset into the backbone feature extraction network; the backbone feature extraction network is used to extract multi-scale geometric features from the sample images of the sample dataset to obtain multi-scale geometric feature maps; the neck feature fusion network is used to fuse the multi-scale geometric feature maps to obtain fused feature maps; the head detection network is used to detect the category of maritime targets based on the fused feature maps and output two-dimensional detection boxes. S4: Train the YOLOv8-IRECG target detection model using a sample dataset to obtain the optimal target detection model. Use the optimal target detection model to detect the category of maritime targets and output a two-dimensional detection box. S5: Based on the improved SGBM stereo matching algorithm, stereo matching is performed between the preprocessed left image and the preprocessed right image to obtain depth feature maps; S6: Based on the pre-calibrated camera intrinsic parameter matrix, perform inverse perspective projection on the depth feature map to generate a global scene 3D point cloud; by using the 2D detection box output in step S4 as the view frustum truncation boundary, perform spatial clipping on the global scene 3D point cloud to obtain a local point cloud subset. A statistical outlier removal algorithm based on spatial distance distribution characteristics is used to remove abnormal outliers from a subset of local point clouds to obtain an optimized target point cloud. Based on the spatial bounding box algorithm, a spatial bounding box representing the three-dimensional point cloud perception of the maritime target is obtained according to the geometric centroid and three-dimensional spatial coordinates of the optimized target point cloud. This enables the detection of maritime targets and the perception of three-dimensional point clouds based on infrared binocular vision.

[0007] Furthermore, the backbone feature extraction network described in S3 includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a first C2f_DCNv2 module, a fourth convolutional layer, a second C2f_DCNv2 module, a fifth convolutional layer, a third C2f_DCNv2 module, and an SPPF module, all connected sequentially with different kernel parameters. The first convolutional layer is used to extract geometric features from the sample images in the sample dataset. The second convolutional layer is used to perform a convolution operation on the output of the first convolutional layer. The first C2f module is used to extract geometric features from the output of the second convolutional layer. The third convolutional layer is used to perform a convolution operation on the output of the first C2f module. The first, second, and third C2f_DCNv2 modules are essentially the C2f modules of the original YOLOv8 network. The standard convolutional layers in the module are replaced with modules obtained by deformable convolution DCNv2; the first C2f_DCNv2 module is used to extract geometric features from the output of the third convolutional layer; the fourth convolutional layer is used to perform convolution operations on the output of the first C2f_DCNv2 module; the second C2f_DCNv2 module is used to extract geometric features from the output of the fourth convolutional layer; the fifth convolutional layer is used to perform convolution operations on the output of the second C2f_DCNv2 module; the third C2f_DCNv2 module is used to extract geometric features from the output of the fifth convolutional layer; the SPPF module is used to perform pooling operations on the output of the third C2f_DCNv2 module; the multi-scale geometric feature map is the output of the first C2f_DCNv2 module, the output of the second C2f_DCNv2 module, and the output of the SPPF module.

[0008] Furthermore, the neck feature fusion network described in S3 includes a first upsampling layer, a first concatenation layer, a fourth C2f_DCNv2 module, a second upsampling layer, a second concatenation layer, a second C2f module, a first EMA attention module, a sixth convolutional layer, a third concatenation layer, a third C2f module, a second EMA attention module, a seventh convolutional layer, a fourth concatenation layer, a fourth C2f module, and a third EMA attention module, all connected sequentially with different kernel parameters. The first upsampling layer performs an upsampling operation on the output of the SPPF module. The first concatenation layer performs a channel concatenation operation on the output of the first upsampling layer and the output of the second C2f_DCNv2 module. The fourth C2f_DCNv2 module performs geometric feature extraction on the output of the first concatenation layer. The second upsampling layer performs an upsampling operation on the output of the fourth C2f_DCNv2 module. The second concatenation layer performs a channel concatenation operation on the output of the second upsampling layer and the output of the first C2f_DCNv2 module. The second C2f module performs geometric feature extraction on the output of the second concatenation layer. The system is structured as follows: The first EMA attention module extracts global contextual features from the output of the second C2f module based on an attention mechanism; the sixth convolutional layer performs a convolution operation on the output of the first EMA attention module; the third concatenation layer performs channel concatenation on the output of the sixth convolutional layer and the output of the fourth C2f_DCNv2 module; the third C2f module extracts geometric features from the output of the third concatenation layer; the second EMA attention module extracts global contextual features from the output of the third C2f module based on an attention mechanism; the seventh convolutional layer performs a convolution operation on the output of the second EMA attention module; the fourth concatenation layer performs channel concatenation on the output of the seventh convolutional layer and the output of the SPPF module; the fourth C2f module extracts geometric features from the output of the fourth concatenation layer; and the third EMA attention module extracts global contextual features from the output of the fourth C2f module based on an attention mechanism. The fused feature map consists of the outputs of the first EMA attention module, the second EMA attention module, and the third EMA attention module.

[0009] Furthermore, the method for obtaining the optimal object detection model in S4 is as follows: S41: Randomly divide the sample dataset into a training set and a validation set according to a preset ratio; S42: Train the constructed YOLOv8-IRECG object detection model based on the training set to obtain the trained object detection model; S43: Use GIoU Loss as the model loss function, and validate the trained object detection model using the validation set; That is, to determine whether the output of the trained object detection model has converged; If the output of the trained object detection model converges, then the trained object detection model is confirmed to be the optimal object detection model. Otherwise, the weight parameters of the trained object detection model are adaptively adjusted based on the backpropagation method, and step S42 is repeated until the weight parameters of the trained object detection model with converged output are confirmed to be the optimal weight parameters, and the object detection model is reconstructed to obtain the optimal object detection model.

[0010] Furthermore, S5 specifically includes the following steps: S51: Construct an image pyramid network containing a first resolution layer and a second resolution layer; the first resolution layer in the image pyramid network preserves the original image resolution of the preprocessed left image and the preprocessed right image; the second resolution layer is used to perform downsampling operations on the preprocessed left image and the preprocessed right image based on the bilinear interpolation algorithm; S52: Perform SGBM stereo matching operation on the output images of the first resolution layer and the second resolution layer respectively to obtain the first initial disparity map D1 corresponding to the first resolution layer and the second initial disparity map D2 corresponding to the second resolution layer; then perform upsampling operation on the second initial disparity map D2 through bilinear interpolation algorithm to obtain the second optimized disparity map D2_up with the same scale as the first initial disparity map D1. S53: Perform a weighted fusion of the first initial disparity map D1 and the second optimized disparity map D2_up to obtain the fused disparity map D_sum: D_sum = W1×D1 + W2×D2_up In the formula: W1 and W2 represent weighting coefficients; S54: Apply the Canny edge detection operator to perform edge detection on the preprocessed left image to obtain an edge binary map Edge_map; traverse the pixels in the fused disparity map D_sum, and take the pixels with a disparity value of 0 as holes. Using the holes as the center point, search for non-hole pixels in a preset neighborhood. Take the non-hole pixels whose distance from the center point is less than a preset distance threshold as the disparity pixel set, and obtain the median disparity of the pixels based on the disparity pixel set as the filling value of the holes to obtain the interpolated disparity map D_inter; S55: Using the preprocessed left image as the guide image, the interpolated disparity map D_inter is filtered based on the adaptive WLS filtering algorithm to obtain the final disparity map D_final; and based on the camera parameters of the infrared stereo camera, the final disparity map D_final is converted into a depth feature map, and the conversion formula is: Z = (f×B) / d; where Z represents the vertical depth from the target to the stereo camera; f represents the effective focal length of the stereo camera; B represents the baseline distance of the stereo camera, i.e., the horizontal distance between the optical centers of the two cameras; and d represents the disparity value, i.e., pixels.

[0011] Furthermore, step S6 specifically includes the following steps: S61: Based on the pre-calibrated intrinsic parameter matrix of the left eye camera, perform inverse perspective projection on all pixels in the depth feature map to generate a global scene 3D point cloud. The coordinate transformation formula for inverse perspective projection is:

[0012] In the formula: , These represent the horizontal, vertical, and lateral coordinates of the target in the camera coordinate system, respectively. This represents the depth value, i.e., the vertical coordinate. ; , This represents the position coordinates of a pixel in the depth feature map; express Focal length along the axial direction; express Focal length along the axial direction; , Indicates the optical center coordinates of the camera; S62: By using the two-dimensional detection box output in step S4 as the view frustum truncation boundary, spatial clipping is performed on the global scene three-dimensional point cloud to obtain a local point cloud subset; S63: A statistical outlier removal algorithm based on spatial distance distribution characteristics is used to remove outliers from a subset of local point clouds to obtain an optimized target point cloud. Based on the spatial bounding box algorithm, the minimum bounding box (OBB) representing the perception of the three-dimensional point cloud of the maritime target is obtained according to the geometric centroid and three-dimensional spatial coordinates of the optimized target point cloud. This enables the detection of maritime targets and the perception of three-dimensional point clouds based on infrared binocular vision.

[0013] Beneficial Effects: This invention provides a method for marine target detection and 3D point cloud perception based on infrared binocular vision. By constructing an improved YOLOv8 network based on EMA attention modules and DCNv2 modules, the resulting YOLOv8-IRECG target detection model significantly improves the detection accuracy and robustness of infrared marine targets, and enhances the ability to identify deformed, occluded, and small targets. By improving the SGBM stereo matching algorithm, depth feature maps are obtained by stereo matching the preprocessed left and right images. This greatly improves the accuracy and completeness of depth estimation for marine targets, and solves the problem of stereo mapping in weakly textured infrared scenes. This approach addresses the issue of volume matching error. It generates a global scene 3D point cloud by performing inverse perspective projection on the depth feature map. By using the 2D detection box output by the optimal target detection model as the view frustum truncation boundary, it spatially prunes the global scene 3D point cloud to obtain a local point cloud subset. Furthermore, based on a statistical outlier removal algorithm using spatial distance distribution characteristics, it removes abnormal outliers from the local point cloud subset to obtain an optimized target point cloud. Finally, based on a spatial bounding box algorithm, it obtains a spatial bounding box representing the perception of the 3D point cloud of maritime targets according to the geometric centroid and 3D spatial coordinates of the optimized target point cloud, thereby achieving maritime target detection and 3D point cloud perception based on infrared binocular vision. This approach provides rich 3D point cloud data of maritime targets, supporting refined target geometry measurement and environmental perception. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the method for marine target detection and three-dimensional point cloud perception based on infrared binocular vision according to the present invention; Figure 2 This is a schematic diagram of the YOLOv8-IRECG target detection model constructed in this embodiment; Figure 3 This is a flowchart of the improved SGBM stereo matching algorithm in this embodiment; Figure 4 This is a simulation diagram of the two-dimensional detection box output by the optimal target detection model in this embodiment; Figure 5 This is a simulation diagram of the depth feature map in this embodiment; Figure 6 This is a simulation diagram of maritime target detection and 3D point cloud perception in this embodiment. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This embodiment provides a method for marine target detection and 3D point cloud perception based on infrared binocular vision, such as... Figure 1 As shown, the specific steps include: S1: Simultaneously acquire left and right views of the maritime target scene using an infrared binocular camera to obtain a pair of original infrared images with timestamp alignment; Specifically, left and right views of a maritime scene are simultaneously acquired using a pre-calibrated infrared binocular camera. The infrared binocular camera consists of two infrared cameras with identical parameters, located on the same plane, and having a fixed baseline distance B. It includes: (1) Synchronous acquisition of infrared cameras: In this embodiment, the left and right infrared cameras are configured with independent acquisition processes. The underlying hardware interfaces of the two processes are called concurrently through a unified global startup configuration file, and image capture is triggered at the same time. At the data stream receiving end, the system timestamp of each frame image is extracted. The left and right views with a timestamp difference of less than a preset threshold (1ms) are paired and encapsulated to achieve high-precision software-level synchronous acquisition. In this embodiment, the original infrared image pairs of the sea area in front are acquired in real time through the above synchronization mechanism. A total of 9152 valid infrared ship images are collected and organized, covering various sea conditions and target attitudes. They are randomly divided into training set (6406 images), validation set (1830 images) and test set (916 images) in a ratio of 7:2:1. A dedicated dataset is constructed and sent to the threshold processing system. (2) Image resolution settings: Set the camera output image resolution to 640×512 pixels and save it in 16-bit RAW format to preserve complete infrared radiation information; (3) Data collection scenario: The sea area near Xiaoping Island in Dalian was selected as the test site. The data collection period covered daytime (10:00-14:00), dusk (17:00-19:00) and nighttime (22:00-02:00). The weather conditions included sunny, foggy and light fog to verify the all-weather adaptability of the method. (4) Image storage: The acquired left and right views are saved by timestamp. The left view file is named "left.raw" and the right view file is named "right.raw" to facilitate image matching in subsequent processing.

[0018] S2: For the left and right views in the original infrared image pair, histogram equalization and gamma correction are performed sequentially to obtain preprocessed left and right images with enhanced contrast and texture features; and target categories and bounding boxes are labeled on the preprocessed left image using a labeling tool to obtain a sample dataset. Specifically, firstly, the gray-level histogram of the original infrared image is statistically analyzed, and the probability density of each gray level k appearing in the image is calculated. for:

[0019] In the formula: Indicates grayscale level The total number of pixels; N represents the total number of pixels in the image; Represents each gray level The probability density of appearance in the image; Calculate the cumulative distribution function of grayscale distribution. for:

[0020] This cumulative distribution function is used to construct a mapping relationship, which non-linearly stretches the gray values ​​of the original pixels to a new gray level. :

[0021] In the formula: L represents the maximum gray level allowed in the image (e.g., for an 8-bit image, L=256); round() represents the function used to round numbers; Then, gamma correction of the nonlinear hue response is performed: Building upon histogram equalization, to further highlight details in dark areas and the subtle texture features of the hull surface, the system introduces gamma correction for nonlinear adjustment. Its underlying pixel-level transformation model is as follows: O

[0022] In the formula: I(x, y) represents the input pixel value after equalization and normalization to the [0,1] interval; O(x, y) represents the corrected output pixel value; c represents a constant scaling factor (c=1 in this embodiment), and the preset gamma exponent parameter γ<1 (γ=0.5 in this embodiment); S3: Improve the YOLOv8 (You Only Look Once version 8, object detection algorithm) network based on the EMA attention module and DCNv2 module, and name the improved YOLOv8 network model the YOLOv8-IRECG object detection model; for example... Figure 2As shown, the model includes an input layer, a backbone feature extraction network, a neck feature fusion network, and a head detection network; and YOLOv8-IRECG is: You Only Look Once version 8 - IR, EMA, C2f_DCNv2, GIoU is an abbreviation for IR, where IR stands for infrared; EMA, C2f_DCNv2, and GIoU are used to improve the modules and loss function introduced in the YOLOv8 network, respectively. Specifically, the backbone feature extraction network includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a first C2f_DCNv2 module, a fourth convolutional layer, a second C2f_DCNv2 module, a fifth convolutional layer, a third C2f_DCNv2 module, and an SPPF module, all connected sequentially with different kernel parameters. The first convolutional layer is used to extract geometric features from sample images in the sample dataset. The second convolutional layer is used to perform a convolution operation on the output of the first convolutional layer. The first C2f module is used to extract geometric features from the output of the second convolutional layer. The third convolutional layer is used to perform a convolution operation on the output of the first C2f module. The first, second, and third C2f_DCNv2 modules are essentially the C2f modules of the original YOLOv8 network. The standard convolutional layers in the module are replaced with deformable convolutional modules (DCNv2); the first C2f_DCNv2 module is used to extract geometric features from the output of the third convolutional layer; the fourth convolutional layer is used to perform convolution operations on the output of the first C2f_DCNv2 module; the second C2f_DCNv2 module is used to extract geometric features from the output of the fourth convolutional layer; the fifth convolutional layer is used to perform convolution operations on the output of the second C2f_DCNv2 module; the third C2f_DCNv2 module is used to extract geometric features from the output of the fifth convolutional layer; the SPPF module is used to perform pooling operations on the output of the third C2f_DCNv2 module; the multi-scale geometric feature map is the output of the first C2f_DCNv2 module, the output of the second C2f_DCNv2 module, and the output of the SPPF module.

[0023] Specifically, the neck feature fusion network includes a first upsampling layer, a first concatenation layer, a fourth C2f_DCNv2 module, a second upsampling layer, a second concatenation layer, a second C2f module, a first EMA attention module, a sixth convolutional layer, a third concatenation layer, a third C2f module, a second EMA attention module, a seventh convolutional layer, a fourth concatenation layer, a fourth C2f module, and a third EMA attention module, all connected sequentially with different kernel parameters. The first upsampling layer performs an upsampling operation on the output of the SPPF module. The first concatenation layer performs a channel concatenation operation on the output of the first upsampling layer and the output of the second C2f_DCNv2 module. The fourth C2f_DCNv2 module performs geometric feature extraction on the output of the first concatenation layer. The second upsampling layer performs an upsampling operation on the output of the fourth C2f_DCNv2 module. The second concatenation layer performs a channel concatenation operation on the output of the second upsampling layer and the output of the first C2f_DCNv2 module. The second C2f module performs geometric feature extraction on the output of the second concatenation layer. The first EMA attention module is used to extract global contextual features from the output of the second C2f module based on the attention mechanism; the sixth convolutional layer is used to perform convolution operations on the output of the first EMA attention module; the third concatenation layer is used to perform channel concatenation operations on the output of the sixth convolutional layer and the output of the fourth C2f_DCNv2 module; the third C2f module is used to extract geometric features from the output of the third concatenation layer; the second EMA attention module is used to extract global contextual features from the output of the third C2f module based on the attention mechanism; the seventh convolutional layer is used to perform convolution operations on the output of the second EMA attention module; the fourth concatenation layer is used to perform channel concatenation operations on the output of the seventh convolutional layer and the output of the SPPF module; the fourth C2f module is used to extract geometric features from the output of the fourth concatenation layer; the third EMA attention module is used to extract global contextual features from the output of the fourth C2f module based on the attention mechanism; the fused feature map is the output of the first EMA attention module, the output of the second EMA attention module, and the output of the first EMA attention module.

[0024] The input layer is used to input sample images of the sample dataset into the backbone feature extraction network; the backbone feature extraction network is used to extract multi-scale geometric features from the sample images of the sample dataset to obtain multi-scale geometric feature maps; the neck feature fusion network is used to fuse the multi-scale geometric feature maps to obtain fused feature maps; the head detection network is used to detect the category of maritime targets based on the fused feature maps and output two-dimensional detection boxes. This embodiment introduces Deformable Convolution v2 (DCNv2) modules into the backbone and neck network of YOLOv8 to replace some standard convolutions, enhancing the ability to extract geometric features of irregular targets. Simultaneously, EMA (Efficient Multi-Scale Attention) modules are embedded in specific layers of the network. By utilizing global context information, channel weights in each parallel processing branch are dynamically adjusted to suppress interference from complex sea surface backgrounds. The model's loss function employs GIoU Loss (Generalized Intersection over Union) to optimize the regression accuracy of bounding boxes.

[0025] S4: Train the YOLOv8-IRECG target detection model using a sample dataset to obtain the optimal target detection model. Use the optimal target detection model to detect the category of maritime targets and output a two-dimensional detection box. The method for obtaining the optimal target detection model in this embodiment is as follows: S41: Randomly divide the sample dataset into a training set and a validation set according to a preset ratio; S42: Train the constructed YOLOv8-IRECG object detection model based on the training set to obtain the trained object detection model; S43: Use GIoU Loss as the model loss function, and validate the trained object detection model using the validation set; That is, to determine whether the output of the trained object detection model has converged; If the output of the trained object detection model converges, then the trained object detection model is confirmed to be the optimal object detection model. Otherwise, the weight parameters of the trained object detection model are adaptively adjusted based on the backpropagation method, and step S42 is repeated until the weight parameters of the trained object detection model with converged output are confirmed to be the optimal weight parameters, and the object detection model is reconstructed to obtain the optimal object detection model.

[0026] Specifically, in this embodiment, the preprocessed left view I_Left is input into the optimal object detection model: (1) Model loading: Load the pre-trained YOLOv8-IRECG model, i.e. the weight file "yolov8_irecg.pt" of the optimal object detection model, and set the model input size to 640×640 pixels; (2) Image preprocessing adaptation model: I_Left (original size 640×512) is adjusted to 640×640 pixels, and letterbox padding is used to keep the aspect ratio of the image unchanged. Missing parts are filled with 0. The image data is normalized to the [0,1] interval and converted to PyTorch tensor format with dimensions [1, 3, 640, 640]. (3) Model inference: The preprocessed tensor is input into the optimal object detection model for forward inference. The internal processing flow of the model is as follows: Backbone Feature Extraction Network: First, features are extracted by introducing the C2f module of DCNv2. For example, for an input image of 640×640, the first DCNv2 module has a convolution kernel size of 3×3, a stride of 2, and an output feature map size of 320×320; Neck Feature Fusion Network: Based on the feature map with embedded EMA attention mechanism, global average pooling and 1×1 convolution are performed to generate channel attention weights. For example, in the layer with a feature map size of 80×80, the EMA module divides the input feature map into 4 groups along the channel dimension, and spatial attention is computed in parallel for each group, and finally weighted and fused; Head Detection Network: Predicts target bounding boxes and categories at three scales (80×80, 40×40, 20×20); Post-processing and output: The model output is a prediction tensor with three scales. Each tensor contains bounding box coordinates, target confidence, and class probability. Redundant detection boxes are removed by applying the non-maximum suppression (NMS) algorithm. The NMS threshold is set to 0.5 and the confidence threshold is set to 0.25.

[0027] Example Output: For a ship in the test image, the model outputs the coordinates of the 2D bounding box as ( , , , (), with a confidence level of 0.92, and the category is "ship".

[0028] Bounding box coordinate transformation: Convert the bounding box coordinates in the 640×640 coordinate system output by the model back to the original image's 640×512 coordinate system. The transformation formula is as follows: x_original = (x_model - pad_left)×(original_width / model_input_width) y_original = (y_model - pad_top)×(original_height / model_input_height) In the formula: x_original and y_original represent the target coordinates (pixels) in the original image coordinate system after transformation; x_model and y_model represent the target coordinates in the model input image (640×640) coordinate system; pad_left and pad_top represent the number of pixels to fill the letterbox; original_width and original_height represent the original image size, which are 640 pixels and 512 pixels respectively; model_input_width and model_input_height represent the model input size, both of which are 640 pixels. Application of the GIoU loss function in training: The loss function is calculated as follows during the model training phase: For the predicted bounding box B_p and the ground truth bounding box B_gt, calculate the area A_c of their minimum convex hull C; calculate = |B_p∩B_gt| / |B_p∪B_gt|; Calculate GIoU = IoU - (A_c - |B_p∪B_gt|) / A_c; Bounding box regression loss L_box = 1–GIoU; When the predicted bounding box does not overlap with the ground truth bounding box, IoU=0, but GIoU is negative, which can still provide gradient guidance to make the predicted bounding box move closer to the ground truth bounding box.

[0029] S5: Based on the improved SGBM stereo matching algorithm, stereo matching is performed between the preprocessed left image and the preprocessed right image to obtain depth feature maps, such as... Figure 3 As shown, the specific steps include: S51: Construct an image pyramid network containing a first resolution layer and a second resolution layer; the first resolution layer in the image pyramid network preserves the original image resolution of the preprocessed left image and the preprocessed right image; the second resolution layer is used to perform downsampling operations on the preprocessed left image and the preprocessed right image based on the bilinear interpolation algorithm; S52: Perform SGBM stereo matching operation on the output images of the first resolution layer and the second resolution layer respectively to obtain the first initial disparity map D1 corresponding to the first resolution layer and the second initial disparity map D2 corresponding to the second resolution layer; then perform upsampling operation on the second initial disparity map D2 through bilinear interpolation algorithm to obtain the second optimized disparity map D2_up with the same scale as the first initial disparity map D1. S53: Perform a weighted fusion of the first initial disparity map D1 and the second optimized disparity map D2_up to obtain the fused disparity map D_sum: D_sum = W1×D1 + W2×D2_up In the formula: W1 and W2 represent weighting coefficients; S54: Apply the Canny edge detection operator to perform edge detection on the preprocessed left image to obtain an edge binary map Edge_map; traverse the pixels in the fused disparity map D_sum, and take the pixels with a disparity value of 0 as holes. Using the holes as the center point, search for non-hole pixels in a preset neighborhood. Take the non-hole pixels whose distance from the center point is less than a preset distance threshold as the disparity pixel set, and obtain the median disparity of the pixels based on the disparity pixel set as the filling value of the holes to obtain the interpolated disparity map D_inter; S55: Using the preprocessed left image as the guide image, the interpolated disparity map D_inter is filtered using the adaptive WLS filtering algorithm to obtain the final disparity map D_final; and based on the camera parameters of the infrared stereo camera, the final disparity map D_final is converted into a depth feature map, such as... Figure 5 As shown, the conversion formula is: Z = (f×B) / d; where Z represents the vertical depth from the target to the stereo camera; f represents the effective focal length of the stereo camera; B represents the baseline distance of the stereo camera, i.e., the horizontal distance between the optical centers of the two cameras; and d represents the parallax value, i.e., pixels.

[0030] In this embodiment, an improved SGBM stereo matching algorithm is applied to the preprocessed left image I_Left and the preprocessed right image I_Right. The specific implementation method is as follows: (1) Multi-resolution parallax fusion: Constructing the image pyramid: Original resolution layer I1, i.e., the first resolution layer: directly use I_Left and I_Right, with a size of 640×512; Low resolution layer I2, i.e., the second resolution layer: downsample I_Left and I_Right by 2 times, and use bilinear interpolation to obtain an image with a size of 320×256. (2) SGBM matching parameter settings: For I1 layer: Set the disparity range min_disp=0, max_disp=128, window size block_size=11, P1=8×3×block_size², P2=32×3×block_size²; uniqueness_ratio=10, speckle_window_size=100, speckle_range=32; For the I2 layer: the disparity range is reduced proportionally to min_disp=0, max_disp=64, the window size block_size=7, and other parameters are adjusted accordingly based on experience; (3) Execute the SGBM stereo matching algorithm: Use the StereoSGBM_create function of the OpenCV library to create a matcher, perform stereo matching on I1 and I2 respectively to obtain the first initial disparity map D1 (640×512) and the second initial disparity map D2 (320×256), and then upsample D2 to 640×512 through bilinear interpolation to obtain D2_up of the same size as D1; (4) Weighted fusion: Set weight coefficients W1=0.6 and W2=0.4. These weights are determined experimentally to achieve a balance between preserving details and suppressing noise. Calculate the fused disparity map D_sum. (5) Edge-preserving interpolation optimization: Edge detection: Apply the Canny edge detection operator to the preprocessed left image I_Left, set a low threshold of 50 and a high threshold of 150, and generate an edge binary map Edge_map; Parallax hole detection: Traverse D_sum and mark pixels with a parallax value of 0 as hole points; Edge-preserving interpolation: For each hole point, non-hole pixels are searched within a 5×5 neighborhood centered on it. Only pixels in the same edge region as the center point (i.e., the edge distance to the center point is less than a threshold) are considered. The edge distance is determined by the Edge_map (i.e., when interpolating disparity holes, the system uses the binary map to determine whether the neighboring pixels and the hole center point are in the same edge region (i.e., they are not separated by an edge line). Only valid pixels that do not cross the edge are retained to calculate the median for filling, thereby ensuring that the interpolation result can accurately restore the boundary structure of the target and avoid edge blurring). The disparity median of the pixels that meet the conditions is taken as the filling value of the hole point. If there are no pixels that meet the conditions, the search range is gradually expanded to 7×7, 9×9 until a valid pixel is found. (6) Adaptive weighted least squares filtering: Constructing the guide image: using the preprocessed left image I_Left as the guide image; Set the WLS filter parameters: Regularization parameter lambda = 0.1, controlling the smoothing intensity; a larger value results in a stronger smoothing effect. Color sensitivity sigma_color = 0.1, controlling the sensitivity to image edges; Performing WLS filtering: Using the `create Disparity WLS Filter` function from the `ximgproc` module of the OpenCV library, the interpolated disparity map `D_inter` is used as input, and the preprocessed left image `I_Left` is used as a guide for filtering. During the filtering process, in image edge regions (large gradients), the smoothing weights are reduced, and the edges are preserved; in flat regions (small gradients), the smoothing weights are increased, and noise is effectively suppressed, thus obtaining... )); (7) Output the final depth map: Obtain the final disparity map D_final, with a size of 640×512 and pixel values ​​of sub-pixel precision floating-point numbers. Then, according to the formula Z = (f×B) / d, convert the disparity map into a depth map Depth_map; where f=779 pixels, B=523.6mm, d represents the disparity value (pixels), and the calculated Z unit is mm.

[0031] In this embodiment, to address the issue of matching errors caused by wave reflection and low illumination, an improved SGBM stereo matching algorithm is adopted. This algorithm involves constructing a two-layer image pyramid to obtain the original resolution image I1 (the output of the first resolution layer) and the low-resolution image I2 (the output of the second resolution layer). Then, SGBM matching is performed on I1 and I2 respectively to generate initial disparity maps D1 and D2_up. Next, D1 and D2_up are weighted and fused using preset adjustable weight coefficients W1 and W2 to obtain a preliminary fused disparity map D_sum. Then, edge-preserving interpolation optimization is performed on D_sum to improve the continuity of target edges, resulting in an interpolated disparity map D_inter. Finally, adaptive weighted least squares filtering is used to smooth and preserve the edges of D_inter, outputting the final high-quality disparity map D_final.

[0032] This embodiment also includes a method for extracting the target's depth information from the depth map based on the two-dimensional bounding box, in order to calculate the target's three-dimensional spatial coordinates: that is, by combining the target detection result (two-dimensional bounding box) in step S4 with the feature map depth map generated in step S5, the target depth information is extracted and the three-dimensional spatial coordinates are calculated. The specific implementation method is as follows: (1) Extract the depth value of the target region: Based on the target bounding box coordinates obtained in step S4 , , , In the depth feature map, a depth value matrix of the corresponding region is extracted. A mean strategy, median strategy, or quartile strategy can be used to extract representative depth values ​​of the target. This embodiment prefers the mean strategy. Specifically, the arithmetic mean of all valid depth values ​​within the target region is calculated as the target's depth value Z_target. Simultaneously, a confidence assessment is performed, calculating the proportion of valid depth values ​​within the target region. If this proportion is lower than a preset threshold (e.g., 50%), the depth extraction is considered unreliable, and the target is marked for re-detection or matching.

[0033] (2) Obtain the pixel coordinates of the target center point, i.e., calculate the coordinates of the center point of the target bounding box:

[0034] (3) Obtain the intrinsic parameter matrix K of the left eye camera through binocular camera calibration:

[0035] In the formula: , Indicates the coordinates of the principal point; These represent the focal lengths along the x-axis and y-axis, respectively, both in pixels. In this example, =330.0867, =255.8351, =778.8785, =779.3975; Based on the principle of binocular vision triangulation, calculate the three-dimensional coordinates of the target in the camera coordinate system:

[0036] In the formula: , These represent the horizontal, vertical, and lateral coordinates of the target in the camera coordinate system, respectively. This represents the depth value, i.e., the vertical coordinate. ; , This represents the position coordinates of a pixel in the depth feature map; express Focal length along the axial direction; express Focal length along the axial direction; , The coordinates of the camera's optical center are represented; and the calculated 3D coordinate information is associated with the target detection results (category, bounding box) from step S4, which can be used for visualization. This embodiment Figure 4 The application effect in a real-world scenario is demonstrated. The ship target in the infrared image is precisely locked by a two-dimensional detection box, and the distance between the target and the system (e.g., "Distance: 25.7m") and the three-dimensional coordinates (e.g., "X: 10.8m, Y: 0.5m, Z: 25.7m") are displayed in real time above or to the side of the image. Figure 5 It displays a depth map that encodes spatial depth information using color. The color directly corresponds to the distance of the object from the lens: warm colors (red, orange, yellow) represent nearby objects, and cool colors (cyan, blue) represent distant objects. The more red the color, the closer the object is, and the more blue the color, the farther away it is.

[0037] S6: Based on the pre-calibrated camera intrinsic parameter matrix, and according to the depth feature map and the two-dimensional detection box output in step S4, the detection of marine targets and the perception of three-dimensional point clouds based on infrared binocular vision are realized. Specifically, the following steps are included: S61: Based on the pre-calibrated camera intrinsic parameter matrix, i.e. the intrinsic parameter matrix of the left eye camera, perform inverse perspective projection on all pixels in the depth feature map to generate a global scene 3D point cloud. Furthermore, the coordinate transformation formula for inverse perspective projection is the same as the formula for calculating the three-dimensional coordinates of the target in the camera coordinate system based on the principle of binocular vision triangulation. All the three-dimensional coordinate points that have undergone the above mapping are collected and encapsulated to generate a global scene three-dimensional point cloud containing the sea background, waves and ship targets. S62: By using the 2D detection box output in step S4 as the view frustum truncation boundary, spatial clipping is performed on the global scene 3D point cloud to obtain a local point cloud subset; in this embodiment, the 2D bounding box of the "ship" category target identified by the optimal target detection model is directly retrieved as a mask (the mask serves two purposes: first, spatial clipping operation: as the projection interval boundary to determine which 3D points belong to the target, thereby eliminating the sea surface and background; second, noise removal reference: as the view frustum truncation boundary, to cooperate with the subsequent use of the SOR algorithm to assist the system in locating specific point cloud regions that need filtering). For example, when the detection box coordinates are ( , , , When the system traverses the global scene 3D point cloud, it retains only the 3D points that fall within the projection range and removes redundant data such as the sea surface and background. S63: The Statistical Outlier Removal (SOR) algorithm based on spatial distance distribution characteristics removes outliers from a subset of local point clouds to obtain an optimized target point cloud. In this embodiment, the initial extracted point cloud contains "flying point" noise due to the strong reflection of local waves at sea. By applying the Statistical Outlier Removal (SOR) algorithm, in this example, the K-nearest neighbor search parameter k = 50 and the standard deviation multiplier threshold α = 1.0 are set. After calculation, the algorithm completely removes outlier noise points caused by sea wave reflection, water mist, etc. Based on the spatial bounding box algorithm, the minimum bounding box (OBB) representing the 3D point cloud perception of maritime targets is obtained by optimizing the geometric centroid and 3D spatial coordinates of the target point cloud. This enables maritime target detection and 3D point cloud perception based on infrared binocular vision. This embodiment not only eliminates the risk of abrupt range changes due to the center of the 2D bounding box falling precisely in the reflective blind zone, but also further stabilizes the range measurement error. Simultaneously, by calculating the OBB of this high-purity point cloud, the physical dimensions and spatial attitude of the target are calculated, achieving maritime target detection and 3D point cloud perception based on infrared binocular vision.

[0038] like Figure 6The image shows a high-purity local 3D point cloud of the target ship extracted based on a 2D detection bounding box and a filtering and denoising algorithm. By using the 2D bounding box output by the optimal target detection model as the view frustum boundary for precise cropping, a large amount of redundant data such as the sea surface and background environment is removed. Furthermore, the Statistical Outlier Removal (SOR) algorithm is applied to effectively remove "flying point" noise caused by complex environmental factors such as wave reflection and water mist, resulting in the final cleaned image.

[0039] Compared with the prior art, the beneficial effects of the method described in this embodiment are as follows: An improved YOLOv8-IRECG target detection model was constructed. This model is based on YOLOv8n and integrates three core improvements: (1) the introduction of the DCNv2 deformable convolution module to replace part of the standard C2f convolution, so that the sampling points of the convolution kernel can be adaptively adjusted according to the actual shape of the target, enhancing the ability to extract geometric features of deformed, irregular, and partially occluded targets; (2) the embedding of the EMA attention mechanism, which uses global context information to dynamically adjust the channel weights, effectively suppressing complex background interference such as sea wave reflection, and making the model focus on the target area; (3) the adoption of the GIoU loss function to replace CIoU, and the introduction of the minimum bounding box penalty term to solve the gradient vanishing problem between the non-overlapping predicted box and the real box, significantly improving the training stability and recall of small and dense targets. Experiments show that compared with the baseline YOLOv8n model, the improved model improves precision by 2.9%, recall by 2.1%, mAP@0.5 by 1.5%, and mAP@0.5:0.95 by 1.9%. It significantly improves the detection accuracy and robustness of infrared maritime targets, and enhances the ability to identify deformed, occluded, and small targets.

[0040] An improved SGBM stereo matching algorithm is proposed to address the weak texture characteristics of infrared images. The algorithm includes: (1) Multi-resolution disparity fusion: Two-layer image pyramids are constructed, and SGBM matching is performed separately and then weighted and fused to balance the contribution of disparity information under different resolutions, while taking into account both detail preservation and noise suppression; (2) Edge-preserving interpolation optimization: Edge-preserving interpolation is performed on the fused disparity map to fill holes while referencing image edge information, thereby improving the continuity and accuracy of the target edge region; (3) Adaptive weighted least squares filtering: WLS filtering is performed on the interpolation results to smooth noise while preserving image edge structure information. Experiments show that the improved SGBM algorithm reduces the average relative ranging error from 1.97% of the traditional SGBM to 1.53%, and improves the accuracy by about 22.3%. It can significantly improve the accuracy and completeness of depth estimation and solve the stereo matching error problem in infrared weak texture scenes.

[0041] The method described in this embodiment, based on obtaining a high-precision depth map, further generates a 3D point cloud by back-projecting the depth map, and extracts a subset of the target point cloud by combining it with target detection boxes. Through point cloud post-processing, the complete 3D contour of the target can be obtained, thereby calculating the ship's length, width, height, draft, heading angle, and other geometric parameters, providing more comprehensive data support for applications such as maritime collision avoidance, berthing assistance, and 3D reconstruction. Simultaneously, the scene point cloud can assist in the fine-grained identification of sea surface obstacles and environmental modeling, expanding the application scope of this system.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for marine target detection and 3D point cloud perception based on infrared binocular vision, characterized in that, The specific steps include: S1: Simultaneously acquire left and right views of the maritime target scene using an infrared binocular camera to obtain a pair of original infrared images with timestamp alignment; S2: For the left and right views in the original infrared image pair, histogram equalization and gamma correction are performed sequentially to obtain preprocessed left and right images with enhanced contrast and texture features; and target categories and bounding boxes are labeled on the preprocessed left image using a labeling tool to obtain a sample dataset. S3: Improve the YOLOv8 network based on the EMA attention module and the DCNv2 module to obtain the YOLOv8-IRECG target detection model; the model includes an input layer, a backbone feature extraction network, a neck feature fusion network, and a head detection network; The input layer is used to input sample images from the sample dataset into the backbone feature extraction network; The backbone feature extraction network is used to extract multi-scale geometric features from sample images in the sample dataset to obtain multi-scale geometric feature maps. The neck feature fusion network is used to fuse multi-scale geometric feature maps to obtain a fused feature map; the head detection network is used to detect the category of maritime targets based on the fused feature map and output a two-dimensional detection box. S4: Train the YOLOv8-IRECG target detection model using a sample dataset to obtain the optimal target detection model. Use the optimal target detection model to detect the category of maritime targets and output a two-dimensional detection box. S5: Based on the improved SGBM stereo matching algorithm, stereo matching is performed between the preprocessed left image and the preprocessed right image to obtain depth feature maps; S6: Based on the pre-calibrated camera intrinsic parameter matrix, perform inverse perspective projection on the depth feature map to generate a global scene 3D point cloud; By using the two-dimensional detection box output in step S4 as the view frustum truncation boundary, the global scene three-dimensional point cloud is spatially clipped to obtain a subset of local point cloud; A statistical outlier removal algorithm based on spatial distance distribution characteristics is used to remove abnormal outliers from a subset of local point clouds to obtain an optimized target point cloud. Based on the spatial bounding box algorithm, a spatial bounding box representing the three-dimensional point cloud perception of the maritime target is obtained according to the geometric centroid and three-dimensional spatial coordinates of the optimized target point cloud. This enables the detection of maritime targets and the perception of three-dimensional point clouds based on infrared binocular vision.

2. The method for marine target detection and 3D point cloud perception based on infrared binocular vision according to claim 1, characterized in that, The backbone feature extraction network described in S3 includes a first convolutional layer, a second convolutional layer, a first C2f module, a third convolutional layer, a first C2f_DCNv2 module, a fourth convolutional layer, a second C2f_DCNv2 module, a fifth convolutional layer, a third C2f_DCNv2 module, and an SPPF module, all connected sequentially with different kernel parameters. The first convolutional layer is used to extract geometric features from the sample images in the sample dataset. The second convolutional layer is used to perform a convolution operation on the output of the first convolutional layer. The first C2f module is used to extract geometric features from the output of the second convolutional layer. The third convolutional layer is used to perform a convolution operation on the output of the first C2f module. The first, second, and third C2f_DCNv2 modules are essentially the C2f modules of the original YOLOv8 network. The standard convolutional layers in the module are replaced with deformable convolutional modules (DCNv2); the first C2f_DCNv2 module is used to extract geometric features from the output of the third convolutional layer; the fourth convolutional layer is used to perform convolution operations on the output of the first C2f_DCNv2 module; the second C2f_DCNv2 module is used to extract geometric features from the output of the fourth convolutional layer; the fifth convolutional layer is used to perform convolution operations on the output of the second C2f_DCNv2 module; the third C2f_DCNv2 module is used to extract geometric features from the output of the fifth convolutional layer; the SPPF module is used to perform pooling operations on the output of the third C2f_DCNv2 module; the multi-scale geometric feature map is the output of the first C2f_DCNv2 module, the output of the second C2f_DCNv2 module, and the output of the SPPF module.

3. The method for marine target detection and 3D point cloud perception based on infrared binocular vision according to claim 2, characterized in that, The neck feature fusion network described in S3 includes, in sequence, a first upsampling layer, a first concatenation layer, a fourth C2f_DCNv2 module, a second upsampling layer, a second concatenation layer, a second C2f module, a first EMA attention module, a sixth convolutional layer, a third concatenation layer, a third C2f module, a second EMA attention module, a seventh convolutional layer, a fourth concatenation layer, a fourth C2f module, and a third EMA attention module. The first upsampling layer performs an upsampling operation on the output of the SPPF module. The first concatenation layer performs channel concatenation on the output of the first upsampling layer and the output of the second C2f_DCNv2 module. The fourth C2f_DCNv2 module extracts geometric features from the output of the first concatenation layer. The second upsampling layer performs an upsampling operation on the output of the fourth C2f_DCNv2 module. The second concatenation layer performs channel concatenation on the output of the second upsampling layer and the output of the first C2f_DCNv2 module. The second C2f module extracts geometric features from the output of the second concatenation layer. The first EMA attention module is used to extract global contextual features from the output of the second C2f module based on the attention mechanism; The sixth convolutional layer is used to perform convolution operations on the output of the first EMA attention module; the third concatenation layer is used to perform channel concatenation operations on the output of the sixth convolutional layer and the output of the fourth C2f_DCNv2 module; the third C2f module is used to extract geometric features from the output of the third concatenation layer. The second EMA attention module is used to extract global contextual features from the output of the third C2f module based on the attention mechanism; The seventh convolutional layer is used to perform convolution operations on the output of the second EMA attention module; the fourth concatenation layer is used to perform channel concatenation operations on the output of the seventh convolutional layer and the output of the SPPF module; the fourth C2f module is used to extract geometric features from the output of the fourth concatenation layer. The third EMA attention module is used to extract global context features from the output of the fourth C2f module based on the attention mechanism; the fused feature map is the output of the first EMA attention module, the output of the second EMA attention module, and the output of the third EMA attention module.

4. The method for marine target detection and three-dimensional point cloud perception based on infrared binocular vision according to claim 3, characterized in that, The method for obtaining the optimal object detection model in S4 is as follows: S41: Randomly divide the sample dataset into a training set and a validation set according to a preset ratio; S42: Train the constructed YOLOv8-IRECG object detection model based on the training set to obtain the trained object detection model; S43: Use GIoU Loss as the model loss function, and validate the trained object detection model using the validation set; That is, to determine whether the output of the trained object detection model has converged; If the output of the trained object detection model converges, then the trained object detection model is confirmed to be the optimal object detection model. Otherwise, the weight parameters of the trained object detection model are adaptively adjusted based on the backpropagation method, and step S42 is repeated until the weight parameters of the trained object detection model with converged output are confirmed to be the optimal weight parameters, and the object detection model is reconstructed to obtain the optimal object detection model.

5. The method for marine target detection and three-dimensional point cloud perception based on infrared binocular vision according to claim 3, characterized in that, S5 specifically includes the following steps: S51: Construct an image pyramid network containing a first resolution layer and a second resolution layer; The first resolution layer in the image pyramid network preserves the original image resolution of the preprocessed left and preprocessed right images; The second resolution layer is used to perform downsampling operations on the preprocessed left image and the preprocessed right image based on the bilinear interpolation algorithm; S52: Perform SGBM stereo matching operation on the output images of the first resolution layer and the second resolution layer respectively to obtain the first initial disparity map D1 corresponding to the first resolution layer and the second initial disparity map D2 corresponding to the second resolution layer; then perform upsampling operation on the second initial disparity map D2 through bilinear interpolation algorithm to obtain the second optimized disparity map D2_up with the same scale as the first initial disparity map D1. S53: Perform a weighted fusion of the first initial disparity map D1 and the second optimized disparity map D2_up to obtain the fused disparity map D_sum: D_sum = W1×D1 + W2×D2_up In the formula: W1 and W2 represent weighting coefficients; S54: Apply the Canny edge detection operator to perform edge detection on the preprocessed left image to obtain an edge binary map Edge_map; traverse the pixels in the fused disparity map D_sum, and take the pixels with a disparity value of 0 as holes. Using the holes as the center point, search for non-hole pixels in a preset neighborhood. Take the non-hole pixels whose distance from the center point is less than a preset distance threshold as the disparity pixel set, and obtain the median disparity of the pixels based on the disparity pixel set as the filling value of the holes to obtain the interpolated disparity map D_inter; S55: Using the preprocessed left image as the guide image, the interpolated disparity map D_inter is filtered based on the adaptive WLS filtering algorithm to obtain the final disparity map D_final; and based on the camera parameters of the infrared stereo camera, the final disparity map D_final is converted into a depth feature map, and the conversion formula is: Z = (f×B) / d; where Z represents the vertical depth from the target to the stereo camera; f represents the effective focal length of the stereo camera; B represents the baseline distance of the stereo camera, i.e., the horizontal distance between the optical centers of the two cameras; and d represents the disparity value, i.e., pixels.

6. The method for marine target detection and three-dimensional point cloud perception based on infrared binocular vision according to claim 5, characterized in that, S6 specifically includes the following steps: S61: Based on the pre-calibrated intrinsic parameter matrix of the left eye camera, perform inverse perspective projection on all pixels in the depth feature map to generate a global scene 3D point cloud. The coordinate transformation formula for inverse perspective projection is: In the formula: , These represent the horizontal, vertical, and lateral coordinates of the target in the camera coordinate system, respectively. This represents the depth value, i.e., the vertical coordinate. ; , This represents the position coordinates of a pixel in the depth feature map; express Focal length along the axial direction; express Focal length along the axial direction; , Indicates the optical center coordinates of the camera; S62: By using the two-dimensional detection box output in step S4 as the view frustum truncation boundary, spatial clipping is performed on the global scene three-dimensional point cloud to obtain a local point cloud subset; S63: A statistical outlier removal algorithm based on spatial distance distribution characteristics is used to remove outliers from a subset of local point clouds to obtain an optimized target point cloud. Based on the spatial bounding box algorithm, the minimum bounding box (OBB) representing the perception of the three-dimensional point cloud of the maritime target is obtained according to the geometric centroid and three-dimensional spatial coordinates of the optimized target point cloud. This enables the detection of maritime targets and the perception of three-dimensional point clouds based on infrared binocular vision.

Images