Multi-scale marine benthos detection method based on global-local trans-attention fusion

Abstract

The invention discloses a multi-scale marine benthos detection method based on global-local cross-attention fusion, and the method comprises the steps: collecting and preprocessing submarine data information through an underwater robot, and building a multi-scale marine benthos data set; preprocessing the multi-scale marine benthic organism data set and dividing the multi-scale marine benthic organism data set into a training set and a test set; building a multi-scale benthic organism detection framework based on global-local cross-attention fusion; adopting the training set to train the multi-scale benthos detection framework to obtain a multi-scale marine benthos detection model; and performing deployment and hyper-parameter adjustment on the trained multi-scale marine benthos detection model, and using the obtained model to detect marine benthos. According to the method, histogram equalization is introduced to pre-process the image, so that the image contrast is effectively enhanced, and the problem of low contrast of the underwater image caused by uneven illumination, color degradation or blurring is solved.

Description

Technical Field

[0001] The invention relates to the field of computer vision technology, and more particularly to a multi-scale marine benthic organism detection method based on global-local cross-attention fusion. Background Technology

[0002] Marine benthic organisms are an important component of marine ecosystems, and the rational development of marine benthic resources is of great significance. Although modern, large-scale marine ranches have provided effective solutions for developing marine benthic resources, improvements are still needed in the seafood harvesting process. There is an urgent need to develop accurate and efficient methods for detecting marine benthic organisms to assist robotic operations in the water and improve harvesting efficiency.

[0003] Marine benthic organism detection is one of the most cutting-edge research areas in computer vision, attracting considerable attention from scholars in recent years. With the rise of deep learning technology, many mainstream detection models have been applied to marine benthic organism detection tasks. Examples include multi-stage anchor-bound detection methods represented by Faster R-CNN, single-stage anchor-bound detection methods represented by YOLO, and single-stage anchor-free detection methods represented by RetinaNet.

[0004] In real underwater environments, marine benthic organisms exhibit significant size variations, coexistence of targets at multiple scales, and a high proportion of small targets. Furthermore, factors such as changing viewing angles, occlusion, and blurring lead to large geometrical deformations of underwater targets, resulting in poor performance of existing detection algorithms. Therefore, there is an urgent need to design a detection method that is highly adaptable to the unique underwater environment to achieve efficient detection of marine benthic organisms. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention discloses a multi-scale marine benthic organism detection method based on global-local cross-attention fusion. This method processes a publicly available dataset from an underwater robot harvesting competition to establish a multi-scale marine benthic organism dataset; it constructs a multi-scale benthic organism detection network based on global-local cross-attention fusion and trains it on the dataset to obtain a multi-scale marine benthic organism detection model; finally, it deploys the trained model on a detection device to achieve efficient detection of multi-scale marine benthic organisms.

[0006] The technical means employed in this invention are as follows: A multi-scale marine benthic organism detection method based on global-local cross-attention fusion includes: S1. Process the publicly available dataset from the underwater robot harvesting competition to establish a multi-scale marine benthic organism dataset; S2. Preprocessing and segmenting the multi-scale marine benthic organism dataset; S3. Establish a multi-scale marine benthic organism detection framework based on global-local cross-attention fusion; S4. Train the detection framework on the preprocessed and partitioned dataset to obtain a multi-scale marine benthic organism detection model; S5. Deploy the training model and adjust the hyperparameters, then use the resulting model for marine benthic organism detection tasks; Furthermore, in S1, the step of processing the publicly available dataset from the underwater robot harvesting competition includes: Use a hash algorithm to evaluate the image duplication rate in the dataset and remove severely blurred, duplicate, and invalid labeled images.

[0007] Furthermore, in S2, the preprocessing and segmentation steps for the multi-scale marine benthic organism dataset include: Images from a multi-scale marine benthic organism dataset are read in, and histogram equalization is used to change the form of image grayscale distribution to generate image set A. Read in the images from image set A, scale the images to a standard size, generate image set B to be trained, and randomly divide the training set and test set in a 7:3 ratio.

[0008] Furthermore, in S3, the step of constructing a multi-scale benthic organism detection network based on global-local cross-attention fusion includes: Build an input layer to receive underwater images; A backbone network using multi-branch downsampling convolutional blocks and cross-channel residual blocks is constructed to extract features from the input image; A neck network employing a global attention mechanism and cross-channel residual blocks is constructed to fuse multi-scale features, and the fused features are then fed into the detection head to generate detection results. Combining the networks constructed in the above steps yields a detection network based on global-local cross-attention fusion.

[0009] Furthermore, in S3, the multi-branch downsampling convolution calculation process includes: Perform 2×2 average pooling on the feature map output by the upper layer network; The input underwater image is divided into four branches along the channel dimension, with each branch having one-quarter of the number of channels as the original input channels; Branch 1 connects the residual to the output; Branch 2 performs 3×3 convolution to extract features; Branch 3 first performs 2×2 average pooling, then performs 1×1 convolution to adjust the number of channels; Branch 4 first performs 2×2 max pooling, then performs 1×1 convolution to adjust the number of channels. Finally, the sampling results of the four branches are concatenated along the channel dimension to obtain the output feature map.

[0010] Furthermore, in S3, the global attention mechanism calculation process includes: The input feature map is reshaped by a three-dimensional arrangement. The output result is input into a two-layer multilayer perceptron and reshaped again to obtain a weight matrix. The weight matrix is ​​then multiplied element-wise with the input feature map to obtain the output feature map. The input feature map is subjected to two 7×7 convolutions to obtain the output feature map.

[0011] Furthermore, in S5, the process of deploying the trained model for the marine benthic organism detection task includes: deploying the multi-scale marine benthic organism detection model on the detection equipment; reading the parameter configuration file and loading the pre-trained model weights; adaptively adjusting the hyperparameters and configuring the model processing rate; reading the real-time input image and preprocessing the input image; sending the processed image into the multi-scale marine benthic organism detection model to perform target prediction; and visualizing the location and category information in the detection results.

[0012] By employing the aforementioned technical solutions, this invention discloses a multi-scale marine benthic organism detection method based on global-local cross-attention fusion. This method introduces histogram equalization for image preprocessing, effectively enhancing image contrast and improving the low contrast problem caused by uneven lighting, color degradation, or blurring in underwater images. Furthermore, this method improves the detection performance of traditional single-stage detection algorithms by introducing multi-branch downsampling convolutional blocks and a global attention mechanism, enhancing the model's adaptability and improving the algorithm's adaptability to changes in the scale and morphology of marine benthic organisms. This method proposes a single-stage anchor-frame-less detection model that, while maintaining high detection performance, meets the computational complexity requirements of embedded devices in real-world environments, achieving high-efficiency and high-precision detection of marine benthic organisms. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of the multi-scale marine benthic organism detection method based on global-local cross-attention fusion provided by the present invention.

[0015] Figure 2 This is a schematic diagram of the network structure of the single-stage anchorless detection algorithm provided by the present invention.

[0016] Figure 3 This is a flowchart of multi-branch convolution.

[0017] Figure 4 This is a flowchart of the global attention mechanism.

[0018] Figure 5 This is a visual representation of the test results.

[0019] Figure 6 These are the comparative experimental results of this detection method compared with other mainstream detection methods. Detailed Implementation To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention: like Figure 1 The method for detecting marine benthic organisms based on global-local cross-attention fusion, as shown, specifically includes: S1. Process the publicly available dataset from the underwater robot harvesting competition to establish a multi-scale marine benthic organism dataset; In a specific implementation, as a preferred embodiment of the present invention, the publicly available dataset of the underwater robot harvesting competition is processed, and a hash algorithm is used to evaluate the image duplication of the dataset, removing images that are severely blurry, duplicate, or have invalid annotations.

[0020] S2. Preprocessing and segmenting the multi-scale marine benthic organism dataset; In a specific implementation, as a preferred embodiment of the present invention, the multi-scale marine benthic organism dataset is preprocessed and divided. Images from the multi-scale marine benthic organism dataset are read in, and histogram equalization is used to change the form of image grayscale distribution to generate image set A. Images from image set A are read in, and the image size is scaled to a standard size to generate image set B to be trained. The training set and test set are randomly divided in a 7:3 ratio.

[0021] S3. Construct a multi-scale benthic organism detection framework based on global-local cross-attention fusion; In a specific implementation, as a preferred embodiment of the present invention, the construction of a multi-scale benthic organism detection network based on global-local cross-attention fusion involves: constructing an input layer to receive input underwater images; constructing a backbone network using multi-branch downsampling convolutional blocks and cross-channel residual blocks to extract input image features; constructing a neck network using a global attention mechanism and cross-channel residual blocks to fuse multi-scale features, and then connecting the fused features to the detection head to generate detection results; and combining the networks constructed in the above steps to obtain a multi-scale benthic organism detection framework based on global-local cross-attention fusion.

[0022] The multi-branch downsampling convolution calculation process involves performing 2×2 average pooling on the output feature map of the upper-layer network; dividing the input underwater image into four branches along the channel dimension, with each branch having one-quarter of the original input channels; connecting the residual of branch one to the output; performing 3×3 convolution to extract features on branch two; performing 2×2 average pooling on branch three, followed by 1×1 convolution to adjust the number of channels; performing 2×2 max pooling on branch four, followed by 1×1 convolution to adjust the number of channels; and finally concatenating the sampling results of the four branches along the channel dimension to output a new feature map.

[0023] The global attention mechanism calculation process involves reshaping the input feature map through a three-dimensional arrangement, inputting the output result into a two-layer multilayer perceptron, reshaping it again to obtain a weight matrix, multiplying the obtained weight matrix element-wise with the input feature map to obtain the output feature map, and performing two 7×7 convolutions on the input feature map to obtain the output feature map.

[0024] S4. Train the detection framework on the preprocessed and partitioned dataset to obtain a multi-scale marine benthic organism detection model; In a preferred embodiment of the present invention, the training process employs an adaptive learning rate adjustment strategy, dynamically adjusting the learning rate according to the loss changes during training to ensure that the model can converge stably during training.

[0025] S5. Deploy the training model and adjust the hyperparameters. The resulting model is used for marine benthic organism detection tasks. In a specific implementation, as a preferred embodiment of the present invention, the process of deploying the trained model for marine benthic organism detection includes: deploying the multi-scale marine benthic organism detection model on the detection device; reading the parameter configuration file and loading the pre-trained model weights; adaptively adjusting the hyperparameters and configuring the model processing rate; reading the real-time input image and preprocessing the input image; sending the processed image into the multi-scale marine benthic organism detection model to perform target prediction; and visualizing the location and category information in the detection results. Example like Figure 1 As shown, this invention provides a multi-scale marine benthic organism detection method based on global-local cross-attention fusion; The dataset was obtained by processing the publicly available dataset from the National Underwater Robot Harvesting Competition. A hash algorithm was used to evaluate the image redundancy in the dataset, removing severely blurry, duplicate, and invalidally labeled images. Ultimately, 5760 images were retained, representing 74% of the original dataset. The dataset includes four types of objects: sea urchins, sea cucumbers, scallops, and starfish, and is randomly divided into training and testing sets in a 7:3 ratio.

[0026] Training Process: All experiments were conducted on an Ubuntu 22.04 workstation equipped with an Intel Core i7 processor, three NVIDIA RTX A6000 GPUs, and 32GB of memory. Experiments were implemented using PyTorch 1.12.1, CUDA 11.6, and cuDNN 8.3.2, with 200 rounds per round. Experimental results show that the multi-scale marine benthic organism detection method based on global-local cross-attention fusion improves the AP, AP50, and AP75 metrics by 2.44%, 1.18%, and 3.8%, respectively, compared to the baseline model.

[0027] like Figure 2 As shown, this embodiment provides a single-stage anchorless frame detection algorithm.

[0028] like Figure 3 As shown, this embodiment provides a multi-branch downsampling convolution module, which performs 2×2 average pooling on the feature map output by the upper layer network; divides the input underwater image into four branches along the channel dimension, with each branch having one-quarter of the original input channels; the residual of branch one is connected to the output; branch two performs 3×3 convolution to extract features; branch three first performs 2×2 average pooling, and then performs 1×1 convolution to adjust the number of channels; branch four first performs 2×2 max pooling, and then performs 1×1 convolution to adjust the number of channels; finally, the sampling results of the four branches are concatenated along the channel dimension to output a new feature map.

[0029] like Figure 4 As shown, this embodiment provides a global attention mechanism, the calculation process of which includes: reshaping the input feature map through three-dimensional arrangement, inputting the output result into a two-layer multilayer perceptron, reshaping it again to obtain a weight matrix, multiplying the obtained weight matrix element-wise with the input feature map to obtain the output feature map; and performing two-layer 7×7 convolution on the input feature map to obtain the output feature map.

[0030] like Figure 5 As shown in the figure, this embodiment provides a visualization diagram of the detection results of a multi-scale marine benthic organism detection method based on global-local cross-attention fusion. The position of the detection box represents the spatial location of the target, and the number above the detection box represents the confidence level of the detection result.

[0031] like Figure 6 As shown, this embodiment provides comparative experimental results of the multi-scale marine benthic organism detection method based on global-local cross-attention fusion and other mainstream detection methods in marine benthic organism detection tasks.

[0032] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-scale marine benthic organism detection method based on global-local cross-attention fusion, characterized in that, include: A multi-scale marine benthic organism dataset was established by collecting and preprocessing seabed data using an underwater robot. The multi-scale marine benthic organism dataset was preprocessed and divided into training and testing sets. A multi-scale benthic organism detection framework based on global-local cross-attention fusion was established. A multi-scale marine benthic organism detection model was obtained by training the multi-scale benthic organism detection framework using a training set. The trained multi-scale marine benthic organism detection model was deployed and its hyperparameters were adjusted, and the resulting model was used to detect marine benthic organisms.

2. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: When preprocessing seabed data: use a hash algorithm to evaluate the repetition of the image dataset and remove blurry, duplicate, and invalid labeled images.

3. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: Images from a multi-scale marine benthic organism dataset are read, and the gray-level distribution of the images is transformed using a histogram equalization method to generate image set A. Read the data information from image set A, scale the image size to a standard size, generate image set B to be trained, and randomly divide the training set and test set according to a 7:3 ratio.

4. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: When building a multi-scale benthic organism detection framework: an input layer is built to receive the input underwater images, and a backbone network based on a multi-branch downsampling convolutional module and a cross-channel residual module is built to extract the features of the input images; A neck network using a global attention mechanism and cross-channel residual blocks is constructed to fuse multi-scale features, and the fused features are then fed into the detection head to generate detection results. The input layer, backbone network, and neck network are combined to obtain a multi-scale marine benthic organism detection model.

5. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: By combining the networks constructed in the above steps, a multi-scale marine benthic organism detection method based on global-local cross-attention fusion is obtained.

6. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: When building a multi-branch downsampling convolution module: Average pooling is performed on the feature maps output by the upper-layer network; The input underwater image is divided into four branches along the channel dimension, with each branch having one-quarter of the number of channels as the input channels; Branch 1 connects the residual to the output; Branch 2 performs convolution to extract features. Branch 3 first performs average pooling, then performs convolution to adjust the number of channels; Branch 4 first performs max pooling, then performs convolution to adjust the number of channels. Finally, the outputs of the four branches are concatenated along the channel dimension to output a new feature map.

7. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: When building a global attention mechanism: The input feature map is reshaped by a three-dimensional arrangement. The output result is input into a two-layer multilayer perceptron and reshaped again to obtain a weight matrix. The weight matrix is ​​then multiplied element-wise with the input feature map to obtain the output feature map. The input feature map is then subjected to two convolutional layers to obtain the output feature map.

8. The multi-scale marine benthic organism detection method based on global-local cross-attention fusion according to claim 1, characterized in that: When deploying multi-scale marine benthic organism detection models: A multi-scale marine benthic organism detection model is deployed on the detection equipment. The parameter configuration file is read, the pre-trained model weights are loaded, the hyperparameters are adaptively adjusted, and the model processing rate is configured. Real-time input images are read and preprocessed. The processed images are fed into a multi-scale marine benthic organism detection model to perform target prediction. Visualize the location and category information in the detection results.

Images