A detection method based on a packet cross-channel attention deformable convolution network

Abstract

The application discloses a detection method based on a grouping cross-channel attention deformable convolution network, and comprises the following steps: acquiring a data set composed of to-be-recognized benthic organism pictures, and dividing the data set into a training set and a test set according to a proportion; constructing a grouping cross-channel attention deformable convolution neural network model for recognizing benthic organisms on the pictures; training the grouping cross-channel attention deformable convolution neural network model based on the training set to obtain a trained grouping cross-channel attention deformable convolution neural network model; and inputting test set data into the trained grouping cross-channel attention deformable convolution neural network model to realize recognition of the benthic organism species on the pictures. By organically integrating a BDC and an SGCA module, an SGCA-BDC framework is established, benthic organism features are enhanced from a space and a channel level, and finally the accuracy and the robustness of benthic organism detection are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of underwater visual intelligent perception, and particularly relates to a detection method based on a grouping cross-channel attention deformable convolutional network. BACKGROUND

[0002] With the continuous progress of underwater machine vision technology, people's demand for high-intelligent underwater robots to autonomously capture benthic organisms (such as sea urchins, scallops, starfish and sea cucumbers) in marine aquaculture is also increasing. It should be noted that in the extremely harsh underwater environment of low contrast, uneven illumination and color distortion, geometric deformation caused by adaptive motion blur, view angle change and occlusion is extremely challenging for benthic organism detection. In this case, by introducing a deformable convolutional neural network with an attention mechanism, the accuracy of benthic organism detection can be improved.

[0003] It should be noted that the benthic organism detection task is largely dependent on the extracted effective features. Specifically, traditional machine learning-based methods (such as SIFT, SURF and HOG, etc.) can only extract limited low-level features such as color, texture and edge, etc., which are obtained through pre-defined hand-crafted descriptors. Unfortunately, it is difficult to obtain effective semantic features in the case of almost the same color between foreground and background. Fortunately, high-level semantic feature extraction frameworks based on convolutional neural networks, such as VGG, GoogLeNet, ResNet and DenseNet, etc., have been widely developed. Accordingly, relevant scholars have proposed many two-stage object detectors, including Faster RCNN, SPPNet, FPN and Cascade RCNN, etc. Specifically, on the basis of combining generative adversarial networks and white balance technology, a benthic organism detection framework based on the Fast RCNN method is established, which can effectively compensate for color cast and color difference, thereby significantly improving the accuracy of detection and recognition. Essentially, the region proposal of the above method is realized through an external selective search method, so an end-to-end training framework cannot be established. Moreover, the aforementioned two-stage benthic organism detection method takes the region proposal as an intermediate step, which obviously increases the computational burden and makes it difficult to provide efficient inference capability.

[0004] To solve the foregoing problems, methods such as SSD, YOLO, G-CNN and OverFeat directly use a regression strategy to calculate coordinates and predict categories. In this case, single-stage-based methods have obvious advantages in benthic detection tasks, and are therefore widely used in underwater robots. Specifically, considering the extremely limited hardware computing resources in the actual environment, relevant scholars have developed a MobileNet-SSD framework based on depth separable convolution, which decomposes standard convolution into depth convolution and point-wise convolution, thereby significantly reducing training parameters and improving nonlinear mapping capabilities. In order to further improve detection speed and reduce redundant calculations, by using a convolution kernel pruning technique to eliminate neurons that contribute less to detection performance, researchers have proposed a pruned SSD method, which can achieve a detection speed of 46.4 FPS and an accuracy of 89.54%. Notably, benthic detection methods based on SSD use a large number of prior boxes with a fixed aspect ratio, and do not fully utilize the prior dimension knowledge of benthic organisms, thereby inevitably leading to poor detection performance. To solve the above problems, a YOLO-based benthic detection framework is established using the K-means dimension clustering method, which uses different scale anchor boxes to predict the coordinates of the bounding box, thereby significantly improving the accuracy of benthic detection.

[0005] The benthic detection technology in a complex multi-domain scene mainly has the following defects: (1) the standard convolutional neural network itself has a fixed geometric sampling structure, so the above detection framework cannot correctly detect and recognize geometrically deformed objects caused by occlusion, motion blur and changes in viewing angle; (2) only the distance information between the offset position and the integer position is linearly utilized, and the feature difference is not fully utilized, which can easily cause feature details to degenerate; (3) the single-hidden-layer feedforward network method using hidden layer dimension reduction can destroy the direct correspondence between channels and weights, and inevitably introduce a large number of weight parameters. On the other hand, using global average pooling or global maximum pooling operations can hardly comprehensively provide channel descriptors with high expression ability to fully describe the statistical information of the entire feature map. SUMMARY

[0006] To solve the above problems, the technical scheme adopted by the present application is: a detection method based on a grouped cross-channel attention deformable convolutional neural network, comprising the following steps:

[0007] Obtain a data set composed of benthic organism pictures to be identified, and divide the data set into a training set and a test set according to a proportion;

[0008] Construct a grouped cross-channel attention deformable convolutional neural network model for identifying benthic organisms on pictures;

[0009] training the grouped cross-channel attention deformable convolutional neural network model based on the training set to obtain a trained grouped cross-channel attention deformable convolutional neural network model;

[0010] inputting the test set data into the trained grouped cross-channel attention deformable convolutional neural network model to realize recognition of the benthic species on the picture.

[0011] Further, the grouped cross-channel attention deformable convolutional neural network model comprises a plurality of cascaded grouped cross-channel attention deformable convolutional neural network units.

[0012] The grouped cross-channel attention deformable convolutional neural network unit comprises a first convolutional layer, a second convolutional layer, a bilateral filtering interpolation deformable convolutional neural network module and a random grouped cross-channel attention module; wherein the first convolutional layer functions to increase the nonlinear mapping capability; the second convolutional layer functions to extract features; the bilateral filtering interpolation deformable convolutional neural network module functions to increase the deformation benthic feature construction capability; and the random grouped cross-channel attention module functions to suppress the seabed background noise.

[0013] The first convolutional layer, the second convolutional layer, the bilateral filtering interpolation deformable convolutional neural network module and the random grouped cross-channel attention module are sequentially cascaded.

[0014] Further, the bilateral filtering interpolation deformable convolutional neural network module is constructed as follows:

[0015] The standard sampling position of the 3x3 convolution kernel is represented as:

[0016] P={(u,v)|u,v∈{-1,0,1}} (1)

[0017] wherein: P is a set of sampling positions (u, v);

[0018] The deformation offset position of each element in the convolution kernel is described as:

[0019]

[0020] wherein: m=1, 2, …, n, n=|P|, p m ∈P, p0 is the center position of the convolution kernel, Δp m represents the offset of the mth element, represents the deformation sampling position of the mth element of the convolution kernel, in addition, the offset coordinates need to meet the following conditions:

[0021]

[0022] wherein, w and h respectively represent the width and height of the current feature map;

[0023] The feature value is calculated by using linear interpolation in the X direction which is expressed as:

[0024]

[0025] where p tl = (x i , y i ), p tr = (x i+1 , y i ), p bl = (x i , y i+1 ) and p br = (x i+1 , y i+1 ) are the integer coordinates of the upper-left, upper-right, lower-left and lower-right corners adjacent to the offset position

[0026] The feature value is expressed as:

[0027]

[0028] The linear interpolation along the Y direction is expressed as:

[0029]

[0030] where is the feature value of the offset position

[0031] Subsequently, the spatial similarity is expressed as:

[0032]

[0033] where ★ ∈ {tl, tr, bl, br}, p ★ denotes the integer coordinates of the upper-left, upper-right, lower-left and lower-right respectively, and σ s is the spatial smoothing parameter;

[0034] The feature similarity is expressed as:

[0035]

[0036] where σ f is the feature smoothing parameter.

[0037] In this case, the combined weight at the offset position can be expressed as:

[0038] ​​

[0039] In this case, the feature value at the offset position can be updated as:

[0040]

[0041] where, is the feature map by considering the feature similarity and spatial similarity at the same time;

[0042] Finally, the whole deformable convolution module can be represented as:

[0043]

[0044] where, is the output feature map, represents the weight of the mth element in the convolution filter, w m represents the modulation scalar of the mth offset position, represents the convolution operation. Obviously, w m can be constrained to (0, 1) using the logical activation function σ(·).

[0045] Further: the construction process of the random grouping cross-channel attention module is as follows:

[0046] In order to capture specific semantic responses, a given X is divided into g groups, that is, where c is the number of channels containing g groups, X n is the nth sub-feature;

[0047] Subsequently, the information of each feature map is counted using the information entropy technique, and the feature probability statistics in a single feature map are represented as:

[0048]

[0049] where, i = 1, 2, …, l, j = 1, 2, …, c / g, n = 1, 2, …, g, and are the number and probability of the ith feature level in the jth feature map in the nth group, respectively;

[0050] Correspondingly, the information entropy of a single feature map can be represented as:

[0051]

[0052] where: represents the information entropy of the jth feature map in the nth group;

[0053] In addition, based on the 1-D convolution technique, a random grouping cross-channel attention module is proposed, and the optimal convolution kernel size k is determined by the following method:

[0054]

[0055] where k represents the filter size in 1-D convolution, i.e., the coverage of cross-channel interaction, and γ and b are hyperparameters, and the high-dimensional and low-dimensional channels correspond to larger and smaller filters, respectively, and exhibit a nonlinear relationship. In addition, the channel weight can be expressed as:

[0056]

[0057] wherein and represent the weights of the convolution kernel and the recalibrated channel weight, respectively.

[0058] Further, the recalibrated output in the nth group of feature maps is expressed as:

[0059]

[0060] wherein represents the jth recalibrated feature map in the nth group; by splicing operation, the recalibrated sub-features are aggregated along the channel dimension, which can be expressed as:

[0061] To strengthen the information exchange between different groups, a channel shuffle operation is used, which can be expressed as:

[0062]

[0063] wherein: represents the output of the reshaping operation R1, represents the output of the transpose operation T, represents the output of the reshaping operation R2.

[0064] A benthic organism detection device based on a grouping cross-channel attention deformable convolutional neural network, comprising:

[0065] An acquisition module for acquiring a data set composed of pictures of benthic organisms to be identified, and dividing the data set into a training set and a test set according to a proportion;

[0066] A construction module for constructing a grouping cross-channel attention deformable convolutional neural network model for identifying benthic organisms on pictures;

[0067] A training module for training the grouping cross-channel attention deformable convolutional neural network model based on the training set to obtain a trained grouping cross-channel attention deformable convolutional neural network model.

[0068] Implementing module: for inputting test set data into the trained group cross-channel attention deformable convolutional neural network model, realizing the identification of benthic species on the picture.

[0069] The detection method based on the group cross-channel attention deformable convolutional network has the following advantages:

[0070] By utilizing the spatial similarity and feature similarity between the offset position and the integer position, a bilateral filter interpolation deformable convolutional neural network module is created, which can effectively preserve high-frequency features and maintain edge characteristics, and thus can adaptively enhance the spatial sampling capability of the convolution kernel;

[0071] In order to respectively enhance the benthic biological features and suppress the underwater background noise, a shuffle group cross-channel attention module is innovatively designed, the information entropy technology is used to depict the information amount of the sub-feature map, and the 1-D convolution technology is used, which is helpful to enhance the benthic biological features from the channel level;

[0072] By organically integrating the BDC and SGCA modules, an SGCA-BDC framework is established, which enhances the benthic biological features from the spatial and channel levels, and finally helps to improve the accuracy and robustness of benthic biological detection. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0074] Figure 1 is a 3x3 convolution kernel sampling position diagram; (a) standard sampling position (green dot); (b) morphing sampling position (red dot);

[0075] Figure 2 is a bilateral filter interpolation diagram;

[0076] Figure 3 is an SGCA module composition diagram;

[0077] Figure 4 is an SGCA-BDC module diagram;

[0078] Figure 5 is a feature sampling position visualization, (a) original image; (b) standard convolution sampling position; (c) BDC module sampling position;

[0079] Figure 6are prediction performance and comparison; (a) P-R curve for detecting sea urchin; (b) P-R curve for detecting scallop; (c) P-R curve for detecting starfish; (d) P-R curve for detecting sea cucumber;

[0080] Figure 7 are comparison graphs of detection results in real underwater environment. DETAILED DESCRIPTION

[0081] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0082] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work belong to the scope of protection of the present application.

[0083] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0084] Unless specifically stated otherwise, the relative arrangement of the components and steps, numerical expressions, and numerical values set forth in the various embodiments described herein are not limiting. It should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn in accordance with the actual proportion relationship. The technology, methods and devices known to those skilled in the relevant art can not be discussed in detail, but should be considered as part of the authorized specification, if appropriate. In all examples shown and discussed herein, any specific value should be interpreted as merely exemplary, and not as a limitation. Therefore, other examples of exemplary embodiments can have different values. It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0085] In the description of the present application, it should be understood that the orientation words such as "front, back, up, down, left, right", "transverse, vertical, perpendicular, horizontal" and "top, bottom" and the like indicate the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate and imply that the devices or elements referred to must have a particular orientation or be constructed and operated in a particular orientation, therefore cannot be understood as a limitation on the scope of protection of the present application: the orientation words "inner, outer" refer to the inner and outer of the contour of each component itself.

[0086] For the convenience of description, spatial relative terms such as "over", "above", "upper surface", "upper" and the like can be used herein to describe the spatial positional relationship of one device or feature with other devices or features as shown in the drawings. It should be understood that the spatial relative terms are intended to include different orientations in use or operation in addition to the orientation of the devices described in the drawings. For example, if the devices in the drawings are inverted, the device described as "above" or "over" other devices or structures will be positioned "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below" orientations. The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein are interpreted accordingly.

[0087] In addition, it should be noted that the use of the words "first", "second" and the like to define parts is only for the convenience of distinguishing the corresponding parts, and the above words have no special meaning unless otherwise stated, and therefore cannot be understood as a limitation on the scope of protection of the present application.

[0088] A detection method based on a shuffled grouping cross-channel attention-based bilateral-filter-interpolation deformable convolutional network (SGCA-BDC) includes the following steps:

[0089] S1: obtaining a data set composed of pictures of benthic organisms to be identified, and dividing the data set into a training set and a test set according to a proportion;

[0090] S2: constructing a shuffled grouping cross-channel attention-based deformable convolutional neural network model for identifying benthic organisms on pictures;

[0091] S3: training the grouping cross-channel attention deformable convolutional neural network model based on the training set to obtain a trained grouping cross-channel attention deformable convolutional neural network model;

[0092] S4: inputting the test set data into the trained grouping cross-channel attention deformable convolutional neural network model to realize the identification of the benthic species on the picture.

[0093] Steps S1 / S2 / S3 / S4 are sequentially executed;

[0094] Further, the benthic organism detection network comprises a plurality of cascaded grouping cross-channel attention deformable convolutional neural network units;

[0095] The grouping cross-channel attention deformable convolutional neural network unit comprises a first convolutional layer, a second convolutional layer, a bilateral filter interpolation deformable convolutional neural network module, and a random grouping cross-channel attention module; wherein the first convolutional layer functions to increase the nonlinear mapping capability; the second convolutional layer functions to extract features; the bilateral filter interpolation deformable convolutional neural network module functions to increase the deformation benthic organism feature construction capability; and the random grouping cross-channel attention module functions to suppress the seabed background noise.

[0096] The first convolutional layer, the second convolutional layer, the bilateral filter interpolation deformable convolutional neural network module, and the random grouping cross-channel attention module are sequentially cascaded.

[0097] Further, the bilateral filter interpolation deformable convolutional neural network module is constructed as follows:

[0098] A bilateral filter interpolation deformable convolutional neural network (BDC) module is designed to enhance the sampling capability of a conventional convolution kernel in space. As shown in Figure 1 The standard sampling positions of a 3x3 convolution kernel can be represented as:

[0099] P={(u,v)|u,v∈{-1,0,1}} (1)

[0100] Where P is a set of sampling positions (u, v).

[0101] The deformation offset position of each element in the convolution kernel can be described as:

[0102]

[0103] Where m = 1, 2, …, n, n = |P|, p m ∈P, p0 is the center position of the convolution kernel, and Δp m represents the offset of the mth element. denotes the warped sampling position of the m-th element of the convolution kernel. In addition, the offset coordinates The following conditions need to be satisfied:

[0104]

[0105] where w and h denote the width and height of the current feature map, respectively.

[0106] In addition, as Figure 2 illustrated, the feature value can be calculated by using linear interpolation in the X direction.

[0107]

[0108] where p tl = (x i , y i ), p tr = (x i+1 , y i ), p bl = (x i , y i+1 ) and p br = (x i+1 , y i+1 ) are the integer coordinates of the top-left, top-right, bottom-left and bottom-right corners adjacent to the offset position

[0109] Similarly, the feature value can be represented as:

[0110]

[0111] Correspondingly, the linear interpolation along the Y direction can be represented as:

[0112]

[0113] where: is the feature value of the offset position

[0114] Subsequently, the spatial similarity can be represented as:

[0115]

[0116] where * e {tl, tr, bl, br}, p ＊ denotes the integer coordinates of the top-left, top-right, bottom-left and bottom-right, respectively, and σ s is the spatial smoothing parameter.

[0117] The feature similarity ​​is denoted as:

[0118]

[0119] where: σ f is a characteristic smoothing parameter.

[0120] In this case, the combined weight at the offset position can be denoted as:

[0121]

[0122] In this case, the characteristic value at the offset position can be updated as:

[0123]

[0124] where, is the feature map by considering the feature similarity and spatial similarity at the same time.

[0125] Finally, the entire deformable convolution module can be denoted as:

[0126]

[0127] where, is the output feature map, denotes the weight of the mth element in the convolution filter, w m denotes the modulation scalar of the mth offset position, denotes the convolution operation. Obviously, w m can be constrained within (0, 1) using the logical activation function σ(·).

[0128] Further: the construction process of the random grouping cross-channel attention module is as follows:

[0129] From the perspective of channels, suppressing the feature map related to the sea bottom background noise can significantly improve the accuracy of sea bottom organism detection. In this case, the random grouping cross-channel attention module is designed to adaptively recalibrate the channel weight. As Figure 3 shown, in order to capture specific semantic responses, a given X is divided into g groups, i.e. where c is the number of channels containing g groups, X n is the nth sub-feature.

[0130] Subsequently, the information entropy technique is used to count the information of each feature map. The feature probability statistics in a single feature map can be denoted as:

[0131]

[0132] Where i = 1, 2, ..., l, j = 1, 2, ..., c / g, n = 1, 2, ..., g, and These represent the quantity and probability of the i-th feature level in the j-th feature map within the n-th group.

[0133] Correspondingly, the information entropy of a single feature map can be expressed as:

[0134]

[0135] in: This represents the information entropy of the j-th feature map in the n-th group;

[0136] Furthermore, a local cross-channel attention method is proposed based on 1-D convolution techniques. It's important to note that directly determining the optimal convolution kernel size k is difficult. To enhance non-linear mapping capabilities, it can be determined as follows:

[0137]

[0138] Where k represents the filter size in the 1-D convolution, i.e., the coverage of cross-channel interaction, and γ and b are hyperparameters. It is clear that high-dimensional and low-dimensional channels correspond to larger and smaller filters, respectively, exhibiting a non-linear relationship. Furthermore, the channel weights can be expressed as:

[0139]

[0140] in, and These represent the weights of the convolution kernel and the recalibrated channel weights, respectively.

[0141] Furthermore, the recalibrated output in the nth feature map can be represented as:

[0142]

[0143] in, Let represent the j-th recalibrated feature map in the n-th group. In this case, the recalibrated sub-features are aggregated along the channel dimension through a concatenation operation, which can be represented as:

[0144] To enhance information exchange between different groups, a channel shuffling operation is adopted, which can be represented as:

[0145]

[0146] in: This represents the output of the reshaping operation R1. This represents the output of the transpose operation T. This represents the output of the reshaping operation R2.

[0147] To improve the accuracy of benthic organism detection from both spatial and channel perspectives, enhancing benthic organism characteristics and suppressing seabed background noise are crucial. For example... Figure 4 As shown, the SGCA-BDC module is formed by sequentially combining bilateral filtering interpolation variable convolution and random grouping cross-channel attention modules.

[0148] A benthic organism detection device based on a grouped cross-channel attention deformable convolutional neural network, comprising:

[0149] Acquisition module: used to acquire a dataset consisting of images of benthic organisms to be identified, and to divide the dataset into a training set and a test set according to a certain ratio;

[0150] Build modules: Used to build grouped cross-channel attention deformable convolutional neural network models for identifying benthic organisms in images;

[0151] Training module: Used to train the grouped cross-channel attention deformable convolutional neural network model based on the training set, and obtain the trained grouped cross-channel attention deformable convolutional neural network model;

[0152] Implementation module: Used to input test set data into a trained grouped cross-channel attention deformable convolutional neural network model to identify benthic organism species in images.

[0153] Detailed Implementation Plan

[0154] 3.1 Performance of Bilateral Filtering Deformable Convolutional Networks

[0155] To effectively demonstrate the effectiveness and superiority of the proposed BDC module, a comprehensive and in-depth comparison is conducted with DCNv1 and DCNv2. For ease of comparison, YOLOv3 is used as the benchmark detector, with a comparison resolution of 416×416. AP and mAP are used as quantification metrics, and the comparison results are shown in Table 1.

[0156] Table 1

[0157]

[0158] As can be clearly seen from the table, detection frameworks based on DCNv1 or DCNv2 achieve higher detection accuracy than the baseline framework because the use of positional offset techniques allows for the extraction of more representative semantic information. In contrast, the DCNv2-based framework achieves higher detection accuracy than the DCNv1-based detection method because DCNv2 fully considers the weighted modulation mechanism at the sampling location, thus significantly reducing the influence of irrelevant factors. Furthermore, it should be noted that the detection performance of the BDC framework is affected by the spatial smoothing parameter σ.s and feature smoothing parameter σ f Due to the combined effects of these factors, it is difficult to achieve optimal detection accuracy for all four categories simultaneously using the same set of parameters. Therefore, in pursuit of comprehensive detection performance, σ was ultimately adopted. s =0.8 and σ f =0.5. Clearly, the mAP@.5, mAP@.75, and mAP@[.5,.95] indices of the BDC framework are improved by 0.75%, 0.53%, and 0.47% respectively compared to the DCNv2 method. This indicates that the proposed BDC framework can achieve superior spatial sampling capabilities by employing bilateral filtering interpolation techniques. Furthermore, Figure 5 The sampling locations of the convolution kernels are visually illustrated. For ease of visualization, only the sampling centers are shown; the corresponding sampling locations are mapped from the 26×26 feature map to the original input image. From... Figure 5 As can be clearly seen, the sampling positions of traditional standard convolutions are tiled regularly in the image, which inevitably limits the sampling capability of the convolution kernel. Notably, by utilizing the proposed BDC module, the sampling positions of the convolution kernel tend to circumferentially surround benthic organisms. In this case, it is beneficial for coordinate regression and category prediction of benthic organisms.

[0159] 3.2 Information Entropy Statistical Performance

[0160] Regarding the statistical evaluation of information entropy, as shown in Table 2,

[0161] Table 2

[0162]

[0163] The proposed method is compared with GMP and GAP techniques. Table 2 clearly shows that regardless of whether GAP or GMP techniques are used to statistically analyze feature map information, the channel attention-based detection method can simultaneously improve the mAP@.5, mAP@.75, and mAP@[.5,.95] values. Most importantly, the information entropy technique generates more realistic feature descriptions, thus achieving superior detection performance based on the information entropy statistical method. Therefore, it can be concluded that the proposed information entropy statistical method has a positive promoting effect on the SGCA-BDC framework.

[0164] 3.3 Ablation Study

[0165] To comprehensively investigate the role of each component in the SGCA-BDC framework, corresponding ablation studies were conducted, including those without the DCN module (W / O DCN), without grouping operations, without information entropy statistics (W / O IES), without cross-channel attention, and without shuffle operations. Comprehensive ablation experiments were performed at 576×576 resolution, and the results are summarized in Table 3.

[0166] Table 3

[0167]

[0168] It is clear that eliminating the DCN module significantly impacts detection performance; spatially, this means the feature sampling capability of the convolutional kernel is greatly affected. It is noteworthy that, based on mAP@.5, mAP@.75, and mAP@[.5,.95] values, removing cross-channel attention leads to a decrease in detection performance because feature maps related to benthic organisms and the seabed background cannot be enhanced and suppressed separately in each subgroup. Furthermore, removing the shuffling operation results in reduced detection accuracy. This indicates that performing shuffling between different subgroups promotes information exchange. Additionally, not using grouping operations and information entropy statistics leads to a decrease in detection accuracy, suggesting that it is difficult to capture specific semantic responses and comprehensive channel statistics.

[0169] 3.4 Comparison of attention performance

[0170] To fully demonstrate the superiority of the proposed SGCA-BDC framework, a comprehensive comparison is conducted using three representative attention-based detection methods (SENet, CBAM, and SKNet). The comparison results are shown in Table 4.

[0171] Table 4

[0172]

[0173] It is clear that any attention mechanism can significantly improve detection accuracy, meaning that attention strategies can adaptively enhance benthic biomarkers and suppress seabed background noise. Furthermore, the detection performance of the SENet-based framework is significantly inferior to other similar methods because SENet only utilizes channel attention. Additionally, although the CBAM method uses both channel and spatial attention, its detection accuracy is still lower than the proposed SGCA-BDC framework. The main reasons can be summarized as follows: in terms of channel attention, using two dimensionally reduced fully connected layers severely disrupts the correspondence between weights and hidden layers; furthermore, using a single convolutional kernel makes it difficult to simultaneously generate regional weight responses at large, medium, and small scales. It is noteworthy that although selective kernel networks can effectively adjust the receptive field, the detection performance of the SKNet framework is still inferior to the proposed SGCA-BDC framework because SKNet uses a standard convolutional strategy, thus providing only very limited spatial modeling capabilities. Furthermore, due to the powerful feature extraction and channel-dependent calibration capabilities of the BDC and SGCA modules in terms of space and channels, the proposed SGCA-BDC framework achieves optimal detection accuracy in terms of mAP@.5, mAP@.75, and mAP@[.5,.95].

[0174] Accordingly, the precision-recall (PR) curve is as follows: Figure 6 As shown, particularly for starfish and sea cucumbers, the proposed SGCA-BDC framework achieves an optimal balance between precision and recall. Furthermore, in detecting sea cucumbers and scallops, although the SGCA-BDC framework has lower precision than other methods, it achieves higher recall, indicating that the SGCA-BDC method can detect more benthic organisms in the images.

[0175] 3.5 Comparison with state-of-the-art detection methods

[0176] To fully demonstrate the superiority of the proposed SGCA-BDC framework, we considered typical two-stage detection methods such as Faster RCNN, anchor-box-based single-stage detection methods such as SSD, YOLOv3, RetinaNet, and YOLOv4, as well as anchor-box-free single-stage detection framework CenterNet. The comparison results are shown in Table 5.

[0177] Table 5

[0178]

[0179] It is evident that at a resolution of 608×608, the proposed SGCA-BDC framework achieves the best mAP@.5, mAP@.75, and mAP@[.5,.95] values. It is worth noting that by combining spatial pyramid pooling and path aggregation networks, the YOLOv4 framework, leveraging the powerful feature extraction network CSPDarkNet53, achieves the best sea cucumber detection accuracy. While CenterNet achieves the most competitive detection speed, due to the lack of prior scale information about benthic organisms (i.e., without using anchor boxes), the CenterNet method struggles to provide detection accuracy consistent with the SGCA-BDC framework. Furthermore, without attention mechanisms, unwanted noise is inevitably introduced during feature extraction, making it difficult for YOLOv3 and SSD frameworks to achieve detection accuracy consistent with the SGCA-BDC framework. It is noteworthy that the MobileNet-based SSD framework achieves the most competitive detection speed of 49.04 FPS at a resolution of 300×300. Furthermore, Faster R-CNN's mAP@.5, mAP@.75, and mAP@[.5,.95] metrics are 21.52%, 4.95%, and 6.8% lower than the SGCA-BDC framework, respectively. This is primarily because Faster R-CNN only uses the final convolutional layer to detect benthic organisms. Additionally, due to the large number of class-independent candidate regions generated by the RPN, the Faster R-CNN framework can only achieve a detection speed of 5.95 FPS.

[0180] 3.6 Real-world environment verification

[0181] To fully verify the superiority of the SGCA-BDC framework in real-world environments, four typical underwater scenarios were considered: 1) motion blur (first column), 2) sea urchin occlusion (second column), 3) sea cucumber occlusion (third column), and 4) benthic organisms of different scales (fourth column). The corresponding detection results are as follows: Figure 7As shown, the proposed SGCA-BDC framework outperforms comparable methods in terms of motion blur. Furthermore, while the SGCA-BDC framework cannot correctly detect and identify all benthic organisms of interest, it effectively detects intra-class occluded sea urchins and sea cucumbers (i.e., columns 2 and 3) thanks to its robust modeling ability for geometrically deformed benthic organisms. Faster R-CNN and CenterNet methods, utilizing RPN and DCN techniques to pre-generate candidate bounding boxes and enhance convolutional kernel sampling locations, achieve only suboptimal detection performance compared to the proposed SGCA-BDC framework. By jointly using the BDC and SGCA modules, the proposed SGCA-BDC framework effectively preserves small-scale benthic semantic features, thus significantly improving detection capability (i.e., column 4).

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A detection method based on a grouped cross-channel attention deformable convolutional network, characterized in that: Includes the following steps: Obtain a dataset consisting of images of benthic organisms to be identified, and divide the dataset into a training set and a test set according to a certain ratio; Construct a grouped cross-channel attention deformable convolutional neural network model for identifying benthic organisms in images; The grouped cross-channel attention deformable convolutional neural network model is trained based on the training set to obtain the trained grouped cross-channel attention deformable convolutional neural network model. The grouped cross-channel attention deformable convolutional neural network model includes multiple cascaded grouped cross-channel attention deformable convolutional neural network units; The grouped cross-channel attention deformable convolutional neural network unit includes a first convolutional layer that increases nonlinear mapping capability, a second convolutional layer that performs feature extraction, a bilateral filtering interpolation variable convolutional neural network module that increases the ability to construct deformable benthic organism features, and a random grouped cross-channel attention module that suppresses seabed background noise. The first convolutional layer, the second convolutional layer, the bilateral filtering interpolation variable convolutional neural network module, and the random grouping cross-channel attention module are cascaded in sequence; The test set data is input into a trained grouped cross-channel attention deformable convolutional neural network model to identify benthic organism species in the image.

2. The detection method based on a grouped cross-channel attention deformable convolutional network according to claim 1, characterized in that: The bilateral filter interpolation variable convolutional neural network module is constructed as follows: The standard sampling position of a 3×3 convolution kernel is represented as follows: P={(u,v)|u,v∈{-1,0,1}}(1) Where: P is the set of sampling positions (u,v); The deformation offset position of each element within the convolution kernel is described as follows: Where: m = 1, 2, ..., n, n = |P|, p m ∈P, p0 is the center position of the convolution kernel, Δp m This represents the offset of the m-th element. This represents the deformation sampling position of the m-th element of the convolution kernel. Additionally, the offset coordinates... The following conditions must be met: Where w and h represent the width and height of the current feature map, respectively; The eigenvalues ​​are calculated by using linear interpolation in the X direction. It is represented as: Where p tl =(x i ,y i ), p tr =(x i+1 ,y i ), p bl =(x i ,y i+1 ) and p br =(x i+1 ,y i+1 ) is relative to the offset coordinates Integer coordinates of adjacent top-left, top-right, bottom-left, and bottom-right corners; Eigenvalues Represented as: Linear interpolation along the Y direction is expressed as: in Offset coordinates eigenvalues; Subsequently, spatial similarity Represented as: Where, *∈{tl,tr,bl,br}, p ＊ Let σ represent the integer coordinates of the top left, top right, bottom left, and bottom right, respectively. s It is a spatial smoothing parameter; Feature similarity The representation is: Where σf is the feature smoothing parameter; In this case, the combined weight at the offset position is expressed as: In this case, offset coordinates The eigenvalue at point is updated as follows: in, It is a feature map that simultaneously considers feature similarity and spatial similarity; Finally, the entire deformable convolutional module is represented as: in, It is the output feature map. w represents the weight of the m-th element in the convolutional filter. m This represents the modulation scalar at the m-th offset position. To represent a convolution operation, obviously, the logistic activation function σ(·) is used to encode w. m Constrained to (0,1).

3. The detection method based on a grouped cross-channel attention deformable convolutional network according to claim 1, characterized in that: The construction process of the random grouping cross-channel attention module is as follows: To capture semantic responses, a given X is divided into g groups, i.e. Where c is the number of channels containing group g, X n It is the nth sub-feature; Subsequently, information entropy techniques are used to statistically analyze the information of each feature map. The feature probability statistics for a single feature map are represented as follows: Where i = 1, 2, ..., l, j = 1, 2, ..., c / g, n = 1, 2, ..., g, and These are the quantity and probability of the i-th feature level in the j-th feature map within the n-th group; Accordingly, the information entropy of a single feature map is expressed as: in: This represents the information entropy of the j-th feature map in the n-th group; Furthermore, based on 1-D convolution technology, a randomized grouped cross-channel attention module is proposed, and the optimal convolution kernel size k is determined as follows: Where k represents the filter size in the 1-D convolution, i.e., the coverage range of cross-channel interaction; γ and b are hyperparameters, with high-dimensional and low-dimensional channels corresponding to larger and smaller filters, respectively, exhibiting a non-linear relationship; furthermore, the channel weights are expressed as: in, and These represent the weights of the convolution kernel and the recalibrated channel weights, respectively. Furthermore, the recalibrated output in the nth feature map is represented as: in, Let represent the j-th recalibrated feature map in the n-th group; through a concatenation operation, the recalibrated sub-features are aggregated along the channel dimension, which is represented as: To enhance information exchange between different groups, a channel shuffling operation is adopted, which is represented as follows: in: This represents the output of the reshaping operation R1. This represents the output of the transpose operation T. This represents the output of the reshaping operation R2.

4. A detection device based on a grouped cross-channel attention deformable convolutional network, characterized in that: include: Acquisition module: used to acquire a dataset consisting of images of benthic organisms to be identified, and to divide the dataset into a training set and a test set according to a certain ratio; The building module is used to construct a grouped cross-channel attention deformable convolutional neural network model for identifying benthic organisms in images; the grouped cross-channel attention deformable convolutional neural network model includes multiple cascaded grouped cross-channel attention deformable convolutional neural network units; The grouped cross-channel attention deformable convolutional neural network unit includes a first convolutional layer that increases nonlinear mapping capability, a second convolutional layer that performs feature extraction, a bilateral filtering interpolation variable convolutional neural network module that increases the ability to construct deformable benthic organism features, and a random grouped cross-channel attention module that suppresses seabed background noise. The first convolutional layer, the second convolutional layer, the bilateral filtering interpolation variable convolutional neural network module, and the random grouping cross-channel attention module are cascaded in sequence; Training module: Used to train the grouped cross-channel attention deformable convolutional neural network model based on the training set, and obtain the trained grouped cross-channel attention deformable convolutional neural network model; Implementation module: Used to input test set data into a trained grouped cross-channel attention deformable convolutional neural network model to identify benthic organism species in images.

Images