Deep learning-based fish detection model construction method and fish detection method and system

Abstract

The invention discloses a construction method of a fish detection model based on deep learning and a fish detection method and system, a residual network ResNet is used as a main network structure of the fish detection model, and the residual network ResNet has a strong feature extraction capability and can capture features such as basic forms and textures of fishes; then a mixed attention mechanism is introduced to extract more effective feature quantities, a mixed attention module is connected behind each residual block of the residual network ResNet, key features can be further extracted and emphasized, and therefore a better effect is achieved on the residual network ResNet; the mixed attention is combined with an attention mechanism of two dimensions of channel attention of GRA and space attention based on CBP, the model can more comprehensively capture feature information of fish, algorithm complexity is effectively reduced, hardware requirements of actual deployment are further reduced, the requirements for depth and different level widths of a deep learning algorithm are reduced, and the method is suitable for being popularized and applied. And the purpose of algorithm weight reduction is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for constructing a fish detection model based on deep learning and a fish detection method and system. Background Art

[0002] Fish detection and classification technology based on deep learning is a research field that uses deep learning algorithms to automatically detect and identify fish. This technology is of great significance for fishery scientific research, marine / freshwater ecological monitoring, and the sustainable development of aquaculture.

[0003] Refer to the fish image recognition method, system and device with patent application publication number CN202110955820.6. This solution directly inserts the CBAM (Convolutional Block Attention Module) hybrid attention architecture into the residual block of the resnet50 deep network to enhance the algorithm effect, but does not perform targeted optimization of the CBAM algorithm, and ultimately only obtains an 80% recognition accuracy rate.

[0004] See the ESB-YOLO model farmed fish detection method based on channel non-dimensionality reduction attention mechanism and improved YOLOv5 with patent application publication number CN202211245772.2. This solution introduces the CBAM architecture into the YOLOv5 algorithm to improve the YOLOv5 backbone network's ability to extract fish features. The YOLOv5 algorithm itself is relatively complex, and the improved design based on YOLOv5 is not very friendly to actual deployment.

[0005] See A method for identifying economic fish by integrating CBAM attention mechanism and pyramid convolution [J]. Journal of Jiangsu Ocean University: Natural Science Edition, 2023, 32(1): 73-80. This scheme uses the ResNet50 deep network as the basic network framework, introduces CBAM and replaces the ordinary convolution in the basic network with pyramid convolution (PyConv) to improve network performance. Compared with the above scheme [1], the additional introduction of PyConv increases the complexity of the algorithm.

[0006] The above-mentioned strategies do not make a targeted design for the CBAM structure when using it, but directly apply it based on the existing deep learning algorithm, which leads to limited improvement in algorithm performance and fails to effectively reduce the algorithm complexity, resulting in high hardware requirements and difficulty in actual deployment. At the same time, they focus on the algorithm and ignore the development of supporting systems, or only develop simple result display interfaces, which have limited actual application value. Summary of the invention

[0007] To this end, it is necessary to provide a fish detection and classification system based on deep learning to solve the problem that the existing algorithms are highly complex, resulting in high hardware requirements and difficulty in actual deployment.

[0008] To achieve the above object, the present invention provides a method for constructing a fish detection model based on deep learning, which comprises the following steps:

[0009] Obtain a sample set of fish images;

[0010] Segment and label the fish images in the fish image sample set, and use the results as labels for training data;

[0011] Establishing a deep learning network, the deep learning network includes a residual network ResNet and a hybrid attention module connected to each residual block in the residual network ResNet; the hybrid attention submodule includes a channel attention submodule based on GRA and a spatial attention submodule based on CBP in sequence;

[0012] The fish images in the fish image sample set are used as training data to train the deep learning network. The output of the deep learning network is calculated through forward propagation. Then the loss function is used to calculate the difference between the output and the label of the training data. The deep learning network parameters are updated through the back propagation algorithm to obtain the fish detection model.

[0013] Furthermore, the residual network ResNet is ResNet-18.

[0014] Furthermore, before the fish image sample set is segmented and labeled and used as training data, data enhancement is also performed on the fish image sample set.

[0015] Furthermore, the GRA-based channel attention submodule performs the following steps:

[0016] Perform channel dimension reduction on the corresponding residual block output feature F;

[0017] The GRA algorithm is used to analyze the mutual correlation between the channels of the feature F after dimensionality reduction, and a correlation matrix of size c′×c′ is obtained;

[0018] The correlation matrix is ​​convolved with a multilayer perceptron with only one hidden layer to obtain a weight vector with a dimension of 1×1×c;

[0019] The weight vector is multiplied channel by channel into the input feature F to obtain the feature F′ with channel attention weights.

[0020] Furthermore, the step of using the GRA algorithm to analyze the mutual correlation between the channels of the feature F after dimensionality reduction to obtain a correlation matrix of size c′×c′ specifically includes the following:

[0021] Flatten each feature channel to obtain a one-dimensional feature [X1, X2, ..., X c ], take each channel as the reference sequence in turn and record it as X0, then calculate the grey relational degree:

[0022]

[0023] Among them, r m Represents the mth comparison sequence X m Grey correlation degree with the observed sequence X0; ξ m (i) represents the i-th observation value x of the m-th comparison sequence m The relative difference between (i) and the corresponding reference sequence value x0(i), also known as X m The correlation coefficient of X0 at time i; n represents the one-dimensional characteristic length obtained by flattening a channel; a∈[0,1] represents the resolution coefficient; min min|x0(i)-x m (i)| is the minimum difference between the two levels, max max|x0(i)-x m (i) | is the maximum difference between the two levels;

[0024] A correlation matrix of size c′×c′ is obtained, in which the i-th row and the j-th column of the matrix represent the correlation between the j-th channel and the i-th channel.

[0025] Furthermore, the CBP-based spatial attention submodule performs the following steps:

[0026] The feature F′ with channel attention weight is subjected to maximum pooling and average pooling along the channel direction respectively;

[0027] Use CBP to fuse the features obtained by maximum pooling and average pooling to obtain fused features;

[0028] Perform a 3×3 convolution operation on the fused features to obtain a spatial weight matrix with a channel dimension of 1;

[0029] The spatial weight matrix is ​​weighted element by element by multiplying the input feature F' with channel attention weights to obtain the feature F" with spatial attention weights.

[0030] Furthermore, the step of using CBP to fuse the features obtained by maximum pooling and average pooling specifically includes the following:

[0031] The features obtained by maximum pooling and average pooling are expanded along the channel respectively to obtain two groups of 4 one-dimensional feature vectors with a length of n, denoted as x 11 , x 12 , x 21 , x 22 ;

[0032] For the above four one-dimensional eigenvectors, we define the sketch function ψ(x,h,s)={(Q x )1,...,(Q x ) n}, where (Q x ) j =Σ t:h(t)＝j s(t)x t , h(t) is a random index uniformly drawn from {1, 2, ..., d}, and s(t) is a randomly drawn +1 or -1 symbol; when h(t) = j, the feature x t is selected and multiplied by its corresponding sign s(t), and the products are summed to obtain (Q x ) j ;

[0033] Get the tensor sketch projection of each one-dimensional eigenvector based on the sketch function in represents element-by-element multiplication, and FFT represents fast Fourier transform;

[0034] Compute feature fusion using tensor sketch projection: CBP(x 11 ,x 21 )=vec(φ TS11 ,φ TS21 ),CBP(x 12 ,x 22 )=vec(φ TS12 ,φ TS22 ), where vec() represents the vectorized operation of the matrix.

[0035] A fish detection method based on deep learning, a fish detection model constructed by applying the method for constructing a fish detection model based on deep learning, comprising the following steps:

[0036] Read the image of fish to be detected;

[0037] The fish image to be detected is input into the fish detection model to obtain the fish classification result.

[0038] A fish detection system, comprising:

[0039] The detection unit reads the fish image to be detected, uses the fish detection model provided by the algorithm setting unit to detect the fish image to be detected, and obtains the fish classification result;

[0040] An algorithm setting unit is used to load a required fish detection model, wherein the required fish detection model includes a fish detection model constructed by the method for constructing a fish detection model based on deep learning;

[0041] A display unit displays the fish classification results.

[0042] Furthermore, it also includes a data storage unit, which stores and displays the fish classification results to a specified path.

[0043] Different from the prior art, the above technical solution uses the residual network ResNet as the main network structure of the fish detection model. The residual network ResNet has a powerful feature extraction capability and can capture the basic morphology, texture and other features of fish; then introduces a hybrid attention mechanism to extract more effective feature quantities. A hybrid attention module is connected after each residual block of the residual network ResNet, which can further extract and emphasize key features, thereby achieving better results on the residual network ResNet; the hybrid attention combines the two-dimensional attention mechanisms of the channel attention of GRA and the spatial attention based on CBP. The model can capture the characteristic information of fish more comprehensively, improve the accuracy and robustness of the features, and can also learn key features faster, thereby shortening the training time; effectively reducing the complexity of the algorithm, and then reducing the hardware requirements for actual deployment, so as to reduce the requirements for the depth of the deep learning algorithm and the width of different levels, and achieve the purpose of lightweight algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the structure of a fish detection model in an embodiment;

[0045] Figure 2 A schematic diagram of the architecture of a hybrid attention module in an embodiment;

[0046] Figure 3 Schematic diagram of the execution flow of the GRA-based channel attention submodule of the embodiment;

[0047] Figure 4 Schematic diagram of the execution flow of the spatial attention submodule based on CBP in the embodiment;

[0048] Figure 5 Schematic diagram of the interactive interface of the fish detection system of the embodiment. DETAILED DESCRIPTION

[0049] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0050] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0051] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0052] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0053] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0054] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0055] Similar to the understanding in the Patent Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0056] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0057] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0058] See also Figure 1-Figure 4 As shown, the present invention provides a method for constructing a fish detection model based on deep learning, using the residual network ResNet as the main network structure of the fish detection model, and the residual network ResNet has a strong feature extraction capability, which can capture the basic morphology, texture and other features of fish; then a hybrid attention mechanism is introduced to extract more effective feature quantities, and a hybrid attention module is connected after each residual block of the residual network ResNet, which can further extract and emphasize key features, thereby achieving better effects on the residual network ResNet; the channel attention submodule based on GRA (Grey Relational Analysis) can focus on the channel features that contribute most to classification, and the channel attention submodule based on CBP (Compact Bilinear The spatial attention submodule of deep learning (CNN, CNN, and compact bilinear pooling) can focus on the key areas of fish in the image. By combining the attention mechanisms in both channel and spatial dimensions, the model can capture the characteristic information of fish more comprehensively, improve the accuracy and robustness of the features, and learn key features faster, thereby shortening the training time. The hybrid attention module is introduced to effectively reduce the complexity of the algorithm, thereby reducing the hardware requirements for actual deployment, thereby reducing the requirements for the depth of the deep learning algorithm and the width of different levels, and achieving the purpose of lightweight algorithm.

[0059] See also Figure 1 and Figure 2 As shown, the following provides an embodiment of a method for constructing a fish detection model based on deep learning of the present invention, which comprises the following steps:

[0060] Obtain a sample set of fish images;

[0061] Segment and label the fish images in the fish image sample set, and use the results as labels for training data;

[0062] Establishing a deep learning network, the deep learning network includes a residual network ResNet and a hybrid attention module connected to each residual block in the residual network ResNet; the hybrid attention submodule includes a channel attention submodule based on GRA and a spatial attention submodule based on CBP in sequence;

[0063] The fish images in the fish image sample set are used as training data to train the residual network ResNet. The output of the residual network ResNet is calculated through forward propagation. Then the loss function is used to calculate the difference between the output and the label of the training data. The parameters of the residual network ResNet are updated through the back propagation algorithm to obtain the fish detection model.

[0064] The above-mentioned fish image sample set is used to train a deep learning network, which may include fish images under different time periods, different lighting conditions, and different fish density conditions, etc., which can cover a wider range of visual features, so that the deep learning network can learn fish information under different conditions during the training process, so that the generated fish detection model can more accurately identify and classify fish under different conditions in scene applications, even in the case of changes in lighting conditions, color distortion, etc., it can still accurately identify and classify. In some embodiments, in order to improve the accuracy of fish detection by the fish detection model, the fish images in the fish image sample set are segmented and labeled and used as training data before the fish images in the fish image sample set are pre-processed, such as denoising, data enhancement, etc. For example, data enhancement, such as rotating, scaling, flipping, etc., the fish images in the fish image sample are operated to improve the generalization ability and adaptability of the fish detection model. When the deep learning network is training, the size of the pictures in the picture sample set can be adjusted according to the size of the deep learning network input. Generally, the size of the fish images in the fish image sample set can be adjusted by scaling, cropping, etc. In order to ensure the important feature area of ​​the image, it is preferred to adjust the size of the fish images in the fish image sample set by scaling. The scaling operation can be achieved by methods such as bilinear interpolation and nearest neighbor interpolation.

[0065] The fish images in the fish image sample set are segmented and labeled, and the results are used as labels for training data. Specifically, the fish images in the fish image sample set can be segmented into fish parts and non-fish parts, wherein the fish parts can be labeled with information including the type, size, growth environment, and living location of the fish. The segmentation can be achieved by manual segmentation (such as Photoshop, GIMP, etc.), semi-automatic segmentation (such as ITK-SNAP, 3D Slicer, etc.), or fully automatic segmentation (such as using pre-trained deep learning algorithms (such as Mask R-CNN, U-Net, etc.) to automatically segment fish images); the annotation tools (such as LabelImg, VIA, etc.) annotate fish images and generate files (such as XML, JSON, etc.) containing annotation information to ensure the accuracy and consistency of the annotations, so that they can be used for subsequent training and evaluation of deep learning networks.

[0066] The above-mentioned deep learning network includes a residual network ResNet and a hybrid attention module connected to each residual block in the residual network ResNet; the fish image is trained by focusing on fish image samples so that the generated fish detection model can obtain fish classification results according to the fish image to be detected. The above-mentioned residual network ResNet can be ResNet-18, ResNet-50, ResNet-101, ResNet-152, etc. In order to achieve the purpose of lightweight algorithm of fish detection model, the residual network ResNet is preferably ResNet-18. Compared with other deeper ResNet variants, ResNet-18 has a lighter network structure and can process image data faster, thereby achieving faster response speed and higher throughput in practical applications, requiring less computing resources and memory usage during training and reasoning, being more hardware-friendly, and having more practical promotion and application value; at the same time, due to the small number of layers of ResNet-18, the number of its parameters is also reduced accordingly, which helps to reduce the complexity of the model and the risk of overfitting.

[0067] See also Figure 3 As shown, the above GRA-based channel attention submodule execution includes the following steps:

[0068] Perform channel dimension reduction on the corresponding residual block output feature F;

[0069] The GRA algorithm is used to analyze the mutual correlation between the channels of the feature F after dimensionality reduction, and a correlation matrix of size c′×c′ is obtained;

[0070] The correlation matrix is ​​convolved with a multilayer perceptron with only one hidden layer to obtain a weight vector with a dimension of 1×1×c;

[0071] The weight vector is multiplied channel by channel into the input feature F to obtain the feature F′ with channel attention weights.

[0072] The above-mentioned channel dimensionality reduction of the corresponding residual block output feature F can be performed by using 1x1 convolution to reduce the channel dimensionality from c to c′ to reduce the amount of calculation (in the present invention, c′=c / 4, and in fact the ratio can be adjusted to other integer multiples of 2 according to specific needs), and then GRA is used to analyze the correlation between the feature channels.

[0073] The step of using the GRA algorithm to analyze the mutual correlation between the channels of the feature F after dimensionality reduction to obtain a correlation matrix of size c′×c′ specifically includes the following:

[0074] Flatten each feature channel to obtain a one-dimensional feature [X1, X2, ..., X c ], take each channel as the reference sequence in turn and record it as X0, then calculate the grey relational degree:

[0075]

[0076] Among them, r m Represents the mth comparison sequence X m Grey correlation degree with the observed sequence X0; ξ m (i) represents the i-th observation value x of the m-th comparison sequence m The relative difference between (i) and the corresponding reference sequence value x0(i), also known as X m The correlation coefficient of X0 at time i; n represents the one-dimensional characteristic length obtained by flattening a channel; a∈[0,1] represents the resolution coefficient, which can be taken as 0.5; min min|x0(i)-x m (i)| is the minimum difference between the two levels, max max|x0(i)-x m (i) | is the maximum difference between the two levels;

[0077] A correlation matrix of size c′×c′ is obtained, in which the i-th row and the j-th column of the matrix represent the correlation between the j-th channel and the i-th channel.

[0078] This value is usually a number between 0 and 1. A larger value indicates a stronger correlation between the two channels, and a smaller value indicates a weaker correlation.

[0079] See also Figure 4 As shown, the above-mentioned CBP-based spatial attention submodule execution includes the following steps:

[0080] The feature F′ with channel attention weight is subjected to maximum pooling and average pooling along the channel direction respectively;

[0081] Use CBP to fuse the features obtained by maximum pooling and average pooling to obtain fused features;

[0082] Perform a 3×3 convolution operation on the fused features to obtain a spatial weight matrix with a channel dimension of 1;

[0083] The spatial weight matrix is ​​weighted element by element by multiplying the input feature F' with channel attention weights to obtain the feature F" with spatial attention weights.

[0084] The feature F′ with channel attention weight is subjected to maximum pooling and average pooling along the channel direction respectively. In order to capture more feature information from different angles, the group pooling method is adopted to divide the feature F′ with channel attention weight into two groups for pooling. Therefore, the channel dimension c′ of the obtained pooled feature is actually equal to 2.

[0085] Among them, the steps of using CBP to fuse the features obtained by maximum pooling and average pooling include the following:

[0086] The features obtained by maximum pooling and average pooling are expanded along the channel respectively to obtain two groups of 4 one-dimensional feature vectors with a length of n, denoted as x 11 , x 12 , x 21 , x 22 ;

[0087] For the above four one-dimensional eigenvectors, we define the sketch function ψ(x,h,s)={(Q x )1,...,(Q x ) n}, where (Q x ) j =Σ t:h(t)＝j s(t)x t , h(t) is a random index uniformly drawn from {1, 2, ..., d}, and s(t) is a randomly drawn +1 or -1 symbol; when h(t) = j, the feature x t is selected and multiplied by its corresponding sign s(t), and the products are summed to obtain (Q x ) j ;

[0088] Get the tensor sketch projection of each one-dimensional eigenvector based on the sketch function Where ° represents element-by-element multiplication, and FFT represents fast Fourier transform;

[0089] Compute feature fusion using tensor sketch projection: CBP(x 11 ,x 21 )=vec(φ TS11 ,φ TS21 ),CBP(x12 ,x 22 )=vec(φ TS12 ,φ TS22 ), where vec() represents the vectorized operation of the matrix.

[0090] The present invention is trained and tested based on the Fish4Knowledge public data set, and the comparative test results are as follows:

[0091]

[0092] It can be seen that the fish detection model of the present invention can effectively improve the network feature extraction capability, so that the shallow network also shows a good classification and detection effect, achieving the purpose of lightweight fish detection and classification algorithms. In addition, the network complexity is much lower than these two network types, which is more hardware-friendly and has more practical promotion and application value.

[0093] The present invention also provides a fish detection method based on deep learning, and a fish detection model constructed by applying the above-mentioned method for constructing a fish detection model based on deep learning comprises the following steps:

[0094] Read the image of fish to be detected;

[0095] The fish image to be detected is input into the fish detection model to obtain the fish classification result.

[0096] Before the above-mentioned fish image to be detected is input into the fish detection model, the fish image to be detected can also be pre-processed. At this time, pre-processing refers to adjusting the size and format of the fish image to be detected according to the size and format input by the fish detection model. The size of the fish image to be detected can usually be adjusted by scaling, cropping and other operations to adjust the size of the picture in the picture sample set. In order to ensure the important feature area of ​​the fish image to be detected, it is preferred to adjust the size of the fish image to be detected by scaling. The scaling operation can be achieved by methods such as bilinear interpolation and nearest neighbor interpolation. Of course, before adjusting the size and format of the fish image to be detected, the fish image to be detected is pre-processed by methods such as denoising.

[0097] See also Figure 5 As shown, the present invention also provides a fish detection system, which includes:

[0098] The detection unit reads the fish image to be detected, uses the fish detection model provided by the algorithm setting unit to detect the fish image to be detected, and obtains the fish classification result;

[0099] An algorithm setting unit is used to load a required fish detection model, wherein the required fish detection model includes a fish detection model constructed by the above-mentioned method for constructing a fish detection model based on deep learning;

[0100] A display unit displays the fish classification results.

[0101] The detection unit can provide an automatic detection mode and a manual detection mode. The automatic detection mode reads the images of fish to be detected in batches from a specified path at a set frequency to complete fish classification, such as reading the latest fish data from a specified path at a frequency of 1 image / s to complete fish classification; the manual detection mode reads the specified detection fish images from a specified path to complete fish classification. The two modes support different application scenarios and improve the applicability of the system. The algorithm setting unit supports the loading of different algorithm models, which improves the flexibility of the system. The display unit supports the display of fish images or fish classification results, which improves the intuitiveness of the system.

[0102] The fish detection system also includes a data storage unit, which stores and displays the fish classification results to a specified path. The data storage unit can provide an automatic saving mode and a manual saving mode. The automatic saving mode automatically records the fish classification results to a specified path by setting an automatic recording cycle; the manual saving mode manually saves the fish classification results to a specified path. The two modes improve the usability of the system.

[0103] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. A method for constructing a fish detection model based on deep learning, characterized in that: The following steps are involved: Obtain a sample set of fish images; Segment and label the fish images in the fish image sample set, and use the results as labels for training data; Establishing a deep learning network, the deep learning network includes a residual network ResNet and a hybrid attention module connected to each residual block in the residual network ResNet; the hybrid attention submodule includes a channel attention submodule based on GRA and a spatial attention submodule based on CBP in sequence; The fish images in the fish image sample set are used as training data to train the deep learning network. The output of the deep learning network is calculated through forward propagation. Then the loss function is used to calculate the difference between the output and the label of the training data. The deep learning network parameters are updated through the back propagation algorithm to obtain the fish detection model.

2. The method for constructing a fish detection model based on deep learning according to claim 1, characterized in that: The residual network ResNet is ResNet-18.

3. The method for constructing a fish detection model based on deep learning according to claim 1, characterized in that: Before the fish image sample set is segmented and labeled and used as training data, the method also includes data enhancement for the fish image sample set.

4. The method for constructing a fish detection model based on deep learning according to claim 1, characterized in that: The GRA-based channel attention submodule execution includes the following steps: Perform channel dimension reduction on the corresponding residual block output feature F; The GRA algorithm is used to analyze the mutual correlation between the channels of the feature F after dimensionality reduction, and a correlation matrix of size c′×c′ is obtained; The correlation matrix is ​​convolved with a multilayer perceptron with only one hidden layer to obtain a weight vector with a dimension of 1×1×c; The weight vector is multiplied channel by channel into the input feature F to obtain the feature F′ with channel attention weights.

5. The method for constructing a fish detection model based on deep learning according to claim 4, characterized in that: The step of using the GRA algorithm to analyze the mutual correlation between the channels of the feature F after dimensionality reduction to obtain a correlation matrix of size c′×c′ specifically includes the following: Flatten each feature channel to obtain a one-dimensional feature [X1, x2, ..., X c ], take each channel as the reference sequence in turn and record it as X0, then calculate the grey relational degree: Among them, r m Represents the mth comparison sequence X m Grey correlation degree with observation sequence X0; ξ m (i) represents the i-th observation value x of the m-th comparison sequence m The relative difference between (i) and the corresponding reference sequence value x0(i), also known as X m The correlation coefficient of X0 at time i; n represents the one-dimensional characteristic length obtained by flattening a channel; a∈[0,1] represents the resolution coefficient; minmin|x0(i)-x m (i)| is the minimum difference between the two levels, maxmax|x0(i)-x m (i) | is the maximum difference between the two levels; A correlation matrix of size c′×c′ is obtained, in which the i-th row and the j-th column of the matrix represent the correlation between the j-th channel and the i-th channel.

6. The method for constructing a fish detection model based on deep learning according to claim 1, characterized in that: The CBP-based spatial attention submodule execution includes the following steps: The feature F′ with channel attention weight is subjected to maximum pooling and average pooling along the channel direction respectively; Use CBP to fuse the features obtained by maximum pooling and average pooling to obtain fused features; Perform a 3×3 convolution operation on the fused features to obtain a spatial weight matrix with a channel dimension of 1; The spatial weight matrix is ​​weighted element by element by multiplying the input feature F' with channel attention weights to obtain the feature F" with spatial attention weights.

7. The method for constructing a fish detection model based on deep learning according to claim 6, characterized in that: The step of using CBP to fuse the features obtained by maximum pooling and average pooling specifically includes the following: The features obtained by maximum pooling and average pooling are expanded along the channel respectively to obtain two groups of 4 one-dimensional feature vectors with a length of n, denoted as x 11 , x 12 , x 21 , x 22 ; For the above four one-dimensional eigenvectors, we define the sketch function ψ(x,h,s)={(Q x )1,...,(Q x ) n }, where (Q x ) j =Σ t:h(t)＝j s(t)x t , h(t) is a random index uniformly drawn from {1, 2, ..., d}, and s(t) is a randomly drawn +1 or -1 symbol; when h(t) = j, the feature x t is selected and multiplied by its corresponding sign s(t), and the products are summed to obtain (Q x ) j ; Get the tensor sketch projection of each one-dimensional eigenvector based on the sketch function in represents element-by-element multiplication, and FFT represents fast Fourier transform; Compute feature fusion using tensor sketch projection: CBP(x 11 ,x 21 )=vec(φ TS11 ,φ TS21 ),CBP(x 12 ,x 22 )=vec(φ TS12 ,φ TS22 ), where vec() represents the vectorized operation of the matrix.

8. A fish detection method based on deep learning, a fish detection model constructed by using the method for constructing a fish detection model based on deep learning as described in any one of claims 1 to 7, characterized in that: The following steps are involved: Read the image of fish to be detected; The fish image to be detected is input into the fish detection model to obtain the fish classification result.

9. A fish detection system, characterized in that: include: The detection unit reads the fish image to be detected, uses the fish detection model provided by the algorithm setting unit to detect the fish image to be detected, and obtains the fish classification result; An algorithm setting unit is used to load a required fish detection model, wherein the required fish detection model includes a fish detection model constructed by the method for constructing a fish detection model based on deep learning as described in any one of claims 1 to 7; A display unit displays fish classification results.

10. The fish detection system according to claim 9, characterized in that: It also includes a data storage unit, which stores and displays the fish classification results to a specified path.

Images