Fish identification method

Abstract

The invention discloses a fish identification method, which is applied to the technical field of image identification, and comprises the following steps: carrying out data preprocessing on fish identification sample data, and carrying out data division on the preprocessed data to obtain a training set and a test set; based on the training set, training a fish identification model based on the improved YOLOV5; wherein the improvement of the YOLOV5 comprises the following steps: introducing a Swi n Transform as a backbone network into the YOLOV5, replacing linear interpolation up-sampling by using a CARAFE up-sampling algorithm, replacing common splicing operation by using BIFPN weighted feature fusion, and replacing a neck network in the YOLOV5 by using a new neck network; and calculating the loss of the improved YOLOV5 by using a Rep-GI oU Loss loss function, and inputting a test set to the trained and converged fish identification model to obtain a fish identification result. According to the invention, the fish identification precision is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a fish recognition method. Background Art

[0002] At present, fish recognition models based on machine vision are widely used in the modern fishery industry. The fish recognition model based on machine vision mainly includes the following stages: image acquisition, image preprocessing, image feature extraction, and image recognition and classification. In the above fish recognition process, the fish recognition model is affected by the contrast and illumination in the water, and the model is limited by the feature extraction capability, resulting in the need to further improve the accuracy of the current fish recognition model.

[0003] Therefore, how to provide a fish identification method that can effectively improve the influence of water contrast and illumination in the process of fish identification based on machine vision, and improve the accuracy of fish identification by effectively improving the feature extraction capability of the fish identification model is an urgent problem that technicians in this field need to solve. Summary of the invention

[0004] In view of this, the present invention proposes a fish identification method.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] A fish identification method, comprising:

[0007] Step 1: Preprocess the fish identification sample data and divide the preprocessed data into training sets and test sets;

[0008] Step 2: Based on the training set, train a fish recognition model based on the improved YOLOV5; the improvements of YOLOV5 include: using Swin Transformer as the backbone network introduced into YOLOV5, using the CARAFE upsampling algorithm to replace the linear interpolation upsampling in YOLOV5, using the BIFPN weighted feature fusion to replace the ordinary splicing operation in YOLOV5, and using a new neck network to replace the neck network in YOLOV5;

[0009] Step 3: Use the Rep-GIoU Loss loss function to calculate the loss of the improved YOLOV5, input the test set into the fish recognition model after training convergence, and obtain the fish recognition result.

[0010] Optionally, in step 1, data preprocessing includes: data cleaning, data labeling, and data enhancement.

[0011] Optionally, in step 2, use Swin Transformer as the backbone network to introduce into YOLOV5, specifically:

[0012] The Swin-Stage module is added after the C3 module of YOLOV5, including:

[0013] The Patch Partition module, the first Swin-Stage module, and the second Swin-Stage module are connected in sequence;

[0014] The Patch Partition module is used to divide the image into blocks of fixed size and straighten and linearly transform each part;

[0015] The first Swin-Stage module includes: Linear Embedding linear embedding operation and Swin-Transformer Block target position learning;

[0016] The second Swin Stage module includes: Patch Merging resolution halving, channel doubling operations, and Swin-Transformer Block target position learning.

[0017] Optionally, in step 2, improve YOLOV5, specifically:

[0018] Modules connected in sequence: 1: CBS module, 2: CBS module, 3: C3 module, 4: Swin-Stage module, 5: CBS module, 6: C3 module, 7: Swin-Stage module, 8: CBS module, 9: C3 module, 10: Swin-Stage module, 11: CBS module, 12: C3 module, 13: Swin-Stage module, 14: SPPF module, 15: CBS module, 16: CBS module, 17: BIFPN_Concat module, 18: C3_F module, 19: CARAFE module, 20: CBS module, 21: BIFPN_Concat module, 22: C 3_F module, module 23: CARAFE module, module 24: CBS module, module 25: BIFPN_Concat module, module 26: C3_F module, module 27: CBS module, module 28: BIFPN_Concat module, module 29: C3_F module, module 30: CBS module, module 31: CBS module, module 32: BIFPN_Concat module, module 33: C3_F module; wherein, the input of module 17 is the output of module 15 and module 16; the input of module 21 is the output of module 19, module 10 and module 20; the input of module 25 is the output of module 23 and module 24; the input of module 28 is the output of module 27 and module 10; the input of module 32 is the output of module 30, module 13 and module 31;

[0019] It also includes: three convolution modules; the output of module 26 is used as the input of the first convolution block Conv1, and the output of the convolution block is used as the first output result; the output of module 29 is used as the input of the second convolution block Conv2, and the output of the convolution block is used as the second output result; the output of module 33 is used as the input of the third convolution block Conv3, and the output of the convolution block is used as the third output result.

[0020] Optionally, a CBS module includes sequentially connected two-dimensional convolution, normalization, and activation function operations.

[0021] Optionally, the C3 module is composed of a CBS module and a BottleNeck with a residual structure, specifically: the input of the C3 module passes through the CBS module to obtain a first output, the first output passes through a structure formed by connecting N BottleNecks with a residual structure in series to obtain a second output, the second output passes through the CBS module to obtain a third output, the second output and the third output are spliced, and the fourth output is obtained through the CBS module as the output of the C3 module; wherein, the C3_F module is composed of a CBS module and a BottleNeck without a residual structure.

[0022] Optionally, the SPPF module consists of a CBS module and a maximum pooling downsampling MaxPool2d, specifically:

[0023] The input of the SPPF module passes through the CBS module and MaxPool2d three times in sequence to obtain the first output, the second output, the third output and the fourth output respectively, and Concat splicing is performed to obtain the fifth output, and then passes through the CBS module to obtain the sixth output as the output of the SPPF module.

[0024] Optional, BIFPN_Concat module, specifically: by setting a learnable weight w i To distinguish the importance of different features in the feature fusion process, and for each weight w i Use the Relu function to ensure that the weight is between 0 and 1; the weight w i ,as follows:

[0025]

[0026] Among them, ∈ is a very small number; I i is the feature vector of the ith input.

[0027] Optionally, the CARAFE module is used for upsampling, specifically:

[0028] Use 1×1 convolution to compress the number of channels of the input feature map to C m ;

[0029] Using a K encoder ×K encoder Size, number of channels is The convolution kernel is H×W× The feature map of

[0030] Expand the channel of the feature map in space to get the shape of The upsampling kernel is used and normalized using the softmax function;

[0031] Map each position of the output feature map to the input feature map and extract the K up ×K up The area of ​​​​the upsampling kernel at this point is predicted in the output feature map of the upsampling kernel, and the dot product is performed to obtain the output value. Different channels at the same position share the same upsampling kernel;

[0032] Finally, the feature map of δH×δW×C is output to complete the upsampling operation.

[0033] Optionally, in step 3, the Rep-GIoU Loss loss function is as follows:

[0034] L Rep-GIoU =L GIoU +αL Rep ;

[0035]

[0036] Among them, L GIoU is GIoU Loss; L Rep is Repulsion Loss; α is the weight coefficient; P is the positive sample, which is the set of detection boxes p divided according to the set IoU threshold; B p The predicted box is obtained after regression offset of the detection box p; is the truth box with the largest GIoU except the truth box matching p; is the true value box with the largest IoU matched by each detection box p; G is all the true value boxes g except Other truth boxes outside.

[0037] It can be seen from the above technical solutions that compared with the prior art, the present invention proposes a fish recognition method. Based on YOLOV5, Swin Transformer is used as the backbone network to introduce into YOLOV5, which suppresses useless feature information in the network, and further integrates multi-scale local features and global feature information, effectively improving the network's ability to extract features from images; using the CARAFE upsampling algorithm to replace the linear interpolation upsampling in YOLOV5, using BIFPN weighted feature fusion to replace the ordinary splicing operation in YOLOV5, and using a new neck network to replace the neck network in YOLOV5, effectively improves the effects of water contrast and illumination in the process of fish recognition; at the same time, the Rep-GIoU Loss loss function is used to further improve the common problems of multiple targets and target occlusion in marine fish recognition models. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0039] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1:

[0042] Embodiment 1 of the present invention discloses a fish identification method, comprising:

[0043] Step 1: Preprocess the fish identification sample data and divide the preprocessed data into training sets and test sets.

[0044] Data preprocessing, including data cleaning, data labeling, and data enhancement.

[0045] Data cleaning and data labeling are to remove and mark empty objects and distorted data in sample data.

[0046] Data augmentation is to expand the number of datasets through geometric transformation and photometric transformation.

[0047] Step 2: Based on the training set, train a fish recognition model based on the improved YOLOV5; the improvements of YOLOV5 include: using Swin Transformer as the backbone network introduced into YOLOV5, using the CARAFE upsampling algorithm to replace the linear interpolation upsampling in YOLOV5, using the BIFPN weighted feature fusion to replace the ordinary splicing operation in YOLOV5, and using a new neck network to replace the neck network in YOLOV5.

[0048] Swin Transformer builds a hierarchical Transformer and introduces the idea of ​​localization, limits the attention calculation within the window, and designs the Transformer hierarchical features. This hierarchical feature enables Swin Transformer to use some multi-scale feature processing techniques for dense prediction, effectively solving the scale problem and high computational complexity problem of high-resolution images in the Transformer architecture. Therefore, Swin Transformer is introduced as the backbone network into the YOLOv5 network model to suppress useless feature information in the network, and further integrate multi-scale local features and global feature information to improve the network's ability to extract features from images.

[0049] Use Swin Transformer as the backbone network to introduce into YOLOV5, specifically:

[0050] The Swin-Stage module is added after the C3 module of YOLOV5, including:

[0051] The Patch Partition module, the first Swin-Stage module, and the second Swin-Stage module are connected in sequence;

[0052] The Patch Partition module is used to divide the image into blocks of fixed size and straighten and linearly transform each part;

[0053] The first Swin-Stage module includes: Linear Embedding linear embedding operation and Swin-Transformer Block target position learning;

[0054] The second Swin Stage module includes: Patch Merging resolution halving, channel doubling operations, and Swin-Transformer Block target position learning.

[0055] Improve YOLOV5, specifically:

[0056] Modules connected in sequence: 1: CBS module, 2: CBS module, 3: C3 module, 4: Swin-Stage module, 5: CBS module, 6: C3 module, 7: Swin-Stage module, 8: CBS module, 9: C3 module, 10: Swin-Stage module, 11: CBS module, 12: C3 module, 13: Swin-Stage module, 14: SPPF module, 15: CBS module, 16: CBS module, 17: BIFPN_Concat module, 18: C3_F module, 19: CARAFE module, 20: CBS module, 21: BIFPN_Concat module, 22: C 3_F module, module 23: CARAFE module, module 24: CBS module, module 25: BIFPN_Concat module, module 26: C3_F module, module 27: CBS module, module 28: BIFPN_Concat module, module 29: C3_F module, module 30: CBS module, module 31: CBS module, module 32: BIFPN_Concat module, module 33: C3_F module; wherein, the input of module 17 is the output of module 15 and module 16; the input of module 21 is the output of module 19, module 10 and module 20; the input of module 25 is the output of module 23 and module 24; the input of module 28 is the output of module 27 and module 10; the input of module 32 is the output of module 30, module 13 and module 31;

[0057] It also includes: three convolution modules; the output of module 26 is used as the input of the first convolution block Conv1, and the output of the convolution block is used as the first output result; the output of module 29 is used as the input of the second convolution block Conv2, and the output of the convolution block is used as the second output result; the output of module 33 is used as the input of the third convolution block Conv3, and the output of the convolution block is used as the third output result.

[0058] The CBS module includes two-dimensional convolution, normalization and activation function operations connected in sequence, and the activation function adopts the Silu activation function.

[0059] The C3 module is composed of a CBS module and a BottleNeck with a residual structure. Specifically, the input of the C3 module passes through the CBS module to obtain the first output, the first output passes through a structure formed by connecting N BottleNecks with a residual structure in series to obtain the second output, the second output passes through the CBS module to obtain the third output, the second output and the third output are spliced, and the fourth output is obtained through the CBS module as the output of the C3 module; among them, the C3_F module is composed of a CBS module and a BottleNeck without a residual structure. BottleNeck has two forms, one with a residual structure and the other without a residual structure. The one with a residual structure is obtained by passing the input through two CBS modules in sequence and adding the output result to the input, and the one without a residual structure is obtained by directly passing the input through two CBS modules in sequence.

[0060] The SPPF module consists of a CBS module and a maximum pooling downsampling MaxPool2d, specifically:

[0061] The input of the SPPF module passes through the CBS module and MaxPool2d three times in sequence to obtain the first output, the second output, the third output and the fourth output respectively, and Concat splicing is performed to obtain the fifth output, and then passes through the CBS module to obtain the sixth output as the output of the SPPF module.

[0062] The BIFPN_Concat module is as follows: By setting a learnable weight w i To distinguish the importance of different features in the feature fusion process, and for each weight w i Use the Relu function to ensure that the weight is between 0 and 1; the weight w i ,as follows:

[0063]

[0064] Among them, ∈ is a very small number; I i is the feature vector of the ith input.

[0065] The CARAFE module is used for upsampling, specifically:

[0066] Use 1×1 convolution to compress the number of channels of the input feature map to C m ;

[0067] Using a K encoder ×K encoder Size, number of channels is The convolution kernel is obtained The feature map of

[0068] Expand the channel of the feature map in space to get the shape of The upsampling kernel is used and normalized using the softmax function;

[0069] Map each position of the output feature map to the input feature map and extract the K up ×K up The area of ​​​​the upsampling kernel at this point is predicted in the output feature map of the upsampling kernel, and the dot product is performed to obtain the output value. Different channels at the same position share the same upsampling kernel;

[0070] Finally, the feature map of δH×δW×C is output to complete the upsampling operation.

[0071] The CARAFE module upsampling formula is as follows:

[0072]

[0073] Where l′ is the position of the output feature map; W l′ is the sampling kernel corresponding to l′ of the upsampling kernel prediction module; (i, j) is the position of the input feature map that maps the position of the output feature map l′; X is the input feature map; X′ is the output feature map.

[0074] Step 3: Use the Rep-GIoU Loss loss function to calculate the loss of the improved YOLOV5, input the test set into the fish recognition model after training convergence, and obtain the fish recognition result.

[0075] The Rep-GIoU Loss loss function not only combines the advantages of GIoU_Loss and IoU_Loss, but also increases the correlation between the regression parameters and IoU, while preventing the proposed box from shifting to the surrounding targets, thereby improving the detection performance of occluded targets. It also introduces Rep to further improve the common problems of multiple targets and target occlusion in marine fish recognition models.

[0076] The Rep-GIoU Loss loss function is as follows:

[0077] L Rep-GIoU =L GIoU +αL Rep ;

[0078]

[0079] Among them, L GIoU is GIoU Loss; L Rep is Repulsion Loss; α is the weight coefficient; P is the positive sample, which is the set of detection boxes p divided according to the set IoU threshold; B p The predicted box is obtained after regression offset of the detection box p; is the truth box with the largest GIoU except the truth box matching p; is the true value box with the largest IoU matched by each detection box p; G is all the true value boxes g except Other truth boxes outside.

[0080] The embodiment of the invention discloses a fish identification method.

[0081] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0082] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fish identification method, characterized in that: include: Step 1: Preprocess the fish identification sample data and divide the preprocessed data into training sets and test sets; Step 2: Based on the training set, a fish recognition model based on the improved YOLOV5 is trained; wherein the improvement of the YOLOV5 includes: using Swin Transformer as a backbone network introduced into the YOLOV5, using the CARAFE upsampling algorithm to replace the linear interpolation upsampling in the YOLOV5, using the BIFPN weighted feature fusion to replace the ordinary splicing operation in the YOLOV5, and using a new neck network to replace the neck network in the YOLOV5; Step 3: Use the Rep-GIoU Loss loss function to calculate the loss of the improved YOLOV5, input the test set into the fish recognition model after training convergence, and obtain the fish recognition result.

2. A fish identification method according to claim 1, characterized in that: In step 1, the data preprocessing includes: data cleaning, data labeling and data enhancement.

3. A fish identification method according to claim 1, characterized in that: In step 2, SwinTransformer is used as the backbone network to introduce into the YOLOV5, specifically: The Swin-Stage module is added after the C3 module of YOLOV5, including: The Patch Partition module, the first Swin-Stage module, and the second Swin-Stage module are connected in sequence; The Patch Partition module is used to divide the image into blocks according to a fixed size, and straighten and linearly transform each part; The first Swin-Stage module includes: a Linear Embedding linear embedding operation and a Swin-Transformer Block target position learning performed sequentially; The second Swin Stage module includes: Patch Merging resolution halving, channel number doubling operations and Swin-Transformer Block target position learning performed in sequence.

4. A fish identification method according to claim 3, characterized in that: In step 2, the improved YOLOV5 is specifically as follows: Modules connected in sequence: 1: CBS module, 2: CBS module, 3: C3 module, 4: Swin-Stage module, 5: CBS module, 6: C3 module, 7: Swin-Stage module, 8: CBS module, 9: C3 module, 10: Swin-Stage module, 11: CBS module, 12: C3 module, 13: Swin-Stage module, 14: SPPF module, 15: CBS module, 16: CBS module, 17: BIFPN_Concat module, 18: C3_F module, 19: CARAFE module, 20: CBS module, 21: BIFPN_Concat module, 22: C 3_F module, module 23: CARAFE module, module 24: CBS module, module 25: BIFPN_Concat module, module 26: C3_F module, module 27: CBS module, module 28: BIFPN_Concat module, module 29: C3_F module, module 30: CBS module, module 31: CBS module, module 32: BIFPN_Concat module, module 33: C3_F module; wherein, the input of module 17 is the output of module 15 and module 16; the input of module 21 is the output of module 19, module 10 and module 20; the input of module 25 is the output of module 23 and module 24; the input of module 28 is the output of module 27 and module 10; the input of module 32 is the output of module 30, module 13 and module 31; It also includes: three convolution modules; the output of module 26 is used as the input of the first convolution block Conv1, and the output of the convolution block is used as the first output result; the output of module 29 is used as the input of the second convolution block Conv2, and the output of the convolution block is used as the second output result; the output of module 33 is used as the input of the third convolution block Conv3, and the output of the convolution block is used as the third output result.

5. A fish identification method according to claim 4, characterized in that: The CBS module includes two-dimensional convolution, normalization and activation function operations connected in sequence.

6. A fish identification method according to claim 4, characterized in that: The C3 module is composed of a CBS module and a BottleNeck with a residual structure. Specifically, the input of the C3 module is passed through the CBS module to obtain a first output, the first output is passed through a structure formed by connecting N BottleNecks with a residual structure in series to obtain a second output, the second output is passed through the CBS module to obtain a third output, the second output and the third output are spliced, and the fourth output is obtained through the CBS module as the output of the C3 module; wherein the C3_F module is composed of a CBS module and a BottleNeck without a residual structure.

7. A fish identification method according to claim 4, characterized in that: The SPPF module is composed of a CBS module and a maximum pooling downsampling MaxPool2d, specifically: The input of the SPPF module passes through the CBS module and MaxPool2d three times in sequence to obtain the first output, the second output, the third output and the fourth output respectively, and Concat splicing is performed to obtain the fifth output, and then passes through the CBS module to obtain the sixth output as the output of the SPPF module.

8. A fish identification method according to claim 4, characterized in that: The BIFPN_Concat module is specifically: by setting a learnable weight w i To distinguish the importance of different features in the feature fusion process, and for each weight w i Use the Relu function to ensure that the weight is between 0 and 1; the weight w i ,as follows: Among them, ∈ is a very small number; I i is the feature vector of the ith input.

9. A fish identification method according to claim 4, characterized in that: The CARAFE module is used for upsampling, specifically: Use 1×1 convolution to compress the number of channels of the input feature map to C m ; Using a K encoder ×K encoder Size, number of channels is The convolution kernel is obtained The feature map of Expand the channel of the feature map in space to get the shape of The upsampling kernel is used and normalized using the softmax function; Map each position of the output feature map to the input feature map and extract the K up ×K up The area of ​​​​the upsampling kernel at this point is predicted in the output feature map of the upsampling kernel, and the dot product is performed to obtain the output value. Different channels at the same position share the same upsampling kernel; Finally, the feature map of δH×δW×C is output to complete the upsampling operation.

10. A fish identification method according to claim 1, characterized in that: In step 3, the Rep-GIoULoss loss function is as follows: L Rep-GIoU =L GIoU +αL Rep ; Among them, L GIoU is GIoU Loss; L Rep is Repulsion Loss; α is the weight coefficient; P is the positive sample, which is the set of detection boxes p divided according to the set IoU threshold; B p The predicted box is obtained after regression offset of the detection box p; is the truth box with the largest GIoU except the truth box matching p; is the true value box with the largest IoU matched by each detection box p; G is all the true value boxes g except Other truth boxes outside.

Images