A fish target identification method for river ecological health assessment

Abstract

The present invention discloses a method for identifying fish targets for river ecological health assessment, and belongs to the technical field of river ecological health assessment and image recognition. The identification method comprises the steps of inputting a fish image into a fish recognition model; extracting features of the fish image in sequence through a plurality of convolution modules, and then performing a first pooling process on the feature graphs output by each convolution module respectively, inputting each secondary feature graph into a fusion module, inputting each secondary feature graph into a regression enhancement module, inputting the feature graphs output by each convolution module and the first weight into a joint adjustment module, fusing the main feature graph, the first weight and the second weight to obtain a purified feature graph; and inputting the purified feature graph into a classification module, etc. The identification method provided by the present invention has better noise resistance and the ability of deep learning abstract features, and is more suitable for identifying the acquired fish target images in the task of river ecological health assessment.

Description

Technical Field

[0001] The invention belongs to the technical field of river ecological health assessment and image recognition, and in particular relates to a fish target recognition method for river ecological health assessment. Background Art

[0002] One of the important aspects of river water ecological health assessment is the assessment of biodiversity. Among them, the traditional method of assessing fish diversity is time-consuming and labor-intensive, and the results are highly unreliable. Now, underwater robots are used to automatically take photos and automatically identify and count fish, which greatly improves efficiency and accuracy. However, rivers usually contain a large amount of suspended matter. In the images taken by the camera, not only is the background complex and diverse, but there may also be occlusion and interference from suspended matter in front of the target fish, which has a significant impact on fish target identification. Therefore, it is necessary to provide a fish target identification method suitable for river ecological health assessment based on actual conditions. Summary of the invention

[0003] In view of the above phenomenon, the present invention provides a fish target recognition method for river ecological health assessment to improve the recognition accuracy of fish images collected in actual scenes.

[0004] In order to achieve the above object, the solution adopted by the present invention is: a fish target identification method for river ecological health assessment, comprising the following steps:

[0005] S100, obtaining a fish image, calling a fish recognition model, and inputting the fish image into the fish recognition model; the fish recognition model is provided with a convolution module, a fusion module, a regression enhancement module, a joint adjustment module and a classification module;

[0006] S200, extracting features of the fish image in sequence through a plurality of the convolution modules, and then performing a first pooling process on the feature graphs output by each of the convolution modules to obtain a plurality of secondary-order feature graphs;

[0007] S300, inputting each of the sub-order feature maps into the fusion module, and fusing each of the sub-order feature maps through the fusion module to obtain a main feature map;

[0008] S400, inputting each of the secondary feature maps into a regression enhancement module, and obtaining a first weight after feature extraction by the regression enhancement module;

[0009] S500, inputting the feature graphs output by each of the convolution modules and the first weights into the joint adjustment module, and obtaining second weights after feature extraction by the joint adjustment module;

[0010] S600, fusing the backbone feature graph, the first weight, and the second weight to obtain a purified feature graph;

[0011] S700, inputting the purified feature map into a classification module, and the classification module outputs the classification result of the fish image to complete the recognition.

[0012] Furthermore, the convolution module includes a convolution layer and a first activation function arranged in sequence.

[0013] Furthermore, when the convolution layer performs a convolution operation, the sliding step size is 1; and the first activation function is a ReLU function.

[0014] It should be noted that, as is known, the feature map includes three dimensions, namely, height, width and number of channels. Further, the first pooling is maximum pooling, and after the first pooling process, the height and width of the feature map are reduced.

[0015] Furthermore, the internal processing process of the regression enhancement module includes:

[0016] K100, performing a global second pooling process in a channel direction on each of the secondary-order feature maps to obtain a plurality of first process weights;

[0017] K200, concatenate all the first process weights obtained by K100 to obtain the second process weights;

[0018] K300, performing a first convolution operation on the second process weight to obtain a third process weight;

[0019] K400, the third process weight is activated by the second activation function to obtain the first weight.

[0020] When the regression enhancement module adopts the above processing flow, it not only integrates the feature information of different depths well, but also can effectively bypass the local noise in the image, focus on learning the feature mapping relationship of the entire image, and further improve the model's anti-interference ability.

[0021] Furthermore, the second pooling is average pooling; and the second activation function is a sigmoid function.

[0022] Furthermore, the internal processing of the joint adjustment module includes:

[0023] M100, concatenating the feature maps output by each of the convolution modules to obtain a first intermediate weight;

[0024] M200, performing a second convolution operation on the first intermediate weight to obtain a second intermediate weight;

[0025] M300, performing a global third pooling process on the second intermediate weight in the channel direction to obtain a third intermediate weight;

[0026] M400, subtract the third intermediate weight from the first weight to obtain a fourth intermediate weight;

[0027] M500. After the fourth intermediate weight is activated by the third activation function, the second weight is obtained.

[0028] Furthermore, during the second convolution operation, the sliding step size is greater than 1; the third pooling is variance pooling; and the third activation function is a sigmoid function.

[0029] Furthermore, the backbone feature map, the first weight and the second weight are fused, including: performing element-wise multiplication of the backbone feature map with the first weight and the second weight in sequence.

[0030] Furthermore, the internal processing of the classification module includes:

[0031] Y100, performing a fourth pooling process on the purified feature map in a global spatial direction to obtain a first vector;

[0032] Y200, input the first vector into a fully connected layer, and then the fully connected layer outputs a second vector;

[0033] Y300, input the second vector into a softmax classifier, and then the softmax classifier outputs a classification result of the fish image.

[0034] The beneficial effects of the present invention are:

[0035] (1) In the present invention, multiple first pooling processes are performed first, and then the backbone feature maps are fused, and then the first weights and the second weights are further generated. The inventor creatively discovered that the use of such a distributed pre-pooling setting can differentiate the random interference information in the image in different dimensions, weaken the proportion of image noise in the extracted feature information, and effectively improve the model's ability to recognize noise;

[0036] (2) Part of the feature information input to the joint adjustment module comes directly from the output of the convolution module (without the first pooling process), and the other part comes from the first weight output by the regression enhancement module. The input information of the regression enhancement module is the second-order feature map after the first pooling process; therefore, the joint adjustment module receives the information before and after the first pooling as input at the same time. After the joint module is fused with the backbone feature map, the missing, deformed and misaligned useful information in the first pooling process can be properly repaired (most of the deformation and misalignment of useful information is caused by image noise), thereby improving the accuracy of effective feature extraction. In addition, the joint adjustment module forms a progressive coordination relationship with the regression enhancement module, which is conducive to further digging deeper feature information and making the feature extraction process more robust. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a visual schematic diagram of the processing process in step S200 of the present invention;

[0038] In the attached figure:

[0039] 1-convolution module, 2-first pooling process. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings:

[0041] Embodiment 1:

[0042] This embodiment provides a fish target identification method for river ecological health assessment, comprising the following steps:

[0043] The fish image is obtained, the fish recognition model is called, and the fish image is input into the fish recognition model. The fish recognition model is pre-trained, and the fish recognition model is provided with a convolution module 1, a fusion module, a regression enhancement module, a joint adjustment module and a classification module.

[0044] In this embodiment, the fish recognition model includes five feature extraction units connected in sequence and a classification module. The feature map output by the previous feature extraction unit is directly input to the next feature extraction unit, and the feature map output by the fifth feature extraction unit is input to the classification module. Each feature extraction unit includes three convolution modules 1, a fusion module, a regression enhancement module and a joint adjustment module. Specifically, the calculation or processing process inside each feature extraction unit is as follows:

[0045] Step 1: If Figure 1As shown, multiple convolution modules 1 are used to extract features of fish images in sequence (the feature map output by the previous convolution module 1 is directly input to the next convolution module 1), and then the feature maps output by each convolution module 1 are respectively subjected to the first pooling process 2 to obtain multiple secondary feature maps. Among them, the convolution module 1 includes a convolution layer and a first activation function arranged in sequence. Exemplarily, the convolution kernel size of the convolution layer can be 3*3, and when the convolution layer performs a convolution operation, the sliding step size can be 1; the first activation function can be a ReLU function. In the same feature extraction unit, the feature map sizes output by each convolution module 1 are equal. The first pooling is the maximum pooling. Exemplarily, its pooling window size can be 2*2, and the sliding step size can be 2. After the first pooling process 2, the height and width of the secondary feature map are respectively half of the height and width of the feature map output by the convolution module 1.

[0046] Step 2: After obtaining the secondary feature map, each secondary feature map is input into the fusion module, and each secondary feature map is fused through the fusion module to obtain the backbone feature map. In the same feature extraction unit, the number of channels of the backbone feature map is usually twice that of the secondary feature map. In this embodiment, the fusion module is implemented using existing technology that can achieve feature fusion. Exemplarily, the fusion module can include splicing operations, convolution operations, and activation operations in sequence. Specifically, the convolution operation can be a convolution operation with a convolution kernel size of 1*1 and a step size of 1, and the ReLU function can be used for activation operation.

[0047] Step 3: Input each sub-order feature map into the regression enhancement module, and obtain the first weight after feature extraction by the regression enhancement module; specifically, in this embodiment, the internal processing process of the regression enhancement module includes:

[0048] K100, perform a global second pooling process in the channel direction on each sub-order feature map to obtain multiple first process weights; specifically, the second pooling process is a global average pooling, and after the second pooling process, the channel of the first process weight becomes 1.

[0049] K200, concatenate all the first process weights obtained by K100 to obtain the second process weights;

[0050] K300, perform the first convolution operation on the second process weight (the convolution kernel can be 3*3) to obtain the third process weight (the channel is 1);

[0051] K400, after the third process weight is activated by the second activation function (such as the sigmoid function), a first weight is obtained; the height of the first weight is equal to the height of the trunk feature map, and the width of the first weight is equal to the width of the trunk feature map.

[0052] Step 4: Input the feature maps output by each convolution module and the first weight into the joint adjustment module, and obtain the second weight after feature extraction by the joint adjustment module; specifically, in this embodiment, the internal processing process of the joint adjustment module includes:

[0053] M100, concatenate the feature maps output by each convolution module to obtain the first intermediate weight;

[0054] M200, performing a second convolution operation on the first intermediate weight (exemplarily, the convolution kernel size of the second convolution operation is 3*3, and the step size is 2) to obtain a second intermediate weight, wherein the length and width of the second intermediate weight are respectively equal to the length and width of the first weight;

[0055] M300, performing a third pooling process on the second intermediate weight in a channel direction globally to obtain a third intermediate weight; specifically, the third pooling is a global variance pooling;

[0056] M400, subtract the third intermediate weight from the first weight (the third intermediate weight minus the first weight) to obtain a fourth intermediate weight;

[0057] After M500 and the fourth intermediate weight are activated by the third activation function (which may be a sigmoid function), the second weight is obtained. The height of the second weight is equal to the height of the trunk feature map, and the width of the second weight is equal to the width of the trunk feature map.

[0058] Step 5: Multiply the backbone feature map with the first weight and the second weight element by element, so as to achieve the fusion of the backbone feature map, the first weight and the second weight and obtain the purified feature map.

[0059] The purified feature graph output by the fifth feature extraction unit is input into the classification module, and then the classification module outputs the classification result of the fish image to complete the recognition. Specifically, in this embodiment, the internal processing process of the classification module includes:

[0060] Y100, performing a fourth pooling process on the input purified feature map in a global spatial direction to obtain a first vector; specifically, the fourth pooling process is a global average pooling process. After the fourth pooling process, the height and width of the purified feature map are compressed to 1, and the length of the first vector is equal to the number of channels of the purified feature map;

[0061] Y200, input the first vector into a fully connected layer, and then the fully connected layer outputs a second vector; wherein the number of nodes in the output layer of the fully connected layer (that is, the length of the second vector) is equal to the total number of categories actually required to be identified;

[0062] Y300, input the second vector into a softmax classifier, and then the softmax classifier outputs a classification result of the fish image.

[0063] Horizontal comparison tests with some existing image recognition models (DenseNet, PULC, MobileNet-v3, etc.) show that the method provided by the present invention has an accuracy rate of more than 8.4% higher than other models for fish target recognition, has significantly stronger anti-noise ability and deep learning abstract feature ability, and is more suitable for identifying acquired fish target images in river ecological health assessment tasks.

[0064] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A fish target identification method for river ecological health assessment, characterized by: The following steps are involved: S100, obtaining a fish image, calling a fish recognition model, and inputting the fish image into the fish recognition model; the fish recognition model is provided with a convolution module, a fusion module, a regression enhancement module, a joint adjustment module and a classification module; S200, extracting features of the fish image in sequence through a plurality of the convolution modules, and then performing a first pooling process on the feature graphs output by each of the convolution modules to obtain a plurality of secondary-order feature graphs; S300, inputting each of the sub-order feature maps into the fusion module, and fusing each of the sub-order feature maps through the fusion module to obtain a main feature map; S400, inputting each of the secondary feature maps into a regression enhancement module, and obtaining a first weight after feature extraction by the regression enhancement module; S500, inputting the feature graphs output by each of the convolution modules and the first weights into the joint adjustment module, and obtaining second weights after feature extraction by the joint adjustment module; S600, fusing the backbone feature graph, the first weight, and the second weight to obtain a purified feature graph; S700, inputting the purified feature graph into a classification module, and the classification module outputs a classification result of the fish image to complete the recognition; The internal processing process of the regression enhancement module includes: K100, performing a global second pooling process in a channel direction on each of the secondary-order feature maps to obtain a plurality of first process weights; K200, concatenate all the first process weights obtained by K100 to obtain the second process weights; K300, performing a first convolution operation on the second process weight to obtain a third process weight; K400, the third process weight is activated by the second activation function to obtain the first weight; The internal processing of the joint adjustment module includes: M100, concatenating the feature maps output by each of the convolution modules to obtain a first intermediate weight; M200, performing a second convolution operation on the first intermediate weight to obtain a second intermediate weight; M300, performing a global third pooling process on the second intermediate weight in the channel direction to obtain a third intermediate weight; M400, subtract the third intermediate weight from the first weight to obtain a fourth intermediate weight; M500. After the fourth intermediate weight is activated by the third activation function, the second weight is obtained.

2. The fish target identification method for river ecological health assessment according to claim 1 is characterized by: The convolution module includes a convolution layer and a first activation function arranged in sequence.

3. The fish target identification method for river ecological health assessment according to claim 2 is characterized by: When the convolution layer performs a convolution operation, the sliding step size is 1; and the first activation function is a ReLU function.

4. The method for identifying fish targets for river ecological health assessment according to claim 1 is characterized by: The first pooling is maximum pooling. After the first pooling process, the height and width of the feature map are reduced.

5. The method for identifying fish targets for river ecological health assessment according to claim 1 is characterized by: The second pooling is average pooling; and the second activation function is a sigmoid function.

6. The method for identifying fish targets for river ecological health assessment according to claim 1 is characterized by: During the second convolution operation, the sliding step size is greater than 1; the third pooling is variance pooling; and the third activation function is a sigmoid function.

7. The method for identifying fish targets for river ecological health assessment according to claim 1 is characterized by: The backbone feature map, the first weight and the second weight are fused, including: performing element-wise multiplication of the backbone feature map with the first weight and the second weight in sequence.

8. The method for identifying fish targets for river ecological health assessment according to claim 1 is characterized by: The internal processing of the classification module includes: Y100, performing a fourth pooling process on the purified feature map in a global spatial direction to obtain a first vector; Y200, input the first vector into a fully connected layer, and then the fully connected layer outputs a second vector; Y300, input the second vector into a softmax classifier, and then the softmax classifier outputs a classification result of the fish image.

Images