Fish recognition system and method for local feature affecting global feature learning

Abstract

The present invention discloses a fish recognition system in which local features affect global feature learning, including a data collection module, a data transmission module, and a fish recognition terminal. The data collection module includes a fixing plate, an underwater camera, a power supply circuit, an underwater lighting device, and a supplementary lighting light wall. The data transmission module includes a data storage device and an image transmission device. The image transmission device sends the image information collected by the underwater camera to the data storage device for storage. The fish recognition terminal includes a server, a real-time monitoring device, and an image receiving device. The real-time monitoring device is arranged above the data collection module. The image receiving device receives the image information from the data storage device and sends it to the server. The fish recognition system and method with the above structure in which local features affect global feature learning have strong generalization ability for recognizing different types of fish bodies, can recognize unfamiliar individuals, and have a high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of fish recognition systems, and in particular to a fish recognition system and method in which local features affect global feature learning. Background Art

[0002] The data explosion brought about by the big data era and the improvement of computer computing power have further promoted the development of fields such as computer vision and deep learning. In recent years, the deterioration of the marine ecological environment has received more attention, and target recognition of marine organisms is of great significance. Underwater fish target recognition provides technical support for protecting the marine ecological environment. Accurately identifying the identity information of each fish is conducive to understanding the status of the ecosystem, recording population density, observing individual growth and physical condition.

[0003] Traditional fish recognition is to learn the characteristics of fish bodies. When recognizing, it is necessary to compare with the pictures of the fish bodies to be detected, and it cannot recognize strange fish bodies. At the same time, problems such as low accuracy of fish body recognition caused by the complex underwater environment have become the key technical problems that need to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of the present invention is to provide a fish recognition system and method in which local features affect global feature learning, which has strong generalization for different types of fish body recognition, can recognize strange individuals, and has high recognition accuracy.

[0005] To achieve the above purpose, the present invention provides a fish recognition system in which local features affect global feature learning, including a data collection module, a data transmission module, and a fish recognition terminal.

[0006] The data collection module includes a fixing plate, an underwater camera, a power supply circuit, an underwater lighting device, and a supplementary lighting light wall. The power supply circuit supplies power to the underwater camera and the underwater lighting device. The fixing plate is vertically arranged under the water surface. The underwater camera and the underwater lighting device are fixed on the fixing plate, and the supplementary lighting light wall is arranged opposite to the fixing plate.

[0007] The data transmission module includes a data storage device and an image transmission device. The image transmission device sends the image information collected by the underwater camera to the data storage device for storage.

[0008] The fish recognition terminal includes a server, a real-time monitoring device, and an image receiving device. The real-time monitoring device is arranged above the data collection module. The image receiving device receives the image information from the data storage device and sends it to the server.

[0009] Preferably, the data collection module further includes a temperature sensor, an oxygen content sensor, and a pressure sensor.

[0010] Preferably, the data storage device is a mechanical hard disk with available space of more than 10G.

[0011] Preferably, the image transmission device, the image receiving device, the real-time monitoring device and the server are communicatively connected via a wireless bridge.

[0012] A fish recognition method in which local features affect global feature learning, comprising the following steps:

[0013] S1. Perform video frame extraction on the image information sent to the server, use the characteristics of video coherence to find different pictures of the same fish, and number the pictures.

[0014] S2. Use the Labelme annotation tool to annotate key points on the fish body.

[0015] S3. Divide the fish body into blocks according to the key points to obtain local pictures of the fish body, and use the local pictures and the overall picture as the input of the neural network.

[0016] S4. Learn the fish body features through the SE-Resnet backbone network, fuse the features learned locally and globally, and use the Adam optimization algorithm for training to obtain the optimal fish recognition model.

[0017] Preferably, in S3, data augmentation is performed on the local pictures of the fish body obtained after segmentation, including random rotation from -7 degrees to 7 degrees and brightness change from 0.7 to 1.3.

[0018] Compared with the prior art, the present invention has the following advantages:

[0019] 1. This system does not require a large amount of fish body information to query the database. As long as a fish individual swims past the underwater camera, an identity information can be assigned to it, and the texture features of the individual can be learned. When it is recorded by the camera again, the assigned identity information will be matched according to the similarity ranking, so as to complete the target recognition task. This perfectly bypasses the disadvantages of difficult collection of underwater fish data and small database, and can achieve the goal of fish individual recognition only through underwater camera shooting.

[0020] 2. Use SE-Resnet, which combines the resnet network and the SE module, as the backbone network. By adding the attention mechanism SE module, the relationship between different channels is concerned, and the importance of channels is found. At the cost of a minimal increase in the number of parameters, the network performance is improved.

[0021] 3. Adopt a learning method in which local features affect global features, combining coarse-grained and fine-grained, and can effectively identify the target object even in the case of occlusion or missing of key information.

[0022] The fish recognition device and system provided by the present invention, which are affected by local features in global feature learning, not only have high accuracy, strong generalization, simple operation and reliable use, but also have strong practical value and improve the underwater fish recognition efficiency.

[0023] The technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic structural diagram of the fish recognition system of the present invention in which local features affect global feature learning;

[0025] Figure 2 It is a network diagram of the fish recognition terminal of the embodiment of the present invention;

[0026] Figure 3 It is a flowchart of the fish recognition method of the embodiment of the present invention.

[0027] REFERENCE SIGNS

[0028] 1, fixing plate; 2, underwater camera; 3, underwater lighting device; 4, supplementary light wall; 5, power supply circuit; 6, image transmission device; 7, image receiving device; 8, data storage device; 9, server; 10, real-time monitoring device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] Embodiment

[0031] As shown in the figure, the fish recognition system affected by local features in global feature learning includes a data collection module, a data transmission module and a fish recognition terminal.

[0032] The data collection module includes a fixing plate 1, an underwater camera, a power supply circuit 5, an underwater lighting device 3, and a supplementary light wall 4. The power supply circuit 5 supplies power to the underwater camera 2 and the underwater lighting device 3. The fixing plate 1 is vertically arranged under the water surface, the underwater camera and the underwater lighting device 3 are fixed on the fixing plate 1, the supplementary light wall 4 is arranged opposite to the fixing plate 1, and the underwater lighting device 3 and the supplementary light wall 4 provide the required light source for the underwater camera 2. The data collection module also includes a temperature sensor, an oxygen content sensor and a pressure sensor, which are respectively used to measure the temperature, oxygen content and pressure in the water body as environmental reference variables in the recognition system.

[0033] The data transmission module includes a data storage device 8 and an image transmission device 6. The image transmission device 6 sends the image information collected by the underwater camera to the data storage device 8 for storage. The data storage device 8 is a mechanical hard disk with an available space of more than 10G.

[0034] The fish recognition terminal includes a server 9, a real-time monitoring device 10, and an image receiving device 7. The real-time monitoring device 10 is arranged above the data collection module. The image receiving device 7 receives image information from the data storage device 8 and sends it to the server 9. The image transmission device 6, the image receiving device 7, the real-time monitoring device 10, and the server 9 are communicatively connected via a wireless bridge.

[0035] A fish recognition method in which local features affect global feature learning includes the following steps:

[0036] S1. Perform video frame extraction on the image information sent to the server 9. Utilize the characteristics of video coherence to find different pictures of the same fish and number them.

[0037] S2. Use the Labelme annotation tool to perform key point annotation on the fish body.

[0038] S3. Divide the fish body into local pictures according to the key points. Perform data augmentation on the local pictures of the fish body obtained after division, including random rotation from -7 degrees to 7 degrees and brightness change from 0.7 to 1.3. Use the local pictures and the overall picture as the input of the neural network.

[0039] The ways of dividing the fish body according to the key points include dividing it in the left-right order and in the up-down order. When dividing in the left-right order, it is successively divided into three parts: head, body, and tail, which is called the first part of the input image. When dividing in the up-down order, it is successively divided into two parts: upper fin and lower fin, which is called the second part of the input image. The overall picture is used as the global feature input into the neural network.

[0040] S4. Use 5 independent SE-Resnet101 pre-trained networks to learn the first part of the input image and the second part of the input image. After the first part of the input image passes through the average pooling layer, 3 feature vectors are generated, and then the 3 vectors are concatenated and fused along the channels into a local feature vector. After the second part of the input image passes through the average pooling layer, 2 feature vectors are generated, and then the 2 vectors are concatenated and fused along the channels into another local feature vector.

[0041] The overall picture with a size of 256×512 extracts the features commonly shared macroscopically by the whole through the SE-Resnet50 pre-trained network, and a 2048-dimensional global feature vector is obtained after passing through the average pooling layer. The advantage of using networks with different numbers of layers to learn different regions is that as the number of layers increases appropriately, the extracted features are more abstract, and using a network with more layers in the spatial dimension can learn more detailed and subtle fish body features.

[0042] Fuse the two generated local feature vectors with the global feature vector respectively to obtain two branches of local feature learning. Finally, perform weighted fusion on the cross-entropy loss functions generated by the above three learning branches, and use the Adam optimization algorithm for training to obtain the optimal fish recognition model.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that they can still modify or equivalently replace the technical solutions of the present invention, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A fish recognition method in which local features affect global feature learning, characterized in that It includes the following steps: S1. Perform video frame extraction on the image information sent to the server. Utilize the characteristics of video coherence to find different pictures of the same fish and number the pictures. S2. Use the Labelme annotation tool to label the key points of the fish body. S3. Divide the fish body into local pictures according to the key points. The local pictures and the overall picture are used as the input of the neural network. The method of dividing the fish body according to the key points includes dividing by left - right order and by up - down order. When dividing by left - right order, it is divided into three parts: head, body, and tail in sequence, which is called the first part of the input image. When dividing by up - down order, it is divided into two parts: upper fin and lower fin in sequence, which is called the second part of the input image. S4. Adopt 5 independent pre - trained SE - Resnet101 networks to learn the first part of the input image and the second part of the input image. After the first part of the input image passes through the average pooling layer, 3 feature vectors are generated, and the 3 feature vectors are concatenated and fused along the channels to form a local feature vector. After the second part of the input image passes through the average pooling layer, 2 feature vectors are generated, and the 2 feature vectors are concatenated and fused along the channels to form another local feature vector. The overall picture extracts the features that are macroscopically common through the pre - trained SE - Resnet50 network, and a 2048 - dimensional global feature vector is obtained after passing through the average pooling layer. Fuse the two generated local feature vectors with the global feature vector respectively to obtain two branches of local feature learning. Finally, perform weighted fusion on the cross - entropy loss functions generated by the three learning branches, and use the Adam optimization algorithm for training to obtain the optimal fish recognition model.

2. The fish recognition method for local feature influencing global feature learning according to claim 1, wherein: In S3, data augmentation is performed on the local pictures of the fish body obtained after segmentation, including random rotation from - 7 degrees to 7 degrees and brightness change from 0.7 to 1.

3.

3. A fish recognition system for implementing the fish recognition method of local feature influencing global feature learning according to any one of claims 1-2, characterized in that: It includes a data collection module, a data transmission module, and a fish recognition terminal. The data collection module includes a fixing plate, an underwater camera, a power circuit, an underwater lighting device, and a supplementary light wall. The power circuit supplies power to the underwater camera and the underwater lighting device. The fixing plate is vertically arranged under the water surface. The underwater camera and the underwater lighting device are fixed on the fixing plate. The supplementary light wall is arranged opposite to the fixing plate. The data transmission module includes a data storage device and an image transmission device. The image transmission device sends the image information collected by the underwater camera to the data storage device for storage. The fish recognition terminal includes a server, a real - time monitoring device, and an image receiving device. The real - time monitoring device is arranged above the data collection module. The image receiving device receives the image information from the data storage device and sends it to the server.

4. The fish recognition system according to claim 3, wherein: The data collection module also includes a temperature sensor, an oxygen content sensor, and a pressure sensor.

5. The fish recognition system according to claim 3, characterized in that: The data storage device is a mechanical hard disk with available space greater than or equal to 10G.

6. The fish recognition system according to claim 3, wherein: The image transmission device, the image receiving device, the real - time monitoring device, and the server are communicatively connected through a wireless bridge.

Images