Fish individual tracking method and device, electronic equipment and storage medium

Through the twin network structure and coordinate attention mechanism in the fish individual tracking method, combined with the AlexNet network and the regional suggestion network, the problems of low accuracy and low efficiency of fish individual tracking in underwater videos are solved, and high accuracy and fast fish individual tracking.

CN118941835APending Publication Date: 2024-11-12CHINA AGRI UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310531575.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing fish individual tracking methods in underwater videos have low accuracy and low efficiency due to high blur, occlusion and fish similarity, which cannot meet the real-time requirements.

Method used

The twin network structure and coordinate attention mechanism are used to combine the AlexNet network to enhance the denoising processing and feature matching through image enhancement, fish characteristics in fish images are extracted, and regional suggestions networks are used to classify the location information of the target fish.

Benefits of technology

It improves the accuracy and speed of individual fish tracking, and can obtain the precise location of the target fish in a fuzzy and severely obstructed environment underwater to meet the requirements of real-time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941835A_ABST
    Figure CN118941835A_ABST
Patent Text Reader

Abstract

The invention provides a fish individual tracking method and device, electronic equipment and a storage medium. The fish individual tracking method comprises the following steps: acquiring a fish video; splitting the fish video into multiple frames of fish images, and performing image enhancement denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; fish features in the multiple frames of clear fish images are extracted; according to the method, the fish features in each frame of clear fish image are classified to obtain the position information of the target fish in each frame of fish image, so that the problems of fuzzy underwater video, serious shielding, high fish similarity and difficult identification and tracking are solved, the accurate position of the target fish can be obtained, the tracking precision is high, the tracking speed is high, and the real-time requirement can be met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a method, device, electronic equipment and storage medium for tracking individual fish. Background Art

[0002] In aquaculture, monitoring the growth conditions and behavior of fish is an important part of the welfare farming process. As an effective way to monitor fish, tracking technology can track early diseased individuals or potential diseased individuals to ensure early detection of diseases and continuous real-time monitoring of the health of fish. By tracking individual fish, we can also obtain information on water quality, biomass estimation, etc. Therefore, the study of individual tracking of fish will become the key to improving the level of refined farming and is of great significance to promoting the development of smart fisheries. Traditional fish tracking and monitoring often use technologies such as acoustics and sensors. Acoustic technology requires a small photoacoustic tag to be placed in the abdominal cavity of the fish (or attached to the outside of the fish) to accurately track the trajectory of the fish. The ultrasonic signal emitted by the tag is received by a receiver placed at a specific location underwater. Acoustics are widely used in spatiotemporal distribution behavior, species detection, fish resource assessment and fish tracking without causing damage to the fish. However, acoustic methods are easily affected by noise, high cost, and technical difficulties further limit their application. Various sensors based on different parameters can be used to track and identify fish behavior, but they need to be fixed to the fish. However, as modern aquaculture is increasingly dominated by welfare farming, such invasive monitoring methods are unlikely to be applicable to large-scale intensive farming.

[0003] In the related art, computer vision technology is applied to fish tracking and monitoring in aquaculture. However, most of the current research on fish tracking is aimed at multiple targets. For example, by using convolutional neural networks to detect fish heads and identifying the head image of each fish in each frame to associate cross-frame data, a large number of fish can be reliably tracked throughout the entire time. However, in practical applications, due to the blur of underwater videos, severe occlusion, and high similarity of fish, it is difficult to track individual fish, and directly applying multi-target tracking algorithms to individual tracking problems may result in low accuracy and efficiency, and cannot meet real-time requirements. Therefore, there is an urgent need to study an effective method for individual fish tracking. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for tracking individual fish, which are used to solve the defects of low accuracy, low efficiency and failure to meet real-time requirements when a computer vision multi-target recognition method is used for tracking individual fish.

[0005] The present invention provides a method for tracking individual fish, comprising:

[0006] Get fish videos;

[0007] Splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images;

[0008] Extracting fish features from the multiple frames of clear fish images;

[0009] The fish features in each frame of clear fish image are classified to obtain the position information of the target fish in each frame of fish image.

[0010] According to a method for tracking individual fish provided by the present invention, the multiple frames of clear fish images include template frame fish images and search frame fish images, and the step of extracting fish features from the multiple frames of clear fish images includes:

[0011] Extracting fish features in the multiple frames of clear fish images through a twin network structure, wherein the twin network structure includes a first branch network and a second branch network;

[0012] The first branch network is used to extract fish features in the template frame fish image;

[0013] The second branch network is used to extract fish features in the search frame fish image.

[0014] According to a method for tracking individual fish provided by the present invention, after the first branch network extracts the fish features in the template frame fish image, the method further includes:

[0015] The fish features output by the first branch network are calculated by the coordinate attention mechanism to extract the deep features of the target fish in the template frame;

[0016] The fish features in the search frame fish image are matched with the target fish depth features to obtain the position information of the target fish in the search frame fish image.

[0017] According to a method for tracking individual fish provided by the present invention, the fish features output by the first branch network are calculated by a coordinate attention mechanism to extract the target fish depth features in the template frame, including:

[0018] Aggregate the fish features of the target fish along the vertical and horizontal directions into a vertical direction perception feature map and a horizontal direction perception feature map respectively;

[0019] Encoding the vertical direction perception feature map and the horizontal direction perception feature map to obtain a vertical direction attention map and a horizontal direction attention map;

[0020] The vertical position information of the target fish is obtained according to the vertical attention map, and the horizontal position information of the target fish is obtained according to the horizontal attention map, and the vertical position information and horizontal position information of the target fish are used as the depth features of the target fish.

[0021] According to a fish individual tracking method provided by the present invention, the first branch network and the second branch network are AlexNet networks.

[0022] According to a method for tracking individual fish provided by the present invention, the feature classification is performed according to the fish features to obtain the position information of the target fish in each frame of the fish image, including:

[0023] The fish features are classified through the classification branch of the region proposal network to obtain the initial location information of the target fish;

[0024] The initial position information of the target fish is regressed by the region proposal network regression branch to obtain the corrected position information of the target fish, and the corrected position information of the target fish is used as the position information of the target fish.

[0025] According to a method for tracking individual fish provided by the present invention, performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images includes:

[0026] Dividing at least one continuous non-overlapping sub-region on each frame of the fish image, wherein the size of the sub-region is determined according to a preset local detail enhancement strength;

[0027] Calculate the grayscale histogram and corresponding transformation function of each sub-region;

[0028] Use histogram equalization to equalize each sub-region;

[0029] Performing bilinear interpolation processing on the pixel points in the equalized sub-region according to the transformation function;

[0030] A clear fish image is obtained based on the grayscale value of the pixel after bilinear interpolation processing.

[0031] The present invention also provides a fish individual tracking device, comprising:

[0032] Acquisition module, used to acquire fish videos;

[0033] A processing module, used for splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images, and obtaining multiple frames of clear fish images;

[0034] An extraction module, used for extracting fish features from the multiple frames of clear fish images;

[0035] The classification module is used to classify the fish features in each frame of clear fish image to obtain the position information of the target fish in each frame of fish image.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for tracking individual fish as described above is implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for tracking individual fish as described above is implemented.

[0038] The individual fish tracking method, device, electronic device and storage medium provided by the present invention obtain fish videos; split the fish videos into multiple frames of fish images, perform image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; extract fish features in the multiple frames of clear fish images; perform feature classification on the fish features in each frame of the clear fish images to obtain the position information of the target fish in each frame of the fish image, thereby overcoming the problems of difficulty in identifying and tracking such as blurred underwater videos, severe occlusion, and high similarity of fish, and the precise position of the target fish can be obtained, with high tracking accuracy and fast speed, and can meet real-time requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is one of the flow charts of the method for tracking individual fish provided by the present invention;

[0041] Figure 2 This is the second flow chart of the method for tracking individual fish provided by the present invention;

[0042] Figure 3 This is the third flow chart of the method for tracking individual fish provided by the present invention;

[0043] Figure 4 This is the fourth flow chart of the method for tracking individual fish provided by the present invention;

[0044] Figure 5 It is a structural schematic diagram of the fish individual tracking device provided by the present invention;

[0045] Figure 6It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Figure 1 A flow chart of a method for tracking individual fish provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method for tracking individual fish provided by the embodiment of the present invention includes:

[0048] Step 101, obtaining fish videos;

[0049] In the embodiment of the present invention, the camera is installed above the side of the fish pond so that the camera angle of view can cover the entire fish pond, and the obtained video image presents the movement state of the entire school of fish. The camera position is changed to shoot fish videos at different angles.

[0050] In some embodiments of the present invention, the camera viewing angle is also controlled by logging into the APP so that the camera viewing angle can cover the entire fish pond.

[0051] Step 102, splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images, and obtaining multiple frames of clear fish images;

[0052] An image dataset is constructed based on video images, which contains fish data in high-density real breeding scenes and experimental scenes. Each image in the image dataset contains multiple fish, and the image data has blur, similarity interference, occlusion, scale changes, etc.

[0053] Step 103, extracting fish features from multiple frames of clear fish images;

[0054] Step 104: classify the fish features in each frame of clear fish image to obtain the position information of the target fish in each frame of fish image.

[0055] In an embodiment of the present invention, feature classification is performed according to fish features to obtain position information of target fish in each frame of fish image, including:

[0056] Step 1041, classifying fish features through the region proposal network classification branch to obtain the initial position information of the target fish;

[0057] Step 1042: regress the initial position information of the target fish through the region proposal network regression branch to obtain the corrected position information of the target fish, and use the corrected position information of the target fish as the position information of the target fish.

[0058] In the embodiment of the present invention, the region proposal network is also a convolutional neural network. Different from the precise classification of the entire network, the region proposal network only distinguishes the regions where targets may appear, that is, they are divided into two categories: with or without targets. The first several layers of the region proposal network can be regarded as the backbone network for feature extraction, and the subsequent several layers are responsible for predicting the scores and positions of possible target regions.

[0059] Traditional fish tracking methods are mostly multi-target. For example, using convolutional neural networks to detect fish heads and associate cross-frame data by identifying the head image of each fish in each frame can reliably track a large number of fish throughout time. However, in practical applications, due to the blur of underwater videos, severe occlusion, and high similarity of fish, it is difficult to track individual fish, and directly applying multi-target tracking algorithms to individual tracking problems may result in low accuracy and efficiency, and cannot meet real-time requirements.

[0060] The individual fish tracking method provided by the embodiment of the present invention obtains a fish video; splits the fish video into multiple frames of fish images, performs image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; extracts fish features in the multiple frames of clear fish images; and performs feature classification on the fish features in each frame of the clear fish image to obtain the position information of the target fish in each frame of the fish image, thereby overcoming the problems of underwater video blur, severe occlusion, high fish similarity and other difficulties in identification and tracking, and can obtain the precise position of the target fish, with high tracking accuracy and fast speed, and can meet real-time requirements.

[0061] Based on any of the above embodiments, multiple frames of clear fish images include a template frame fish image and a search frame fish image, the template frame is usually the first frame of the video, and the search frame is usually the image frame after the first frame. In an embodiment of the present invention, fish features in multiple frames of clear fish images are extracted, including: extracting fish features in multiple frames of clear fish images through a twin network structure, the twin network structure includes a first branch network and a second branch network; the first branch network is used to extract fish features in the template frame fish image; the second branch network is used to extract fish features in the search frame fish image.

[0062] In the embodiment of the present invention, the first branch network and the second branch network are AlexNet networks.

[0063] The twin network algorithm can solve the problems of insufficient data and poor real-time performance in the field of target tracking. The twin network algorithm simplifies the similarity calculation process and greatly improves the tracking speed.

[0064] like Figure 2 As shown, after the first branch network extracts the fish features in the template frame fish image, it also includes:

[0065] Step 201, performing coordinate attention mechanism calculation on the fish features output by the first branch network to extract the target fish depth features in the template frame;

[0066] Step 202: feature matching is performed between the fish features in the search frame fish image and the depth features of the target fish to obtain the position information of the target fish in the search frame fish image.

[0067] In an embodiment of the present invention, the fish features output by the first branch network are calculated by a coordinate attention mechanism, and the deep features of the target fish in the template frame are extracted, including:

[0068] Step 211, aggregating the fish features of the target fish along the vertical and horizontal directions into a vertical direction perception feature map and a horizontal direction perception feature map respectively;

[0069] Step 212: Encode the vertical direction perception feature map and the horizontal direction perception feature map to obtain a vertical direction attention map and a horizontal direction attention map;

[0070] Step 2013: obtain the vertical position information of the target fish according to the vertical attention map, obtain the horizontal position information of the target fish according to the horizontal attention map, and use the vertical position information and horizontal position information of the target fish as the depth feature of the target fish.

[0071] Based on any of the above embodiments, Figure 3 As shown, multiple frames of fish images are subjected to image enhancement and denoising, and multiple frames of clear fish images are obtained, including:

[0072] Step 301: Divide each frame of the fish image into at least one continuous non-overlapping sub-region, where the size of the sub-region is determined according to a preset local detail enhancement strength;

[0073] Step 302: Calculate the grayscale histogram and corresponding transformation function of each sub-region;

[0074] Step 303: use histogram equalization to equalize each sub-region;

[0075] Step 304: perform bilinear interpolation processing on the pixels in the equalized sub-region according to the transformation function;

[0076] In the embodiment of the present invention, the bilinear interpolation formula is as follows:

[0077] G(i)=a[bF 1 (i)+(1-b)F 2 (i)]+(1-a)[bF 3 (i)+(1-b)F 4 (i)]

[0078] Among them, G(i) is the gray value of the pixel point (x, y) in the fish image, and the coordinates of the pixels around this point are A 1 (x 1 ,y 1 ),A 2 (x 2 ,y 1 ),A 3 (x 2 ,y 2 ) and A 4 (x 1 ,y 2 ), where A 1 (x 1 ,y 1 ) is the pixel closest to point (x, y), A 3 (x 2 ,y 2 ) is the pixel farthest from point (x, y), A 2 (x 2 ,y 1 ) and A 4 (x 1 ,y 2 ) are the other two pixel points. These four points can form a rectangle. F 1 (i),F 2 (i),F 3 (i) and F 4 (i) represents A 1 (x 1 ,y 1 ),A 2 (x 2 ,y 1 ),A 3 (x 2 ,y 2 ) and A 4 (x 1 ,y 2 )’s grayscale value.

[0079] Step 305: Obtain a clear fish image according to the grayscale values ​​of the pixels after bilinear interpolation processing.

[0080] In the embodiment of the present invention, Figure 4 As shown, the individual fish tracking method specifically includes: obtaining fish data images, dividing the fish data images into template frames and search frames, performing image enhancement processing on the template frames and search frame images, inputting the enhanced images into different branches of the AlexNet network respectively, performing coordinate attention mechanism calculation on the template frame processing result output by one branch, and then inputting into the region proposal network, and the search frame processing result output by another branch is input into the region proposal network, and finally obtaining the position information of the target fish in each search frame image.

[0081] In some embodiments of the present invention, the trained twin network and region proposal network are implanted into a monitoring platform, and tracking data management is completed through the monitoring platform. Specifically, the monitoring platform can be used to visualize the fish tracking data, and the display content includes but is not limited to:

[0082] (1) Fish distribution map: The spatial position and movement trajectory of fish schools are displayed on the underwater map, so that users can quickly understand the distribution and activities of fish schools.

[0083] (2) Fish density map: Use color coding to display the density distribution of fish schools. Different colors represent different density levels to help users quickly analyze and understand the activity patterns of fish schools.

[0084] (3) Fish population chart: Use bar charts or line charts to show the population trends of different fish populations, so that users can understand the population and changes of fish populations.

[0085] (4) Monitoring charts: Use chart types such as line charts or time series charts to display the changing trends of various sensor data to help users understand the changes in the underwater environment.

[0086] The monitoring platform establishes a search index for fish tracking data to facilitate users to quickly search. The specific search methods include:

[0087] (1) Time search: Users can select a specific time period to view the fish behavior data within that time period.

[0088] (2) Spatial search: Users can select a specific area and view the fish behavior data in that area.

[0089] (3) Fish species screening: Users can quickly search for specific fish behavior data by selecting a specific fish species or fish type.

[0090] (4) Sorting and filtering: Users can sort and filter data based on specific attributes, such as fish population, density, activity patterns, etc., to quickly find specific fish behavior data.

[0091] The individual fish tracking method provided by the embodiment of the present invention can be used for real-time tracking and monitoring of underwater fish in factory-scale recirculating aquaculture. In order to solve the problems of similarity between underwater targets and backgrounds and blurred images, a contrast-limited adaptive histogram equalization method is used to enhance the contrast of underwater fish images while suppressing noise; a coordinate attention mechanism is added to fuse the shallow features and deep features of the model to further enrich the features and increase the model accuracy; the two indicators of accuracy and success rate are used to measure the effect of fish tracking, which can overcome the problems of blurred underwater images of fish in aquaculture, severe occlusion, high similarity, etc., and obtain accurate recognition of individual fish behaviors. The method has high tracking accuracy and fast speed and can meet real-time requirements.

[0092] The individual fish tracking device provided by the present invention is described below. The individual fish tracking device described below and the individual fish tracking method described above can be referenced to each other.

[0093] Figure 5 A schematic diagram of a fish individual tracking device provided by an embodiment of the present invention, such as Figure 5 As shown, the fish individual tracking device provided by the embodiment of the present invention includes:

[0094] An acquisition module 501 is used to acquire fish videos;

[0095] A processing module 502 is used to split the fish video into multiple frames of fish images, and perform image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images;

[0096] An extraction module 503 is used to extract fish features from the multiple frames of clear fish images;

[0097] The classification module 504 is used to classify the fish features in each frame of clear fish image to obtain the position information of the target fish in each frame of fish image.

[0098] In some embodiments of the present invention, the individual fish tracking device is arranged in a monitoring platform, and the monitoring platform also includes a camera system unit, an image processing unit, a tracking algorithm unit, a data storage and management unit, and a user interface unit.

[0099] Camera system unit: The quality of the camera system unit is directly related to the tracking accuracy and system performance. Select a camera with relatively high resolution, good frame rate and sensitivity to obtain underwater video stream data.

[0100] Image processing unit: The image processing unit is responsible for extracting fish features from image data, such as shape, color, motion trajectory, etc., for subsequent tracking and analysis. Considering the impact of the underwater environment and the characteristics of fish, the contrast-limited adaptive histogram equalization method is used to enhance the contrast of underwater fish images while suppressing noise. Then the AlexNet network is used to extract fish features.

[0101] Tracking algorithm unit: The tracking algorithm unit is responsible for real-time tracking and recording of fish. In view of the fish needs in aquaculture environment, a region proposal network is used to output individual fish tracking results.

[0102] Data storage and management unit: The data storage and management unit is responsible for storing the tracked fish data in a database or file system, and providing corresponding query and statistical functions for subsequent analysis and research.

[0103] User interface unit: Provides users with a friendly and easy-to-use graphical interface to facilitate user operation and management of the system. In addition, the user interface unit can also support data visualization and quick search functions to facilitate users to analyze and study system data.

[0104] The individual fish tracking device provided by the embodiment of the present invention obtains a fish video; splits the fish video into multiple frames of fish images, performs image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; extracts fish features in the multiple frames of clear fish images; and performs feature classification on the fish features in each frame of the clear fish image to obtain the position information of the target fish in each frame of the fish image, thereby overcoming the problems of difficulty in identifying and tracking such as blurred underwater videos, severe occlusion, and high similarity of fish, and can obtain the precise position of the target fish, with high tracking accuracy and fast speed, and can meet real-time requirements.

[0105] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for tracking individual fish as described above is implemented.

[0106] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the fish individual tracking method, which includes: obtaining a fish video; splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; extracting fish features in the multiple frames of clear fish images; performing feature classification on the fish features in each frame of clear fish images to obtain the position information of the target fish in each frame of fish images, overcoming the problems of underwater video blur, severe occlusion, high fish similarity and other difficult identification and tracking problems, and obtaining the precise position of the target fish, with high tracking accuracy and fast speed, which can meet the real-time requirements.

[0107] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0108] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the individual fish tracking method provided by the above-mentioned methods. The method includes: obtaining a fish video; splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; extracting fish features in the multiple frames of clear fish images; performing feature classification on the fish features in each frame of the clear fish image to obtain the position information of the target fish in each frame of the fish image, thereby overcoming the problems of underwater video blur, severe occlusion, high fish similarity and other difficulties in identification and tracking, and being able to obtain the precise position of the target fish, with high tracking accuracy and fast speed, and being able to meet real-time requirements.

[0109] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for tracking individual fish, characterized in that: include: Get fish videos; Splitting the fish video into multiple frames of fish images, performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images; Extracting fish features from the multiple frames of clear fish images; The fish features in each frame of clear fish image are classified to obtain the position information of the target fish in each frame of fish image.

2. The method for tracking individual fish according to claim 1, wherein: The multiple frames of clear fish images include template frame fish images and search frame fish images, and extracting fish features from the multiple frames of clear fish images includes: Extracting fish features from the multiple frames of clear fish images through a twin network structure, wherein the twin network structure includes a first branch network and a second branch network; The first branch network is used to extract fish features in the template frame fish image; The second branch network is used to extract fish features in the search frame fish image.

3. The method for tracking individual fish according to claim 2, wherein: After the first branch network extracts the fish features in the template frame fish image, the method further includes: The fish features output by the first branch network are calculated using a coordinate attention mechanism to extract the depth features of the target fish in the template frame; The fish features in the search frame fish image are matched with the target fish depth features to obtain the position information of the target fish in the search frame fish image.

4. The method for tracking individual fish according to claim 3, wherein: The performing of coordinate attention mechanism calculation on the fish features output by the first branch network to extract the target fish depth features in the template frame includes: Aggregating the fish features of the target fish along the vertical and horizontal directions into a vertical direction perception feature map and a horizontal direction perception feature map respectively; Encoding the vertical direction perception feature map and the horizontal direction perception feature map to obtain a vertical direction attention map and a horizontal direction attention map; The vertical position information of the target fish is obtained according to the vertical attention map, and the horizontal position information of the target fish is obtained according to the horizontal attention map, and the vertical position information and horizontal position information of the target fish are used as the depth features of the target fish.

5. The method for tracking individual fish according to claim 2, wherein: The first branch network and the second branch network are AlexNet networks.

6. The method for tracking individual fish according to claim 1, wherein: The performing feature classification according to the fish features to obtain the position information of the target fish in each frame of the fish image includes: Fish features are classified through the classification branch of the region proposal network to obtain the initial location information of the target fish; The initial position information of the target fish is regressed by the regression branch of the region proposal network to obtain the corrected position information of the target fish, and the corrected position information of the target fish is used as the position information of the target fish.

7. The method for tracking individual fish according to claim 1, wherein: The performing image enhancement and denoising processing on the multiple frames of fish images to obtain multiple frames of clear fish images comprises: Dividing each frame of the fish image into at least one continuous non-overlapping sub-region, wherein the size of the sub-region is determined according to a preset local detail enhancement strength; Calculate the grayscale histogram and corresponding transformation function of each sub-region; Use histogram equalization to equalize each sub-region; Performing bilinear interpolation processing on the pixel points in the equalized sub-region according to the transformation function; A clear fish image is obtained based on the grayscale value of the pixel after bilinear interpolation processing.

8. A device for tracking individual fish, characterized in that: include: Acquisition module, used to acquire fish videos; A processing module is used to split the fish video into multiple frames of fish images, and perform image enhancement and denoising on the multiple frames of fish images to obtain multiple frames of clear fish images; An extraction module, configured to extract fish features from the multiple frames of clear fish images; The classification module is used to classify the fish features in each frame of clear fish image to obtain the position information of the target fish in each frame of fish image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the individual fish tracking method according to any one of claims 1 to 7 is implemented.

10. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for tracking individual fish as claimed in any one of claims 1 to 7 is implemented.

Citation Information

Cited By

  • Multi-view underwater sheltered fish behavior identification and tracking method and device

    CN121305616A