An apparatus and method for fish biological intelligent recognition tracking and state determination

Abstract

The embodiment of the present disclosure provides a device and method for fish biological intelligent recognition tracking and state judgment, which is used for collecting and recognizing fish group information of a water area to be measured, tracking and recognizing a platform, and a water area processing assembly, a bait throwing assembly and an identification assembly arranged on the tracking and recognizing platform respectively; the tracking and recognizing platform is arranged on the bank of the water area to be measured; the bait throwing assembly is used for throwing bait to the water area to be measured to attract fish groups; the water area processing assembly is arranged in the water area to be measured to disturb the water flow of the water area; and the identification assembly is used for collecting fish group videos of the water area to be measured and identifying the species and growth state information of each fish in the fish group videos. The device and method of the present application obtain a large amount of original data through long-time identification tracking, and efficiently obtain the fish species and fish state information in the water area to be measured based on the identification assembly.

Description

Technical Field

[0001] This disclosure relates to the field of fish identification technology, and in particular to an apparatus and method for intelligent identification, tracking and status determination of fish, as well as a computer-readable storage medium. Background Technology

[0002] In the fish farming industry, the growth status of fish and the water quality directly affect the final yield of aquatic products. The growth status of fish can only be detected through manual sampling, which is difficult to implement and the results obtained from the samples are not universally applicable.

[0003] On the other hand, for larger rivers and lakes, it is impossible to monitor fish species using traditional manual sampling methods. Existing dynamic image monitoring technology can only make a certain judgment on the number of fish, but cannot obtain effective information on the actual status of fish species. It is difficult to judge the growth of fish species and water quality in a certain body of water based on the collected data. Summary of the Invention

[0004] The purpose of this disclosure is to provide an apparatus and method for intelligent identification, tracking, and status determination of fish organisms, as well as a computer-readable storage medium, thereby solving the aforementioned problems existing in the prior art.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this disclosure are as follows:

[0006] This disclosure provides an apparatus for intelligent identification, tracking, and status assessment of fish organisms, used to collect and identify fish population information in a water area to be tested. The apparatus includes a tracking and identification platform, and a water treatment component, a bait delivery component, and an identification component respectively disposed on the tracking and identification platform.

[0007] A tracking and identification platform, designed to be installed on the shoreline of the water area to be measured;

[0008] The bait delivery device is used to deliver bait into the waters to be tested in order to attract fish.

[0009] A water treatment component is used to be installed in the water area to drive the water flow.

[0010] The identification component is used to collect videos of fish schools in the waters to be tested and to identify the species and growth status of each fish in the video.

[0011] Optionally, the tracking and identification platform includes: a vertical support and a horizontal support, one end of the vertical support is used to be set on the bank of the water area to be measured, and the other end is connected to the horizontal support;

[0012] The horizontal support is located above the water area, and the horizontal support is equipped with the identification component and the bait delivery component.

[0013] Optionally, the tracking and identification platform further includes: a vertical mounting plate, one end of which is connected to the horizontal support, and the other end of which is used to extend into the water.

[0014] A connecting plate is provided between the vertical mounting plate and the vertical bracket, so a support plate is provided on the other end of the vertical mounting plate away from the shore of the water.

[0015] The water treatment component includes a water pump, a pumping pipe, and a drain pipe; the water pump is mounted on the support plate, the water inlet of the water pump is connected to one end of the pumping pipe, the water outlet is connected to one end of the drain pipe, the other end of the pumping pipe is close to the shore of the water area, and the other end of the drain pipe is far away from the shore of the water area.

[0016] Optionally, the axial direction of the water pump is perpendicular to the vertical mounting plate, its inlet faces the shore of the water area, and its outlet is away from the shore of the water area.

[0017] The other end of the pumping pipe is threaded through the support plate and extends along the direction of the waterway shoreline.

[0018] The other end of the drainage pipe extends away from the shore of the water, and the drainage pipe is at a preset angle to the vertical mounting plate.

[0019] Optionally, the bait delivery assembly includes: a bait trough, a channel, a solenoid valve, a float, and a partition net;

[0020] The channel passes through the horizontal support; the feed trough is provided at one end of the channel above the horizontal support, the other end of the channel is used to extend into the water, the float is provided at the other end of the channel, and the partition net is provided at the other end of the channel; the solenoid valve is provided in the other end of the channel for adjusting the opening and closing of the channel.

[0021] Optionally, the identification component includes: an image acquisition device, a lighting device, and a scanner respectively disposed on the horizontal support, wherein the image acquisition device is used to acquire video of the water area to be measured;

[0022] The lighting device is set within a preset distance range of the image acquisition device and is used to provide lighting for the image acquisition device when acquiring video or to attract fish.

[0023] The scanner is electrically connected to both the lighting device and the image acquisition device to control the lighting device to start and stop, control the image acquisition device to acquire video, and perform identification and analysis on the video to obtain the species and growth status of each fish in the video. The scanner is equipped with a visual recognition module. When the bait delivery component includes a solenoid valve, the scanner is electrically connected to the solenoid valve to control the opening and closing of the solenoid valve.

[0024] Another aspect of this disclosure provides a method for intelligent identification, tracking, and status determination of fish organisms, the method comprising:

[0025] The aforementioned device is installed, and the tracking and identification platform is set up on the shore of the water area to be measured;

[0026] The water treatment component and the identification component are activated respectively. The water treatment component drives the water flow in the water area, so that the water near the shore is pumped in and discharged in the direction away from the shore.

[0027] The bait delivery component releases bait into the water, and the water treatment component drives the water flow, causing the bait to spread throughout the water to attract fish.

[0028] The identification component collects video of the water area to be tested, and controls the frequency of bait delivery by the bait delivery component based on the acquisition frequency of the original images to fully obtain fish information.

[0029] Another aspect of this disclosure provides a method for intelligent identification, tracking, and status determination of fish organisms, the method comprising:

[0030] Acquire fish videos of a school of fish in a body of water for which fish to be tracked and identified, wherein the fish videos are acquired using an identification component of a device constructed on the shore of the water as described above;

[0031] Each image frame in the video is sequentially input into a pre-trained fish state comparison model to obtain the species and state result of each fish in the corresponding image; wherein, the fish state comparison model is trained based on a YOLOv5 network;

[0032] Determine if the fish are in an abnormal state. If so, mark the abnormal fish and save the image of the abnormal fish; otherwise, save the image of the normal fish.

[0033] Optionally, the training process of the fish state control model includes:

[0034] Obtain an image dataset containing at least one fish species, and label each fish in the dataset with its bounding rectangle, species or type, and status, where the status includes normal and abnormal.

[0035] The convolutional neural network is trained using the labeled image dataset of fish schools to obtain a trained fish state comparison model.

[0036] Another aspect of this disclosure provides an apparatus for intelligent identification, tracking, and status determination of fish organisms, the apparatus comprising:

[0037] The acquisition module is used to acquire fish videos of the water area to be tested, wherein the fish videos are acquired based on the recognition components of the device described above, which is built on the shore of the water area to be tested.

[0038] The detection module is used to sequentially input each image frame in the video into a pre-trained fish state comparison model to obtain the species and state result of each fish in the corresponding image; wherein, the fish state comparison model is trained based on a YOLOv5 network.

[0039] The judgment module is used to determine whether the fish are in an abnormal state. If so, the abnormal fish are marked and the image of the abnormal fish is saved; otherwise, the image of the normal fish is saved.

[0040] Another aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for intelligent identification, tracking, and status determination of fish organisms as described above.

[0041] The beneficial effects of the embodiments disclosed herein are:

[0042] The device for intelligent identification, tracking, and status judgment of fish organisms in this embodiment has a simple structure and efficiently realizes the judgment of the growth status of fish organisms in the water area to be tested. The water treatment component, namely the identification area processing system, performs flow guidance treatment on the bottom of the water area to be tested, and the bait delivery component delivers bait to attract fish, which improves the efficiency of obtaining effective data of the water area to be tested. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of a structure for intelligent identification, tracking, and status determination of fish organisms according to an embodiment of this disclosure;

[0044] Figure 2 for Figure 1 Schematic diagram of the water treatment component structure;

[0045] Figure 3 for Figure 1 Schematic diagram of the bait delivery component structure;

[0046] Figure 4 This is a schematic flowchart of a method for intelligent identification, tracking, and status determination of fish organisms proposed in an embodiment of this disclosure;

[0047] Figure 5 This is a schematic flowchart of another method for intelligent identification, tracking, and status determination of fish organisms proposed in this embodiment of the present disclosure;

[0048] Figure 6 This is a schematic diagram of another device structure for intelligent identification, tracking, and status determination of fish organisms proposed in an embodiment of this disclosure.

[0049] In the diagram: 1. Tracking and identification platform; 2. Water treatment components; 3. Bait delivery components; 4. Image acquisition equipment; 5. Scanner; 6. Lighting equipment; 7. Tray; 8. Water pump; 9. Pumping pipe; 10. Drainage pipe; 11. Bait trough; 12. Channel; 13. Float; 14. Net. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this disclosure and are not intended to limit the embodiments of this disclosure.

[0051] like Figure 1 As shown, one embodiment of this disclosure proposes a device for intelligent identification, tracking, and status judgment of fish organisms, used to collect and identify fish information in a test water area. The device includes: a tracking and identification platform 1, and a water area processing component 2, a bait delivery component 3, and an identification component (not shown in the figure) respectively disposed on the tracking and identification platform; the tracking and identification platform is used to be disposed on the shore of the test water area; the bait delivery component is used to deliver bait into the test water area to attract fish; the water area processing component 2 is used to be disposed in the test water area to disturb the water flow; and the identification component is used to collect video of fish in the test water area and identify the species and growth status information of each fish in the video.

[0052] The device of this embodiment is set in the water area to be tested. Multiple devices can be set according to the actual situation. The device is convenient for collecting videos of fish in the water area to be tested, and can identify the species and growth status information of each fish from the video. Specifically, it identifies the species and size of each fish, and obtains the corresponding fish growth status according to the species and size of each fish. The growth status refers to abnormal or normal.

[0053] The water treatment component drives the water environment of the test area according to the actual situation, keeping the water continuously flowing. It can also use a baiting component to release bait, spreading it and increasing the probability of fish approaching. An identification component collects images of fish in the test area for subsequent identification and tracking. The identification component collects raw images of the fish in the test area. Specifically, the identification component acquires video and extracts images from it to identify each fish and track them within the captured video footage. The identification component acquires and saves video in real time, and then performs identification and analysis on the acquired video to extract and archive relevant content such as captured fish.

[0054] As a specific example of a tracking and identification platform, the tracking and identification platform includes: a vertical support (not shown in the figure) and a horizontal support (not shown in the figure). One end of the vertical support is used to be set on the shore of the water area to be measured, and a rib is provided on the side facing the shore. The other end is connected to the horizontal support. The horizontal support is located above the water area, and the horizontal support is provided with the identification component and the bait delivery component 3.

[0055] The device of this embodiment is installed on the shore of the water area to be measured to collect and identify fish schools near the shore. The tracking and identification platform has a simple structure and is easy to assemble. The vertical support has ribs on the side facing the shore; there can be two ribs, one on each side of the vertical support. The ribs on the shore strengthen the structure of the vertical support and prevent bending. The ribs can be right-angled triangles, with the two right-angled sides respectively located on the vertical support and the ground on the shore. The vertical support can be a vertical frame, or a horizontal frame. The holes in the triangles improve balance and make it more stable during use. One end of the tracking and identification platform can be directly embedded into the ground on the shore of the water area to be measured using long bolts. The specific fixing method can be determined according to the actual situation and will not be elaborated here.

[0056] like Figure 2 As shown, as a specific example of a tracking and identification platform, the tracking and identification platform further includes: a vertical mounting plate, one end of which is connected to the horizontal support, and the other end of which is used to extend into the water area; a connecting plate is provided between the vertical mounting plate and the vertical support, so a support plate is provided on the side of the other end of the vertical mounting plate away from the water area shore; the water treatment component includes: a water pump, a pumping pipe, and a draining pipe; the water pump is disposed on the support plate, the inlet of the water pump is connected to one end of the pumping pipe, the outlet of the water pump is connected to one end of the draining pipe, the other end of the pumping pipe is close to the water area shore, and the other end of the draining pipe is far away from the water area shore.

[0057] In this embodiment, the vertical mounting plate should be longer than the vertical support. One end of the vertical support is positioned on the shore, and the other end of the vertical mounting plate is positioned in the water. The length of the vertical mounting plate is at least longer than the length of the vertical support above the ground. The vertical mounting plate is also a vertical mounting plate. Specifically, the lengths of the vertical support and the vertical mounting plate can be set according to actual conditions, as can the distance between the vertical mounting plate and the shore, and the distance between the vertical mounting plate and the vertical support.

[0058] The connecting plate ensures the stability of the tracking and identification platform structure. Regardless of the water pump configuration, to increase the flow area of ​​the driving water, the pumping outlet of the pumping pipe is close to the shore, while the drain outlet of the draining pipe is far from the shore.

[0059] Preferably, there are two connecting plates, with one end of each connecting plate close to one of the vertical bracket and the vertical mounting plate, and the other end of each connecting plate spaced at a preset distance from the other of the vertical mounting plate.

[0060] As a specific example of a water treatment component, the pump's axis is perpendicular to the vertical mounting plate, its inlet faces the water's edge, and its outlet is away from the water's edge; the other end of the pumping pipe passes through the support plate and extends along the water's edge; the other end of the drainage pipe extends away from the water's edge, and the drainage pipe is at a predetermined angle to the vertical mounting plate.

[0061] Specifically, the extension line of the drainage pipe is perpendicular to the vertical mounting plate to increase the area for driving water flow. The water treatment component uses a water pump to drive water flow to the area to be tracked and identified. The water pump draws water from near the intake of the pumping pipe and discharges it towards the area to be tracked and identified. Ideally, the intake should not be directly opposite the bank. Multiple drainage outlets can be provided at the other end of the drainage pipe, distributed away from the bank; this is not limited. After the water pump is placed on a support plate, the pumping and drainage pipes at both ends are connected to the water area at the bottom. The water pump draws water near the bank and discharges it outwards into the water area.

[0062] The vertical mounting plate of the tracking and identification platform has a support plate at the bottom. A water pumping pipe is connected to the side of the support plate. One end of the water pumping pipe is fixedly connected to the inlet of the water pump, and the outlet of the water pump is connected to one end of the drain pipe. The water pump draws water from the side of the support plate through the water pumping pipe and then discharges it out through the drain pipe, thereby achieving the effect of water flow in this part.

[0063] like Figure 3As shown, as a specific example of a bait delivery component, the bait delivery component 3 includes: a bait trough 11, a channel 12, a solenoid valve, a float 13, and a partition net 14;

[0064] The channel 12 passes through the horizontal support; the feed trough 11 is provided at one end of the channel 12 above the horizontal support, the other end of the channel 12 is used to extend into the water, the float 13 is provided at the other end of the channel 12, and the partition net 14 is provided at the other end of the channel. The solenoid valve is provided in the other end of the channel for adjusting the opening and closing of the channel.

[0065] When using the bait delivery component of this embodiment, the bait is placed into the bait trough at the top, and the bait inside enters the inner side of the partition net along the bottom channel, directly immersing below the water surface. The bait delivery can be automatically controlled by setting a solenoid valve in the channel, which is electrically connected to the scanner at the back.

[0066] Specifically, the partition net can be positioned directly opposite the outlet of the drainage pipe. The float can reduce the tension on the horizontal support after bait is added. A channel 12 is inserted into the horizontal support of the tracking and identification platform. A bait trough 11 is located at the top of the channel 12. After bait is added into the bait trough 11, it can travel along the channel 12 into the bottom partition net 14. Because the bait is directly submerged in water, it can be melted by the water flow. After melting, the melted bait is dispersed by the water flow from the drainage outlet of the rear drainage pipe, spreading it outwards to attract fish. This process can be controlled based on the efficiency of the identification component in collecting video of fish in the water area. The usage status of the bait delivery component is controlled according to the frequency of the collected original images. This process can be controlled based on the efficiency of the image acquisition camera 4 in collecting photos of fish at the bottom. If no fish photos are detected for an extended period, the solenoid valve inside the channel can be opened to release bait.

[0067] As a specific example of the identification component, the identification component includes: an image acquisition device 4, a lighting device 6, and a scanner 5, respectively disposed on the horizontal support. The image acquisition device 4 is used to acquire video of the water area to be tested; the lighting device 6 is disposed within a preset distance range of the image acquisition device 4 to provide illumination or attract fish when the image acquisition device 4 acquires video; the scanner 5 is electrically connected to both the lighting device and the image acquisition device 4 to control the start and stop of the lighting device, control the image acquisition device 4 to acquire video, and perform identification and analysis on the video to obtain the species and growth status of each fish in the video. The scanner is equipped with a visual recognition module. When the bait delivery component includes a solenoid valve, the scanner is electrically connected to the solenoid valve to control the opening and closing of the solenoid valve.

[0068] like Figure 1 As shown, the horizontal support structure includes a first-stage support and a second-stage support. One end of the first-stage support is connected to both the vertical support and the vertical mounting plate. The other end of the first-stage support, facing the water surface, is connected to one end of the second-stage support via two fixing plates. The second-stage support is lower than the first-stage support, meaning it is closer to the water surface than the first-stage support. The two fixing plates can be tilted and arranged parallel to each other. The double-layer design of the horizontal support structure increases structural strength and improves aesthetics.

[0069] The image acquisition device and the lighting device's lamps are positioned on the side of the horizontal support facing the water surface, while the scanner is positioned on the side of the horizontal support away from the water surface, close to the image acquisition device. Specifically, the image acquisition device and the lighting device's lamps are positioned on the side of the second-stage support facing the water surface, and the scanner is positioned on the side of the second-stage support away from the water surface. The channel of the bait delivery component passes through the second-stage support. A fixing pipe is provided at the other end of the first-stage support, on the side away from the water surface. This fixing pipe is used to pass through a connecting pipe, and the connecting cable of the image acquisition device is located inside the connecting pipe and connected to the power supply. The connecting pipe is wound around the scanner and passes through the fixing pipe. The connecting pipe can be a flexible hose or a rigid pipe. The lighting device can also be connected to the power supply via a cable, and similarly, the cable can also be connected to the power supply via the connecting pipe.

[0070] The scanner is equipped with a recognition module, such as the K210 visual recognition module. This module uses edge computing to analyze and recognize images. The scanner can also embed a wireless communication module for wireless connection to a host computer. The image acquisition device may include a camera; the lighting device may include a searchlight.

[0071] In this embodiment, a searchlight 6 is installed at the end of the horizontal support of the tracking and identification platform, that is, at the end of the second-stage support away from the shore. The searchlight 6 illuminates the area to be collected at the bottom of the camera 4. A scanner 5 is placed on the horizontal support of the tracking and identification platform. After the camera 4 collects and captures videos of fish in the water below it, the scanner 5 completes subsequent feature comparison and status judgment.

[0072] The tracking and identification platform can also be equipped with water quality testing equipment, which can be installed on a vertical mounting plate or a horizontal support and suspended above the water surface. The water quality monitoring probe is immersed in the water to obtain water quality data.

[0073] like Figure 4 As shown, another aspect of this disclosure proposes a method for intelligent identification, tracking, and status determination of fish organisms, the method comprising:

[0074] Step S110: Set up the above-mentioned device and place the tracking and identification platform on the shore of the water area to be measured.

[0075] The method of this disclosure involves fixing a vertical bracket to the shore of the water area to be tracked and identified, and adjusting the image acquisition device of the identification component to cover the water area to be measured.

[0076] Step S120: Activate the water treatment component and the identification component respectively. The water treatment component drives the water flow in the water area, so that the water near the shore is pumped in and discharged in the direction away from the shore.

[0077] In step S120, the image acquisition device of the recognition component is turned on. Alternatively, the lighting device, i.e., the searchlight, installed at the end of the tracking and recognition platform can be turned on to illuminate the bottom of the image acquisition device of the recognition component, i.e., the camera, facilitating video capture by the camera. The lighting can also attract schools of fish.

[0078] Step S130: The bait delivery component delivers bait into the water area. The water treatment component drives the water flow, causing the bait to spread in the water area to attract fish.

[0079] This embodiment of the disclosure uses a baiting component to attract fish by releasing bait, and also utilizes the phototaxis of organisms to gather fish around the bottom of the image acquisition device, i.e., the camera. The lighting equipment and baiting can be turned on as needed.

[0080] Step S140: The identification component collects video of the water area to be tested, and controls the frequency of bait release by the bait release component according to the acquisition frequency of the original images to obtain fish information fully and accurately.

[0081] A conventional solenoid valve structure is installed inside the channel of the bait delivery component. The scanner is electrically connected to the solenoid valve. When the scanner detects that it cannot identify a fish photo after a preset time, it controls the solenoid valve inside the channel to open, releasing the bait from the bait trough into the water to attract fish. The preset time can be set according to actual conditions.

[0082] like Figure 5 As shown, another aspect of this disclosure proposes a method for intelligent identification, tracking, and status determination of fish organisms, the method comprising:

[0083] Step S210: Obtain a video of a school of fish in the waters where the fish to be tracked and identified are located, wherein the video of the school of fish is obtained based on the identification component of the device described above, which is installed on the shore of the waters.

[0084] Before step S210, fish can be attracted to the water area to be tested by lighting or baiting, and video of the water area to be tested can be obtained by the image acquisition device of the recognition component.

[0085] Step S220: Input each image frame in the video into the pre-trained fish state comparison model in sequence to obtain the species and state result of each fish in the corresponding image; wherein, the fish state comparison model is trained based on a YOLOv5 network.

[0086] In this embodiment, images can also be extracted from the video at a preset frame extraction frequency, which can speed up the detection process. The extracted images are then analyzed to obtain the species of each fish and its corresponding target detection box. The target detection box is the bounding rectangle of the fish, and the fish state is determined based on the size and species of the target detection box. The target detection box is considered the size of the fish. Images where no fish are detected are discarded, and the next image is selected.

[0087] Step S230: Determine if the fish are in an abnormal state. If the fish are in an abnormal state, mark the abnormal fish and save the image of the abnormal fish. If the fish are in a normal state, save the image of the normal fish.

[0088] As a specific example, after sequentially inputting the image frames from the video into the pre-trained fish state comparison model, the method further includes: determining whether fish state detection is in progress; if so, feeding back the fish state image; otherwise, continuing to input the next frame image to detect the state of each fish in the next frame image, and saving the final execution result.

[0089] This embodiment of the disclosure can perform detection and identification after the fish have grown up. Determining whether fish status detection is in progress can prevent the image from mistakenly identifying other branches or debris. Steps S220 and S230 of this embodiment of the disclosure are both implemented by the video recognition module in the scanner. The species and size of each fish in each frame of the image are determined. Based on the species and corresponding size, the growth status of the fish is judged. If a fish is too large or too small, exceeding the normal range, then the fish is growing abnormally. Every fish in a frame of the image needs to be detected, and abnormal fish are marked. If one fish is abnormal, the abnormal image is saved. If all the fish in the image are normal, the detection continues to the next frame of the image. If no fish are detected in the image, then the image is not saved.

[0090] As a specific example of a fish identification method, after acquiring a video of the fish school in the waters where the fish to be tracked and identified are located, the method further includes:

[0091] The frequency of bait deployment is controlled based on the acquisition frequency of the original images to regulate the number of fish collected. This continues until the acquisition frequency of the original images falls within a preset time range. Fish approaching the test area are then identified and tracked, and original images of the fish in that area are acquired. The original image frequency is the number of times the image acquisition device captures video.

[0092] As a specific example of a fish state control model, the training process of the fish state control model includes:

[0093] Step S221: Obtain an image dataset containing at least one fish species, and label each fish in the fish dataset with its bounding rectangle, species or type, and status, wherein the status includes normal and abnormal.

[0094] In step S221 of this embodiment, each fish in each image is independently labeled, and different anomaly types are labeled according to the acquired fish anomaly big data model. An image contains at least one fish; if there are multiple fish, each is labeled. The bounding rectangle is labeled based on experience, and the fish size is determined based on the area of ​​the labeled rectangle. Depending on the fish species, fish that are too large or too small, or that are incomplete, are considered abnormal. That is, for a certain type of fish, a fish whose size is greater than a first preset size or less than a second preset size, or that is incomplete, is considered abnormal; otherwise, it is considered normal. The size can be the length or width of the fish. This embodiment can label adult fish.

[0095] Step S222: Train the YOLOv5 network with the labeled image dataset to obtain a trained fish state control model.

[0096] The embodiments of this disclosure, based on the constructed device, can not only acquire videos of fish schools in water, but also identify the species and detect the status of individual fish in the fish school videos. The growth status of individual fish can be determined according to the size of the detection frame of each fish. Here, the growth status mainly refers to whether the individual is normal or abnormal, and the individual size is determined by the area of ​​the detection frame.

[0097] like Figure 6 As shown in the embodiments of this disclosure, a device for intelligent identification, tracking, and status determination of fish organisms is proposed, the device comprising:

[0098] The acquisition module 210 is used to acquire a video of a school of fish in the water area to be tested, wherein the video of the school of fish is acquired based on the recognition component of the device described above, which is built on the shore of the water area to be tested.

[0099] The detection module 220 is used to sequentially input each image frame in the video into a pre-trained fish state comparison model to obtain the species and state result of each fish in the corresponding image; wherein, the fish state comparison model is trained based on a YOLOv5 network.

[0100] The judgment module 230 is used to determine whether the fish are in an abnormal state. If so, the abnormal fish are marked and the image of the abnormal fish is saved; otherwise, the image of the normal fish is saved.

[0101] Both the detection module 220 and the judgment module 230 are located in the scanner.

[0102] Another aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above for intelligent identification, tracking, and status determination of fish organisms.

[0103] By adopting the above-described technical solutions disclosed in the embodiments of this disclosure, the following beneficial effects are obtained:

[0104] 1. In this method for intelligent tracking, identification and status judgment of fish organisms, the tracking and identification platform is set up on the shore of the water area to be tested. A large amount of raw video data is obtained through long-term tracking and identification, and detection is performed based on the trained fish status comparison model, so as to efficiently realize the judgment of the growth status of fish organisms and water quality data in the water area.

[0105] 2. The method for intelligent tracking, identification, and status judgment of fish organisms utilizes the identification area processing system on the tracking and identification platform to guide the flow at the bottom of the water body under test, thereby continuously flowing the water surface in the top identification area. This flow effect prevents non-fish organisms from obstructing or covering the tracking and identification area, providing protection for the tracking and identification area and thus improving the efficiency of the method in obtaining effective data.

[0106] 3. The method for intelligent tracking, identification, and status judgment of fish organisms is implemented by monitoring the water area near the shore. Therefore, identification is easy and makes it difficult for fish to approach. By setting a bait delivery component at the bottom of the tracking and identification area, the gathering of fish is accelerated, thereby improving the efficiency of the sample video acquired by the image acquisition device, i.e., the camera. The results of fish biological status acquired within a certain period of time are more accurate and reliable.

[0107] The following is a detailed description of the fish tracking, identification, and status determination device using embodiments of this disclosure, which is mainly implemented according to the following steps:

[0108] S1. First, collect and prepare the dataset required for training, which includes images of at least one fish.

[0109] The quality and quantity of the dataset significantly impact the training results. This process includes data acquisition and data preprocessing. Data acquisition involves obtaining data through various means, including downloading from the internet, directly importing existing professional datasets, or extracting images from a large number of videos using the device described in this embodiment. To ensure the data source is reliable and meets the training objectives, this embodiment uses a large number of fish state images downloaded from the internet. Data preprocessing involves building a suitable model architecture. The model architecture used in this embodiment includes convolutional neural networks (CNNs) and recurrent neural networks (RNNs).

[0110] S2. Manually label the dataset: Define the types of information to be labeled, including: labeling the outer bounding box, type, and status of each fish. Use the fish's outer bounding box as the specified rule, with the labeling format: {(class), (x1, y1), (x2, y2)}, where (x1, y1) is the coordinate of the top-left corner of the maximum bounding box of a single fish, and (x2, y2) is the coordinate of the bottom-right corner of the maximum bounding box of the same single fish. Also label the fish's type and status.

[0111] S3. Establish a big data model. Train the convolutional neural network model based on the dataset to obtain a high-accuracy fish state control model. The fish state control model can be trained based on the YOLOv5 network. The training process includes model initialization: pre-training the convolutional neural network model parameters; defining the loss function using cross-entropy loss, mean squared error, etc.; and using the Adam optimization algorithm to adjust the model parameters to minimize the loss function.

[0112] S4. Construct a tracking and identification device. First, select the water area for subsequent fish tracking and identification, and install a tracking and identification platform 1 on the bank beside this water area. The bottom of the tracking and identification platform 1 is continuously circulated by a water treatment component 2 to maintain the water environment in the tracking and identification area. Simultaneously, a baiting component 3 is used to increase the probability of fish approaching. The identification component image acquisition device 4, i.e., a camera, is used to identify and track approaching fish and acquire corresponding raw video images. The acquired data is video; images extracted from the video are used for target detection.

[0113] Specifically, the tracking and identification platform 1 is constructed by fixing a vertical support frame to the shore of the water area to be tracked and identified. The water treatment component 2 uses a water pump 8 to drive the water flow in the water area to be tracked and identified. The water pump 8 draws water into the area near the water inlet of the water pumping pipe 9 and discharges it towards the area to be tracked and identified. A searchlight 6 is installed at the end of the horizontal support frame of the tracking and identification platform 1 to illuminate the bottom of the image acquisition device 4, i.e., the camera. A scanner 5 is placed on the top of the horizontal support frame of the tracking and identification platform 1. After the image acquisition device 4, i.e., the camera, captures video of the fish at the bottom of the horizontal support frame, the scanner 5 completes the subsequent feature comparison and status judgment processes.

[0114] The tracking and identification platform 1 has a support plate 7 installed at the bottom of its vertical support. A water pumping pipe 9 is connected to the side of the support plate 7. One end of the water pumping pipe 9 is fixedly connected to the inlet of the water pump 8, and the outlet of the water pump 8 is connected to the drain pipe 10. The water pump 8 draws water from the side through the water pumping pipe 9 and then discharges it outward from the drain pipe 10, thereby achieving the flow effect of this part of the water.

[0115] In this embodiment, fish are attracted to the bottom of the image acquisition device 4 (camera) by utilizing the phototaxis of organisms. At the same time, a bait delivery component 3 is built at the bottom of the tracking and recognition platform 1. The usage of the bait delivery component 3 is controlled according to the frequency of the acquired original images. After the bait is placed inside the bait trough 11, it accumulates inside the bottom mesh 14. The bait is spread by melting and the water treatment component 2, thereby controlling the number of fish in the image acquisition device camera 4 until the image acquisition frequency of the image acquisition device camera 4 is controlled within the set range.

[0116] A channel 12 is inserted at the bottom of the tracking and identification platform 1, and a bait trough 11 is set at the top of the channel 12. After bait is put into the channel 12, it can enter the bottom net 14 along the channel 12. Since the bait is directly soaked in the water, it can be melted by the water flow. After the bait is melted, the water pump 8 at the rear can be used to discharge the melted bait from the drain pipe 10, which will spread the melted bait far away to attract fish. This process can be used and controlled according to the efficiency of the image acquisition device 4, i.e., the camera, in acquiring video of the fish at the bottom.

[0117] S7, based on dynamic recognition technology, utilizes a self-attention mechanism. It captures bottom video footage from a camera, ensuring that images are extracted from the video.

[0118] S8. Feature comparison based on a fish state comparison model. Local extrema in the acquired raw video can form feature points, so a comparison is made between each pixel and all its adjacent pixels; gradient histograms are used to statistically analyze the orientation of key points. The main orientation is determined by analyzing the distribution of key orientations of pixels within a specific region, using the key point as the origin. The detection bounding box (outer border) of the fish in each frame of the video is obtained using the aforementioned method.

[0119] S9. Using the trained fish state comparison model to judge the obtained state, display the tracking and identification results of the species and living state in each frame of the image, and save the abnormal state frame images of the fish. Determine whether fish state detection is in progress. If so, feed back the fish state and the abnormal state frame images. Otherwise, continue to extract images from the video and repeat the recognition process, and save the final execution result.

[0120] To determine whether fish state detection is in progress, it's crucial to avoid misidentifying other objects like branches or debris in the image. The outer bounding boxes of fish in different motion states serve as the primary reference direction for the big data model. Fish state determination is achieved by comparing the collected real-time fish motion states with the outer bounding boxes in different states from the established training data.

[0121] The above description is only a preferred embodiment of the present disclosure. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present disclosure, and these improvements and modifications should also be considered within the protection scope of the present disclosure.

Claims

1. A device for intelligent identification, tracking, and status assessment of fish organisms, used to collect and identify fish population information in a water area to be measured, characterized in that, The device includes: a tracking and identification platform, and a water treatment component, a bait delivery component, and an identification component respectively disposed on the tracking and identification platform; A tracking and identification platform, designed to be installed on the shoreline of the water area to be measured; The bait delivery device is used to deliver bait into the waters to be tested in order to attract fish. Water treatment components are used to disturb the water flow in the water area to be tested; The identification component is used to collect videos of fish schools in the waters under test and identify the species and growth status of each fish in the video. The tracking and identification platform includes a vertical support and a horizontal support. One end of the vertical support is set on the bank of the water area to be measured, and the other end is connected to the horizontal support. The horizontal support is located above the water area and is equipped with the identification component and the bait delivery component. The tracking and identification platform further includes: a vertical mounting plate, one end of which is connected to the horizontal support, and the other end of which is used to extend into the water; a connecting plate is provided between the vertical mounting plate and the vertical support, and a support plate is provided on the other end of the vertical mounting plate away from the shore of the water; The water treatment component includes: a water pump, a pumping pipe, and a drain pipe; the water pump is mounted on the support plate, the water inlet of the water pump is connected to one end of the pumping pipe, the water outlet is connected to one end of the drain pipe, the other end of the pumping pipe is close to the shore of the water area, and the other end of the drain pipe is far away from the shore of the water area. The bait delivery assembly includes: a bait trough, a channel, a solenoid valve, a float, and a partition net; the channel passes through the horizontal support; the bait trough is provided at one end of the channel above the horizontal support, the other end of the channel extends into the water, the float is provided at the other end of the channel, and the partition net is provided at the other end; the solenoid valve is provided inside the other end of the channel for adjusting the opening and closing of the channel; The identification component includes: an image acquisition device, a lighting device, and a scanner, each respectively mounted on the horizontal support. The image acquisition device is used to acquire video of the water area to be tested. The lighting device is positioned within a preset distance range of the image acquisition device to provide illumination or attract fish when the image acquisition device acquires video. The scanner is electrically connected to both the lighting device and the image acquisition device to control the lighting device's start and stop, control the image acquisition device's video acquisition, and perform identification and analysis on the video to obtain the species and growth status of each fish in the video. The scanner is equipped with a visual recognition module. When the bait delivery component includes a solenoid valve, the scanner is electrically connected to the solenoid valve to control its opening and closing, and controls the usage status of the bait delivery component based on the acquisition frequency of the acquired original images.

2. The apparatus according to claim 1, characterized in that, The pump's axis is perpendicular to the vertical mounting plate, its inlet faces the shore of the water area, and its outlet is far from the shore of the water area. The other end of the pumping pipe is threaded through the support plate and extends along the direction of the waterway shoreline. The other end of the drainage pipe extends away from the shore of the water, and the drainage pipe is at a preset angle to the vertical mounting plate.

3. A method for intelligent identification, tracking, and status determination of fish organisms, characterized in that, The method includes: Construct the apparatus as described in any one of claims 1 to 2, and place the tracking and identification platform on the shore of the water area to be measured; The water treatment component and the identification component are activated respectively. The water treatment component drives the water flow in the water area, so that the water near the shore is pumped in and discharged in the direction away from the shore. The bait delivery component releases bait into the water, and the water treatment component drives the water flow, causing the bait to spread throughout the water to attract fish. The identification component collects video of the water area to be tested, and controls the frequency of bait release by the bait release component based on the acquisition frequency of the original images to fully obtain fish information.

4. A method for intelligent identification, tracking, and status determination of fish organisms, characterized in that, The method includes: Acquire a video of a school of fish in a body of water for which fish to be tracked and identified, wherein the video of the school of fish is acquired by an identification component of the device described in any one of claims 1 to 2, which is constructed on the shore of the body of water. Each image frame in the video is sequentially input into a pre-trained fish state comparison model to obtain the species and state of each fish in the corresponding image; wherein, the fish state comparison model is trained based on the YOLOv5 network; Determine if the fish are in an abnormal state. If so, mark the abnormal fish and save the image of the abnormal fish; otherwise, save the image of the normal fish. The training process of the fish state comparison model includes: acquiring an image dataset containing at least one fish, and labeling each fish in the image dataset with its bounding box, species or type, and state, wherein the state includes normal and abnormal; training a YOLOv5 network with the labeled image dataset to obtain a trained fish state comparison model.

5. A device for intelligent identification, tracking, and status determination of fish organisms, characterized in that, The device includes: An acquisition module is used to acquire a video of a school of fish in the water area to be tested, wherein the video of the school of fish is acquired according to the identification component of the device described in any one of claims 1 to 2, which is built on the shore of the water area to be tested; The detection module is used to sequentially input each image frame in the video into a pre-trained fish state comparison model to obtain the species and state result of each fish in the corresponding image; wherein, the fish state comparison model is trained based on a YOLOv5 network. The judgment module is used to determine whether the fish are in an abnormal state. If so, the abnormal fish are marked and the image of the abnormal fish is saved; otherwise, the image of the normal fish is saved. The training process of the fish state comparison model includes: acquiring an image dataset containing at least one fish, and labeling each fish in the image dataset with its bounding box, species or type, and state, wherein the state includes normal and abnormal; training a YOLOv5 network with the labeled image dataset to obtain a trained fish state comparison model.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in claim 4.

Images