Fish individual identification system and method based on fluorescent staining and image target detection

Abstract

The invention relates to the technical field of machine vision, and discloses a fish individual identification system and method based on fluorescent staining and image target detection. The system comprises an anesthesia device, a conveying device, a marking device, a resuscitation device, a recognition device and a control terminal, the anesthesia device is connected with one side of the conveying device, the marking device is arranged on the conveying device, the resuscitation device is connected with the other side of the conveying device, and the recognition device is connected with the resuscitation device. The control terminal is connected with the anesthesia device, the conveying device, the marking device, the resuscitation device and the recognition device. The fish individual identification method comprises the following steps: acquiring a plurality of target fish body images of a target fish body under various illumination conditions, various shooting angles and various postures from a fish individual identification module by using a control terminal; training an image classification model in combination with the target fish body image and the corresponding tag file; and fish individual identification is carried out based on the image classification model. According to the invention, individual identification of pattern-free fish species is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a fish individual identification system and method based on fluorescent staining and image target detection. BACKGROUND

[0002] Accurate identification of fish individuals is crucial for fish behavior research, aquaculture, and rare species protection. It not only helps to refine breeding management and improve breeding efficiency, but also plays an important role in monitoring, protecting, and researching rare fish populations. Through individual identification, information such as growth, reproduction, and health status of fish can be tracked, providing a solid basis for scientific decision-making.

[0003] In recent years, with the rapid development of artificial intelligence technology, deep learning algorithms using convolutional neural networks have made some progress in fish individual identification. Especially for koi fish, which have rich and unique patterns on their bodies. Through the extraction and analysis of koi fish pattern features, deep learning algorithms can accurately distinguish different individuals, providing effective technical support for koi fish breeding and trading.

[0004] However, for many non-patterned fish such as the four major carp species, individual identification faces great challenges. Since these fish do not have obvious landmark markers on their body surfaces, pattern feature-based identification methods are not applicable. This leads to the inability to achieve accurate automatic identification of these fish individuals in practical applications. SUMMARY

[0005] Therefore, the present application provides a fish individual identification system and method based on fluorescent staining and image target detection to achieve individual identification of non-patterned fish species.

[0006] In a first aspect, the present application provides a fish individual identification system based on fluorescent dyeing and image target detection, which comprises a narcotizing device, a conveying device, a marking device, a recovery device, an identification device and a control terminal 16, wherein the narcotizing device is connected to one side of the conveying device, the marking device is arranged on the conveying device, the recovery device is connected to the other side of the conveying device, the identification device is connected to the recovery device, and the control terminal 16 is connected to the narcotizing device, the conveying device, the marking device, the recovery device and the identification device, wherein the narcotizing device comprises a narcotizing pool 1, a narcotizing injection module and a narcotizing state monitoring module, the narcotizing injection module is arranged on the narcotizing pool 1, and the narcotizing state monitoring module is arranged above the narcotizing pool 1; the conveying device is connected to the narcotizing pool 1; the marking device comprises a fixing module, an injection module and a marking state monitoring module, the fixing module is arranged above the conveying device, the injection module and the marking state monitoring module are arranged above the fixing module, and the injection module is used for injecting a target marking pattern on a fish body; the recovery device comprises a recovery pool 12 and a recovery state monitoring module, the recovery pool 12 is connected to the conveying device, and the recovery state monitoring module is arranged above the recovery pool 12; the identification device comprises a breeding pool 15 and a fish individual identification module, the breeding pool 15 is connected to the recovery device, and the fish individual identification module is arranged above the breeding pool 15; and the control terminal 16 is used for controlling the conveying device.

[0007] In an optional implementation, the narcotizing pool 1 is provided with a first water valve 5; the narcotizing injection module is configured with a high-precision metering pump 2 and a concentration sensor 3, the high-precision metering pump 2 and the concentration sensor 3 are connected, the high-precision metering pump 2 is used for calculating the dosage of the narcotic agent according to the volume of the breeding water body and injecting the narcotic agent into the breeding water body, and the concentration sensor 3 is used for monitoring the concentration of the narcotic agent in the breeding water body; the narcotizing state monitoring module is used for acquiring a first fish body image of a target fish body in the narcotizing pool 1; and the control terminal 16 is used for controlling the opening and closing of the first water valve 5 based on the first fish body image.

[0008] In an optional implementation, the fixing module comprises an air bag clamp 8, the injection module comprises a mechanical arm 9, a syringe 10 and a force control sensor 11, one end of the mechanical arm 9 is connected to the side edge of the conveying device, the other end is connected to the syringe 10 and the force control sensor 11, and the syringe 10 and the force control sensor 11 are arranged above the fixing module, wherein the air bag clamp 8 is used for fixing a fish body; the marking state monitoring module is used for acquiring a second fish body image of a target fish body fixed by the air bag clamp 8; the mechanical arm 9 is used for controlling the syringe 10 to inject a target fluorescent dye according to a target marking pattern; the force control sensor 11 is used for controlling the dye injection depth; and the control terminal 16 is used for controlling the air bag clamp 8, the mechanical arm 9 and the force control sensor 11 based on the second fish body image and the target marking pattern.

[0009] In an alternative embodiment, a second water valve 13 is arranged on the recovery tank 12; the recovery state monitoring module is configured to acquire a third fish image of the target fish in the recovery tank 12; and the control terminal 16 is configured to control the opening and closing of the second water valve 13 based on the third fish image.

[0010] In an alternative embodiment, the fish individual identification system based on fluorescent dyeing and image target detection further comprises a fish conveying pipeline 14, wherein the breeding tank 15 is connected to the second water valve 13 on the recovery tank 12 through the fish conveying pipeline 14; the fish individual identification module is configured to acquire a fish video of the target fish in the breeding tank 15; and the control terminal 16 is configured to perform fish individual identification based on the fish video.

[0011] In a second aspect, the present application provides a fish individual identification method based on fluorescent dyeing and image target detection, which is applied to the above-mentioned control terminal 16. The fish individual identification method based on fluorescent dyeing and image target detection comprises the following steps: acquiring a first fish image from the anesthesia state monitoring module; performing motion recognition on the target fish in the first fish image, and determining that the target fish is in an anesthesia state when the target fish is static for more than a preset time; controlling the first water valve 5 to open, and controlling the conveying device to start, so as to transfer the target fish to the marking device.

[0012] In an alternative embodiment, the fish individual identification method based on fluorescent dyeing and image target detection further comprises the following steps: acquiring a second fish image from the marking state monitoring module; controlling the air bag clamp 8 to fix the target fish based on the second fish image; generating a pattern trajectory point based on a target marking pattern and the second fish image, wherein the target marking pattern is a unique marking pattern generated based on the number of fish individuals; controlling the syringe 10 on the mechanical arm 9 to inject a target fluorescent dye on the target fish according to the pattern trajectory point; controlling the force control sensor 11 to adjust the dye injection depth; and controlling the conveying device to start, so as to transfer the target fish to the recovery device.

[0013] In an alternative embodiment, the fish individual identification method based on fluorescent dyeing and image target detection further comprises the following steps: acquiring a third fish image from the recovery state monitoring module; performing operculum opening and closing recognition on the target fish in the first fish image, and determining that the target fish is in a recovery completion state when the operculum opening and closing frequency of the target fish is greater than a preset frequency; and controlling the second water valve to open, so as to transfer the target fish to the identification device.

[0014] In an alternative embodiment, the fish individual identification method based on fluorescent staining and image target detection further comprises: obtaining multiple target fish images of the target fish under multiple lighting conditions, multiple shooting angles and multiple postures from the fish individual identification module; labeling a target mark pattern on the multiple target fish images to generate a label file; training an image classification model in combination with the target fish images and the corresponding label file; and identifying the fish individual based on the image classification model.

[0015] In an alternative embodiment, obtaining multiple target fish images of the target fish under multiple lighting conditions, multiple shooting angles and multiple postures from the fish individual identification module comprises: obtaining a target fish video of the target fish under multiple lighting conditions, multiple shooting angles and multiple postures from the fish individual identification module; performing frame segmentation on the target fish video and segmenting the fish region to obtain multiple single-frame fish images; and extracting frames from the multiple single-frame fish images according to a preset frame interval to obtain the multiple target fish images.

[0016] In this implementation, by providing the fish individual identification system based on fluorescent staining and image target detection comprising an anesthesia device, a conveying device, a marking device, a resuscitation device, an identification device and a control terminal, the control terminal is used to monitor each stage of subcutaneous staining, the subcutaneous staining method is combined with the neural network algorithm, the advantages of methods and means from two fields are fully utilized, and the technical advantages of interdisciplinary integration are embodied.

[0017] In a third aspect, the present application provides a computer readable storage medium, which stores computer instructions for causing a computer to execute the fish individual identification method based on fluorescent staining and image target detection of the first aspect or any of the corresponding embodiments thereof.

[0018] In a fourth aspect, the present application provides a computer program product comprising computer instructions for causing a computer to execute the fish individual identification method based on fluorescent staining and image target detection of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0020] Figure 1 is a schematic diagram of a fish individual identification system based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 2 is a schematic diagram of a head fluorescent marking of a target fish body according to an embodiment of the present application; Figure 3 is a flow chart of a fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 4 is a flow chart of another fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 5 is a flow chart of yet another fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 6 is a flow chart of yet another fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 7 is a flow chart of a specific fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application; Figure 8 is a hardware structure schematic diagram of a computer device according to an embodiment of the present application.

[0021] Marked with reference numerals: 1, anesthesia pool; 2, high-precision metering pump; 3, concentration sensor; 5, first water valve; 6, conveying belt; 7, structured light camera; 8, air bag clamp; 9, mechanical arm; 10, syringe; 11, force control sensor; 12, recovery pool; 13, second water valve; 14, fish conveying pipeline; 15, breeding pool; 16, control terminal; 41, first high-speed camera; 42, second high-speed camera; 43, third high-speed camera. DETAILED DESCRIPTION

[0022] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0023] For many fish species such as four common carp, individual identification is a great challenge. Because these fish have no obvious markers on their body surface, the identification method based on pattern features is difficult to apply. Therefore, the accurate automatic identification of these fish individuals cannot be realized in practical application. Therefore, the present application provides a fish individual identification system and method based on fluorescent staining and image target detection. The fish individual identification system based on fluorescent staining and image target detection is set up, including anesthetic device, conveying device, marking device, recovery device, identification device and control terminal. The control terminal is used for monitoring each stage of subcutaneous staining. The subcutaneous staining method is combined with neural network algorithm, which fully utilizes the advantages of methods from two fields and reflects the technical advantages of interdisciplinary integration.

[0024] According to the embodiment of the present application, a fish individual identification system based on fluorescent staining and image target detection is provided. Please refer to Figure 1 , Figure 1 is a schematic diagram of a fish individual identification system based on fluorescent staining and image target detection according to the embodiment of the present application.

[0025] As shown in Figure 1 , the fish individual identification system based on fluorescent staining and image target detection includes anesthetic device, conveying device, marking device, recovery device, identification device and control terminal 16. The anesthetic device is connected to one side of the conveying device. The marking device is arranged on the conveying device. The recovery device is connected to the other side of the conveying device. The identification device is connected to the recovery device. The control terminal 16 is connected to the anesthetic device, conveying device, marking device, recovery device and identification device. The control terminal 16 is used for controlling the conveying device.

[0026] In an implementation manner, the conveying device is provided with a conveying belt 6. The control terminal 16 is used for controlling the conveying belt 6 of the conveying device.

[0027] The anesthetic device is connected to one side of the conveying device. The anesthetic device includes anesthetic tank 1, anesthetic injection module and anesthetic state monitoring module. The anesthetic injection module is arranged on the anesthetic tank 1. The anesthetic state monitoring module is arranged above the anesthetic tank 1.

[0028] Further, the anesthetic tank 1 is provided with a first water valve 5. The anesthetic tank 1 is connected to the conveying device through the first water valve 5. The anesthetic injection module is provided with high-precision metering pump 2 and concentration sensor 3. The high-precision metering pump 2 and the concentration sensor 3 are connected. The anesthetic state monitoring module includes a first high-speed camera 41.

[0029] The high-speed camera is a waterproof starlight night vision camera. The minimum available illumination range is 0.01 Lux. The observation angle is downward.

[0030] The high-precision metering pump 2 is used to calculate the amount of anesthetic according to the volume of the aquaculture water in the anesthetic tank 1 and inject the anesthetic into the aquaculture water, and the concentration sensor 3 is used to monitor the anesthetic concentration of the aquaculture water in the anesthetic tank 1.

[0031] The first high-speed camera 41 of the anesthesia state monitoring module is used to obtain a first fish image of the target fish in the anesthetic tank 1. The control terminal 16 is used to control the opening and closing of the first water valve 5 based on the first fish image.

[0032] Specifically, the target fish is placed in the anesthetic tank 1, the amount of anesthetic is automatically calculated by the high-precision metering pump 2 according to the volume of the aquaculture water, the anesthetic is injected into the water through a pipeline and stirred uniformly, the concentration of the anesthetic in the aquaculture water is detected in real time by the concentration sensor 3, when the concentration of the anesthetic in the aquaculture water is less than a preset concentration range, the high-precision metering pump 2 is controlled to inject the anesthetic, and when the concentration of the anesthetic in the aquaculture water is greater than the preset concentration range, the high-precision metering pump 2 is controlled to stop injecting the anesthetic.

[0033] Further, the first high-speed camera 41 of the anesthesia state monitoring module is used to capture the swimming behavior and body swing amplitude of the target fish in real time, determine the first fish image, and the control terminal 16 is used to perform motion recognition on the target fish in the first fish image, when the target fish is static for more than a preset time, it is judged that the target fish is in an anesthesia state, the control terminal 16 controls the first water valve 5 to open, and the control terminal 16 controls the conveyor belt 6 of the conveying device to start, and the target fish is transferred to the marking device.

[0034] Illustratively, the yolo target detection model is used to detect and analyze the reaction of the target fish, when the target fish is static for more than 15 seconds, the first water valve 5 is triggered to open, the conveyor belt 6 on the conveying device is started, and the target fish is transferred to the next station.

[0035] In one implementation, the anesthetic is MS222.

[0036] The marking device is arranged on the conveying device, and the marking device comprises a fixing module, an injection module and a marking state monitoring module. The fixing module is arranged above the conveying device, and the injection module and the marking state monitoring module are arranged above the fixing module.

[0037] Further, the fixing module comprises an air bag clamp 8, the injection module comprises a mechanical arm 9, a syringe 10 and a force control sensor 11, one end of the mechanical arm 9 is connected to the side edge of the conveying device, the other end is connected to the syringe 10 and the force control sensor 11, the syringe 10 and the force control sensor 11 are arranged above the fixing module, and the marking state monitoring module comprises a structured light camera 7.

[0038] In an implementation manner, the injector 10 is a micro-injector, and the force control sensor 11 has a precision of ±0.1 μL.

[0039] The air bag clamp 8 is used to fix the fish body; the marking state monitoring module is used to obtain a second fish body image of the target fish body fixed by the air bag clamp 8; the mechanical arm 9 is used to control the injector 10 to inject the target fluorescent dye according to the target marking pattern; the force control sensor 11 is used to control the dye injection depth; and the control terminal 16 is used to control the air bag clamp 8, the mechanical arm 9 and the force control sensor 11 based on the second fish body image and the target marking pattern.

[0040] In an implementation manner, the target fluorescent dye is a dyeing agent and a diluent, which are non-toxic and harmless, and the dyeing agent and the diluent are mixed at a ratio of 1:1000 to obtain the target fluorescent dye. After the target fluorescent dye is injected subcutaneously into the fish, it will solidify into a solid to prevent falling off.

[0041] Specifically, the control terminal 16 generates a corresponding number of unique marking patterns according to the number of fish individuals, and stores a mapping relationship between the target marking pattern and the fish individual ID in the database. The unique marking pattern includes “△”, “○”, “×” and the like.

[0042] Further, the control terminal 16 controls the air bag clamp 8 to fix the target fish body, uses the structured light camera 7 of the state monitoring module to scan a second fish body image of the target fish body in real time, controls the control terminal 16 to perform three-dimensional modeling on the target fish body according to the second fish body image, determines a subcutaneous injection area, and uses the target marking pattern to design a pattern path, decomposes the target marking pattern into continuous pattern track points, controls the injector 10 carried on the mechanical arm 9 to inject according to the pattern track points, and controls the dye injection depth through the force control sensor 11. The control terminal 16 controls the transmission belt 6 of the conveying device to start, and transfers the target fish body to the recovery device.

[0043] Exemplarily, the head curve of the target fish body is scanned by using the structured light camera 7, the pattern is adjusted to adapt to different individual sizes, the head position is fixed by using the air bag clamp 8 without damaging the fish body, a three-dimensional model of the fish body head is constructed, and injection is performed on the fish body head. After the injection is completed, the transmission belt 6 is triggered to start, and the target fish body is transferred to the next station.

[0044] Please refer to Figure 2 , Figure 2 is a schematic diagram of fluorescent marking of the head of a target fish body according to an embodiment of the present application. As shown in Figure 2 , five different target fish bodies are marked with “△”, “×”, “|”, “=” and “○” patterns respectively.

[0045] The resuscitation device is connected to the other side of the conveying device, and the resuscitation device comprises a resuscitation tank 12 connected to the conveying device and a resuscitation state monitoring module arranged above the resuscitation tank 12.

[0046] Further, a second water valve 13 is arranged on the resuscitation tank 12, and the resuscitation state monitoring module comprises a second high-speed camera 42. The resuscitation state monitoring module is configured to acquire a third fish image of the target fish in the resuscitation tank 12; and the control terminal 16 is configured to control the opening and closing of the second water valve 13 based on the third fish image.

[0047] Specifically, the target fish is moved into the resuscitation tank 12, the second high-speed camera 42 of the resuscitation state monitoring module is used to continuously monitor the balance ability and swimming trajectory of the fish, a third fish image is determined, the control terminal 16 performs gill cover opening and closing identification on the target fish in the third fish image, when the gill cover opening and closing frequency of the target fish is greater than a preset frequency, it is judged that the target fish is in a resuscitation completion state, and the control terminal 16 controls the second water valve 13 to open and transfers the target fish to the identification device.

[0048] In an implementation manner, the system further comprises a fish conveying pipeline 14, and the control terminal 16 controls the second water valve 13 to open, and the fish conveying pipeline 14 transfers the target fish to the identification device.

[0049] The fish conveying pipeline 14 is made of PVC material with a smooth inner surface.

[0050] Exemplarily, the gill cover region is located by using a yolov5s target detection model, the gill cover opening and closing frequency of the target fish is detected and analyzed to determine the resuscitation completion degree. When the gill cover opening and closing frequency is less than or equal to a preset frequency, it is judged that the resuscitation completion degree is low, and the monitoring is continued. When the gill cover opening and closing frequency is greater than the preset frequency, it is judged that the resuscitation is completed, the second water valve 13 is triggered to open, and the target fish is transferred to the next station.

[0051] The identification device comprises a breeding tank 15 and a fish individual identification module, the breeding tank 15 is connected to the resuscitation device, and the fish individual identification module is arranged above the breeding tank 15.

[0052] Further, the breeding tank 15 is connected to the second water valve 13 on the breeding tank 15 through the fish conveying pipeline 14, and the fish individual identification module comprises a third high-speed camera 43. The fish individual identification module is configured to acquire a fish video of the target fish in the breeding tank 15; and the control terminal 16 is configured to perform fish individual identification based on the fish video.

[0053] Specifically, the target fish is moved into the breeding tank 15, the third high-speed camera 43 of the fish individual identification module is used to acquire a plurality of target fish images of the target fish under a plurality of illumination conditions and a plurality of shooting angles; and fish individual identification is performed by using the plurality of target fish images.

[0054] In the embodiment, a fish individual identification method based on fluorescent staining and image target detection is provided, which can be used in the control terminal 16, Figure 3 is a flowchart of a fish individual identification method based on fluorescent staining and image target detection according to an embodiment of the present application. It should be noted that the flow sequence shown in the embodiment is not limited. As shown in the embodiment, the flow includes the following steps: Figure 3 Figure 3 Step S301, obtaining a plurality of target fish images of a target fish under a plurality of illumination conditions, a plurality of shooting angles and a plurality of postures from a fish individual identification module.

[0055] The third high-speed camera 43 of the fish individual identification module is used to shoot fish videos covering different illumination conditions, different shooting angles and a plurality of swimming postures as much as possible, and a plurality of target fish images are extracted from the fish videos.

[0056] The illumination conditions include natural light, artificial light, night infrared light, etc., and the shooting angles include top view, side view, overhead view, etc.

[0057] Specifically, the above step S301 includes: Step S3011, obtaining a target fish video of a target fish under a plurality of illumination conditions, a plurality of shooting angles and a plurality of postures from a fish individual identification module.

[0058] The third high-speed camera 43 of the fish individual identification module is used to continuously record and collect the behavior state and distribution of the target fish under a plurality of swimming postures under a plurality of illumination conditions and a plurality of shooting angles, and the data is stored in a video format to obtain a fish video, and the control terminal 16 collects the fish video.

[0059] Among them, the collection frame rate of the fish video is set according to the behavior intensity of the target fish, and when the behavior intensity of the target fish is larger, the corresponding collection frame rate is larger. Exemplarily, the collection frame rate is about 30 frames / second.

[0060] Step S3012, frame segmentation is performed on the target fish video, and the fish region is segmented to obtain a plurality of single-frame fish images.

[0061] The target fish video is frame segmented at a preset interval to be split into single-frame images, and the fish region is segmented from the single-frame images by using a deep learning model to generate single-frame fish images.

[0062] ​​Specifically, the captured target fish video is frame segmented at a fixed interval by using the OpenCV library of Python, the fish region is segmented by using the U-Net model, a binary mask is generated and background noise is removed, and a plurality of single-frame fish images are obtained.

[0063] Step S3013, a plurality of target fish images are obtained by extracting frames from the plurality of single-frame fish images at a preset frame interval.

[0064] A plurality of target fish images are obtained by extracting representative frames capable of representing the entire shooting environment and the state of the target fish from the plurality of single-frame fish images at a preset frame interval.

[0065] Specifically, invalid frames are removed from the plurality of single-frame fish images, and representative frames are extracted from all frame pictures at a preset frame interval to obtain a plurality of target fish images. Fe= Ft / IFS.

[0066] Wherein: Fe is the number of target fish images; Ft is the total number of frames after removing invalid frames; IFS is the preset frame interval.

[0067] Step S302, labeling a target marking pattern on a plurality of target fish images to generate a label file.

[0068] Key feature points on the target marking pattern drawn by the fluorescent dye are recognized on the target fish image by using a recognition tool, and are labeled to generate a label file.

[0069] Specifically, the data labeling software labelimg is used to import the target fish image by "Open Dir", a standard directory structure is established, and a predefined fish class label is set. The polygon labeling tool is used to carefully outline the target fish contour, ensure that the contour fits the fish edge, keep the line continuous and non-intersecting during labeling, further label the key feature points of the target marking pattern drawn by the fluorescent dye, click "Save" to generate an XML format label file after completing the single image labeling, and the file automatically associates the image path and stores the information of the class and coordinate position of the target fish.

[0070] Step S303, training an image classification model in combination with the target fish image and the corresponding label file.

[0071] The target fish image and the corresponding label file are used to construct a data set, which is divided into a detection data set, a verification data set and a training data set according to a proportion, and the image classification model is trained by using the training data set.

[0072] Specifically, the total dataset is divided into a detection dataset, a validation dataset, and a training dataset in a 1:1:8 ratio, ensuring a uniform distribution of each category within each set. Data augmentation operations such as rotation and brightness adjustment are applied to the training dataset using the Albumentations library to improve generalization.

[0073] Furthermore, the YOLOv5s image classification model was trained multiple times using train.py in a Python environment. The YOLOv5s image classification model was tested using a detection dataset, and the training parameters were adjusted continuously based on the test results. The YOLOv5s image classification model was validated using a validation dataset. Finally, an image classification model and classification weight file were generated that can be used for individual fish classification. The weight file format is ".pt".

[0074] Step S304: Individual fish identification is performed based on an image classification model.

[0075] Specifically, in a Python environment, a trained YOLOv5s image classification model and a classification weight file are used to perform target detection and classification of individual fish samples.

[0076] In this implementation, the image classification model can accurately identify and classify individual fish based on the fluorescent marker target features of individual fish learned during multiple rounds of training.

[0077] This embodiment provides another method for fish individual identification based on fluorescence staining and image target detection, which can be used in the aforementioned control terminal 16. Figure 4 This is a flowchart of another fish individual identification method based on fluorescence staining and image target detection according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily use it as the standard. Figure 4 The illustrated process sequence is limited. For example... Figure 4 As shown, the process includes the following steps: Step S401: Obtain the first fish image from the anesthesia monitoring module.

[0078] Step S402: Motion recognition is performed on the target fish in the first fish image. When the target fish remains still for a longer than a preset time, it is determined that the target fish is in anesthetized state.

[0079] Step S403: Control the first water valve to open, control the transmission device to start, and transfer the target fish to the marking device.

[0080] This embodiment provides another method for fish individual identification based on fluorescence staining and image target detection, which can be used in the aforementioned control terminal 16. Figure 5This is a flowchart of another fish individual identification method based on fluorescence staining and image target detection according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily use it. Figure 5 The illustrated process sequence is limited. For example... Figure 5 As shown, the process includes the following steps: Step S501: Obtain the second fish image from the tagging status monitoring module.

[0081] Step S502: Based on the second fish image, control the airbag clamp to fix the target fish.

[0082] Step S503: Generate pattern trajectory points based on the target marker pattern and the second fish body image. The target marker pattern is a unique marker pattern generated based on the number of individual fish.

[0083] Step S504: Control the syringe on the robotic arm to inject the target fluorescent dye onto the target fish body according to the pattern trajectory; control the force control sensor to adjust the dye injection depth.

[0084] Step S505: Control the start of the transfer device to transfer the target fish to the resuscitation device.

[0085] This embodiment provides another method for fish individual identification based on fluorescence staining and image target detection, which can be used in the aforementioned control terminal 16. Figure 6 This is a flowchart of another fish individual identification method based on fluorescence staining and image target detection according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily use it. Figure 6 The illustrated process sequence is limited. For example... Figure 6 As shown, the process includes the following steps: Step S601: Obtain the image of the third fish body from the recovery status monitoring module.

[0086] Step S602: The gill cover opening and closing of the target fish in the first fish image is identified. When the gill cover opening and closing frequency of the target fish is greater than the preset frequency, it is determined that the target fish is in the recovery completed state.

[0087] Step S603: Control the second water valve to open and transfer the target fish to the identification device.

[0088] This embodiment provides a specific method for fish individual identification based on fluorescence staining and image target detection, which can be used in the aforementioned fish individual identification system based on fluorescence staining and image target detection. Figure 7 This is a flowchart illustrating a specific method for fish individual identification based on fluorescence staining and image target detection according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that method. Figure 7 The illustrated process sequence is limited. For example...Figure 7 As shown, the flow includes the following steps: Step S701, fish is anesthetized.

[0089] The anesthetic tank 1 includes an anesthetic injection module and an anesthetic state monitoring module. The anesthetic injection module is configured with a high-precision metering pump 2 and a concentration sensor 3. The high-precision metering pump 2 automatically calculates the amount of anesthetic according to the volume of the aquaculture water body, injects it into the water body through a pipeline, and stirs it evenly. The concentration sensor 3 monitors the concentration of the anesthetic in the water body in real time to ensure that the concentration is within the preset range. The anesthetic state monitoring module is to capture the fish swimming behavior and body swing amplitude in real time through the deployment of a high-speed camera 41, and analyze the fish body reaction based on yolo target detection. When the static time exceeds 15 seconds, the first water valve 5 is triggered to open, and the conveyor belt 6 is started to move the fish body to the next station.

[0090] Step S702, mark pattern design.

[0091] The number of fish individuals is input, and the system automatically generates a unique pattern combination (the pattern includes “△”, “○”, “×”, etc.). The “pattern-individual ID” mapping relationship is stored in the database to facilitate subsequent tracing. The fish head curved surface is scanned by a structured light camera 7, and the pattern is adjusted to adapt to different individual sizes.

[0092] Step S703, fluorescent dye marking.

[0093] An air bag clamp 8 is used to fix the head position without damaging the fish body. A structured light camera 7 is used to construct a three-dimensional model of the fish head to determine the subcutaneous injection area. According to the designed pattern path, the symbol is decomposed into continuous path points. A mechanical arm 9 carries a micro-injector 10 to inject according to the path points, and a force control sensor 11 is used to control the dye injection depth.

[0094] Step S704, fish water immersion recovery.

[0095] The conveyor belt 6 moves the marked fish body to the recovery tank 12. The top high-speed camera 42 continuously monitors the fish body balance ability and swimming trajectory, and locates the operculum area through the yolov5s target detection model to calculate the operculum opening and closing frequency to judge the recovery completion degree. When the standard is met, the second water valve 13 of the recovery tank 12 outlet is automatically opened, and the fish is released to the breeding tank 15 through the fish conveying pipeline 14.

[0096] Step S705, fish image shooting.

[0097] The appearance of all marked fish individuals is imaged using a high-definition camera, and the imaging covers different lighting conditions (natural light, artificial light, night infrared light), different angles (top view, side view, overhead view), and different swimming postures of the fish body as much as possible. The behavior state and distribution of the fish are continuously recorded and collected, and the data are stored in video format. The frame rate (fps) of the video is determined according to the intensity of the specific biological behavior, and the range is about 30 frames per second; Step S706, fish image frame segmentation.

[0098] The fish appearance images are segmented into single-frame images at a fixed interval using the OpenCV library of Python. The fish body region is segmented using a deep learning model U-Net, and a binary mask is generated to remove background noise.

[0099] Step S707, extracting representative frames.

[0100] According to the activity intensity of the fish individual and the total number of remaining frames (Ft) after invalid frames are removed, representative frames are extracted from all frame pictures at a specific frame interval (IFS), and the number of representative frames obtained is: Fe= Ft / IFS.

[0101] Wherein: Fe is the number of extracted representative frames; Ft is the total number of frames; IFS is the frame interval.

[0102] Step S708, labeling fish appearance feature information.

[0103] The image target detection special data labeling software labelimg is used to import the representative frame images that have been segmented and extracted by “Open Dir”, establish a standard directory structure and set the predefined fish class label. The labeling work starts from basic information labeling, and the polygon labeling tool is used to carefully outline along the fish body contour, ensuring that the contour fits the fish body edge, and the line is continuous and without intersection during labeling. After completing the contour labeling, the next step is to label the key feature points, mainly labeling the pattern drawn using fluorescent dye, which will be used for subsequent posture analysis and individual identification. After completing the labeling of a single image, click “Save” to generate an XML format labeling file, which automatically associates the image path and stores the class and coordinate position information of the target.

[0104] Step S709, constructing an image classification data set.

[0105] The extracted representative frame pictures and the label file obtained after labeling the target detection information jointly constitute a target detection data set. The total data set is divided into a detection data set, a verification data set and a training data set in a ratio of 1:1:8. That is, the training data set accounts for 8 / 10 of the total data set file quantity, and the verification data set and the test data set each account for 1 / 10, and the distribution of each category in each set is ensured to be uniform. Through the Albumentations library, rotation, brightness adjustment and other transformations are added to perform data enhancement operations on the training set, thereby improving the generalization.

[0106] Step S710, training an image classification model.

[0107] Using the constructed training data set and verification data set, the yolov5s image classification model is trained multiple times under the python environment using train.py. The test data set is used to test the training result, and the training parameters are adjusted in real time according to the test result, and finally an image classification model and a classification weight file that can be used for fish individual classification are generated, and the weight file format is “.pt”.

[0108] Step S711, fish individual recognition and classification.

[0109] The trained yolov5s image classification model and classification weight file are used to detect and classify the fish individuals to be tested under the python environment. The image classification model can accurately recognize and classify the fish individuals to be tested according to the fish individual fluorescently labeled target features learned in the multiple training processes.

[0110] The embodiment proposes a fish individual recognition system and method based on fluorescent staining and image target detection, which provides an effective technical means for monitoring the health status of individuals in the process of aquaculture and the process of rare species in captivity. For most Chinese breeding varieties such as four common carp and rare fish species, it is difficult to identify individuals due to the lack of body patterns. Through subcutaneous fluorescent staining, each individual is given a unique visual marker, thereby realizing the individual identification of non-patterned fish species. The subcutaneous staining method traditionally used for fish restocking marking is combined with the target detection algorithm based on convolutional neural network, which fully utilizes the advantages of methods from both fields and reflects the technical advantages of interdisciplinary integration.

[0111] The further function description of each module and unit is the same as the above-mentioned corresponding embodiment, which will not be repeated here.

[0112] In this embodiment, the control terminal is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0113] This invention also provides a computer device having the above-described features. Figure 1 The control terminal shown.

[0114] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 100, memory 200, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 100 as an example.

[0115] Processor 100 may be a central processing unit, a network processor, or a combination thereof. Processor 100 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0116] The memory 200 stores instructions executable by at least one processor 100 to cause the at least one processor 100 to perform the method shown in the above embodiments.

[0117] The memory 200 can include a program storage area and a data storage area, where the program storage area can store an operating system, at least one application required by a function, and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 200 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some optional embodiments, the memory 200 can optionally include a memory disposed remotely with respect to the processor 100, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The memory 200 can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state disk; and the memory 200 can also include a combination of the above-mentioned kinds of memories.

[0119] The computer device also includes an input device 300 and an output device 400. The processor 100, the memory 200, the input device 300, and the output device 400 can be connected through a bus or other means, Figure 8 For example, by way of example, through a bus connection.

[0120] The input device 300 can receive inputted digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 400 can include a display device, an auxiliary lighting device (e.g., an LED), a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes, but is not limited to, a liquid crystal display, a light-emitting diode, a display, and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0121] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded to a local storage medium through network, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0122] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, the operation of the computer can invoke or provide the method and / or technical solutions according to the present application. Those skilled in the art should understand that the form of computer program instructions in computer readable medium includes but is not limited to source file, executable file, installation package file, etc. Correspondingly, the way of computer program instructions executed by computer includes but is not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0123] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A fish individual identification system based on fluorescent staining and image target detection, characterized in that, The system comprises an anesthesia device, a conveying device, a marking device, a recovery device, an identification device and a control terminal (16), wherein the anesthesia device is connected with one side of the conveying device, the marking device is arranged on the conveying device, the recovery device is connected with the other side of the conveying device, the identification device is connected with the recovery device, and the control terminal (16) is connected with the anesthesia device, the conveying device, the marking device, the recovery device and the identification device, wherein The anesthesia device comprises an anesthesia pool (1), an anesthesia injection module and an anesthesia state monitoring module, the anesthesia injection module is arranged on the anesthesia pool (1), and the anesthesia state monitoring module is arranged above the anesthesia pool (1); the conveying device is connected with the anesthesia pool (1); The marking device comprises a fixing module, an injection module and a marking state monitoring module, the fixing module is arranged above the conveying device, the injection module and the marking state monitoring module are arranged above the fixing module, and the injection module is used for injecting a target marking pattern on a fish body; The recovery device comprises a recovery pool (12) and a recovery state monitoring module, the recovery pool (12) is connected with the conveying device, and the recovery state monitoring module is arranged above the recovery pool (12); The identification device comprises a breeding pool (15) and a fish individual identification module, the breeding pool (15) is connected with the recovery device, and the fish individual identification module is arranged above the breeding pool (15); The control terminal (16) is used for controlling the conveying device.

2. The fish individual identification system based on fluorescent staining and image target detection according to claim 1, wherein A first water valve (5) is arranged on the anesthesia pool (1); The anesthesia injection module is configured with a high-precision metering pump (2) and a concentration sensor (3), the high-precision metering pump (2) and the concentration sensor (3) are connected, the high-precision metering pump (2) is used for calculating the dosage of anesthetic according to the volume of the breeding water body and injecting the anesthetic into the breeding water body, and the concentration sensor (3) is used for monitoring the concentration of the anesthetic in the breeding water body; The anesthesia state monitoring module is used for acquiring a first fish body image of a target fish body in the anesthesia pool (1); The control terminal (16) is used for controlling the opening and closing of the first water valve (5) based on the first fish body image.

3. The fish individual recognition system based on fluorescent dyeing and image target detection according to claim 2, characterized in that, The fixing module comprises an air bag clamp (8), the injection module comprises a mechanical arm (9), a syringe (10) and a force control sensor (11), one end of the mechanical arm (9) is connected with the side edge of the conveying device, the other end is connected with the syringe (10) and the force control sensor (11), and the syringe (10) and the force control sensor (11) are arranged above the fixing module, wherein The air bag clamp (8) is used for fixing a fish body; The marking state monitoring module is used for acquiring a second fish body image of a target fish body fixed by the air bag clamp (8); The mechanical arm (9) is used for controlling the syringe (10) to inject a target fluorescent dye according to a target marking pattern. The force control sensor (11) is used for controlling the dye injection depth; The control terminal (16) is used for controlling the air bag clamp (8), the mechanical arm (9) and the force control sensor (11) based on the second fish body image and the target mark pattern.

4. The fish individual identification system based on fluorescent dyeing and image target detection according to claim 3, characterized in that, A second water valve (13) is arranged on the recovery tank (12); The recovery state monitoring module is used for acquiring a third fish body image of a target fish body in the recovery tank (12); The control terminal (16) is used for controlling the opening and closing of the second water valve (13) based on the third fish body image.

5. The fish individual recognition system based on fluorescent dyeing and image target detection according to claim 4, characterized in that, The system further comprises a fish conveying pipeline (14), wherein, The breeding tank (15) is connected with the second water valve (13) on the recovery tank (12) through the fish conveying pipeline (14); The fish individual identification module is used for acquiring a fish body video of a target fish body in the breeding tank (15); The control terminal (16) is used for fish individual identification based on the fish body video.

6. A fish individual identification method based on fluorescent staining and image target detection, characterized in that, The method applied to the control terminal (16) of claim (5) comprises: acquiring a first fish body image from an anesthesia state monitoring module; performing motion recognition on a target fish body in the first fish body image, and determining that the target fish body is in an anesthesia state when a still time of the target fish body is greater than a preset time; controlling a first water valve (5) to open and controlling a conveying device to start, so as to transfer the target fish body to a marking device.

7. The fish individual recognition method based on fluorescent staining and image target detection according to claim 6, characterized in that, The method further comprises: acquiring a second fish body image from a marking state monitoring module; controlling an air bag clamp (8) to fix the target fish body based on the second fish body image; generating a pattern track point based on a target mark pattern and the second fish body image, wherein the target mark pattern is a unique mark pattern generated based on the number of fish individuals; controlling an injector (10) on a mechanical arm (9) to inject a target fluorescent dye on the target fish body according to the pattern track point; and controlling a force control sensor (11) to adjust the dye injection depth; controlling the conveying device to start, so as to transfer the target fish body to a recovery device.

8. The fish individual recognition method based on fluorescent staining and image target detection according to claim 6, characterized in that, The method further comprises: acquiring a third fish body image from a recovery state monitoring module; performing operculum opening and closing recognition on a target fish body in the first fish body image, and determining that the target fish body is in a recovery completion state when an operculum opening and closing frequency of the target fish body is greater than a preset frequency; controlling a second water valve to open, so as to transfer the target fish body to an identification device. 9.The fish individual recognition method based on fluorescent staining and image target detection according to claim 6, characterized in that, The method further comprises: acquiring a plurality of target fish body images of a target fish body under a plurality of illumination conditions, a plurality of shooting angles and a plurality of postures from the fish individual identification module; labeling a target mark pattern on a plurality of the target fish body images to generate a label file; training an image classification model in combination with the target fish body images and the corresponding label file; performing fish individual identification based on the image classification model.

10. The fish individual recognition method based on fluorescent staining and image target detection according to claim 9, characterized in that, The acquiring of the plurality of target fish body images of the target fish body under the plurality of illumination conditions, the plurality of shooting angles and the plurality of postures from the fish individual identification module comprises: obtain target fish videos of the target fish under various illumination conditions, various shooting angles and various postures from the fish individual recognition module; perform frame segmentation on the target fish videos and segment fish regions to obtain a plurality of single-frame fish images; frame extraction is performed on the plurality of single-frame fish images according to a preset frame interval to obtain a plurality of target fish images.

Images