Machine vision-based shrimp feeding behavior prediction and automatic feeding system and method

Abstract

The invention relates to a shrimp feeding behavior prediction and automatic feeding method based on machine vision, which comprises the following steps: S1, identifying and analyzing the behaviors of shrimps: identifying the dynamic and overall behaviors of the steps of the shrimps through feature labeling and model training of the steps of the shrimps, and judging the resting, swimming and foraging behavior states of the shrimps; s2, identifying and analyzing the behavior state of the shrimps through the body labeling features, and combining with identification of residual feed in water to adjust a feeding strategy, reduce waste and reduce water treatment load; s3, the computing power center obtains real-time image frames of the underwater high-definition camera, after computing power analysis is applied, signals are sent to the feeder control center, feedback information is received, and signal linkage is achieved; and S4, video real-time monitoring and breeding data real-time display. According to the method, the training model for accurately identifying the prawn foraging behavior can be realized, the method can adapt to the complex and changeable underwater environment of the circulating water culture pond, the technical robustness is high, accurate identification can be realized, and the intelligent degree is high.

Description

Technical Field

[0001] The invention relates to an automatic shrimp feeding system and method, belonging to the technical field of fishery breeding. Background Art

[0002] Green, pollution-free and healthy food has become the first choice for people. As an environmentally friendly and efficient farming method, factory-scale recirculating water shrimp farming has gradually attracted widespread attention. However, in a high-density factory farming environment, the way and amount of feeding is one of the key factors that determine the survival rate of shrimp and the farming efficiency.

[0003] When the feed is insufficient, it will lead to uneven growth among the shrimps, and even cause them to fight for food and hurt each other, seriously affecting the survival rate; while overfeeding not only increases the breeding cost, but also increases the load on the water body and increases the risk of disease. Although traditional artificial feeding can control the amount of residual feed through experience, its high labor cost, strong dependence, low efficiency and other problems limit its scope of application and effect.

[0004] In the prior art, in order to save feed, a technology of adaptive feeding of shrimp based on vision has emerged. However, the existing visual feeding strategy lacks the extraction of key points of the shrimp itself, lacks attention to the characteristics of the shrimp's own foraging behavior, and there are still deficiencies in the design of the feeding device, resulting in the actual feeding strategy still lacking specificity. Summary of the invention

[0005] The purpose of the present invention is to provide a shrimp feeding behavior prediction and automatic feeding system and method based on machine vision, which can realize a training model for accurately identifying the foraging behavior of shrimps, can adapt to the complex and changeable underwater environment of the circulating aquaculture pond, has high technical robustness, can realize accurate identification, and has a high degree of intelligence.

[0006] The present invention adopts the following technical solutions:

[0007] A method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision, comprising the following steps:

[0008] S1. Identify and analyze shrimp behavior: Identify the dynamics and overall behavior of shrimp legs through feature annotation and model training of shrimp legs, and judge their resting, swimming, and foraging behavior states;

[0009] S2, analyzing the behavior of shrimps through body marking features, combined with identifying residual bait in the water, to adjust feeding strategies, reduce waste and reduce water treatment load;

[0010] S3, the computing center obtains the real-time image frame of the underwater high-definition camera, uses computing power to analyze it, sends a signal to the feeding machine control center, and receives feedback information to achieve signal linkage;

[0011] S4. Real-time video monitoring and real-time display of breeding data.

[0012] Preferably, step S1 specifically includes:

[0013] S1a, Video acquisition using a high-definition underwater camera: Videos of shrimps in different water environments and under night vision mode were captured using a high-definition camera;

[0014] S1b, the key points of the shrimp annotation in the model training are: left eye, right eye, head, waist segment, tail segment, three points of the first step of the left and right legs, three points of the second step of the left and right legs, and three points of the third step of the left and right legs;

[0015] S1c, training model and key point prediction: Use the labeled data set to train the DLC model. After the model training is completed, use the model to predict the key points of the new video, which can form the skeleton of the shrimp. The left eye and the right eye of the skeleton are connected to the head respectively, the head is connected to the waist, the waist is connected to the tail, and the three points of the three pairs of walking legs of the shrimp are connected respectively; the body skeleton and three pairs of walking legs are shown in the video;

[0016] S1d, extracting a time series from the key point data for building a behavior recognition model, obtaining output data after recognition, which includes the key point position and confidence of each frame, and processing the output data into a format suitable for the behavior recognition model;

[0017] S1e, build behavior recognition model and behavior labeling training: use the extracted key point data as input, create a sequence model to identify different behaviors, assign labels to the data of each time period according to the labeled behaviors, and use these labels to train the sequence model;

[0018] S1f. Real-time behavior prediction: In real-time monitoring, the key point data extracted by training the DLC model using the data set is input into the sequence model for behavior recognition.

[0019] Preferably, step S2 specifically includes:

[0020] S2a. Use YOLOv11 to detect shrimps in the video: By continuously monitoring the position of each shrimp in the video, comparing the position information of the same shrimp in consecutive frames, calculating its displacement and speed, and using a delayed judgment mechanism, when an action is detected, wait for a certain period of time to confirm whether the behavior continues, so as to decide whether to push it to DLC for analysis; based on the calculated speed or displacement information, screen out shrimps that meet the set condition threshold. The initial threshold is set by observing the data distribution and the characteristics of the target behavior, and adjusted according to the actual environment of the shrimp farming; if the result exceeds the predetermined threshold, it is regarded as a significant change and pushed to DLC for feature analysis;

[0021] S2b, pass the screened shrimp to the data set to train the DLC model: after the DLC model analysis, if it is foraging behavior, send a signal to the Raspberry Pi, and the Raspberry Pi controls the feeder to feed once; when the shrimp is eating, use YOLOv11 to identify the shrimp eating and bait, then turn off the DLC analysis. When YOLOv11 cannot identify the residual bait and thinks that the feed has been eaten, then turn on the DLC analysis. When YOLOv11 recognizes the shrimp that meets the moving threshold condition again, the above procedure of step S2b is repeated until the shrimp foraging behavior cannot be captured, then the shrimp enters the swimming or resting state, and the feeder control center thinks that all the shrimp have finished eating.

[0022] Furthermore, the setting condition in step S2a is: the speed change exceeds the threshold value of the set percentage range, and the speed change percentage is evaluated by the following formula:

[0023]

[0024] Among them, V new Indicates the speed after the change, V old Indicates the speed before the change.

[0025] Furthermore, the set percentage range is from 10% to 30%.

[0026] An automatic feeding system based on the above method comprises a fish pond 7, a feeding machine 4, a computing center 3 with a GPU, an underwater network camera 1, a water surface collector 5, and a router 2; the feeding machine 4 is fixedly arranged above the fish pond 7, and the position below the feeding position corresponds to the position of the water surface collector 5, and the position directly opposite the lower part of the water surface collector 5 is a feeding area 6, and the underwater network camera 1 is aimed at the feeding area 6; when the shrimp enters the feeding area 6, the underwater network camera 1 captures the picture, which is transmitted to the computing center 3 via the router 2, and the YOLOv11 in the computing center 3 starts to recognize the picture, and the recognized picture shows the state of the shrimp, including "swimming", "resting", and "moving". When the "moving" shrimp is recognized, after threshold analysis, it meets the preset conditions, and the data set training DLC ​​model is pushed for further feature recognition Analysis, after the DLC model analysis of the data set training, if it is foraging behavior A1, a signal is sent to the feeding machine 4 with the control center, and the control center controls the feeding machine to feed once A3, and the feed is put into the water surface collector 5 to avoid the fluctuation of the water flow. The feed sinks and is fed to the feeding area 6 at a fixed point. The picture of the shrimp eating will be captured by the camera 1. When the shrimp eats A2, YOLOv11 is used to identify the shrimp eating and bait, and the DLC analysis is turned off. When YOLOv11 cannot identify the residual bait, the control center believes that the feed has been eaten, and then the data set training DLC ​​model analysis is turned on; when YOLOv11 recognizes the shrimp that meets the "moving" threshold condition again A4, the above program is repeated until the shrimp foraging behavior is no longer captured, then the shrimp enters the "swimming" or "resting" state, and it is believed that all the shrimp have eaten.

[0027] Preferably, the control center of the feeding machine 4 is a Raspberry Pi.

[0028] Preferably, all data is stored and displayed in a computing center 3 with a GPU.

[0029] Preferably, it also includes a guide rail, which is arranged across the top of the fish pond 7, and the feeding machine 7 can slide along the guide rail 4.

[0030] The beneficial effects of the present invention are:

[0031] 1) Improve feeding efficiency: Use a variety of machine vision fusion technologies to monitor shrimp behavior in real time to achieve precise feeding, reduce feed waste, and improve shrimp growth and survival rates.

[0032] 2) Reduce labor costs: Reduce manual monitoring and feeding, reduce labor costs and improve management efficiency.

[0033] 3) Improve water environment: Appropriate feeding amount can effectively reduce water pollution and improve the breeding environment.

[0034] 4) Predict and optimize growth plans: Infer the growth of shrimp through the collected behavior and feeding data, and optimize the breeding plan.

[0035] 5) Development of unmanned farming technology: The implementation of intelligent feeding technology is an important step in unmanned farming. Through the use of automated equipment and systems to manage and feed feed, human intervention can be reduced, and farming efficiency and accuracy can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The key points and skeleton diagram of the shrimp are marked for the model training in the present invention.

[0037] Figure 2 It is a structural schematic diagram of the shrimp feeding behavior prediction and automatic feeding system based on machine vision of the present invention.

[0038] Figure 3 It is a logical flow chart of intelligent feeding identification.

[0039] In the picture, 1. Underwater network camera, 2. Router, 3. Computing center, 4. Feeding machine, 5. Water surface collector, 6. Feeding area, 7. Fish pond. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0041] 1. Accurately identify and analyze shrimp behavior: Through feature annotation and model training of shrimp walking legs, accurately identify the dynamics of shrimp walking legs and overall behavior, and judge its resting, swimming, foraging and other behavioral states.

[0042] DeepLabCut (DLC for short) is mainly used in animal behavior research, usually using deep learning technology to identify and track body parts, and can be used to analyze and explain animal movement patterns and behaviors. The DLC model training is as follows:

[0043] 1. Use high-definition underwater cameras for video acquisition: Use high-definition cameras to capture videos of shrimps in different water environments and night vision modes.

[0044] 2. Key points of shrimp body marking:

[0045] Prawns have three pairs of walking legs for foraging and ingesting food. When the prawns are hungry and need to forage, the three pairs of walking legs behave differently from other behaviors. Therefore, if the three pairs of walking legs can be accurately identified, the behavior patterns of the prawns can be accurately distinguished. The key points of the model for annotating prawns are: left eye, right eye, head, waist segment, tail segment, three points of the first and second walking legs, three points of the third and left walking legs, see Figure 1 .

[0046] 3. Training model and key point prediction

[0047] The DLC model is trained using the labeled data set. After the model training is completed, the model is used to predict key points of the new video, and the skeleton of the shrimp can be formed. The left and right eyes of the skeleton are connected to the head, the head is connected to the waist, the waist is connected to the tail, and the three points of the three pairs of legs of the shrimp are connected. The body skeleton and the three pairs of legs can be displayed in the video.

[0048] 4. Extract time series from key point data to build behavior recognition models (such as LSTM)

[0049] After recognition, output data (usually a CSV file) is obtained, which contains the key point locations and confidences for each frame. Next, this data is processed into a format suitable for action recognition models (such as LSTM).

[0050] 5. Build behavior recognition models (such as LSTM) and behavior tagging training

[0051] Using the extracted keypoint data as input, create a sequence model (such as LSTM) to identify different behaviors. Assign labels to each time segment of data based on the labeled behavior (such as "foraging", "resting", "swimming", etc.). Use these labels to train the LSTM model.

[0052] 6. Real-time behavior prediction

[0053] In real-time monitoring, the key point data extracted by DLC is input into the LSTM model for behavior recognition.

[0054] 2. Multiple fusion recognition technologies: Combining Deeplabcut (DLC) and YOLOv11 model recognition technology, the behavioral state of shrimp is analyzed through body annotation feature recognition, combined with the identification of residual bait in the water to adjust the feeding strategy, reduce waste and reduce water treatment load.

[0055] DLC is very good at extracting and tracking specific points, and is suitable for biological and behavioral research. It is not as fast as YOLO in processing speed, especially for real-time applications. YOLO is known for its high-speed object detection and can quickly identify multiple objects in real-time applications, but YOLO's accuracy may not be as good as algorithms specifically designed for certain tasks. The online real-time method combining DLC ​​and YOLOv11 improves the robustness of the model.

[0056] 1. Use YOLOv11 to detect shrimp in the video.

[0057] By continuously monitoring the position of each shrimp in the video, comparing the position information of the same shrimp in consecutive frames, and calculating its displacement and speed. A delayed judgment mechanism is adopted. When an action is detected to start, a certain period of time is waited to confirm whether the behavior continues, so as to decide whether to push it to DLC analysis. According to the calculated speed or displacement information, shrimps that meet specific conditions (such as speed changes exceeding a certain threshold in the range of 10% to 30%) are screened out. The preliminary threshold can be set by observing the data distribution and the characteristics of the target behavior, and can be adjusted according to the actual environmental conditions of the farmed shrimp. The percentage of speed change can be evaluated by the following formula:

[0058]

[0059] If the result exceeds a predetermined threshold, it is considered a significant change and is pushed to the DLC for feature analysis.

[0060] 2. Pass the screened shrimp to DeepLabCut.

[0061] After the DLC model analysis, if it is foraging behavior, a signal is sent to the Raspberry Pi, and the Raspberry Pi controls the feeder to feed once. When the shrimps are eating, YOLOv11 can be used to identify the shrimps eating and the bait, and the DLC analysis is turned off. When YOLOv11 cannot identify the remaining bait, it is considered that the feed has been eaten, and the DLC analysis is turned on. When YOLOv11 recognizes the shrimps that meet the moving threshold conditions again, the above procedures are repeated until the shrimps foraging behavior is no longer captured, then the shrimps enter the swimming or resting state, and it is considered that all the shrimps have eaten.

[0062] YOLOv11 has the ability to identify shrimps swimming, shrimps stationary, shrimps moving, shrimps eating and feeding.

[0063] 3. Equipment integration and signal transmission: The computing center obtains the real-time image frames of the underwater high-definition camera, uses computing power to analyze, sends signals to the feeding machine control center, and receives feedback information to achieve signal linkage and seamless connection. Build an intelligent control system that integrates cameras, processing units, and feeding machines to ensure the effective transmission of data and signals.

[0064] 4. Real-time video monitoring and real-time display of breeding data: Set up a GUI interface with real-time monitoring, data display, early warning, manual control of feeding and other functions, and add real-time water quality conditions and other functions according to needs. Set daily feeding times, feeding amount and growth curve. The interface can add water quality conditions and has an oversaturation early warning function.

[0065] See also Figure 2 The intelligent automatic feeding system is composed of the following: an underwater network camera 1, a router 2, a computing center (with GPU) 3, a feeding machine (with Raspberry Pi) 4, a fish pond 7, etc.

[0066] The feeding machine 4 is arranged on a guide rail and can move along the guide rail to adjust the feeding position.

[0067] Combination Figure 2 and Figure 3 When the shrimp enters the feeding area 6, the underwater network camera 1 captures the picture and transmits it to the computing center (with GPU) 3 through the router 2. The YOLOv11 in the computing center 3 starts to recognize the picture. The recognized picture shows the state of the shrimp, including "swimming", "resting" and "moving". When the "moving" shrimp is recognized, it meets the conditions after threshold analysis and is pushed to the DLC for further feature recognition analysis. After the DLC model analysis, if it is a foraging behavior A1, a signal is sent to the feeder (with Raspberry Pi) 4. The Raspberry Pi controls the feeder to feed once A3. The feed will be put into the water surface collector 5, so that it will not be affected by the fluctuation of the water flow. The feed sinks and is fed to the feeding area 6 at a fixed point. The picture of the shrimp eating will be accurately captured by the camera 1. When the shrimp eats A2, YOLOv11 can be used to recognize the shrimp eating and bait, and the DLC analysis is turned off. When YOLOv11 cannot recognize the residual bait and believes that the feed has been eaten, the DLC analysis is turned on. When YOLOv11 recognizes the shrimp A4 that meets the moving threshold condition again, the above procedure is repeated until the shrimp foraging behavior is no longer captured. Then the shrimp enters the "swimming" or "resting" state, and it is considered that all shrimp have eaten. All data is stored and displayed in the computing center (with GPU) 3.

[0068] The artificial intelligence feeding technology of the present invention is based on the living habits and behavior patterns of shrimps, and can dynamically adjust the feeding amount, which not only effectively improves the feeding efficiency, but also can intelligently optimize the feeding strategy according to the real-time water quality data and the number of shrimps, reducing the waste of residual bait and the deterioration of water quality. The precise and efficient intelligent feeding system provides strong technical support for the further promotion of factory-scale circulating water shrimp farming, and at the same time has significant advantages in reducing costs, improving breeding benefits and ensuring food safety.

Claims

1. A method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision, characterized in that: The following steps are involved: S1. Identify and analyze shrimp behavior: Identify the dynamics and overall behavior of shrimp legs through feature annotation and model training of shrimp legs, and judge their resting, swimming, and foraging behavior states; S2, analyzing the behavior of shrimps through body marking features, combined with identifying residual bait in the water, to adjust feeding strategies, reduce waste and reduce water treatment load; S3, the computing center obtains the real-time image frame of the underwater high-definition camera, uses computing power to analyze it, sends a signal to the feeding machine control center, and receives feedback information to achieve signal linkage; S4. Real-time video monitoring and real-time display of breeding data.

2. The method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision according to claim 1, characterized in that: Step S1 specifically includes: S1a, Video acquisition using a high-definition underwater camera: Videos of shrimps in different water environments and under night vision mode were captured using a high-definition camera; S1b, the key points of the shrimp annotation in the model training are: left eye, right eye, head, waist segment, tail segment, three points of the first step of the left and right legs, three points of the second step of the left and right legs, and three points of the third step of the left and right legs; S1c, training model and key point prediction: Use the labeled data set to train the DLC model. After the model training is completed, use the model to predict the key points of the new video, which can form the skeleton of the shrimp. The left eye and the right eye of the skeleton are connected to the head respectively, the head is connected to the waist, the waist is connected to the tail, and the three points of the three pairs of walking legs of the shrimp are connected respectively; the body skeleton and three pairs of walking legs are shown in the video; S1d, extracting a time series from the key point data for building a behavior recognition model, obtaining output data after recognition, which includes the key point position and confidence of each frame, and processing the output data into a format suitable for the behavior recognition model; S1e, build behavior recognition model and behavior labeling training: use the extracted key point data as input, create a sequence model to identify different behaviors, assign labels to the data of each time period according to the labeled behaviors, and use these labels to train the sequence model; S1f. Real-time behavior prediction: In real-time monitoring, the key point data extracted by training the DLC model using the data set is input into the sequence model for behavior recognition.

3. The method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision according to claim 1, characterized in that: Step S2 specifically includes: S2a. Use YOLOv11 to detect shrimps in the video: By continuously monitoring the position of each shrimp in the video, comparing the position information of the same shrimp in consecutive frames, calculating its displacement and speed, and using a delayed judgment mechanism, when an action is detected, wait for a certain period of time to confirm whether the behavior continues, so as to decide whether to push it to DLC for analysis; based on the calculated speed or displacement information, screen out shrimps that meet the set condition threshold. The initial threshold is set by observing the data distribution and the characteristics of the target behavior, and adjusted according to the actual environment of the shrimp farming; if the result exceeds the predetermined threshold, it is regarded as a significant change and pushed to DLC for feature analysis; S2b, pass the screened shrimp to the data set to train the DLC model: after the DLC model analysis, if it is foraging behavior, send a signal to the Raspberry Pi, and the Raspberry Pi controls the feeder to feed once; when the shrimp is eating, use YOLOv11 to identify the shrimp eating and bait, then turn off the DLC analysis. When YOLOv11 cannot identify the residual bait and thinks that the feed has been eaten, then turn on the DLC analysis. When YOLOv11 recognizes the shrimp that meets the moving threshold condition again, the above procedure of step S2b is repeated until the shrimp foraging behavior cannot be captured, then the shrimp enters the swimming or resting state, and the feeder control center thinks that all the shrimp have finished eating.

4. The method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision according to claim 3, characterized in that: The set condition in step S2a is: the speed change exceeds the threshold of the set percentage range, and the speed change percentage is evaluated by the following formula: Among them, V new Indicates the speed after the change, V old Indicates the speed before the change.

5. The method for predicting shrimp feeding behavior and automatically feeding shrimp based on machine vision according to claim 4, characterized in that: The setting percentage ranges from 10% to 30%.

6. An automatic feeding system based on the method according to any one of claims 1 to 5, characterized in that: It includes a fish pond (7), a feeding machine (4), a computing center with a GPU (3), an underwater network camera (1), a water surface collector (5), and a router (2); The feeding machine (4) is fixedly arranged above the fish pond (7), and the position below the feeding position corresponds to the position of the water surface collector (5). The position directly opposite the lower part of the water surface collector (5) is the feeding area (6), and the underwater network camera (1) is aimed at the feeding area (6); when the shrimp enters the feeding area (6), the underwater network camera (1) captures the picture, which is transmitted to the computing center (3) via the router (2), and the YOLOv11 in the computing center (3) starts to recognize the picture. The recognized picture shows the states of the shrimp, including "swimming", "resting" and "moving". When the "moving" shrimp is recognized, after threshold analysis, it meets the preset conditions, then the data set training DLC ​​model is pushed for further feature recognition analysis. After the data set training DLC ​​model analysis, if it is foraging behavior (A1), then a signal is sent to the feeding machine (4) with the control center. The control center controls the feeding machine to feed once (A3). The feed is put into the water surface collector (5) to avoid being affected by the fluctuation of the water flow. The feed sinks and is fed to the feeding area (6) at a fixed point. The picture of the shrimp eating will be captured by the camera (1). When the shrimp eats (A2), YOLOv11 is used to identify the shrimp eating and the bait, and the DLC analysis is turned off. When YOLOv11 cannot identify the residual bait, the control center believes that the feed has been eaten, and then the data set training DLC ​​model analysis is turned on; when YOLOv11 recognizes the shrimp that meets the "moving" threshold condition again (A4), the above procedures are repeated until the shrimp foraging behavior cannot be captured, then the shrimp enters the "swimming" or "resting" state, and it is believed that all the shrimp have eaten.

7. The automatic feeding system according to claim 6, characterized in that: The control center of the feeding machine (4) is a Raspberry Pi.

8. The automatic feeding system according to claim 6, characterized in that: All data is stored and displayed in a computing center (3) equipped with GPU.

9. The automatic feeding system according to claim 6, characterized in that: It also comprises a guide rail which is arranged across the top of the fish pond (7), and the feeding machine (7) can slide along the guide rail (4).

Images