A feeding system and method for bottom-feeding fish based on machine vision

Abstract

The present invention discloses a machine vision-based feeding system and method for bottom-feeding fish, relating to the technical field of aquaculture equipment. The system comprises: a recirculating aquaculture barrel, an underwater feeding frame, a video acquisition device, a video image processing and computing device, and an automatic feeding machine. The underwater feeding frame is located at the bottom of the recirculating aquaculture barrel and is used to define a feed delivery area. The video acquisition device is used to capture feeding videos of fish within the underwater feeding frame. The video image processing and computing device is used to run OpenCV image processing tools and a fish feeding desire judgment model to calculate a quantitative result of the fish feeding desire based on screenshots of the fish feeding video. The automatic feeding machine is located at the edge of the recirculating aquaculture barrel and is used to quantitatively deliver feed into the underwater feeding frame based on the quantitative result of the fish feeding desire. The present invention can achieve efficient and precise feeding of bottom-feeding fish, reduce the labor intensity during the factory farming of bottom-feeding fish, and improve work efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of aquaculture equipment, and in particular to a bottom-feeding fish feeding system and method based on machine vision. Background Art

[0002] With the rapid development of aquaculture and the increasing demand for high-quality proteins with different flavors, aquaculture species are becoming increasingly abundant. Bottom-feeding fish, as an important part of economic fish, have made significant contributions to meeting the demand for high-quality proteins with different flavors. Since the feeding process of bottom-feeding fish often occurs at the bottom of the water, compared with surface-feeding fish, the feeding process is difficult to observe, the feeding time is long, overfeeding is difficult to clean, and inaccurate feeding can easily cause water pollution. In order to improve the above problems, there is an urgent need for a feeding system and method suitable for bottom-feeding fish to improve aquaculture efficiency, avoid water pollution, and reduce the labor intensity of practitioners. Summary of the Invention

[0003] The purpose of the present invention is to provide a bottom-feeding fish feeding system and method based on machine vision, so as to achieve efficient and accurate feeding of bottom-feeding fish, reduce the labor intensity in the factory farming process of bottom-feeding fish, and improve work efficiency.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A bottom-feeding fish feeding system based on machine vision, comprising: a circulating water aquaculture tank, an underwater feeding frame, a video acquisition device, a video image processing and calculation device, and an automatic feeding machine;

[0006] The underwater feeding frame is located at the bottom of the circulating aquaculture barrel and is used to limit the feed delivery area; the video acquisition device is located above the underwater feeding frame and is used to collect the feeding video of the fish in the underwater feeding frame; the video image processing and calculation device is located outside the circulating aquaculture barrel and is used to run the OpenCV image processing tool and the fish feeding desire judgment model, and calculate the quantitative results of the fish feeding desire based on the screenshots of the fish feeding video; the automatic feeding machine is located at the edge of the circulating aquaculture barrel and is used to quantitatively deliver feed into the underwater feeding frame based on the quantitative results of the fish feeding desire.

[0007] Optionally, the automatic feeding machine includes: a controller, a stepping motor, a pressure sensor, a barrel and an auger;

[0008] The discharge port of the barrel is connected to the feed port of the auger, and the tail end of the auger is connected to the stepper motor to form a feeding assembly; the feeding assembly is connected to a support frame installed on the edge of the circulating aquaculture barrel; the pressure sensor is installed between the support frame and the feeding assembly; the pressure sensor, the stepper motor and the video image processing and calculation device are all connected to the controller; the pressure sensor is used to measure the remaining feed amount in the barrel; the controller is used to control the stepper motor to drive the auger to rotate according to the quantitative results of the fish feeding desire and the remaining feed amount, so as to quantitatively release feed into the underwater feeding frame.

[0009] Optionally, the video image processing computing device includes: a tower server; the tower server is installed with an OpenCV image processing tool and is deployed with a fish feeding desire judgment model;

[0010] The tower server is used to run the OpenCV image processing tool to take a screenshot of the fish feeding video at a set time node and perform image quality enhancement, compression and segmentation processing on the screenshot image to obtain a fish feeding image, and to run the fish feeding desire judgment model to calculate a quantitative result of the fish feeding desire based on the fish feeding image.

[0011] Optionally, the tower server is further used to train a machine vision model to obtain a fish feeding desire judgment model; the process of determining the fish feeding desire judgment model includes:

[0012] Get several sample videos;

[0013] Selectively capturing the sample video to obtain original images with different amounts of remaining feed in the underwater feeding frame, and performing image quality enhancement, compression, and segmentation processing on the original images to obtain a plurality of sample images;

[0014] Classifying and labeling the sample images and constructing a sample data set; the sample data set includes sample images of several different fish feeding desire classifications, and each sample image of the fish feeding desire classification is labeled with a corresponding fish feeding desire quantification value;

[0015] The sample data set is used to train a machine vision model to obtain a fish feeding desire judgment model.

[0016] Optionally, the corresponding relationship between the fish school feeding desire classification and the fish school feeding desire quantification value is:

[0017] Very weak - unblocked: -72, weak - unblocked: 2.00, medium - unblocked: 4.00, strong - unblocked: 6.00;

[0018] Very weak-weak blocking: -72, weak-weak blocking: 1.75, medium-weak blocking: 3.75, strong-weak blocking: 5.75;

[0019] Very weak-medium blocking: -54, weak-medium blocking: 1.50, medium-medium blocking: 3.50, strong-medium blocking: 5.50;

[0020] Very weak-strong block: -36, weak-strong block: 1.25, medium-strong block: 3.25, strong-strong block: 5.25;

[0021] Super Weak-Super Strong Block: -36, Weak-Super Strong Block: 1.00, Medium-Super Strong Block: 3.00, Strong-Super Strong Block: 5.00.

[0022] Optionally, the automatic feeder is set to feed several times a day, with a set duration for each feeding, and each feeding is carried out in several rounds; wherein the amount of feed in the first round of feeding is a set proportion of the total planned feeding amount each time, and the amount of feed in each subsequent round of feeding is determined by comparing the quantified result of the fish feeding desire at the set time node with the fish feeding desire trigger threshold corresponding to different feeding amounts.

[0023] Optionally, the fish feeding desire triggering thresholds corresponding to the different feeding amounts include: 0, 24 and 48;

[0024] When the quantitative result of the fish feeding desire is less than or equal to 0, the feeding amount level is level 1, and no feeding is performed;

[0025] When the quantitative result of the feeding desire of the fish school is greater than 0 and less than or equal to 24, the feeding amount level is level 2, and 2 / 10 of the total planned feeding amount is fed each time;

[0026] When the quantitative result of the feeding desire of the fish school is greater than 24 and less than or equal to 48, the feeding amount level is level 3, and 3 / 10 of the total planned feeding amount is fed each time;

[0027] When the quantitative result of the feeding desire of the fish school is greater than 48, the feeding amount level is level 4, and 4 / 10 of the total planned feeding amount is fed each time.

[0028] Optionally, the automatic feeding machine is set to feed 2 or 3 times a day, each feeding lasts 20 minutes, and each feeding is divided into 4 rounds, specifically including:

[0029] The first round of trial feeding is conducted, and the amount of feed fed is 1 / 10 of the total planned feeding amount per feeding. At 2 minutes and 30 seconds after the automatic feeder is turned on, the video image processing and calculation device receives the quantitative results of the fish feeding desire in the second minute, and compares the results with the fish feeding desire triggering thresholds corresponding to different feeding amounts to determine the amount of feed for the second round of feeding. After the amount of feed for the second round of feeding is determined, the second round of feeding is carried out at the third minute.

[0030] After the second round of feeding is completed, the quantified result of the fish feeding desire in the 9th minute is received from the video image processing and calculation device at 9 minutes and 30 seconds, and the result is compared with the triggering threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the third round of feeding. After the amount of feed for the third round of feeding is determined, the third round of feeding is carried out at the 10th minute;

[0031] After the third round of feeding is completed, the quantification result of the fish feeding desire in the 16th minute is received from the video image processing and calculation device at 16 minutes and 30 seconds. The result is compared with the trigger threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the fourth round of feeding. After the amount of feed for the fourth round of feeding is determined, the fourth round of feeding is carried out at the 17th minute. After the feeding is completed, the automatic feeder is turned off to complete the feeding task.

[0032] Optionally, the bottom-feeding fish feeding system further comprises: an information transmission module and a distribution box;

[0033] The video image processing and computing device is connected to the video acquisition device and the automatic feeding machine respectively through the information transmission module; the distribution box is connected to the video acquisition device, the video image processing and computing device and the automatic feeding machine respectively.

[0034] A bottom-feeding fish feeding method based on machine vision is applied to the bottom-feeding fish feeding system based on machine vision. The bottom-feeding fish feeding method comprises:

[0035] Using a video acquisition device to capture the feeding video of the fish in the underwater feeding frame located at the bottom of the circulating aquaculture barrel;

[0036] Using a video image processing computing device to run an OpenCV image processing tool and a fish feeding desire judgment model, and calculating a quantitative result of the fish feeding desire based on the fish feeding video screenshot;

[0037] An automatic feeding machine is used to quantitatively feed the feed into the underwater feeding frame according to the quantitative results of the feeding desire of the fish school.

[0038] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] The machine vision-based feeding system for bottom-feeding fish provided by the present invention first uses a video capture device to capture feeding videos of fish within a bottom-feeding frame. Next, the captured feeding videos are captured, enhanced, compressed, and segmented using the OpenCV image processing tool running on a video image processing and computing device. A fish feeding desire judgment model running on the video image processing and computing device further determines and quantifies the feeding desire of the captured images. Finally, an automatic feeder quantitatively releases feed into the bottom-feeding frame based on the quantified results of the fish feeding desire in the captured images, thereby achieving unmanned automatic feeding of bottom-feeding fish. This machine vision-based feeding strategy is more efficient and accurate than traditional feeding strategies based on worker experience. It can effectively address the problems of slow fish growth and feed waste caused by inaccurate manual feeding, reduce labor intensity in factory-farming bottom-feeding fish, and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 This is a structural diagram of the machine vision-based bottom-feeding fish feeding system provided by the present invention.

[0042] Figure 2 This is a flow chart of the one-time feeding operation of the automatic feeding machine provided by the present invention.

[0043] Explanation of symbols: circulating water aquaculture barrel-1, underwater feeding frame-2, underwater camera-3, NVR recorder-4, controller-5, pressure sensor-6, stepper motor-7, barrel-8, auger-9, video image processing and computing device-10. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] The purpose of the present invention is to provide a bottom-feeding fish feeding system and method based on machine vision, so as to achieve efficient and accurate feeding of bottom-feeding fish, reduce the labor intensity in the factory farming process of bottom-feeding fish, and improve work efficiency.

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] The present invention provides a feeding system for bottom-feeding fish based on machine vision. Figure 1 This is a structural diagram of the bottom-feeding fish feeding system based on machine vision provided by the present invention, such as Figure 1 As shown, the system includes: a recirculating aquaculture barrel 1, an underwater feeding frame 2, a video acquisition device, a video image processing and computing device 10, and an automatic feeding machine. The underwater feeding frame 2 is located at the bottom of the recirculating aquaculture barrel 1 and is used to define the feed delivery area. The video acquisition device is located above the underwater feeding frame and is used to capture feeding videos of fish in the underwater feeding frame 2. The video image processing and computing device 10 is located outside the recirculating aquaculture barrel 1 and is used to run the OpenCV image processing tool and the fish feeding desire judgment model to calculate the quantitative results of the fish feeding desire based on the screenshots of the fish feeding video. The automatic feeding machine is located at the edge of the recirculating aquaculture barrel 1 and is used to quantitatively deliver feed into the underwater feeding frame 2 based on the quantitative results of the fish feeding desire transmitted by the video image processing and computing device 10. Preferably, the fish feeding desire judgment model is trained based on the MobileNetV3 machine vision model.

[0048] The system further includes an information transmission module and a power distribution box. The video image processing and computing device 10 is connected to the video acquisition device and the automatic feeding machine via the information transmission module. The power distribution box is connected to the video acquisition device, the video image processing and computing device 10, and the automatic feeding machine. The information transmission module is used for data transmission; the power distribution box is used to provide power to the aforementioned devices. Preferably, the information transmission module includes a Wi-Fi module and a Bluetooth module.

[0049] The overall operation process of the system includes:

[0050] Step S1: The automatic feeding machine is turned on at a predetermined time according to the pre-set feeding logic.

[0051] Step S2: The video acquisition device acquires the fish feeding video in the underwater feeding frame in real time.

[0052] Step S3: The video image processing computing device is connected to the video acquisition device via a WIFI module. The OpenCV image processing tool running on it will take screenshots of the fish feeding video at the specified time node and perform image quality enhancement, reduction and cutting on the captured images.

[0053] Step S4: After a screenshot is cut into several parts in equal proportion, all the cut images are input into the fish feeding desire judgment model (i.e., the trained machine vision model) in sequence to judge the fish feeding desire. After the judgment is completed, the judgment results are digitally labeled and stored; after all the cut images of a screenshot have completed the judgment, digital labeling and storage of the feeding desire, all the stored numbers are added and summed, and finally the summation result is transmitted via Bluetooth to the automatic feeding machine installed on the edge of the circulating aquaculture barrel.

[0054] Step S5: At a fixed time point, the automatic feeding machine will receive the quantitative results of the fish feeding desire from the video image processing and calculation device (specifically, the sum of the digital annotations of all cut images in a screenshot), and compare it with the fish feeding desire trigger threshold corresponding to different feeding amounts. According to the comparison result, the stepper motor is controlled to rotate for quantitative feeding or standby without feeding.

[0055] The structure and function of each of the above parts are introduced in detail below.

[0056] The underwater feeding frame 2 is a rectangular, submerged rubber frame placed underwater. The frame measures 80cm x 60cm, with a 3cm edge height. This frame limits the sinking of the submerged feed, preventing it from being scattered across the bottom of the recirculating aquaculture tank 1 by the current.

[0057] The video capture device is specifically an underwater camera 3. This camera features IP68 waterproofing, 5-megapixel image capture resolution, a 2.8mm lens, and infrared fill-light capture. The camera is mounted 40-80 cm above the underwater feeding frame 2. The captured image covers the entire feeding frame 2, providing a clear view of the underwater feed. Furthermore, the camera must be connected to a Wi-Fi-enabled NVR 4, which connects the camera 3 and the video image processing and computing device 10 to the same local area network.

[0058] The video image processing computing device 10 specifically consists of a tower server, a monitor, a keyboard, and a mouse. The tower server's primary functions include training machine vision models, running OpenCV image processing tools, executing a fish feeding desire determination model, and transmitting and receiving signals. The monitor, keyboard, and mouse facilitate subsequent management and maintenance of the server. The device is installed within 50 meters of the recirculating aquaculture tank 1, ensuring stable signal transmission and low latency.

[0059] Specifically, the tower server is installed with an OpenCV image processing tool, which can take screenshots of fish feeding videos at set time nodes and enhance, compress and cut the screenshot images; the fish feeding desire judgment model running on it will judge and digitally label the feeding desire of each image cut from the screenshot. After the digital labeling of the feeding desire of all images cut from a screenshot is completed, the digital labeling of the feeding desire of all images is added together to obtain the quantitative result of the fish feeding desire corresponding to this screenshot. This result will be transmitted to the automatic feeder as the output of the tower server; before running the fish feeding desire judgment model, the tower server is also used to train the machine vision model to obtain the fish feeding desire judgment model.

[0060] The process of determining the fish feeding desire judgment model includes: obtaining a number of sample videos; selectively taking screenshots of the sample videos to obtain original images with different amounts of feed remaining in the underwater feeding frame, and performing image quality enhancement, compression and segmentation processing on the original images to obtain a number of sample images; classifying and annotating the sample images to construct a sample data set; the sample data set includes sample images of a number of different fish feeding desire classifications, and the sample images of each fish feeding desire classification are annotated with a corresponding fish feeding desire quantification value; and using the sample data set to train a machine vision model to obtain the fish feeding desire judgment model.

[0061] Furthermore, the specific determination process of the fish feeding desire judgment model is as follows:

[0062] Step S4-1: A large number of videos of fish feeding underwater, ie, sample videos, are collected by underwater cameras.

[0063] Step S4-2: selectively capture the collected video of the fish feeding underwater, and capture images of different feed remaining amounts at the bottom of the water during the entire feeding process, i.e., original images.

[0064] Step S4-3: After the original image is obtained, it will undergo three steps of processing: the first step is to enhance the image quality to make the image clearer; the second step is to compress the image. The original size of the video screenshot is 2560×1920 pixels, which is not conducive to image segmentation processing, so the image must be non-proportionally compressed, and the compressed image size is 2048×1536 pixels; after compression, the image is processed in the third step, that is, proportional segmentation, and divided into 12 pictures of 512×512 pixels, namely, sample images.

[0065] Step S4-4: Classify and label the segmented 512×512 pixel image. The labeling rules are as follows:

[0066] If there is no fish blocking the field of view, the feeding desire of the fish corresponding to the picture is divided into four categories according to the amount of feed in the field of view: ultra-weak, weak, medium and strong. That is, four classifications of fish feeding desire without fish blocking are obtained: ultra-weak-no blocking, weak-no blocking, medium-no blocking and strong-no blocking.

[0067] If there are fish blocking the field of view, the obstruction will be divided into 4 categories according to the proportion of the fish blocking the field of view: weak obstruction: blocked field of view ≤ 1 / 4 of the overall field of view; medium obstruction: 1 / 4 of the overall field of view < blocked field of view ≤ 2 / 4 of the overall field of view; strong obstruction: 2 / 4 of the overall field of view < blocked field of view ≤ 3 / 4 of the overall field of view; super strong obstruction: 3 / 4 of the overall field of view < blocked field of view.

[0068] The above four types of vision blockage classification and the four types of fish feeding desire classification described above are multiplied to form 16 types of fish feeding desire classification in the presence of fish blockage: super weak-weak blockage, weak-weak blockage, medium-weak blockage, strong-weak blockage; super weak-medium blockage, weak-medium blockage, medium-medium blockage, strong-medium blockage; super weak-strong blockage, weak-strong blockage, medium-strong blockage, strong-strong blockage; super weak-super strong blockage, weak-super strong blockage, medium-super strong blockage, strong-super strong blockage.

[0069] By adding the 16 types of fish feeding desire classifications with fish blocking the school and the 4 types of fish feeding desire classifications without fish blocking the school, a total of 20 types of fish feeding desire classifications are obtained. This classification will be used as the output of the machine vision model, and each classification corresponds to a quantitative value of fish feeding desire.

[0070] Specifically, the marking rule for the quantitative value of fish feeding desire is:

[0071] Very weak - unblocked: -72, weak - unblocked: 2.00, medium - unblocked: 4.00, strong - unblocked: 6.00;

[0072] Very weak-weak blocking: -72, weak-weak blocking: 1.75, medium-weak blocking: 3.75, strong-weak blocking: 5.75;

[0073] Very weak-medium blocking: -54, weak-medium blocking: 1.50, medium-medium blocking: 3.50, strong-medium blocking: 5.50;

[0074] Very weak-strong block: -36, weak-strong block: 1.25, medium-strong block: 3.25, strong-strong block: 5.25;

[0075] Super Weak-Super Strong Block: -36, Weak-Super Strong Block: 1.00, Medium-Super Strong Block: 3.00, Strong-Super Strong Block: 5.00.

[0076] Step S4-5: Divide the classified and labeled images into a training set and a test set in a ratio of 8:2; input the images in the training set into the MobileNetV3 machine vision model for training, and after the model training is completed, test it on the test set. The test classification accuracy rate must be higher than 90%.

[0077] The automatic feeding machine specifically consists of a controller 5, a stepper motor 7, a pressure sensor 6, a barrel 8, and an auger (i.e., a screw conveyor, also known as an auger) 9. The discharge port (i.e., the lower outlet) of the barrel 8 is connected to the feed port of the auger 9, and the tail end (i.e., the end) of the auger 9 is connected to the stepper motor 7, forming a feeding assembly. The feeding assembly is connected to a support frame mounted on the edge of the circulating aquaculture tank 1. A pressure sensor 6 is installed between the support frame and the feeding assembly. The pressure sensor 6, stepper motor 7, and video image processing and computing device 10 are all connected to the controller 5. The pressure sensor 6 is used to measure the amount of feed remaining in the barrel 8. The controller 5 controls the stepper motor 7 to drive the auger 9 to rotate based on the quantitative results of the fish feeding desire transmitted by the video image processing and computing device 10 and the amount of feed remaining in the barrel 8, thereby quantitatively releasing feed into the underwater feeding frame 2.

[0078] Specifically, the pressure sensor 6 measures the amount of remaining feed in the barrel 8 in real time, and the controller 5 performs precise quantitative feeding based on the data difference fed back by the pressure sensor 6 during the feeding process.

[0079] Preferably, the controller 5 is a low-power single-chip microcomputer with multi-IO port control, serial communication, SPI communication, I2C communication, WiFi communication, Bluetooth communication, PWM wave control, and AD analog-to-digital conversion functions.

[0080] Furthermore, the automatic feeding machine is set to feed several times a day, with a set duration for each feeding, and each feeding is carried out in several rounds; among them, the amount of feed in the first round of feeding is a set proportion of the total planned feeding amount each time, and the amount of feed in each subsequent round of feeding is determined by comparing the quantification result of the fish feeding desire transmitted by the video image processing and calculation device at the set time node with the fish feeding desire trigger threshold corresponding to different feeding amounts.

[0081] The following is a detailed introduction to the different feeding amount setting rules of the automatic feeder.

[0082] The automatic feeder sets the feeding amount to 4 levels. The feeding amount for each level is different. The following is the feeding amount for each level:

[0083] Level 1: No feeding;

[0084] Level 2: 2 / 10 of the total planned feed amount per feeding;

[0085] Level 3: 3 / 10 of the total planned feed amount per feeding;

[0086] Level 4: 4 / 10 of the total planned feed amount per feeding.

[0087] The automatic feeder also assigns a corresponding trigger threshold for the fish's feeding desire for each feeding level. If the quantified feeding desire result of the image it receives is within this range, it will feed the corresponding amount of feed for this level. The following is the quantified value range of the fish's feeding desire corresponding to each feeding level:

[0088] Level 1: Food desire ≤ 0;

[0089] Level 2: 0<Food Desire≤24;

[0090] Level 3: 24<Food Desire≤48;

[0091] Level 4: Food desire > 48.

[0092] That is: when the quantified result of the fish feeding desire is less than or equal to 0, the feeding amount level is level 1, and no feeding is done; when the quantified result of the fish feeding desire is greater than 0 and less than or equal to 24, the feeding amount level is level 2, and 2 / 10 of the total planned feeding amount is fed each time; when the quantified result of the fish feeding desire is greater than 24 and less than or equal to 48, the feeding amount level is level 3, and 3 / 10 of the total planned feeding amount is fed each time; when the quantified result of the fish feeding desire is greater than 48, the feeding amount level is level 4, and 4 / 10 of the total planned feeding amount is fed each time.

[0093] As a specific implementation method, the automatic feeding machine is set to feed 2 or 3 times a day, each feeding time is 20 minutes, and each feeding is divided into 4 rounds. At a fixed time node, the quantification result of the fish feeding desire transmitted by the video image processing and calculation device is received, and the result is compared with the fish feeding desire trigger threshold corresponding to different feeding amounts to determine the feeding amount for the next round. The above-mentioned fixed time node is determined through big data statistical analysis during the manual feeding process. Figure 2 The operation flow chart of the automatic feeding machine provided by the present invention is as follows: Figure 2 As shown in the figure, the feeding logic of the automatic feeding machine is set as follows:

[0094] The first round of trial feeding is conducted, and the amount of feed fed is 1 / 10 of the total planned feeding amount each time; at 2 minutes and 30 seconds after the automatic feeder is turned on, the quantitative result of the fish school's feeding desire in the second minute is received from the video image processing and calculation device, and the result is compared with the fish school's feeding desire trigger threshold corresponding to different feeding amounts to determine the amount of feed for the second round of feeding. After the amount of feed for the second round of feeding is determined, the second round of feeding is carried out at the 3rd minute.

[0095] After the second round of feeding is completed, the quantification result of the fish feeding desire in the 9th minute is received from the video image processing and calculation device at 9 minutes and 30 seconds. The result is compared with the fish feeding desire trigger threshold corresponding to different feeding amounts to determine the amount of feed for the third round of feeding. After the amount of feed for the third round of feeding is determined, the third round of feeding is carried out at the 10th minute.

[0096] After the third round of feeding is completed, the quantification result of the fish feeding desire in the 16th minute is received from the video image processing and calculation device at 16 minutes and 30 seconds. The result is compared with the trigger threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the fourth round of feeding. After the amount of feed for the fourth round of feeding is determined, the fourth round of feeding is carried out at the 17th minute. After the feeding is completed, the automatic feeder is turned off to complete the feeding task.

[0097] The following is a more detailed explanation using the equipment selection, installation and use during the bottom-feeding fish fry cultivation period as an example.

[0098] 1. The details of the construction of each equipment part are as follows:

[0099] (1) Select a circulating water aquaculture barrel of appropriate size. The diameter of the barrel is 4m. This size can not only ensure the comfort of the fry's living space, but also ensure that all fry can sense the feed falling into the feeding frame during the feeding process.

[0100] (2) Install an underwater feeding frame. The size of the feeding frame is 80cm×60cm, and the edge height of the feeding frame is 3cm. The installation of the feeding frame can not only limit the sinking position of the submerged feed, but also prevent the feed from being scattered on the entire bottom of the breeding barrel under the influence of the water flow.

[0101] (3) Build a video acquisition device and install a SK5-3P5X10 underwater camera on a stainless steel bracket. The underwater camera is connected to the stainless steel support frame through a rotatable connecting rod. The camera is installed at a height of 50 cm from the bottom of the water. The picture taken by the camera can just cover the entire feeding frame. In addition, the camera is connected to the DS-7808N-K1 / C video recorder of Hikvision to realize WiFi connection function and video recording and saving function.

[0102] (4) Build a video image processing computing device, which consists of a DIY tower server (its main internal components are an ASUS Z10PA-D8 motherboard, an Intel E52680V4 CPU, and a Gigabyte RTX2080Ti 11G graphics card), a Redmi V24FAB-RA monitor, and a Shuangfeiyan KR-9276UU keyboard and mouse set. The main functions of the tower server are to train machine vision models, run programs (i.e., OpenCV image processing tools and trained machine vision models), and send and receive signals. The main functions of the monitor, keyboard, and mouse are to facilitate the subsequent management and maintenance of the server. The server is installed within 50 meters of the fish fry breeding tank, which can ensure the stability and low latency of signal transmission.

[0103] (5) A DIY automatic feeder is installed on the edge of the circulating aquaculture barrel. The feeder is based on the Qingdao Haixing intelligent fixed feeder, on which an ESP32-S3 microcontroller and a pressure sensor are installed to achieve precise intelligent control of the feeder.

[0104] 2. The steps for data collection and machine vision model training are as follows:

[0105] The bottom-feeding fish fry in the breeding tank are artificially fed for 7 days, 3 times a day, and the feeding amount and feeding time are recorded each time.

[0106] After the 7-day feeding period, the fish fry in the tanks undergo a full cycle of artificial feeding until they enter the next rearing stage. The feeding methods during this stage have both similarities and differences to the previous 7-day feeding period. Similarities: the number of feedings per day and the feeding time period remain the same; differences: the entire feeding process is recorded by underwater cameras.

[0107] The feeding amount for each feeding is the average of the feeding amounts at the same feeding time point in the previous 7 days. Each feeding is fixed to 4 rounds, and the feeding amount for each round is 1 / 4 of the total feeding amount for each feeding. If the feed in a round is not eaten up, the next round of feeding will not be carried out.

[0108] In addition, the amount of feed should be adjusted once a month as the seedlings grow.

[0109] Time data collection: The time from the time when all the feed in each round of feeding lands in the feeding frame at the bottom of the water to the time when the feed is basically eaten is counted. This time will be used as a reference for setting the time point when the feeding machine receives the quantitative results of the fish feeding desire transmitted by the video image processing and calculation device.

[0110] Image Data Collection, Processing, and Annotation: OpenCV image processing tools were used to selectively capture footage of bottom-feeding fish larvae feeding, focusing on images captured at varying levels of bottom feed. These images were then enhanced. After image quality enhancement, the 2560×1920 pixel images were scaled down to 2048×1536 pixels. Finally, these scaled images were segmented proportionally into 12 images of 512×512 pixels each. A total of 5,000 images were collected, resulting in a total of 60,000 images after segmentation.

[0111] The cropped images were classified and labeled using the following rules: If there were no fish blocking the field of view, the corresponding fish feeding desire was categorized into four categories based on the amount of feed in the field of view: very weak, weak, medium, and strong. If there were fish blocking the field of view, the blocking was categorized into four categories based on the proportion of fish blocking the field of view: weak blocking (blocked field of view ≤ 1 / 4 of the total field of view); medium blocking (1 / 4 of the total field of view < blocked field of view ≤ 2 / 4 of the total field of view); strong blocking (2 / 4 of the total field of view < blocked field of view ≤ 3 / 4 of the total field of view); and very strong blocking (3 / 4 of the total field of view < blocked field of view).

[0112] Multiplying the four visual field obstruction classifications with the four previously described fish feeding desire classifications yields 16 fish feeding desire classifications with fish obstruction. Adding the 16 fish feeding desire classifications with fish obstruction and the four fish feeding desire classifications without fish obstruction yields a total of 20 fish feeding desire classifications, which serve as the output of the machine vision model.

[0113] The classified and labeled images are divided into training set and test set in a ratio of 8:2; the images in the training set are input into the MobileNetV3 machine vision model for training. After the model training is completed, it is tested on the test set. The accuracy of the test classification results must be higher than 90%.

[0114] 3. The settings for the image digital annotation and the triggering thresholds for the fish feeding desire corresponding to different feeding amounts of the feeder are as follows:

[0115] After the manual food desire classification is completed, each image used for machine vision model training will be added with a digital label representing the food desire. The labeling rules are: very weak-no obstruction: -72, weak-no obstruction: 2.00, medium-no obstruction: 4.00, strong-no obstruction: 6.00; very weak-weak obstruction: -72, weak-weak obstruction: 1.75, medium-weak obstruction: 3.75, strong-weak obstruction: 5.75; very weak-medium obstruction: -54, weak-medium obstruction: 1.50, medium-medium obstruction: 3.50, strong-medium obstruction: 5.50; very weak-strong obstruction: -36, weak-strong obstruction: 1.25, medium-strong obstruction: 3.25, strong-strong obstruction: 5.25; very weak-very strong obstruction: -36, weak-very strong obstruction: 1.00, medium-very strong obstruction: 3.00, strong-very strong obstruction: 5.00.

[0116] After setting the numerical labeling rules, the feeding amount for the feeder and the corresponding feeding desire trigger thresholds for different feeding amounts need to be set. The setting rules are as follows: There are four different feeding levels: Level 1: No feeding; Level 2: 2 / 10 of the total planned feeding amount; Level 3: 3 / 10 of the total planned feeding amount; Level 4: 4 / 10 of the total planned feeding amount. Each feeding level corresponds to a range of quantitative values ​​for the fish's feeding desire. If the quantitative value of the fish's feeding desire received by the video image processing device falls within this range, the feeder will feed the corresponding amount of feed. The following are the quantitative feeding desire values ​​corresponding to each level: Level 1: Feeding desire ≤ 0; Level 2: 0 < Feeding desire ≤ 24; Level 3: 24 < Feeding desire ≤ 48; Level 4: Feeding desire > 48.

[0117] 4. The feeding logic of the automatic feeder is set as follows:

[0118] The first round of trial feeding is conducted, and the amount of feed fed is 1 / 10 of the total planned feeding amount each time; at 2 minutes and 30 seconds after the automatic feeder is turned on, the quantitative result of the fish school's feeding desire in the second minute is received from the video image processing and calculation device, and the result is compared with the fish school's feeding desire trigger threshold corresponding to different feeding amounts to determine the amount of feed for the second round of feeding. After the amount of feed for the second round of feeding is determined, the second round of feeding is carried out at the 3rd minute.

[0119] After the second round of feeding is completed, the quantification result of the fish feeding desire in the 9th minute is received from the video image processing and calculation device at 9 minutes and 30 seconds. The result is compared with the fish feeding desire trigger threshold corresponding to different feeding amounts to determine the amount of feed for the third round of feeding. After the amount of feed for the third round of feeding is determined, the third round of feeding is carried out at the 10th minute.

[0120] After the third round of feeding is completed, the quantification result of the fish feeding desire in the 16th minute is received from the video image processing and calculation device at 16 minutes and 30 seconds. The result is compared with the trigger threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the fourth round of feeding. After the amount of feed for the fourth round of feeding is determined, the fourth round of feeding is carried out at the 17th minute. After the feeding is completed, the automatic feeder is turned off to complete the feeding task.

[0121] Furthermore, the present invention also provides a bottom-feeding fish feeding method based on machine vision, which is applied to the above-mentioned bottom-feeding fish feeding system based on machine vision. The method includes: using a video acquisition device to collect fish feeding videos in an underwater feeding frame located at the bottom of a circulating aquaculture barrel; using a video image processing and computing device to run the OpenCV image processing tool and a fish feeding desire judgment model, and calculating the quantitative results of the fish feeding desire based on the screenshots of the fish feeding video; and using an automatic feeding machine to quantitatively feed the bottom feeding frame according to the quantitative results of the fish feeding desire.

[0122] In summary, the present invention provides a system and method for feeding bottom-feeding fish based on machine vision. A video acquisition device is used to capture fish feeding videos in real time. A video image processing and calculation device periodically intercepts the fish feeding videos captured in real time by the video acquisition device to obtain fish feeding images, and quantifies the fish feeding desire corresponding to the images to obtain a quantified result of the fish feeding desire. An information transmission module transmits the quantified result of the fish feeding desire corresponding to the images. An automatic feeder quantitatively feeds feed based on the quantified result of the fish feeding desire corresponding to the images, thereby achieving unmanned and precise feeding of bottom-feeding fish. The present invention applies machine vision technology and automatic control technology to the development of automatic feeding equipment for bottom-feeding fish. The system uses machine vision technology to assess the feeding desire of fish and uses this as a control signal to control the automatic feeder to feed, thereby achieving precise feeding of bottom-feeding fish, improving work efficiency, reducing labor intensity, and avoiding water pollution caused by excessive feed feeding. Compared with existing technologies, the present invention can significantly reduce the time investment in feeding bottom-feeding fish; the feeding strategy based on machine vision is more efficient and accurate than the feeding strategy based on traditional worker experience, and can effectively reduce slow fish growth or feed waste caused by inaccurate manual feeding.

[0123] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A bottom-feeding fish feeding system based on machine vision, characterized in that: The bottom-feeding fish feeding system comprises: a circulating water culture barrel, an underwater feeding frame, a video acquisition device, a video image processing and computing device, and an automatic feeding machine; the video acquisition device is specifically an underwater camera; The underwater feeding frame is a rectangular submerged rubber frame placed underwater; it is located at the bottom of the recirculating aquaculture barrel and is used to define the feed delivery area; the video acquisition device is located above the underwater feeding frame and is used to capture the feeding video of the fish in the underwater feeding frame; the video image processing and calculation device is located outside the recirculating aquaculture barrel and is used to run the OpenCV image processing tool and the fish feeding desire judgment model to calculate the quantitative results of the fish feeding desire based on the screenshots of the fish feeding video; the automatic feeding machine is located at the edge of the recirculating aquaculture barrel and is used to quantitatively deliver feed into the underwater feeding frame based on the quantitative results of the fish feeding desire; The automatic feeding machine includes: a controller, a stepper motor, a pressure sensor, a barrel, and an auger; the discharge port of the barrel is connected to the feed port of the auger, and the tail end of the auger is connected to the stepper motor, forming a feeding assembly; the feeding assembly is connected to a support frame installed on the edge of the circulating aquaculture barrel; the pressure sensor is installed between the support frame and the feeding assembly; the pressure sensor, the stepper motor, and the video image processing and computing device are all connected to the controller; the pressure sensor is used to measure the remaining feed amount in the barrel; the controller is used to control the stepper motor to drive the auger to rotate based on the quantified result of the fish feeding desire and the remaining feed amount, so as to quantitatively release feed into the underwater feeding frame; The overall operation process of the bottom-feeding fish feeding system includes: Step S1: The automatic feeding machine is turned on according to the preset feeding logic; Step S2: The video acquisition device acquires the fish feeding video in the underwater feeding frame in real time; Step S3: The video image processing computing device is connected to the video acquisition device via a WIFI module. The OpenCV image processing tool running on the device takes screenshots of the fish feeding video at a specified time point and performs image quality enhancement, reduction, and cropping on the captured images. Step S4: After a screenshot of a school of fish feeding video is cut into several parts in equal proportion, all the cut images are sequentially input into the fish feeding desire judgment model to judge the fish feeding desire, and after the judgment is completed, the judgment results are digitally marked and stored; after all the cut images of a screenshot of a school of fish feeding video are judged, digitally marked and stored, all the stored numbers are added and summed, and finally the summation result is transmitted via Bluetooth to the automatic feeding machine installed on the edge of the circulating aquaculture barrel; wherein the marking rule is: if there is no fish blocking the field of view, then according to the field of view The amount of feed in the wild divides the feeding desire of the fish corresponding to the picture into four categories: ultra-weak, weak, medium, and strong. That is, the feeding desire of the fish is classified into four categories when there is no fish blocking: ultra-weak-no blockage, weak-no blockage, medium-no blockage, and strong-no blockage. If there is a fish blocking the field of view, the blockage is divided into four categories according to the proportion of the fish blocking the field of view: weak blockage: blocked field of view ≤ 1 / 4 of the entire field of view; medium blockage: 1 / 4 of the entire field of view < blocked field of view ≤ 2 / 4 of the entire field of view; strong blockage: 2 / 4 of the entire field of view < blocked field of view ≤ 3 / 4 of the entire field of view; super strong blockage: 3 / 4 of the entire field of view < blocked field of view; Step S5: The automatic feeding machine receives the quantitative results of the fish feeding desire from the video image processing and calculation device at a fixed time point, and compares it with the fish feeding desire trigger threshold corresponding to different feeding amounts. According to the comparison result, the stepper motor is controlled to rotate for quantitative feeding or standby without feeding.

2. The bottom-feeding fish feeding system based on machine vision according to claim 1, characterized in that: The video image processing computing device includes: a tower server; the tower server is installed with an OpenCV image processing tool and a fish feeding desire judgment model; The tower server is used to run the OpenCV image processing tool to take a screenshot of the fish feeding video at a set time node and perform image quality enhancement, compression and segmentation processing on the screenshot image to obtain a fish feeding image, and to run the fish feeding desire judgment model to calculate a quantitative result of the fish feeding desire based on the fish feeding image.

3. The bottom-feeding fish feeding system based on machine vision according to claim 2, characterized in that: The tower server is further used to train a machine vision model to obtain a fish feeding desire judgment model; the process of determining the fish feeding desire judgment model includes: Get several sample videos; Selectively capturing the sample video to obtain original images with different amounts of remaining feed in the underwater feeding frame, and performing image quality enhancement, compression, and segmentation processing on the original images to obtain a plurality of sample images; Classifying and labeling the sample images and constructing a sample data set; the sample data set includes sample images of several different fish feeding desire classifications, and each sample image of the fish feeding desire classification is labeled with a corresponding fish feeding desire quantification value; The sample data set is used to train a machine vision model to obtain a fish feeding desire judgment model.

4. The machine vision-based bottom-feeding fish feeding system according to claim 3, wherein the corresponding relationship between the fish feeding desire classification and the fish feeding desire quantification value is: Very weak - no block: -72, weak - no block: 2.00, medium - no block: 4.00, Strong-Unblocked: 6.00; Super weak-weak block: -72, weak-weak block: 1.75, Medium-Weak Block: 3.75, strong-weak blocking: 5.75; Very weak-medium block: -54, weak-medium block: 1.50, medium-medium block: 3.50, Strong-Medium Block: 5.50; Very weak-strong block: -36, weak-strong block: 1.25, medium-strong block: 3.25, strong-strong block: 5.25; Very weak to very strong block: -36, weak to very strong block: 1.00, medium to very strong block: 3.00, Strong-Super Strong Block: 5.

00.

5. The bottom-feeding fish feeding system based on machine vision according to claim 4, characterized in that: The automatic feeding machine is set to feed several times a day, with a set duration for each feeding, and each feeding is carried out in several rounds; the feed amount of the first feeding round is a set proportion of the total planned feeding amount each time, and the feed amount of each subsequent feeding round is determined by comparing the quantified results of the fish feeding desire at the set time node with the fish feeding desire trigger threshold corresponding to different feeding amounts.

6. The bottom-feeding fish feeding system based on machine vision according to claim 5, characterized in that: The fish feeding desire triggering thresholds corresponding to different feeding amounts include: 0, 24 and 48; When the quantitative result of the fish feeding desire is less than or equal to 0, the feeding amount level is level 1, and no feeding is performed; When the quantitative result of the feeding desire of the fish school is greater than 0 and less than or equal to 24, the feeding amount level is level 2, and 2 / 10 of the total planned feeding amount is fed each time; When the quantitative result of the feeding desire of the fish school is greater than 24 and less than or equal to 48, the feeding amount level is level 3, and 3 / 10 of the total planned feeding amount is fed each time; When the quantitative result of the feeding desire of the fish school is greater than 48, the feeding amount level is level 4, and 4 / 10 of the total planned feeding amount is fed each time.

7. The bottom-feeding fish feeding system based on machine vision according to claim 5, characterized in that: The automatic feeding machine is set to feed 2 or 3 times a day, each feeding lasts 20 minutes, and each feeding is divided into 4 rounds, specifically including: The first round of trial feeding is conducted, and the amount of feed fed is 1 / 10 of the total planned feeding amount per feeding. At 2 minutes and 30 seconds after the automatic feeder is turned on, the video image processing and calculation device receives the quantitative results of the fish feeding desire in the second minute, and compares the results with the fish feeding desire triggering thresholds corresponding to different feeding amounts to determine the amount of feed for the second round of feeding. After the amount of feed for the second round of feeding is determined, the second round of feeding is carried out at the third minute. After the second round of feeding is completed, the quantified result of the fish feeding desire in the 9th minute is received from the video image processing and calculation device at 9 minutes and 30 seconds, and the result is compared with the triggering threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the third round of feeding. After the amount of feed for the third round of feeding is determined, the third round of feeding is carried out at the 10th minute; After the third round of feeding is completed, the quantification result of the fish feeding desire in the 16th minute is received from the video image processing and calculation device at 16 minutes and 30 seconds. The result is compared with the trigger threshold of the fish feeding desire corresponding to different feeding amounts to determine the amount of feed for the fourth round of feeding. After the amount of feed for the fourth round of feeding is determined, the fourth round of feeding is carried out at the 17th minute. After the feeding is completed, the automatic feeder is turned off to complete the feeding task.

8. The bottom-feeding fish feeding system based on machine vision according to claim 1, characterized in that: The bottom-feeding fish feeding system further comprises: an information transmission module and a distribution box; The video image processing and computing device is connected to the video acquisition device and the automatic feeding machine respectively through the information transmission module; the distribution box is connected to the video acquisition device, the video image processing and computing device and the automatic feeding machine respectively.

9. A method for feeding bottom-feeding fish based on machine vision, applied to the bottom-feeding fish feeding system based on machine vision according to any one of claims 1 to 8, characterized in that: The bottom feeding fish feeding method comprises: Using a video acquisition device to capture the feeding video of the fish in the underwater feeding frame located at the bottom of the circulating aquaculture barrel; Using a video image processing computing device to run an OpenCV image processing tool and a fish feeding desire judgment model, and calculating a quantitative result of the fish feeding desire based on the fish feeding video screenshot; An automatic feeding machine is used to quantitatively feed the feed into the underwater feeding frame according to the quantitative results of the feeding desire of the fish school.

Images