A precision feeding system and method based on machine vision

Abstract

The application discloses a kind of precision feeding systems and methods based on machine vision, system includes image acquisition module, image processing module, satiation degree evaluation module, automatic feeding machine and control module;Image acquisition module is used to collect the water surface water image of fish pond in real time;Image processing module is signal connected with image acquisition module, and the water surface water image of real-time collection is segmented using convolutional neural network, feature extraction is carried out, and the water feature of water surface water image is obtained Water feature includes the size and duration of water flower;And the water feature before feeding, feeding, after feeding is stored in feeding water image database;Satiation degree evaluation module is based on satiation degree evaluation model, and the feeding activity and satiation degree of fish population are evaluated according to feeding water feature.The invention adjusts feeding speed according to the collection and analysis of water surface water feature, so that the feeding speed is coordinated with the feeding speed, precision feeding is realized, and the generation of residual bait is reduced.

Description

A Precision Feeding System and Method Based on Machine Vision Technical Field

[0001] This invention relates to the field of aquaculture technology, specifically a precision feeding system and method based on machine vision. Background Technology

[0002] my country is the world's largest aquaculture nation, accounting for approximately 70% of global aquaculture production. This massive scale of aquaculture has a profound and positive impact on my country's national economy and social development. Particularly in freshwater fish farming, my country's geographical and climatic conditions provide an excellent environment, making freshwater fish farming a crucial component of my country's aquaculture industry. This not only helps meet domestic protein demand but also creates significant economic value for exports.

[0003] However, despite the large scale of aquaculture in my country, serious challenges exist in actual production, such as low feeding efficiency and environmental pollution. Under current feeding methods, approximately 15% of the feed is not effectively consumed and settles at the bottom of the pond or dissolves in the water, increasing the carbon, nitrogen, and phosphorus content in the aquaculture water, leading to environmental pollution and increased aquaculture risks. Accurately analyzing the feeding status of fish and precisely feeding them is crucial for reducing pollution and aquaculture costs, and ensuring fish survival rates.

[0004] To better leverage the advantages of aquaculture and promote the sustainable development of the industry, it is necessary to adopt innovative methods and technological means to improve aquaculture efficiency, reduce environmental burden, and ensure that aquaculture plays a more positive role in the national economy and society. Summary of the Invention

[0005] To address the challenges of precise feeding in existing fish farming practices, which often result in uneaten feed and negatively impact water quality, this invention provides a machine vision-based precision feeding system and method. This invention achieves precise feeding by real-time acquisition and analysis of water surface splash characteristics, reducing uneaten feed. Furthermore, the image processing module analyzes water splash characteristics before, during, and after feeding to further refine the feeding process and prevent overfeeding that could lead to uneaten feed or incomplete satiation.

[0006] The technical solution adopted in this invention is:

[0007] A machine vision-based precision feeding system includes an image acquisition module, an image processing module, a feeding splash image database, a satiety assessment module, an automatic feeder, and a control module.

[0008] The image acquisition module is used to acquire images of the water surface and splashes in the fishpond in real time.

[0009] The image processing module is connected to the image acquisition module. It uses a convolutional neural network to segment and extract features from the real-time acquired water surface splash images, and obtains the splash features of the water surface splash images, including the size and duration of the splashes. The splash features before, during, and after feeding are stored in the feeding splash image database.

[0010] The satiety assessment module, based on the satiety assessment model, evaluates the feeding activity and satiety of fish groups according to the characteristics of feeding splashes;

[0011] The control module is connected to the image processing module, the satiety assessment module, and the automatic feeder. Based on the water splash characteristics of the real-time water surface image obtained by the image processing module, and the feeding activity and satiety of the fish obtained by the satiety assessment module, the control module determines the feeding start time and feeding end time. Based on the satiety of the fish and the interval between the last feeding, the control module determines the estimated feeding amount and feeding speed, and sends feeding control commands to the automatic feeder to control the operation of the automatic feeder.

[0012] An automatic feeder is used to feed fish into a fishpond according to the control instructions of the control module.

[0013] Furthermore, the image acquisition module includes a camera for acquiring images and an LED light source for supplementing the lighting of the fishpond.

[0014] Furthermore, "before feeding" refers to 10-20 minutes before the scheduled feeding time; "after feeding" refers to 10-20 minutes after the automatic feeder stops feeding.

[0015] Furthermore, the satiety assessment model is an LSTM (Long Short-Term Memory) neural network model constructed based on the biological characteristics of fish, the experience of fishery experts, and the characteristics of water splashes on the water surface during the fish farming process.

[0016] The process of constructing the satiety assessment model is as follows:

[0017] (1) Data preprocessing: Clean the collected data and remove invalid or outlier values; normalize the data to ensure that different features have equal weights in the model; convert the time series data into the shape required by the LSTM model, i.e., [samples, time steps, features].

[0018] (2) Model building

[0019] Choose a library, use the Python programming language, and install NumPy for mathematical operations, TensorFlow, and Keras;

[0020] Build an LSTM model using Keras' Sequential model, adding LSTM layers and fully connected layers; select activation and loss functions, and compile the model;

[0021] (3) Model training: Train the LSTM model using the preprocessed data, specifying the number of iterations and batch size; during the training process, monitor the model's performance and prevent overfitting.

[0022] Furthermore, the activation function is ReLU or sigmoid, and the loss function is mean squared error or cross-entropy.

[0023] The feeding method of the precision feeding system based on machine vision includes the following steps:

[0024] S1. The image acquisition module acquires images of water surface splashes in real time. The image processing module uses a convolutional neural network to segment and extract features from the acquired water surface splash images to obtain the splash features of the water surface splash images.

[0025] S2. The satiety assessment module determines the satiety level of the fish group based on the water splash characteristics in the water surface image. The control module determines the feeding start time, estimated feeding amount, and initial feeding speed based on the satiety level and feeding interval. It also adjusts the feeding speed and feeding end time of the automatic feeder based on the water splash characteristics during feeding.

[0026] Furthermore, the control module has a built-in threshold for water splash characteristics before feeding. When the size of the water splash characteristics in the water surface image before feeding is greater than the set threshold and lasts for 4 to 6 minutes, the automatic feeder is controlled to start feeding. The estimated feeding amount and initial feeding speed are determined based on the interval between the last feeding and the satiety assessment model.

[0027] Furthermore, the water splash image characteristics, feeding activity, satiety level, actual feeding amount and feeding speed within 10-20 minutes before feeding begins, during feeding and 10-20 minutes after feeding stops are used to update and optimize the satiety assessment module.

[0028] Furthermore, when the feeding interval is greater than 6 hours, the duration of a single feeding should be 30 to 60 minutes.

[0029] Furthermore, the convolutional neural network includes convolutional layers and connection layers. The convolutional layer includes convolution, activation function, and pooling layer. The image processing module uses a unified convolution kernel to perform sliding convolution on the image. The activation function is the rectified linear function ReLU, with the function expression f(x) = max(0,x). The pooling layer uses max pooling or mean pooling. When building the convolutional neural network model, gradient descent and backpropagation algorithms are used, and the convolutional neural network model is first trained using a subset of samples.

[0030] The beneficial effects of this invention are:

[0031] This invention achieves precise feeding by acquiring real-time images of water surface splashes and analyzing their characteristics, thereby reducing uneaten food. Furthermore, the image processing module analyzes the splash characteristics of the fish before, during, and after feeding to further enhance precision feeding and prevent overfeeding, which can lead to uneaten food or incomplete satiation. Excessive uneaten food affects water quality and wastes feed, while incomplete satiation hinders fish growth.

[0032] In this invention, the need for advance feeding and the initial feeding speed are determined based on the characteristics of the water splash before feeding. The feeding time of the fish is controlled by a floating mechanism, which facilitates precise feeding control and makes the feeding more in line with the actual feeding state of the fish. The feeding speed is adjusted in real time according to the feeding state of the fish, which facilitates the control of feeding and achieves precise feeding. This reduces the generation of uneaten food, avoids excessive uneaten food from affecting water quality, and reduces the frequency of water changes due to deterioration of water quality. At the same time, it reduces the waste of fish food. The feeding speed is adjusted in real time according to the characteristics of the water splash during feeding, and the need for refeeding is determined according to the characteristics of the water splash after feeding, so as to avoid the situation where the fish are not fully satiated, which is not conducive to the growth of the fish.

[0033] The water splash image features, feeding activity, satiety level, actual feeding amount and feeding speed in the 10-20 minutes before feeding begins, during feeding and 10-20 minutes after feeding stops in this invention can also be used to update and optimize the satiety assessment module. Attached Figure Description

[0034] Figure 1 is a flowchart of the machine vision-based precision feeding system of the present invention, which adjusts the feeding strategy according to the water splash.

[0035] Figure 2 is a schematic diagram of the structure of the machine vision-based precision feeding system described in this invention.

[0036] Figure 3 is a side view of the machine vision-based precision feeding system described in this invention.

[0037] In the diagram, 11. First camera, 12. LED light source, 2. Computer, 3. Control module, 31. Embedded processing unit, 4. Automatic feeder. Detailed Implementation

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0039] Figures 1-3 illustrate a specific embodiment of the machine vision-based precision feeding system of the present invention. The system includes an image acquisition module, an image processing module, a feeding splash image database, a satiety assessment module, an automatic feeder 4, and a control module 13. The automatic feeder 4 is installed at the edge of the fishpond and is used to continuously feed the fishpond with bait at a certain feeding rate.

[0040] An image acquisition module is used to acquire images of the water surface and splashes in the fishpond in real time. The image acquisition module includes a camera 11 for acquiring images and an LED light source 12 for supplementing the lighting of the fishpond. The LED light source 12 can provide supplementary lighting at night or on cloudy days or other conditions with poor lighting, so that the camera 11 can acquire images of the water surface.

[0041] The image processing module, connected to the image acquisition module, can use computer 2 to process images. The image processing module uses a convolutional neural network to segment and extract features from the water surface images transmitted from the image acquisition module, analyzing water splash characteristics, including the size and duration of the splashes. The image processing module continuously analyzes the water surface splash characteristics before, during, and after feeding, and stores these characteristics in a feeding splash image database. "Before feeding" refers to 10-20 minutes before the feeding time, and "after feeding" refers to 10-20 minutes after the automatic feeder 4 stops feeding.

[0042] Convolutional Neural Networks (CNNs) consist of convolutional layers and connection layers. Convolutional layers include convolution, activation functions, and pooling layers. Using a uniform convolution kernel for sliding convolution on an image ensures that the weight parameters remain unchanged, thus reducing the number of parameters to be trained. In CNNs, the parameters are the elements of the convolution kernel; using uniform convolution on the entire image achieves parameter sharing during the convolution process. The combination of parameter sharing and pooling layers enables CNNs to effectively process image data and is robust to changes in the position of objects within the image. Parameter sharing allows the output to still extract features even if the input object has moved, although the extracted features may have different positions. For example, using an edge-extraction convolution kernel on an original image means that even if the original image is translated, the features in the convolutioned image will only be partially shifted, yet image extraction can still be achieved.

[0043] The activation function used in the second layer is the rectified linear ReLU function, expressed as f(x) = max(0,x). Compared to the Tanh function, the ReLU function is less prone to gradient saturation during backpropagation, meaning the gradient approaches zero when the neuron input is too large or too small. This is because the gradient is multiplied by the derivative of the activation function during backpropagation. Clearly, the derivative tends to zero when the input is too large, and gradient saturation slows down the convergence of gradient descent. Furthermore, the ReLU function is a non-linear function, and the choice of a non-linear activation function has a significant impact on the network. If all activation functions in the second layer are linear, the entire network will also be linear. This means the network cannot solve linearly inseparable problems, limiting its expressive power. For example, a single-hidden-layer neural network with a linear activation function cannot solve the XOR problem, but one with a non-linear activation function like ReLU can.

[0044] The pooling layers in the third stage employ max pooling or mean pooling to reduce the dimensionality of the feature maps, decrease the number of parameters in the network, avoid overfitting, and improve the computational speed and efficiency of the model. The pooling layers then output the pooled features to the connection layers.

[0045] In convolutional neural networks (CNNs), sparse connections in the connection layers reduce the number of computational parameters and steps. For example, if two layers of a neural network are fully connected, and each layer has M neurons, then M × N parameters are required. However, with sparse connections, not every neuron is connected to all neurons in the next layer, but only a subset. Assuming each neuron is connected to only K neurons in the next layer, and since K is generally much smaller than M, the required number of parameters is K × N, significantly reducing the total number of parameters. In CNNs, sparse interactions are achieved by controlling the size of the convolutional kernel, which determines the degree of sparse interaction.

[0046] When building a convolutional neural network model, it is necessary to first train the convolutional neural network model using a subset of samples. In this embodiment, gradient descent and backpropagation algorithms are used to train the convolutional neural network model.

[0047] The satiety assessment module, based on a satiety assessment model, evaluates the feeding activity and satiety of fish populations according to the characteristics of feeding splashes. The satiety assessment model is an LSTM (Long Short-Term Memory) neural network model constructed based on the biological characteristics of fish, the experience of fisheries experts, and the characteristics of surface splashes during fish farming.

[0048] The process of constructing the satiety assessment model is as follows:

[0049] (1) Data preprocessing: Clean the collected data and remove invalid or outliers; normalize the data to ensure that different features have equal weights in the model; convert the time series data into the shape required by the LSTM model, i.e., [samples, time steps, features].

[0050] (2) Model building

[0051] Choose a library, use the Python programming language, and install NumPy for mathematical operations, TensorFlow, and Keras;

[0052] Construct an LSTM model using Keras' Sequential model, adding LSTM layers and fully connected layers; select the activation function and loss function, and compile the model. The activation function is ReLU or sigmoid, and the loss function is mean squared error or cross-entropy.

[0053] (3) Model training: Train the LSTM model using the preprocessed data, specifying the number of iterations and batch size; during the training process, monitor the model's performance and prevent overfitting.

[0054] The control module 13 is connected to the image processing module, the satiety assessment module, and the automatic feeder 4. Based on the water splash characteristics of the real-time water surface image obtained by the image processing module, and the feeding activity and satiety of the fish obtained by the satiety assessment module, it determines the feeding start time, estimated feeding amount, feeding speed, and feeding end time, and sends feeding control commands to the automatic feeder 4 to control its operation. Specifically, the image acquisition module acquires real-time water surface splash images of the fishpond, and the image processing module obtains the water splash characteristics of the water surface splash images. When the satiety assessment module determines that the water splash characteristics at that moment meet the pre-feeding water splash characteristics and this condition persists for 4-6 minutes, the control module 13 controls the automatic feeder to start feeding; otherwise, pre-feeding is not performed. The estimated feeding amount and initial feeding speed are determined based on the interval between the last feeding and the satiety assessment model. Specifically, the estimated feeding amount and initial feeding speed are determined based on the analyzed water splash characteristics before feeding and the fish's feeding activity level obtained from the satiety assessment module. For example, if the feeding time is set at 8:00 AM, and the water splash image captured before the feeding time meets the feeding initiation conditions, feeding is started earlier; if the water splash image captured at 8:00 AM still does not meet the feeding initiation conditions, the feeding time is postponed. Feeding is stopped when the water splash characteristics reach the levels after feeding. This floating control of the fish's feeding time facilitates precise feeding control, making the feeding more consistent with the fish's actual feeding state. The time interval between feedings can reflect the satiation level of the fish to some extent. The control module 13 determines the estimated feeding amount and initial feeding speed based on the feeding activity and satiation level of the fish obtained by the satiation assessment module. It also assesses the feeding status of the fish in real time based on the changes in water splash during the process and adjusts the feeding speed in real time. This facilitates the control of feeding, achieves precise feeding, and reduces the generation of uneaten feed.

[0055] The control module 13 determines whether to feed again based on the water splash characteristics after feeding analyzed by the image processing module. If, based on the water splash characteristics obtained by the image processing module, the satiation assessment module determines that the fish still have a feeding desire for 4-6 minutes, the control module 13 controls the automatic feeder 4 to feed again at the corresponding feeding speed. Because the bait sinks, the fish dive to the bottom to feed, and the camera cannot capture the splash image, mistakenly believing that feeding has ended. If, after a period of time, the splash image is captured and it is found that the fish have risen again and still have a feeding desire, feeding is repeated to avoid the fish not being fully satiated. Additionally, the splash duration is set to 4-6 minutes to avoid misjudgment. When the feeding interval is greater than 6 hours, the automatic feeder 4 feeds the fishpond for 30-60 minutes at a time, corresponding to the fish's feeding time, facilitating observation of the fish's feeding status.

[0056] In this embodiment, the control module 3 adopts an embedded processing unit 31. In addition, the water splash image characteristics, feeding activity, satiety level, actual feeding amount and feeding speed within 10-20 minutes before feeding begins, during feeding and 10-20 minutes after feeding stops are used for updating and optimizing the satiety assessment module.

[0057] The feeding method of the machine vision-based precision feeding system includes the following steps:

[0058] S1. The image acquisition module acquires water surface splash images in real time. The image processing module uses a convolutional neural network to segment and extract features from the acquired water surface splash images to obtain the splash features of the water surface splash images.

[0059] S2. The satiety assessment module determines the satiety of the fish group based on the water splash characteristics of the water surface image. The control module 13 determines the feeding start time, estimated feeding amount, and initial feeding speed based on the satiety and feeding time interval. It also adjusts the feeding speed and feeding end time of the automatic feeder 4 based on the water splash characteristics during feeding after the start of feeding.

[0060] The control module incorporates a pre-feeding splash feature threshold. If the splash size in the pre-feeding water surface image exceeds the set threshold and persists for 4-6 minutes, the automatic feeder is activated to begin feeding. The estimated feeding amount and initial feeding speed are determined based on the interval between feedings and the previous feeding, as well as a satiety assessment model. Furthermore, the splash image features, feeding activity, satiety level, actual feeding amount, and feeding speed observed 10-20 minutes before, during, and after feeding are used to update and optimize the satiety assessment module. When the feeding interval is greater than 6 hours, each feeding session lasts 30-60 minutes.

[0061] This invention uses real-time image acquisition and analysis of water surface splashes to determine feeding characteristics, achieving precise feeding and reducing uneaten food. Furthermore, the image processing module analyzes water splash characteristics before, during, and after feeding to further refine feeding and prevent uneaten food from resulting in fish not being fully satiated or being overfed. The invention determines whether to feed early and the initial feeding speed based on pre-feeding splash characteristics, using a floating control system to precisely control feeding time and ensure it aligns with the fish's actual feeding state. Real-time adjustments to the feeding speed based on the fish's feeding status facilitate precise feeding control, reducing uneaten food and minimizing water quality impact and the frequency of water changes due to deterioration. It also reduces food waste. Real-time adjustments to the feeding speed based on water splash characteristics during feeding and determining whether to feed again based on post-feeding splash characteristics prevent fish from being fully satiated, as this is detrimental to fish growth. Furthermore, the water splash image characteristics, feeding activity, satiety level, actual feeding amount and feeding speed within 10-20 minutes before feeding begins, during feeding and 10-20 minutes after feeding stops can also be used to update and optimize the satiety assessment module.

[0062] The examples described are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention are within the protection scope of the present invention.

Claims

1. A precision feeding system based on machine vision, characterized in that: It includes an image acquisition module, an image processing module, a feeding splash image database, a satiety assessment module, an automatic feeder (4), and a control module (13); the image acquisition module is used to acquire images of water splashes on the surface of the fishpond in real time; the image processing module is connected to the image acquisition module by signal, and uses a convolutional neural network to segment and extract features from the real-time acquired images of water splashes to obtain the splash features of the water splash images, including the size and duration of the splashes; The water splash characteristics before, during, and after feeding are stored in the feeding splash image database; the satiety assessment module assesses the feeding activity and satiety of the fish based on the feeding splash characteristics according to the satiety assessment model; the control module (13) is connected to the image processing module, the satiety assessment module, and the automatic feeder (4) by signal. Based on the water splash characteristics of the real-time water surface splash image obtained by the image processing module, the feeding activity and satiety of the fish obtained by the satiety assessment module are used to determine the feeding start time and feeding end time. The estimated feeding amount and feeding speed are determined according to the satiety of the fish and the interval between the last feeding. Feeding control commands are sent to the automatic feeder (4) to control the operation of the automatic feeder (4); the control module has a built-in feeding system. The water splash feature threshold is set before feeding. When the size of the water splash feature in the water surface image before feeding is greater than the set threshold and continues for a preset time, the automatic feeder is controlled to start feeding. The estimated feeding amount and initial feeding speed are determined according to the interval between feeding and the last feeding and the degree of satiety. The satiety assessment model is an LSTM neural network model constructed based on the biological characteristics of fish, the experience of fish farmers and the water splash feature in the process of fish farming. The control module determines the feeding start time, estimated feeding amount and initial feeding speed according to the degree of satiety and the feeding interval. The automatic feeder (4) is used to put feed into the fish pond according to the control instructions of the control module.

2. The machine vision-based precision feeding system according to claim 1, characterized in that, The image acquisition module includes a camera (11) for acquiring images and an LED light source (12) for supplementing the lighting of the fishpond.

3. The machine vision-based precision feeding system according to claim 1, characterized in that, Before feeding is 10 to 20 minutes before the feeding time; after feeding is 10 to 20 minutes after the automatic feeder (4) stops feeding.

4. The machine vision-based precision feeding system according to claim 1, characterized in that, The process of constructing the satiety assessment model is as follows: (1) Data preprocessing: Clean the collected data and remove invalid or outlier values; normalize the data to ensure that the weights of different features in the model are equal; convert the time series data into the shape required by the LSTM model, i.e., [samples, timesteps, features]; (2) Model construction: Select the library, use the Python programming language, and install NumPy for mathematical operations, TensorFlow and Keras; construct the LSTM model, use the Keras Sequential model, add LSTM layers, add fully connected layers; select the activation function and loss function, and compile the model; (3) Model training: Train the LSTM model using the preprocessed data, specify the number of iterations and batch size; monitor the performance of the model and prevent overfitting during the training process.

5. The machine vision-based precision feeding system according to claim 4, characterized in that, The activation function is ReLU or sigmoid, and the loss function is mean squared error or cross-entropy.

6. The feeding method of the machine vision-based precision feeding system according to any one of claims 1-5, characterized in that, Includes the following steps: S1. The image acquisition module acquires images of water surface splashes in real time. The image processing module uses a convolutional neural network to segment and extract features from the acquired water surface splash images to obtain the splash features of the water surface splash images. S2. The satiety assessment module determines the satiety of the fish population based on the water splash characteristics of the water surface image. The control module (13) determines the feeding start time, estimated feeding amount, and initial feeding speed based on the satiety and feeding time interval. It also adjusts the feeding speed and feeding end time of the automatic feeder (4) based on the water splash characteristics during feeding after the feeding begins. The control module has a built-in water splash characteristic threshold before feeding. When the water splash characteristic size of the water surface image before feeding is greater than the set threshold and continues for a preset time, the automatic feeder is controlled to start feeding. The estimated feeding amount and initial feeding speed are determined based on the time interval with the last feeding and the satiety. The satiety assessment model is an LSTM neural network model constructed based on the biological characteristics of fish, the experience of fishery experts, and the water splash characteristics of the water surface during the fish farming process.

7. The feeding method according to claim 6, characterized in that, The characteristics of water splashes, feeding activity, satiety level, actual feeding amount and feeding speed within 10-20 minutes before feeding begins, during feeding and 10-20 minutes after feeding stops are used to update and optimize the satiety assessment module.

8. The feeding method according to claim 6, characterized in that, When the feeding interval is greater than 6 hours, the duration of a single feeding should be 30 to 60 minutes.

9. The feeding method according to claim 6, characterized in that, The convolutional neural network includes convolutional layers and connection layers. The convolutional layer includes convolution, activation function, and pooling layer. The image processing module uses a unified convolution kernel to perform sliding convolution on the image. The activation function is the rectified linear function ReLU, with the function expression f(x) = max(0, x). The pooling layer uses max pooling or mean pooling. When building the convolutional neural network model, gradient descent and backpropagation algorithms are used, and the convolutional neural network model is first trained using a subset of samples.

Images