Fish self-adaptive feeding method and device based on vision

Abstract

The invention relates to the technical field of fish culture feeding, in particular to a vision-based fish self-adaptive feeding method and device, and the method comprises the following steps: sequentially carrying out attraction feeding operation and formal feeding operation on fishes in a culture area; after the formal feeding operation, obtaining a feeding video of the fish; according to the ingestion video, obtaining an average ingestion behavior quantitative index of the fish; and obtaining a feeding judgment result according to the average feeding behavior quantitative index, and performing adaptive feeding on the fish according to the feeding judgment result. According to the method, the ingestion desire of the fish school can be represented on the basis of the quantized fish school ingestion behavior, and accurate feeding of the fishes in the breeding area is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of fish farming and feeding, and more particularly to a vision-based adaptive fish feeding method and device. Background Art

[0002] Precision feeding of fish is a key link in aquaculture. It is a method and process of accurately and scientifically providing fish with suitable feed according to their growth needs, environmental conditions and feeding behavior. It can improve feed utilization and promote the healthy growth of fish, thereby reducing breeding costs and achieving an important guarantee for the sustainable development of aquaculture.

[0003] Among existing feeding technologies, manual feeding relies primarily on the experience and judgment of fish farmers, who rely on observation of fish schools to determine the feeding amount, feeding time, and feeding location. This offers flexibility but lacks precision and consistency. Machine feeding, on the other hand, uses programmed and sensor-controlled feeding equipment to feed according to preset parameters such as time and feed amount. While this method achieves a certain degree of automation, it lacks sensitivity to the fish's real-time feeding behavior. Quantitative behavioral feeding incorporates advanced monitoring technologies, such as video surveillance and image analysis, attempting to precisely adjust feeding strategies through quantitative analysis of fish feeding behavior. However, in practice, it is subject to numerous interferences from the outdoor aquaculture environment, which directly affects the accuracy of fish feeding control. Summary of the Invention

[0004] The present invention provides a vision-based adaptive fish feeding method and device, which are used to characterize the feeding desire of fish schools based on quantified fish feeding behavior, thereby achieving precise feeding of fish in aquaculture areas.

[0005] According to a first aspect of the present application, a vision-based adaptive feeding method for fish is provided, the method comprising: Performing an attracting feeding operation and a formal feeding operation on the fish in the culture area in sequence; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed in sequence; After the formal feeding operation, obtaining a feeding video of the fish; the feeding video includes a first feeding video and a second feeding video, the first feeding video is obtained after the first formal feeding operation, and the second feeding video is obtained after the second formal feeding operation; Obtaining an average feeding behavior quantitative index of the fish according to the feeding video; the average feeding behavior quantitative index includes a first average feeding behavior quantitative index corresponding to the first feeding video and a second average feeding behavior quantitative index corresponding to the second feeding video; A feeding judgment result is obtained according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and the fish is adaptively fed according to the feeding judgment result.

[0006] It is understandable that the two-stage feeding design of attraction feeding and formal feeding can use attraction feeding to attract and gather fish, and then calculate the quantitative index of fish feeding behavior during formal feeding, which can reduce the quantitative error caused by water surface fluctuations caused by fish gathering; based on vision, specifically in the form of feeding videos, the quantitative index of fish feeding behavior is collected and extracted, and the biological characteristics of fish feeding intensity, speed, group distribution, etc. are converted into calculable parameters to improve the accuracy of calculation and judgment; through comparative analysis of the first feeding video and the second feeding video, the judgment conditions of fish feeding desire and satiety are constructed, which can not only reflect the immediate feeding needs of fish, but also capture the changing trend of fish feeding ability. The adaptive feeding of the present application can dynamically match the feeding amount to the actual needs of fish, reduce feed waste while ensuring nutritional supply, reduce the risk of water pollution, and improve breeding efficiency and ecological benefits.

[0007] Optionally, the attraction and feeding operation continues for a preset attraction and feeding time to complete the attraction and aggregation of the fish; The first formal feeding operation lasts for a preset first feeding time, and the second formal feeding operation lasts for a preset second feeding time; The suction feeding operation and the first formal feeding operation are separated by a preset first interval time; After completing the first formal feeding operation, a preset second interval time is left before starting the second formal feeding operation, and the duration of the first feeding video is the second interval time; The duration of the second feeding video is a preset third interval time.

[0008] It is understandable that a scientific feeding rhythm for fish is constructed through the refined design of time parameters: first, the fish are gathered at the preset attraction and feeding time to ensure the effectiveness of subsequent feeding; then the initial feeding interference is eliminated through the first interval time, allowing the fish to enter a natural feeding state; after the first formal feeding, a second interval time is set, and the first feeding video is collected synchronously, which can ensure the smooth collection of the first feeding video and avoid excessive waiting of the fish after the first formal feeding; and the third interval time also ensures the smooth collection of the second feeding video, which can be compared with the first feeding video later to monitor the feeding behavior of the fish. By comparing the quantitative index of feeding behavior at different time nodes and combining the feeding data at different stages, a multi-dimensional feeding behavior and satiety judgment condition is constructed, so that adaptive feeding has both real-time responsiveness and physiological adaptability of fish, and ultimately achieves the dual goals of precise feeding and resource optimization.

[0009] Optionally, obtaining an average feeding behavior quantitative index of the fish according to the feeding video includes: Extracting a corresponding number of frames of feeding images according to each feeding video; Preprocessing each frame of the feeding image to obtain a preprocessed feeding image; Processing each frame of the pre-processed feeding image using an optical flow method to obtain a feeding behavior quantitative index of each frame of the pre-processed feeding image; The average feeding behavior quantification index corresponding to each feeding video is obtained according to the feeding behavior quantification indexes of all the pre-processed feeding images corresponding to each feeding video.

[0010] It is understandable that the preprocessing step effectively eliminates environmental interference such as water reflections and shadow noise, enhances image features, and improves the reliability of feature extraction; the optical flow method tracks the motion vector of each pixel in the feeding image, and converts complex behaviors such as fish feeding intensity, swimming rhythm, and group cooperation into quantifiable dynamic indicators, which are more spatiotemporally continuous than traditional manual observation or static feature recognition; finally, by summing and averaging the feeding images of each frame of the feeding video, the errors caused by instantaneous behavioral fluctuations are suppressed while retaining the overall trend of the fish group's feeding characteristics. This multi-level quantitative system enables feeding decisions to be based on the actual feeding needs and behavioral feedback of the fish, ensuring nutrient supply while avoiding overfeeding, thereby improving feed utilization, reducing the risk of aquaculture pollution, and achieving refined aquaculture management.

[0011] Optionally, the using an optical flow method to process each frame of the pre-processed feeding image to obtain a quantitative feeding behavior index of each frame of the pre-processed feeding image includes: For each of the ingestion videos: Obtain information of each pixel point of each frame of the pre-processed feeding image; The second frame of pre-processed food intake image and other frames of pre-processed food intake images thereafter are used as processing images; Obtaining a pre-processed feeding image immediately preceding each processed image as a corresponding comparison image; Based on the optical flow method, obtaining the corresponding pixel displacement amplitude according to each pixel information of the processed image and the corresponding pixel information of the comparison image; Preset the weight corresponding to each pixel information; Obtaining the feeding behavior quantitative index corresponding to each pixel point information according to the pixel point displacement amplitude, weight and the total number of pixels of the corresponding processed image; The feeding behavior quantitative index of the processed image is obtained according to the feeding behavior quantitative index of all pixels.

[0012] It is understandable that by using optical flow vector analysis to capture the instantaneous motion characteristics of fish during feeding, behaviors such as fish body swing frequency and swimming trajectory are converted into calculable physical parameters; a pixel weight mechanism is introduced to set differentiated contribution levels for areas at different distances from the feeding equipment, which not only highlights the dynamic characteristics of areas closer to the feeding area, but also suppresses the interference of redundant data from areas farther away from the feeding area; through time series analysis of adjacent frame comparisons, the continuous changes in feeding behavior are fully recorded, which can better reflect the dynamic growth or decay process of fish feeding willingness than single-frame static analysis. This quantitative method enables feeding behavior assessment to break through the subjective limitations of traditional visual observation, accurately identify the hunger / satiation state of fish, provide millisecond-level behavioral data support for adaptive feeding strategies, significantly improve the timeliness and accuracy of feeding decisions, and reduce the risk of misjudgment caused by environmental interference, ultimately achieving dual optimization of aquaculture resource utilization and ecological benefits.

[0013] Optionally, the weight is expressed as: in They are the horizontal and vertical coordinates of the feeding and throwing equipment, 、 are respectively the major semi-axis and the minor semi-axis of the pre-defined elliptical feeding area with the feeding and throwing device as the origin; 、 are the horizontal and vertical coordinates of the pixel points, For the pixel The corresponding weight.

[0014] It is understandable that the spatial differentiation of fish feeding areas is achieved. An elliptical weight field is formed with the throwing equipment as the center, and the parameters 、 It flexibly adapts to the morphology of fish feeding areas in actual aquaculture areas and strengthens the representation of feeding behavior in key areas around the equipment; by constructing an elliptical gradient weight through an improved Gaussian attenuation function, the weight decays smoothly from the center to the periphery, which not only highlights the pixel displacement contribution of fish in the feeding area, but also suppresses edge noise interference and improves the quantitative signal-to-noise ratio of feeding behavior; the spatial heterogeneous weight mechanism deeply integrates the spatial distribution characteristics of organisms with computer vision features, making the quantitative index of feeding behavior more in line with the actual feeding patterns of fish, providing a more accurate behavioral basis for subsequent feeding decisions, and ultimately achieving a dynamic match between feeding strategies and fish physiological needs.

[0015] Optionally, the feeding behavior quantification index of each processed image frame is Expressed as: in, is the total number of pixels in the processed image; is the pixel displacement amplitude corresponding to the pixel information, is the weight corresponding to the pixel information; And / or, obtaining the average feeding behavior quantitative index corresponding to each feeding video according to the feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video includes: The feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video are summed and averaged to obtain the average feeding behavior quantitative index.

[0016] It is understandable that the quantization formula combines the pixel displacement amplitude with spatial weights, where the weights are used to increase the contribution of pixels closer to the feeding area, highlighting the feeding intensity characteristics of the fish feeding area. By summing and averaging each feeding image frame in the feeding video, the average feeding behavior quantification index corresponding to the feeding video is obtained. This not only smoothes the noise of instantaneous swimming interference in the single-frame feeding image, but also preserves the continuity of the fish's feeding trend. This multi-level computational architecture makes the feeding behavior quantification both spatially sensitive and temporally robust, providing a high-precision, low-interference behavioral characteristic indicator for subsequent feeding decisions, ultimately achieving a dynamic matching of feeding strategies with the actual needs of the fish, effectively improving feed utilization and reducing environmental load.

[0017] Optionally, obtaining a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and performing adaptive feeding on the fish according to the feeding judgment result, includes: Calculating a first feeding behavior quantitative index of a preset first multiple as a first comparison threshold; If the second feeding behavior quantitative index is greater than or equal to the first comparison threshold, continuing the next formal feeding operation on the fish after the third interval time; If the second feeding behavior quantitative index is less than the first comparison threshold, obtaining the current total amount of feed fed to the breeding area and the current total mass of the fish; Calculating a preset second multiple of the current total mass as a second comparison threshold; If the current total feeding amount is greater than the second comparison threshold, feeding the fish is stopped.

[0018] It is understandable that precise adaptive feeding control is achieved by quantifying the feeding behavior and judging the dynamic threshold of the first and second feeding videos. Based on the first average feeding index, the first comparison threshold is used to judge the feeding status of the fish after the second formal feeding. If the second average feeding behavior quantification index meets the standard, the feeding rhythm is continued to meet the continuous feeding needs of the fish. When the fish's feeding willingness is insufficient, the second comparison threshold based on the current total feeding amount and the total weight of the fish is combined to impose long-term total quantity constraints, preventing both short-term misjudgment-induced insufficient feeding and long-term resource waste caused by excessive feeding. The dual threshold condition combines short-term behavioral responses with long-term growth needs, significantly improving feed utilization and reducing residual bait pollution while ensuring the nutritional intake of fish, ultimately achieving the coordinated optimization of breeding efficiency and ecological benefits.

[0019] Optionally, continuing the next formal feeding operation on the fish after the third interval time includes: Using the next formal feeding operation as the current formal feeding operation; The current formal feeding operation lasts for a certain feeding time, and a current feeding video of a certain length is obtained after the current formal feeding operation, and an average feeding behavior quantitative index corresponding to the current formal feeding operation is obtained based on the current feeding video; Whether to continue feeding the fish is determined according to the average feeding behavior quantitative index corresponding to the current formal feeding operation and the average feeding behavior quantitative index corresponding to the last formal feeding operation.

[0020] It is understandable that by incorporating each formal feeding operation into an iterative loop, and by comparing and analyzing the quantitative indices of the "current" and "last" feeding behaviors in real time, a dynamic judgment basis based on historical behavior is formed, effectively avoiding misjudgments caused by single feeding fluctuations; by continuously acquiring the current feeding video and calculating the quantitative index of the current feeding behavior, the feeding strategy and changes in fish behavior are tracked synchronously, and the changes in fish feeding needs over time and in status are responded to in a timely manner. This recursive decision-making architecture can automatically optimize the feeding rhythm according to the dynamic evolution of fish feeding patterns, ultimately achieving a precise match between the amount of feed input and the actual needs of the fish, while reducing resource waste and ensuring a continuous supply for the healthy growth of fish.

[0021] Optionally, the method further includes: Preset multiple feeding time periods, and perform the adaptive feeding on the fish at the beginning of the feeding time period; The stopping of feeding the fish includes: If the feeding of the fish is stopped in the current feeding time period, the next feeding time period is waited for to start, all the average feeding behavior quantitative indexes and feeding times are reset to zero, and the fish are fed again by the adaptive feeding method.

[0022] It is understandable that multiple feeding time periods are preset to match the biological rhythms of fish, and segmented management can accurately match the differences in feeding needs in different time periods such as dawn and dusk, day and night; the data clearing mechanism between different feeding time periods effectively isolates historical data interference, avoids misjudgment caused by the accumulation of cross-time behaviors, and ensures that each feeding cycle is independently optimized; periodic reset is combined with adaptive algorithms, while ensuring the rationality of current feeding, it realizes continuous monitoring of breeding strategies through implicit correlation of cross-cycle data, and ultimately achieves a spiral improvement in resource utilization and breeding benefits.

[0023] According to a second aspect of the present application, a vision-based adaptive fish feeding device is provided, comprising: A feeding module is used to sequentially perform an attracting feeding operation and a formal feeding operation on the fish in the culture area; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed sequentially; A camera module, configured to obtain a feeding video of the fish after the formal feeding operation; the feeding video includes a first feeding video and a second feeding video, the first feeding video being obtained after the first formal feeding operation, and the second feeding video being obtained after the second formal feeding operation; a processing module, configured to obtain an average feeding behavior quantitative index of the fish according to the feeding video; the average feeding behavior quantitative index comprising a first average feeding behavior quantitative index corresponding to the first feeding video and a second average feeding behavior quantitative index corresponding to the second feeding video; A judgment module is used to obtain a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and adaptively feed the fish according to the feeding judgment result.

[0024] Based on any one of the above aspects, the embodiment of the present application provides a vision-based adaptive feeding method and device for fish, which sequentially performs an attraction feeding operation and a formal feeding operation on the fish in the breeding area; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed in sequence; after the formal feeding operation, a feeding video of the fish is obtained; the feeding video includes a first feeding video and a second feeding video, the first feeding video is obtained after the first formal feeding operation, and the second feeding video is obtained after the second formal feeding operation; the average feeding behavior quantitative index of the fish is obtained according to the feeding video; the average feeding behavior quantitative index includes a first average feeding behavior quantitative index corresponding to the first feeding video, and a second average feeding behavior quantitative index corresponding to the second feeding video; a feeding judgment result is obtained according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and the fish is adaptively fed according to the feeding judgment result. The method can achieve the following benefits: Highly accurate quantification of fish feeding behavior: Pre-attraction feeding is used to attract and aggregate fish, reducing errors caused by water surface fluctuations when quantifying feeding behavior. By calculating the feeding behavior quantification index for each feeding video, and accurately measuring the displacement fluctuations of each pixel in each frame of each feeding video, the feeding desire of fish at each interval can be accurately captured and expressed as accurate quantitative index data, which can improve the efficiency and accuracy of judgment, better achieve adaptive feeding of fish, and ensure favorable environmental conditions for fish growth.

[0025] Simple structure and easy implementation: The method for quantifying fish feeding behavior in this application only requires a camera for recording, a processor for processing feeding videos, and other conventional devices to ensure feeding in the aquaculture area. The structure and installation work are very simple. During the operation and calculation process, only the processor needs to be used to process the feeding video and control the feeding work, without too many complicated operation procedures, ensuring simplicity and practicality in specific implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0027] Figure 1 This is a schematic diagram of a vision-based adaptive fish feeding structure provided in this embodiment.

[0028] Figure 2This is a flow chart of a vision-based adaptive fish feeding method provided in this embodiment.

[0029] Figure 3 This embodiment provides a process for obtaining the average feeding behavior quantitative index of fish. Figure 1 .

[0030] Figure 4 This embodiment provides a process for obtaining the average feeding behavior quantitative index of fish. Figure 2 .

[0031] Figure 5 This is a flow chart of the next formal feeding operation provided in this embodiment.

[0032] Figure 6 This is a schematic diagram of the functional modules of a vision-based adaptive fish feeding device provided in this embodiment.

[0033] Icons: 1-PLC controller, 2-camera equipment, 3-throwing equipment, 4-silo, 5-oxygenation equipment, 6-processor. DETAILED DESCRIPTION

[0034] The figures in this application are for illustrative purposes only and are not to be construed as limiting the present application. To better illustrate the following embodiments, some components in the figures may be omitted, enlarged, or reduced in size, and do not represent actual product dimensions. Those skilled in the art will appreciate that some well-known structures and their descriptions may be omitted from the figures.

[0035] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] In today's fish farming industry, precise control of feeding strategies is crucial for ensuring fish nutrition and preventing water pollution. Traditionally, fish monitoring and feeding are typically performed manually. However, this method relies on empirical judgment, resulting in large errors in feeding amounts and timings, and a lack of objective standards, making scientific management difficult. While current quantitative behavioral feeding technologies based on video monitoring and image analysis can optimize feeding strategies based on fish feeding behavior data, they face significant challenges in large-scale outdoor ponds (typically larger than four acres). Dynamic interference factors such as wind disturbances, wave motion caused by aerator operation, and swaying vegetation and human activity severely hinder the accurate extraction of feeding behavior features. In these complex scenarios, mitigating background noise interference in non-feeding areas is a key breakthrough in improving the accuracy of fish feeding behavior recognition in dynamic environments.

[0038] This embodiment provides a technical solution that can solve the above-mentioned problem. The specific implementation methods of this application are described in detail below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, this embodiment provides a schematic diagram of a vision-based adaptive fish feeding system. The diagram shows the relevant structures of a fish farming area. Preferably, the fish farming area is a rectangular pond of at least four mu (approximately 1.5 acres). A camera 2 is installed in the middle of the shore of the fish farming area, ensuring real-time images of the entire fish farming area. The camera 2 is connected to the input of a processor 6, enabling the processor 6 to process the feeding video captured by the camera 2. Preferably, the camera 2 can be a high-definition waterproof camera, and the processor 6 can be a digital signal processor capable of processing the feeding video as a digital signal.

[0040] The throwing device 3 is fixed at the center of the breeding area rectangle, and the position line connecting the throwing device 3 and the camera device 2 is parallel to the shore of one side of the breeding area rectangle, so that the camera device 2 can capture the feeding video centered on the throwing device 3.

[0041] The feed silo 4 is installed on the right side of the outer part of the breeding area and is used to replenish feed for the throwing equipment 3; Aerator 5 is installed in a corner of the aquaculture area. A PVC pipe (Polyvinyl Chloride Pipe) for oxygen circulation is connected to the aerator 5 and fixed to one side of the aerator 5 to reduce water surface fluctuations caused by the aerator 5 during feeding, which could interfere with the quantification of water surface information in the feeding area. The output end of the processor 6 is connected to the input end of the PLC controller 1 (Programmable Logic Controller), so that the processor 6 processes the input information received from the camera device 2 accordingly, analyzes the real-time feeding desire of the fish school through image processing technology, and then transmits the processing result to the PLC controller 1. The PLC controller 1 controls the working state of the throwing device 3 according to the processing result.

[0042] It is understandable that, in actual application, the installation structure can be adapted to breeding areas of other shapes and sizes, and the equipment included in the installation structure can be appropriately adjusted according to the shape and size of the breeding area.

[0043] like Figure 2 As shown, this embodiment provides a vision-based adaptive feeding method for fish, which can be subdivided into the following steps: S100, sequentially performing an attracting feeding operation and a formal feeding operation on the fish in the culture area; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed sequentially; In this embodiment, the attraction feeding operation can release a feeding signal to the fish, causing them to gather in the feeding area, thereby reducing the interference of subsequent water surface fluctuations caused by the gathering of fish on the quantification of feeding video information. The formal feeding operation is the operation of formally feeding the fish, through which the fish obtain feed and nutrition. By setting a first formal feeding operation and a second formal feeding operation, which are performed sequentially, the feeding strategy can be adjusted by analyzing and comparing the changes in the fish's feeding appetite after the two formal feeding operations, thereby achieving adaptive feeding of the fish.

[0044] S200, after the formal feeding operation, obtaining a feeding video of the fish; the feeding video includes a first feeding video and a second feeding video, the first feeding video being obtained after the first formal feeding operation, and the second feeding video being obtained after the second formal feeding operation; In this embodiment, after the first formal feeding operation, a corresponding first feeding video needs to be obtained; similarly, after the second formal feeding operation, a corresponding second feeding video needs to be obtained. The first and second feeding videos can serve as the data basis for extracting the fish's feeding desire and provide data basis for subsequent comparison of fish's feeding desire.

[0045] Specifically, the attraction and feeding operation continues for a preset attraction and feeding time to complete the attraction and aggregation of the fish; The first formal feeding operation lasts for a preset first feeding time, and the second formal feeding operation lasts for a preset second feeding time; The suction feeding operation and the first formal feeding operation are separated by a preset first interval time; After completing the first formal feeding operation, a preset second interval time is left before starting the second formal feeding operation, and the duration of the first feeding video is the second interval time; The duration of the second feeding video is a preset third interval time.

[0046] In this embodiment, corresponding feeding times and interval times need to be preset to make the entire adaptive feeding process more organized and scientific, and comparative data can be compared in the same dimension to improve the accuracy of the comparison.

[0047] For example, in actual operation, the PLC controller 1 can first control the throwing device 3 to perform suction feeding for the suction feeding time. Preferably, the suction feeding time can be set to 10 seconds; After the suction feeding operation, a first interval time is required before the first formal feeding operation is performed; preferably, the first interval time can be set to 40 seconds; After the first interval, the first formal feeding operation begins and lasts for the first feeding time; preferably, the first feeding time can be set to 4 seconds; After the first formal feeding operation, a second formal feeding operation needs to be performed after a second interval; preferably, the second interval can be set to 40 seconds; at the same time, after the first formal feeding operation, the camera device 6 starts to shoot a first feeding video lasting for the second interval to monitor the feeding desire of the fish after the first formal feeding operation; After the second interval, the second formal feeding operation begins and lasts for the second feeding time; preferably, the first feeding time can be set to 4 seconds; After the second formal feeding operation, a third interval time is required before a subsequent judgment step is performed to determine whether the next formal feeding is required; preferably, the third interval time can be set to 40 seconds; at the same time, after the second formal feeding operation, the camera device 6 is used to shoot a second feeding video lasting for the third interval time to monitor the feeding desire of the fish after the second formal feeding operation, ensuring that the duration is the same as that of the first feeding video, and focusing on comparing the feeding desire of the fish.

[0048] S300, obtaining an average feeding behavior quantitative index of the fish according to the feeding video; the average feeding behavior quantitative index includes a first average feeding behavior quantitative index corresponding to the first feeding video, and a second average feeding behavior quantitative index corresponding to the second feeding video; In this embodiment, it is necessary to obtain the corresponding average feeding behavior quantitative index based on each feeding video, and convert the fish's feeding desire into an observable and accurate numerical value. This can avoid errors in the main board's judgment under certain circumstances and improve the accurate extraction of the fish's feeding desire.

[0049] Specifically, if Figure 2 As shown, obtaining the average feeding behavior quantitative index of the fish according to the feeding video may include the following steps: S310, extracting a corresponding number of frames of eating images according to each of the eating videos; In this embodiment, it is necessary to quantify the feeding desire of fish in each frame of the feeding image, so it is necessary to obtain the sampling rate of the shooting device 3. , that is, in a 1-second feeding video, it is possible to extract Frame video, so we need to extract the total duration of the feeding video* Frame of feeding image.

[0050] S320, pre-processing each frame of the food intake image to obtain a pre-processed food intake image; In this embodiment, the optical flow method is subsequently used to extract the displacement amplitude of each pixel point in the feeding image. The principle of the optical flow method is to track the displacement according to the brightness of each pixel point. Therefore, some brightness noise in the feeding image will seriously affect the extraction of the displacement. Therefore, it is necessary to pre-process each frame of the feeding image to repair the image quality caused by the reflection of the water surface that affects the extraction of the fish feeding behavior, as well as the blurred features of the water surface fluctuations caused by feeding, so as to enhance the fluctuation information features in each frame of the feeding image.

[0051] For example, in actual preprocessing, it is necessary to first convert the RGB (Red, Green, Blue) color space of each frame of the food image into the HSV (Hue, Saturation, Value) color space to separate the brightness (Value) and color information (Hue and Saturation), so as to independently process the brightness channel without interfering with the color information.

[0052] In this embodiment, block histogram equalization is used to process the ingested image after the color space conversion. By stretching the grayscale distribution of each block image, the overall brightness of the sub-block is made more uniform, which facilitates subsequent optical flow processing.

[0053] First, each frame of the feeding image is divided into a preset number of blocks, and histogram equalization is performed on each sub-block using the following formula: in, is the total number of pixels in each sub-block, is the grayscale level of each sub-block, is the cutoff coefficient, is the grayscale mean of each sub-block, is the mean square error of each sub-block, is the maximum allowable slope, is the upper limit of clipping for each subhistogram.

[0054] S330, processing each frame of the pre-processed feeding image using an optical flow method to obtain a feeding behavior quantitative index of each frame of the pre-processed feeding image; Specifically, if Figure 3 As shown, the process of processing each frame of the pre-processed feeding image using the optical flow method to obtain the feeding behavior quantitative index of each frame of the pre-processed feeding image may include the following steps: For each of the ingestion videos: S331, obtaining information of each pixel point of each frame of the pre-processed food image; In this embodiment, the optical flow method needs to process each pixel of each frame of the food image, so it is necessary to first obtain the information of each pixel of each frame of the food image after pre-processing. Preferably, the position information corresponding to the pixel information in the food image frame includes the horizontal and vertical coordinates.

[0055] S332, using the second frame of pre-processed food-ingestion image and other frames of pre-processed food-ingestion images thereafter as processed images; It's understood that optical flow refers to the instantaneous velocity vector of pixels in an image due to object or camera motion, expressed as pixel displacement in consecutive frames. Optical flow relies on three prerequisites: the brightness of pixels is constant across consecutive frames; each pixel undergoes a small, continuous temporal movement between two consecutive frames; and the movement of each pixel between two consecutive frames is spatially consistent.

[0056] Therefore, the above-mentioned pre-processing and restoration of the feeding image satisfies the premise of the optical flow method, so that the displacement amplitude of each pixel can be extracted more accurately.

[0057] In this embodiment, because the optical flow method requires two adjacent frames to be compared and the displacement is extracted, it is assumed that the first frame of the food image is not processed, and the pre-processed food images of the second frame and other frames thereafter are used as the processed images, thereby extracting the displacement amplitude of each pixel point of each frame of the processed image.

[0058] S333, obtaining the pre-processed feeding image of the previous frame of each processed image as a corresponding comparison image; In this embodiment, since the optical flow method needs to process two consecutive frames of food intake images, it is necessary to obtain the pre-processed food intake image of the previous frame of each processed image as the corresponding comparison image to complete the displacement amplitude of each pixel point in the optical flow method.

[0059] S334. Based on the optical flow method, obtain the corresponding pixel displacement amplitude according to the pixel information of each pixel in the processed image and the pixel information of the corresponding comparison image; S335. Preset the weight corresponding to each pixel information; In this embodiment, the larger the Euclidean distance between a pixel point in the feeding image and the origin, the smaller the corresponding weight value, indicating that the pixel point has a lower importance in quantifying the activeness of the fish feeding behavior. The origin is set to the location of the feeding device. It is understandable that if the fish have a strong appetite, they will continue to gather towards the feeding device, and the fish's behavior will be active, causing large fluctuations in the water surface near the feeding device. Since the area near the feeding device is mostly inhabited by fish, the closer the pixel point is to the feeding device, the higher the weight corresponding to the pixel point. Similarly, if the fish have a low appetite, usually in a state of fullness, they will not gather towards the feeding device, and their behavior will be stable, causing little fluctuation in the water surface near the feeding device or even the entire aquaculture area. Therefore, focusing on the area near the feeding device with a high weight can accurately reflect the fish's appetite.

[0060] In this embodiment, each pixel The original formula of the corresponding weight is: in, is the distance between the pixel and the origin, Is the coefficient that adjusts the decay speed of the weight coefficient; However, due to the natural environment of the breeding area, the camera is located on one side of the breeding area, so the feeding area captured is approximately an ellipse. With the origin The distance is calculated as: in and are respectively the lengths of the major and minor axes of the elliptical feeding area; preferably, the feeding area is divided by artificial judgment in advance, and The adaptation length needs to be set in advance to divide the feeding area into appropriate sizes.

[0061] Finally, the weights are adjusted using the improved Gaussian decay function. The original formula is transformed to obtain the optimized weight .

[0062] Specifically, the weight is expressed as: in They are the horizontal and vertical coordinates of the feeding and throwing equipment, 、 are respectively the major semi-axis and the minor semi-axis of the pre-defined elliptical feeding area with the feeding and throwing device as the origin; 、 are the horizontal and vertical coordinates of the pixel points, For the pixel The corresponding weight.

[0063] S336, obtaining a quantitative index of feeding behavior corresponding to each pixel point information according to the pixel point displacement amplitude and weight corresponding to each pixel point information and the total number of pixels of the corresponding processed image; S337 . Obtain the feeding behavior quantitative index of the processed image according to the feeding behavior quantitative index of all pixels.

[0064] Specifically, the feeding behavior quantitative index of each processed image is Expressed as: in, is the total number of pixels in the processed image; is the pixel displacement amplitude corresponding to the pixel information, is the weight corresponding to the pixel information.

[0065] S340: Obtain the average feeding behavior quantitative index corresponding to each feeding video according to the feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video.

[0066] Specifically, obtaining the average feeding behavior quantitative index corresponding to each feeding video according to the feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video includes: The feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video are summed and averaged to obtain the average feeding behavior quantitative index.

[0067] In this embodiment, the above steps have calculated the feeding behavior quantitative index corresponding to the feeding video, and the average feeding behavior quantitative index of each feeding video needs to be obtained by summing and averaging the corresponding feeding behavior quantitative indexes of all the pre-processed feeding images.

[0068] For example, as mentioned above, the duration of both the first and second feeding videos can be set to 40 seconds, with the time counting starting at the first official feeding operation. That is, at 0 seconds, the first official feeding operation begins, and the first official feeding operation lasts for 4 seconds. Therefore, at 4 seconds, the first feeding video begins, and the first feeding video ends at 44 seconds.

[0069] Similarly, at 44 seconds, the second formal feeding operation is started, and the second formal feeding operation lasts for 4 seconds. Therefore, at 48 seconds, the second feeding video is started, and the shooting of the second feeding video is ended at 88 seconds.

[0070] Therefore, the average feeding behavior quantitative index corresponding to the first feeding video is The average feeding behavior quantitative index corresponding to the second feeding video for: in, is the camera sampling rate, for -1 frame feeding behavior quantitative index, for -1 frame feeding behavior quantitative index.

[0071] S400: Obtain a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and perform adaptive feeding on the fish according to the feeding judgment result.

[0072] Specifically, obtaining a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and adaptively feeding the fish according to the feeding judgment result, includes: Calculating a first feeding behavior quantitative index of a preset first multiple as a first comparison threshold; If the second feeding behavior quantitative index is greater than or equal to the first comparison threshold, continuing the next formal feeding operation on the fish after the third interval time; If the second feeding behavior quantitative index is less than the first comparison threshold, obtaining the current total amount of feed fed to the breeding area and the current total mass of the fish; Calculating a preset second multiple of the current total mass as a second comparison threshold; If the current total feeding amount is greater than the second comparison threshold, feeding the fish is stopped.

[0073] In this embodiment, it is necessary to set a corresponding judgment and control strategy based on the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and perform adaptive feeding on the fish according to the feeding desire of the fish.

[0074] For example, the first multiple can be set to 0.9; therefore, if the second feeding behavior quantitative index is greater than or equal to the first comparison threshold, specifically: When the third interval time is reached, the fish are fed for the next time; If the second feeding behavior quantitative index is less than the first comparison threshold, specifically: When the total amount of feed T currently fed in the breeding area and the total weight of the fish currently fed are obtained, ; For example, the second multiple can be set to 0.3; therefore, if the current total amount of feed fed is greater than the second comparison threshold, specifically: Stop feeding the fish.

[0075] By obtaining the fish's desire after the first and second formal feedings based on the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, the fish's appetite can be accurately judged. Through the corresponding judgment strategy, the number and time of feeding can be controlled, so that the fish can be adaptively fed, the precise nutrition supply to the fish can be completed, and the pollution of the water body can be avoided to the greatest extent.

[0076] Specifically, if Figure 4 As shown, the process of continuing the next formal feeding operation on the fish after the third interval time includes: S410, taking the next formal feeding operation as the current formal feeding operation; In this embodiment, if it is determined that the fish still has a strong appetite and the average quantitative index of feeding behavior after the second formal feeding is still high, the next formal feeding is required to ensure the nutritional supply of the fish.

[0077] S420: The current formal feeding operation continues for a certain feeding time, and a current feeding video of a certain length is obtained after the current formal feeding operation, and an average feeding behavior quantitative index corresponding to the current formal feeding operation is obtained based on the current feeding video; In this embodiment, the current formal feeding operation lasts for a certain feeding time, and the setting of the certain feeding time can be the same as the feeding time of the first formal feeding or the second formal feeding, and can be appropriately adjusted according to actual conditions; In the acquisition of the current feeding video of a certain duration, the certain duration needs to be the same as the shooting duration of the feeding video shot last time, so as to ensure that the average feeding behavior quantitative index of the feeding video is focused on during comparison. In this embodiment, the certain duration can be set to 40s.

[0078] S430: Determine whether to continue feeding the fish according to the average feeding behavior quantitative index corresponding to the current formal feeding operation and the average feeding behavior quantitative index corresponding to the last formal feeding operation.

[0079] In this embodiment, the same judgment method as above is used to judge whether to perform the next formal feeding, and the iterative processing of judgment and feeding is completed, with the same scientific processing method, to ensure that the processing and judgment steps are carried out in an orderly manner.

[0080] Specifically, the method further includes: Preset multiple feeding time periods, and perform the adaptive feeding on the fish at the beginning of the feeding time period; The stopping of feeding the fish includes: If the feeding of the fish is stopped in the current feeding time period, the next feeding time period is waited for to start, all the average feeding behavior quantitative indexes and feeding times are reset to zero, and the fish are fed again by the adaptive feeding method.

[0081] It is understandable that in general fish farming feeding operations, fish are generally fed in multiple feeding time periods, such as dividing the feeding time periods into morning, afternoon, evening and other time nodes, which is in line with the biological feeding characteristics of fish.

[0082] Therefore, in this embodiment, the adaptive feeding method of the present application is used to feed the fish in the breeding area in each feeding time period. In a feeding time period, if it is determined that the fish has a low appetite, it means that the fish is almost in a full state in the feeding time period, so feeding the fish is stopped in the feeding time period.

[0083] At the beginning of the next feeding period, the fish start a new round of feeding behavior due to metabolic consumption. In this case, all the average feeding behavior quantitative indexes and feeding times need to be reset to zero so as not to be disturbed by the previous round of feeding information. The adaptive feeding method of the present application can be used again to carry out a new round of adaptive feeding for the fish, complete multiple rounds of feeding for the fish, and ensure sufficient nutritional supply for the fish.

[0084] like Figure 5 As shown, the embodiment of the present application also provides a vision-based adaptive fish feeding device. Optionally, the device includes: Feeding module 511, camera module 512, processing module 513, judgment module 514, wherein: The feeding module 511 is used to sequentially perform an attracting feeding operation and a formal feeding operation on the fish in the culture area; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed sequentially; In this embodiment, the feeding module 511 can be used to perform Figure 2 As shown in step S100, for a detailed description of the feeding module 511, reference may be made to the description of step S100.

[0085] The camera module 512 is configured to obtain a feeding video of the fish after the formal feeding operation; the feeding video includes a first feeding video and a second feeding video, the first feeding video being obtained after the first formal feeding operation, and the second feeding video being obtained after the second formal feeding operation; In this embodiment, the camera module 512 can be used to perform Figure 2 As shown in step S200, for a detailed description of the camera module 512, reference may be made to the description of step S200.

[0086] A processing module 513 is configured to obtain an average feeding behavior quantitative index of the fish based on the feeding video; the average feeding behavior quantitative index includes a first average feeding behavior quantitative index corresponding to the first feeding video and a second average feeding behavior quantitative index corresponding to the second feeding video; In this embodiment, the processing module 513 can be used to perform Figure 2 As shown in step S300, for a detailed description of the processing module 513, reference may be made to the description of step S300.

[0087] The judgment module 514 is configured to obtain a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and perform adaptive feeding on the fish according to the feeding judgment result.

[0088] In this embodiment, the judgment module 514 can be used to perform Figure 2 As shown in step S400 , for a detailed description of the judgment module 514 , reference may be made to the description of step S400 .

[0089] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the technical solutions of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A vision-based adaptive fish feeding method, characterized in that: The method comprises: Performing an attracting feeding operation and a formal feeding operation on the fish in the culture area in sequence; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed in sequence; After the formal feeding operation, obtaining a feeding video of the fish; the feeding video includes a first feeding video and a second feeding video, the first feeding video is obtained after the first formal feeding operation, and the second feeding video is obtained after the second formal feeding operation; Obtaining an average feeding behavior quantitative index of the fish according to the feeding video; the average feeding behavior quantitative index includes a first average feeding behavior quantitative index corresponding to the first feeding video and a second average feeding behavior quantitative index corresponding to the second feeding video; A feeding judgment result is obtained according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and the fish is adaptively fed according to the feeding judgment result.

2. The method according to claim 1, characterized in that The attraction and feeding operation continues for a preset attraction and feeding time to complete the attraction and aggregation of the fish; The first formal feeding operation lasts for a preset first feeding time, and the second formal feeding operation lasts for a preset second feeding time; The suction feeding operation and the first formal feeding operation are separated by a preset first interval time; After completing the first formal feeding operation, a preset second interval time is left before starting the second formal feeding operation, and the duration of the first feeding video is the second interval time; The duration of the second feeding video is a preset third interval time.

3. The method according to claim 1, characterized in that The step of obtaining an average feeding behavior quantitative index of the fish according to the feeding video includes: Extracting a corresponding number of frames of feeding images according to each feeding video; Preprocessing each frame of the feeding image to obtain a preprocessed feeding image; Processing each frame of the pre-processed feeding image using an optical flow method to obtain a feeding behavior quantitative index of each frame of the pre-processed feeding image; The average feeding behavior quantification index corresponding to each feeding video is obtained according to the feeding behavior quantification indexes of all the pre-processed feeding images corresponding to each feeding video.

4. The method according to claim 3, characterized in that The method of processing each frame of the pre-processed feeding image using the optical flow method to obtain a feeding behavior quantitative index of each frame of the pre-processed feeding image includes: For each of the ingestion videos: Obtain information of each pixel point of each frame of the pre-processed feeding image; The second frame of pre-processed food intake image and other frames of pre-processed food intake images thereafter are used as processing images; Obtaining a pre-processed feeding image immediately preceding each processed image as a corresponding comparison image; Based on the optical flow method, obtaining the corresponding pixel displacement amplitude according to each pixel information of the processed image and the corresponding pixel information of the comparison image; Preset the weight corresponding to each pixel information; Obtaining the feeding behavior quantitative index corresponding to each pixel point information according to the pixel point displacement amplitude, weight and the total number of pixels of the corresponding processed image; The feeding behavior quantitative index of the processed image is obtained according to the feeding behavior quantitative index of all pixels.

5. The method according to claim 4, characterized in that The weight is expressed as: in They are the horizontal and vertical coordinates of the feeding and throwing equipment, 、 are respectively the major semi-axis and the minor semi-axis of the pre-defined elliptical feeding area with the feeding and throwing device as the origin; 、 are the horizontal and vertical coordinates of the pixel points, For the pixel The corresponding weight.

6. The method according to claim 4, characterized in that The quantitative index of feeding behavior of each processed image Expressed as: in, is the total number of pixels in the processed image; is the pixel displacement amplitude corresponding to the pixel information, is the weight corresponding to the pixel information; And / or, obtaining the average feeding behavior quantitative index corresponding to each feeding video according to the feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video includes: The feeding behavior quantitative indexes of all the pre-processed feeding images corresponding to each feeding video are summed and averaged to obtain the average feeding behavior quantitative index.

7. The method according to claim 1, characterized in that The step of obtaining a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and adaptively feeding the fish according to the feeding judgment result, includes: Calculating a first feeding behavior quantitative index of a preset first multiple as a first comparison threshold; If the second feeding behavior quantitative index is greater than or equal to the first comparison threshold, continuing the next formal feeding operation on the fish after the third interval time; If the second feeding behavior quantitative index is less than the first comparison threshold, obtaining the current total amount of feed fed to the breeding area and the current total mass of the fish; Calculating a preset second multiple of the current total mass as a second comparison threshold; If the current total feeding amount is greater than the second comparison threshold, feeding the fish is stopped.

8. The method according to claim 7, characterized in that The method of continuing the next formal feeding operation on the fish after the third interval time includes: Using the next formal feeding operation as the current formal feeding operation; The current formal feeding operation lasts for a certain feeding time, and a current feeding video of a certain length is obtained after the current formal feeding operation, and an average feeding behavior quantitative index corresponding to the current formal feeding operation is obtained based on the current feeding video; Whether to continue feeding the fish is determined according to the average feeding behavior quantitative index corresponding to the current formal feeding operation and the average feeding behavior quantitative index corresponding to the last formal feeding operation.

9. The method according to claim 7, characterized in that The method further comprises: Preset multiple feeding time periods, and perform the adaptive feeding on the fish at the beginning of the feeding time period; The stopping of feeding the fish includes: If the feeding of the fish is stopped in the current feeding time period, the next feeding time period is waited for to start, all the average feeding behavior quantitative indexes and feeding times are reset to zero, and the fish are fed again by the adaptive feeding method.

10. A vision-based adaptive fish feeding device, characterized in that: The device comprises: A feeding module is used to sequentially perform an attracting feeding operation and a formal feeding operation on the fish in the culture area; the formal feeding operation includes a first formal feeding operation and a second formal feeding operation performed sequentially; A camera module, configured to obtain a feeding video of the fish after the formal feeding operation; the feeding video includes a first feeding video and a second feeding video, the first feeding video being obtained after the first formal feeding operation, and the second feeding video being obtained after the second formal feeding operation; a processing module, configured to obtain an average feeding behavior quantitative index of the fish according to the feeding video; the average feeding behavior quantitative index comprising a first average feeding behavior quantitative index corresponding to the first feeding video and a second average feeding behavior quantitative index corresponding to the second feeding video; A judgment module is used to obtain a feeding judgment result according to the first average feeding behavior quantitative index and the second average feeding behavior quantitative index, and adaptively feed the fish according to the feeding judgment result.

Images