Intelligent pond feeding method and device based on machine vision

Abstract

The invention discloses an intelligent pond feeding method and device based on machine vision, and the method comprises the steps: installing a high-definition waterproof camera at the bank of a pond, and enabling the installation position of the high-definition waterproof camera to guarantee that the real-time image of the whole culture water body can be shot; the method comprises the following steps: processing an image shot by a high-definition waterproof camera after feeding, determining a feeding desire value of cultured organisms in combination with an information entropy and an influence weight, quantifying a numerical relationship between active degrees of the cultured organisms after two times of feeding, and taking the numerical relationship as a basis for judging whether feeding is needed or not subsequently, and the working state of the material throwing device is automatically controlled. The working state of the feeder is controlled according to the quantitative result of two times of feeding, accurate feeding is achieved, under the condition that nutritional conditions needed by fish growth are guaranteed, more attention is paid to the welfare problem of fish, and good environmental conditions can be provided for fish school growth.

Description

Technical Field

[0001] The present invention belongs to the technical field of aquaculture, and relates to a method and device for intelligent feeding in a pond based on machine vision. This device can particularly quantify the feeding desire of fish populations in a pond environment and determine whether to feed. Background Art

[0002] With the rapid development of the economic society and the remarkable improvement of people's living standards, the demand for fish protein by people is increasing continuously. The global fish price will be on an upward trend. The growth of residents' income in the aquatic product consumption market, as well as factors such as population growth and arable land limitation, make the increasing global demand for aquatic products such as protein the main reason driving the price increase. In addition, China's fishery policy is to ensure the sustainability of the ocean and the environment, and the output of its wild capture fishery will gradually decline. At the same time, the growth rate of the output of the aquaculture fishery will also slow down under the condition of continuous increase in costs (such as labor, feed, energy, etc.).

[0003] Currently, the aquaculture modes in China mainly include industrialized aquaculture, deep-water cage aquaculture, pond aquaculture, etc. Among them, due to the influence of environmental factors, the mechanization degree of pond aquaculture is relatively low. There are two common pond feeding methods: manual feeding and pond feeding machines.

[0004] Manual feeding mainly uses manual tools such as spoons and shovels to sprinkle feed manually, and judges the amount of feed required by the aquaculture objects by eyes based on experience. Usually, it is difficult to master the most suitable demand level of the aquaculture objects. Manual feeding cannot guarantee the uniformity of feeding, is time-consuming, labor-intensive, has a low efficiency, and even wastes fish feed. At the same time, it pollutes the pond water quality, affects the growth and development of fish, and increases the aquaculture cost. The feeding machine integrates fixed-point, timing and quantitative feeding, and has advantages such as a wide feeding area and uniform feeding. It not only reduces the labor intensity of fishermen, but also increases the fish yield. However, most of the existing feeding machines adopt a simple control system of mechanical timing and cannot automatically adjust the feeding amount according to the actual feeding demand of fish. When the feeding amount is less than the actual feeding demand of fish, there will be a serious phenomenon of scrambling for food, causing fish to collide with each other and even causing damage to the fish body surface. And fish with damaged surfaces and weak fish are more likely to be infected with certain fish diseases, putting greater pressure on the aquaculture water environment and having an adverse impact on the growth of fish. When the feeding amount is greater than the actual feeding demand of fish, it will not only increase the aquaculture cost, but also the excess feed will seriously pollute the aquaculture environment. Therefore, the feeding amount of feed should be as consistent as possible with the actual feeding demand of fish. Summary of the Invention

[0005] The purpose of the present invention is to propose a method and device for intelligent feeding in a pond based on machine vision in view of the deficiencies of the prior art, which can perform precise feeding according to the feeding activity degree of aquaculture organisms.

[0006] The technical solution adopted by the present invention is as follows:

[0007] An intelligent pond feeding method based on machine vision, which installs a high-definition waterproof camera on the pond bank, and its installation position ensures that the real-time picture of the entire aquaculture water body can be captured; the method includes: processing the images captured by the high-definition waterproof camera after feeding, determining the feeding desire value of the aquaculture organisms by combining information entropy and influence weight, quantifying the numerical relationship between the activity levels of the aquaculture organisms after two consecutive feedings, and using it as the basis for whether to feed subsequently, and automatically controlling the working state of the feeding device.

[0008] In the above technical solution, further, the method includes the following:

[0009] 1) First, control the feeder to feed for t1 seconds, and after an interval of T seconds, feed for the second time for t2 seconds, and the high-definition waterproof camera collects the video data after feeding;

[0010] 2) Preprocess each frame of the video data to extract the reflective area of the aquaculture water body:

[0011]

[0012] Where I s (x, y) and I v (x, y) respectively represent the saturation and brightness of the image at (x, y), T s and T v are the saturation threshold and brightness threshold respectively, and f(x, y) represents the value of the pixel point (x, y) after binarization processing;

[0013] 3) Use the optical flow method to extract the change characteristics of the water surface reflective area generated by the movement of aquaculture organisms. The change amplitude of the characteristic point position in the target area is expressed as:

[0014]

[0015] F is the optical flow between two consecutive frames of images, (x, y) represents the coordinates of the reflective area of the current frame, and M is the total number of non-zero motion vectors in the current frame;

[0016] Use perspective transformation to correct the optical flow error caused by the installation position of the camera, and correct each point in the calculation result v in the motion vector field. After correction, it is:

[0017] v' = v a λ a ;

[0018] v a is the value of any point a in the motion vector field v, and λ a is the correction coefficient of this point;

[0019] 4) The grid method is adopted to extract the motion blur of cultured organisms. Each frame of the image is divided into m×n grid regions. The motion characteristics of each particle are estimated by averaging the optical flow of all pixels within the individual grid. The optical flow information of each individual is calculated as the average optical flow of all pixels within the grid. Then, we have:

[0020]

[0021]

[0022] i ∈ [1, m*n],

[0023] where b and c are the side lengths of the image, v″ is the optical flow of the i-th particle, N is the number of pixel points belonging to each grid, represents the optical flow of the i'-th adjacent pixel belonging to the i-th particle;

[0024] 5) Calculate the influence weight of the moving target i on the particle j:

[0025]

[0026] i, j ∈ [1, b*c],

[0027] d ij is the Euclidean distance between particles i and j;

[0028] 6) Use information entropy to measure the degree of irregularity of the probability distribution of the change characteristics of the water body reflection area, so as to analyze the degree of irregularity of the movement of cultured organisms. The information entropy calculation is as follows:

[0029]

[0030] z is the number of intervals into which v″ is divided, P(j) is the probability of the motion vector falling into the divided velocity interval. The feeding desire value of the fish school is calculated by combining the information entropy and the influence weight:

[0031] D = I D *ζ ij ,

[0032] The larger D is, the stronger the feeding desire of the current fish school. Whether to continue feeding is judged by comparing the feeding activity levels of the fish school during two feedings.

[0033] Furthermore, the t takes 2 seconds to 6 seconds, and the T takes 5 seconds to 100 seconds.

[0034] Furthermore, the correction coefficient λ in 3) a is determined by the distance from point a to the vanishing point V:

[0035]

[0036] Let V be the vanishing point, O be the point in the image that is farthest from the vanishing point, d(V, O) be the distance from the vanishing point to point O, and d(V, P a ) be the distance from the vanishing point to point a;

[0037] Let l1 and l2 be two parallel line segments in the image. Arbitrarily take points B and C on l1 and points D and E on l2 respectively. Calculate the coordinates of the vanishing point V according to the coordinates of these four points:

[0038] V = (B × C) × (D × E).

[0039] Furthermore, judging whether to continue feeding by comparing the feeding activity levels of the fish population during two feedings as described in 6) specifically includes the following:

[0040] Calculate the average feeding desire value of the fish population during the first feeding time period t1 and the average feeding desire value of the fish population during the second feeding time period t2 and compare them, where:

[0041]

[0042] D γ is the feeding desire value at the γ-th second during the first feeding process, and D δ is the feeding desire value at the δ-th second during the second feeding process;

[0043] If then control the feed dispenser to perform the next round of feeding;

[0044] If then stop the feeding instruction and wait for the start of the next feeding operation.

[0045] An intelligent pond feeding device based on machine vision, including a high-definition waterproof camera, an aerator, a feed dispenser, a PLC, and a digital signal processor; the feed dispenser includes a feed bin and a throwing device;

[0046] The high-definition waterproof camera is installed on the pond bank, and its installation position needs to ensure that the real-time picture of the entire aquaculture water body can be captured;

[0047] The throwing device is fixed at the center of the pond, and the connection line between the position of the throwing device and the high-definition waterproof camera is parallel to one side bank of the pond;

[0048] The feed bin is installed outside the pond;

[0049] The aerator is installed in the pond and should minimize the water surface fluctuation caused by its operation to interfere with the quantification of the water surface information in the feeding area;

[0050] The high-definition waterproof camera is connected to the digital signal processor, the digital signal processor is connected to the PLC, the PLC controls the feeding device, and controls the silo to feed the feeding device; the machine vision-based intelligent pond feeding method described in any one of the above is realized.

[0051] The present invention forms the basis for subsequent feeding needs by quantifying the numerical relationship between the activity levels of the fish population after two consecutive feedings, so as to automatically control the working state of the feed dispenser, providing good reference and technical support for the rational feeding operation in pond aquaculture.

[0052] The beneficial effects of the present invention are:

[0053] The machine vision-based intelligent pond feeding device of the present invention has a simple structure and a convenient control method. The device uses machine vision-related technologies to analyze and compare the feeding activity levels of the fish population after two feedings, and controls the working state of the feed dispenser according to the magnitude of the two quantification results, enabling precise feeding. While ensuring the nutritional conditions required for fish growth, it pays more attention to the welfare of the fish, and can provide good environmental conditions for the growth of the fish population. Description of the Drawings

[0054] Figure 1 It is a schematic structural diagram of a machine vision-based fish intelligent feeding device applied to a pond.

[0055] In the figure: 1 - PLC; 2 - high-definition waterproof camera; 3 - feeding device; 4 - silo; 5 - aerator; 6 - digital signal processor Detailed Embodiments

[0056] The technical solutions of the present invention will be further described below with reference to the drawings and specific embodiments.

[0057] Refer to Figure 1 , which is a specific example of the machine vision-based pond intelligent feeding device of the present invention, including PLC 1, high-definition waterproof camera 2, feeding device 3, silo 4, aerator 5, and digital signal processor 6;

[0058] The high-definition waterproof camera 2 is installed in the middle of the pond bank, and the high-definition waterproof camera 2 is connected to the input end of the digital signal processor 6; the installation position of the camera ensures that the real-time picture of the entire aquaculture water body can be captured;

[0059] The feeding device 3 is fixed at the center of the pond, and the connection line between the position of the feeding device 3 and the high-definition waterproof camera 2 is parallel to one side bank of the pond;

[0060] The silo 4 is installed on the right side outside the pond;

[0061] The aerator 5 is installed at a corner of the pond, and the PVC pipe is fixed on one side of the aerator 5 to reduce the interference of the water surface fluctuation caused by the operation of the aerator during feeding on the quantification of the water surface information in the feeding area.

[0062] The output end of the digital signal processor 6 is connected to the input end of the PLC 1; the digital signal processor 6 receives the image information input by the high-definition waterproof camera 2 and makes corresponding processing. First, it analyzes the real-time feeding desire of the fish school through image processing technology, and then the processor transmits the processing result to the PLC 1 to control the working state of the feeding device 3.

[0063] Apply the above device for intelligent feeding of fish schools. The feeding method includes the following steps:

[0064] 1) The PLC 1 controls the feeding device 3 to work for 10 seconds, and feeds for the second time after an interval of 100 seconds. The high-definition waterproof camera 2 transmits the collected video data to the digital signal processor 6 in real time, and analyzes the feeding activity degree of the cultured organisms in the previous two feedings to determine the subsequent feeding state;

[0065] 2) Specifically, the digital signal processor 6 preprocesses the received video image to extract the reflective area of the aquaculture water body: Where I s (x,y) and I v (x,y) respectively represent the saturation and brightness of the image at (x,y), T s and T v are the saturation threshold and brightness threshold respectively, and f(x,y) represents the value of the pixel point (x,y) after binarization processing;

[0066] 3) Use the optical flow method to extract the change characteristics of the water surface reflective area generated by the movement of the cultured organisms. The change amplitude of the feature point position in the target area is expressed as F is the optical flow between two consecutive frames of images, (x,y) represents the coordinate of the reflective area of the current frame, and M is the total number of non-zero motion vectors in the current frame;

[0067] Use the perspective transformation method to correct the optical flow error caused by the camera installation position, and correct each point in the calculation result v in the motion vector field. After correction, it is: v' = v a λ a ; v a is a point a in the motion vector field, and λ a is the correction coefficient of this point; the vanishing point V = (B × C) × (D × E), and the vanishing point can be determined by calculating the intersection point of two line segments. Let l1 and l2 be two parallel line segments in the image, which are defined by points B and C on l1 and points D and E on l2 respectively; the correction coefficient λ a is determined by the distance from point a to the vanishing point V where d(V, O) is the distance from the vanishing point to the point O which is the farthest from the vanishing point in the image, and d(V, P a ) is the distance from the vanishing point to point a;

[0068] 4) The grid method is used to extract the motion blur of the cultured organisms. Grid-shaped particles are placed in each frame of the image to simulate individuals in the fish school. The grid size is 15*15, and the optical flow information of each individual is calculated as the average optical flow of all pixels within the grid: i′∈[1, b / 15*

[0069] c / 15], i∈[1, 15*15], where b and c are the side lengths of the image, v″ is the optical flow of the i-th particle, N is the number of neighboring pixel points belonging to each particle, represents the optical flow of the i′-th neighboring pixel belonging to the i-th particle;

[0070] 5) Calculation of the influence weight of the moving target i on the particle j:

[0071] i, j∈[1, b*c],

[0072] d ij is the Euclidean distance between particles i and j;

[0073] 6) The information entropy is used to measure the degree of irregularity of the probability distribution of the change characteristics of the water body reflection area, so as to analyze the degree of irregularity of the movement of the cultured organisms;

[0074]

[0075] z is the number of intervals into which v″ is divided, P(j) is the probability of the motion vector falling into the divided velocity interval, and the feeding desire value of the fish school is calculated by combining the information entropy and the influence weight: d = I D *ζ ij , the larger D is, the stronger the feeding desire of the current fish school is. Compare the feeding activity degrees of the fish schools during the two feedings to judge whether to continue feeding;

[0076] 7) Calculate the average feeding desire of the fish school during the first feeding time period t1 and the average feeding desire of the fish school during the second feeding time period t2 and compare them, where W γ is the feeding desire value at the γ-th second during the first feeding process, and D δ is the feeding desire value at the δ-th second during the second feeding process; if then the digital signal processor 6 inputs the processing result to the PLC 1, and the PLC 1 controls the feeder to perform the next round of feeding;

[0077] 8) If the digital signal processor 6 sends a stop feeding instruction to the PLC 1 and waits for the start of the next feeding operation.

[0078] The above-disclosed are only specific embodiments of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, any modifications made without departing from the spirit of the present invention shall be considered as falling within the protection scope of the present invention.

Claims

1. An intelligent feeding method for a pond based on machine vision, characterized in that, Install a high-definition waterproof camera on the bank of the pond, and its installation position is guaranteed to be able to capture the real-time picture of the entire aquaculture water body; the method includes: processing the images captured by the high-definition waterproof camera after feeding, determining the feeding desire value of the aquaculture organisms by combining information entropy and influence weight, quantifying the numerical relationship between the activity levels of the aquaculture organisms after two consecutive feedings, and using it as the basis for whether to feed subsequently, and automatically controlling the working state of the feeding device.

2. The intelligent pond feeding method based on machine vision according to claim 1, characterized in that, The method includes the following: 1) First, control the feeding machine to feed for t1 seconds, and after an interval of T seconds, feed for the second time for t2 seconds, and the high-definition waterproof camera collects the video data after feeding; 2) Preprocess each frame of the video data to extract the reflective area of the aquaculture water body: Among them, I s (x, y) and I v (x, y) represent the saturation and brightness of the image at (x, y) respectively, T s and T v are the saturation threshold and brightness threshold respectively, and f(x, y) represents the value of the pixel point (x, y) after binarization processing; 3) Use the optical flow method to extract the change characteristics of the water surface reflective area generated by the movement of aquaculture organisms. The change amplitude of the characteristic point position in the target area is expressed as: F is the optical flow between two consecutive frames of images, (x, y) represents the coordinates of the reflective area of the current frame, and M is the total number of non-zero motion vectors in the current frame; Use perspective transformation to correct the optical flow error caused by the installation position of the camera, and correct each point in the calculation result v in the motion vector field. After correction, it is: v, = v a λ a ; v a is the value of any point a in the motion vector field v, and λ a is the correction coefficient of this point; 4) Adopt a grid method to extract the motion blur of aquaculture organisms. Each frame of the picture is divided into m×n grid areas, and the motion characteristics of each particle are estimated by averaging the optical flow of all pixels within the individual grid. The optical flow information of each individual is calculated as the average value of the optical flow of all pixels within the grid. Then there is: i∈[1,m*n], where b and c are the side lengths of the image, and v ′′ is the optical flow of the i-th particle, N is the number of pixel points belonging to each grid, represents the optical flow of the i-th neighboring pixel belonging to the i-th particle ′ ; 5) Calculate the influence weight of the moving target i on the particle j: i,j∈[1,b*c], d ij is the Euclidean distance between particles i and j; 6) Use information entropy to measure the degree of irregularity of the probability distribution of the change characteristics of the water surface reflective area, so as to realize the analysis of the degree of irregular movement of aquaculture organisms. The information entropy is calculated as: z is v ′′ The number of divided intervals, P(j) is the probability of motion vectors falling within the divided velocity intervals. The fish school feeding desire value is calculated by combining information entropy and influence weight: D = I D *ζ ij , The larger D is, the stronger the feeding desire of the current fish group. Whether to continue feeding is judged by comparing the feeding activity levels of the fish groups in the two feedings.

3. The intelligent pond feeding method based on machine vision according to claim 2, wherein The t takes 2 seconds to 6 seconds, and the T takes 5 seconds to 100 seconds.

4. The intelligent pond feeding method based on machine vision according to claim 2, characterized in that 3) Correction coefficient λ a Determined by the distance from point a to the vanishing point V: V is the vanishing point, O is the point in the image that is farthest from the vanishing point, d(V, O) is the distance from the vanishing point to point O, and d(V, P a ) is the distance from the vanishing point to point a; Let l1 and l2 be two parallel line segments in the image. Arbitrary points B and C are taken from l1 respectively, and arbitrary points D and E are taken from l2. Calculate the coordinates of the vanishing point V according to the coordinates of the above four points: V=(B×C)×(D×E).

5. The intelligent pond feeding method based on machine vision according to claim 2, characterized in that The judgment of whether to continue feeding by comparing the feeding activity levels of the fish groups in the two feedings described in 6) specifically includes the following: Calculate the average feeding desire of the fish school during the first feeding time period t1 and the average feeding desire of the fish school during the second feeding time period t2 and compare them, where: D γ is the feeding desire value at the γ-th second during the first feeding process, D δ is the feeding desire value at the δ-th second during the second feeding process; If then control the feeding machine to perform the next round of feeding; If then stop the feeding instruction and wait for the start of the next feeding operation.

6. An intelligent pond feeding device based on machine vision, characterized in that, It includes a high-definition waterproof camera, an aerator, a feeding machine, a PLC, and a digital signal processor; the feeding machine includes a feed bin and a feeding device; The high-definition waterproof camera is installed on the bank of the pond, and its installation position needs to ensure that the real-time picture of the entire aquaculture water body can be captured; The feeding device is fixed at the center of the pond, and the connection line between the feeding device and the position of the high-definition waterproof camera is parallel to one side bank of the pond; The feed bin is installed outside the pond; The aerator is installed in the pond and should minimize the interference of the water surface fluctuation caused by its operation on the quantification of the water surface information in the feeding area; The high-definition waterproof camera is connected to the digital signal processor, the digital signal processor is connected to the PLC, and the PLC controls the feeding device and controls the feed bin to convey materials to the feeding device; Implement the machine vision-based intelligent pond feeding method according to any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the machine vision-based intelligent pond feeding method according to any one of claims 1-5.

8. An electronic device, characterized in that, The device includes: One or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the machine vision-based intelligent pond feeding method according to any one of claims 1-5.

Images