Intelligent pond feeding method and device based on machine vision

Abstract

The application discloses a pond intelligent feeding method and device based on machine vision, which is characterized by installing a high-definition waterproof camera on the bank of a pond, and the installation position of the camera is ensured to be able to shoot real-time pictures of the whole breeding water body; the method comprises the following steps: processing the images shot by the high-definition waterproof camera after feeding, determining the feeding desire value of the breeding organisms by combining information entropy and influence weight, quantifying the numerical relationship between the activity degrees of the breeding organisms after the first and second feeding, and taking the numerical relationship as the basis for whether the subsequent feeding is needed, and automatically controlling the working state of the material throwing device. According to the size of the quantification results of the two feedings, the working state of the feeding machine is controlled, accurate feeding is realized, and in the case of ensuring the nutritional conditions required for the growth of fishes, more attention is paid to the welfare of the fishes, and good environmental conditions can be provided for the growth of the fish population.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of aquaculture, and relates to a pond intelligent feeding method and device based on machine vision, which can especially quantify the feeding desire of fish groups and determine whether to feed in a pond environment. BACKGROUND

[0002] With the rapid development of the economic society and the significant improvement of people's living standards, people's demand for fish protein is increasing, and the global fish price will be in an upward channel. The growth of residents' income in the consumption market of aquatic products, as well as the factors of population growth and arable land restriction, make the increasing demand for protein and other aquatic meat in the world the main reason driving the price up. In addition, China's fishery policy aims to ensure the sustainability of the ocean and the environment, and the yield of wild catch fisheries will gradually decrease. At the same time, the yield of aquaculture fisheries will also slow down under the condition of rising costs (labor, feed, energy, etc.).

[0003] At present, the main modes of aquaculture in China include factory farming, deep water net cage culture, pond culture, etc. Among them, due to the influence of environmental factors, the degree of mechanization of pond culture is relatively low. The common pond feeding methods are manual feeding and pond feeding machine.

[0004] Manual feeding mainly uses tools such as spoons and shovels to manually throw feed, and the amount of feed is determined by experience through the eyes. Generally, it is difficult to grasp the most suitable demand level of the cultured objects. Manual feeding cannot guarantee the uniformity of feeding, and it is time-consuming, labor-intensive, low in efficiency, and even wastes fish feed and pollutes the pond water quality, affecting the growth and development of fish and increasing the cost of aquaculture. The feeding machine integrates fixed point, fixed time and fixed quantity, has the advantages of wide feeding area, uniform feeding, etc. It not only reduces the labor intensity of fishermen, but also increases the yield of fish. However, the existing feeding machines mostly use simple mechanical timing control systems and cannot automatically adjust the feeding amount according to the actual feeding demand of fish. When the feeding amount is less than the actual feeding demand of fish, serious food fighting will occur, causing fish to collide with each other and even causing damage to the surface of the fish. Fish with surface damage and small fish are more susceptible to certain fish diseases, which puts a lot of pressure on the aquaculture water environment and has a negative impact on the growth of fish. When the feeding amount is greater than the actual feeding demand of fish, not only will the cost of aquaculture increase, but the excess feed will also seriously pollute the aquaculture environment. Therefore, the feeding amount of feed should be consistent with the actual feeding demand of fish as much as possible. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art and provides a pond intelligent feeding method and device based on machine vision, which can accurately feed according to the feeding activity of the cultured organisms.

[0006] The technical solutions adopted by the present application are as follows:

[0007] A pond intelligent feeding method based on machine vision, a high-definition waterproof camera is installed on the bank of the pond, and the installation position ensures that the real-time picture of the entire breeding water body can be shot; the method comprises: processing the image shot by the high-definition waterproof camera after feeding, determining the feeding desire value of the breeding organism by combining information entropy and influence weight, quantifying the numerical relationship between the activity levels of the breeding organisms before and after the two feedings, and taking it as the basis for whether to feed subsequently, and automatically controlling the working state of the material throwing device.

[0008] In the above technical solution, further, the method comprises the following:

[0009] 1) First, control the feeding machine to feed for t1 seconds, feed for the second time for t2 seconds after an interval of T seconds, and the high-definition waterproof camera collects video data after feeding;

[0010] 2) Preprocess the video data frame by frame to extract the reflective light area of the breeding water body:

[0011]

[0012] Where I s (x,y) and I v (x,y) represent the saturation and brightness of the image at (x,y), respectively, T s and T v are the saturation threshold and brightness threshold, respectively, and f(x,y) represents the value of the pixel point (x,y) after binaryzation processing;

[0013] 3) Use the optical flow method to extract the water surface reflective light area change characteristics generated by the movement of the breeding organisms, and the change amplitude of the feature point position of the target area is represented as:

[0014]

[0015] F is the optical flow between two consecutive images, (x,y) represents the reflective light area coordinates of the current frame, and M is the total number of non-zero motion vectors in the current frame;

[0016] The optical flow error caused by the installation position of the camera is corrected by using perspective transformation, and each point in the calculation result v in the motion vector field is corrected, and after correction, it is:

[0017] v'=v a λ a ;

[0018] v a is the value of any point a in the motion vector field v, and λ a is the correction coefficient of the point;

[0019] 4) Using the grid method to extract the movement of the breeding organisms, each frame of picture is divided into m*n grid areas, and the motion characteristics of each particle are estimated by averaging the optical flow of all pixels in the individual grid. Each individual optical flow information is calculated by the average value of all pixel optical flow in the grid, that is,

[0020]

[0021]

[0022] i∈[1,m*n],

[0023] Where b and c are the image side length, v" is the optical flow of the i-th particle, N is the number of pixels belonging to each grid, Indicates the optical flow of the i'th adjacent pixel belonging to the i-th particle;

[0024] 5) Calculate the influence weight of moving target i on particle j:

[0025]

[0026] i,j∈[1,b*c],

[0027] d ij The Euclidean distance between particles i and j;

[0028] 6) Use information entropy to measure the irregularity of the distribution probability of the change characteristics of the water body reflection area, so as to realize the analysis of the irregularity of the breeding organism movement. The information entropy is calculated as:

[0029]

[0030] Z is the number of intervals v" is divided into, P(j) is the probability of falling into the divided speed interval, and the fish feeding desire value is calculated by combining the information entropy and the influence weight:

[0031] D=I D *ζ ij ,

[0032] The greater D is, the stronger the feeding desire of the current fish school is. By comparing the feeding activity of the fish school twice, it is determined whether to continue feeding.

[0033] Further, t is 2 to 6 seconds, and T is 5 to 100 seconds.

[0034] Further, the correction coefficient λ a is determined by the distance from point a to the vanishing point V:

[0035]

[0036] V is the vanishing point, O is the point in the image farthest from the vanishing point, d(V, O) is the distance from the vanishing point to O, d(V, P a ) is the distance from the vanishing point to a;

[0037] Let l1 and l2 be two parallel line segments in the image, and let B and C be any points on l1, and let D and E be any points on l2. The coordinates of the vanishing point V are calculated according to the coordinates of the four points as follows:

[0038] V = (B x C) x (D x E).

[0039] Further, the step 6) of determining whether to continue feeding by comparing the feeding activity of the fish population in two feeding processes, specifically includes the following steps:

[0040] calculating the average fish feeding desire value in the first feeding time period t1 and the average fish feeding desire value in the second feeding time period t2 and comparing them, wherein:

[0041]

[0042] D γ is the feeding desire value at the γth second during the first feeding process, and D δ is the feeding desire value at the δth second during the second feeding process;

[0043] If , the feeding machine is controlled to perform the next round of feeding.

[0044] If , the feeding instruction is stopped, and the next feeding operation is waited for.

[0045] An intelligent pond feeding device based on machine vision, comprising a high-definition waterproof camera, an oxygenator, a feeding machine, a PLC, and a digital signal processor; the feeding machine comprises a silo and a material throwing device.

[0046] The high-definition waterproof camera is installed on the bank of the pond, and its installation position needs to ensure that the real-time image of the entire breeding water body is captured.

[0047] The material throwing device is fixed at the center position of the pond, and the position line of the material throwing device and the high-definition waterproof camera is parallel to one side of the bank of the pond.

[0048] The silo is installed outside the pond.

[0049] The oxygenator is installed inside the pond and should minimize the water surface disturbance caused by its operation to interfere with the quantification of water surface information in the feeding area.

[0050] The high-definition waterproof camera is connected with a digital signal processor, the digital signal processor is connected with a PLC, the PLC controls a material throwing device, and the PLC controls a material bin to feed the material to the material throwing device; and the machine vision-based pond intelligent feeding method is realized.

[0051] The application forms a basis for whether subsequent feeding is needed by quantifying the numerical relationship between the activity levels of the fish groups after the two times of feeding in sequence, so as to automatically control the working state of the feeding machine and provide good reference and technical support for rationalized feeding operation of the pond culture.

[0052] The application has the following beneficial effects:

[0053] The machine vision-based pond intelligent feeding device based on deep learning has simple structure and simple control mode, uses machine vision related technology to analyze and compare the feeding activity levels of the fish groups after the two times of feeding, controls the working state of the feeding machine according to the size of the two times of quantitative results, can realize accurate feeding, pays more attention to the welfare of the fish under the condition of ensuring the nutritional conditions required for the growth of the fish, and can provide good environmental conditions for the growth of the fish groups. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 It is a machine vision-based fish intelligent feeding device structure diagram applied to a pond.

[0055] In the figure: 1-PLC; 2-high-definition waterproof camera; 3-material throwing device; 4-material bin; 5-oxygenator; 6-digital signal processor DETAILED DESCRIPTION

[0056] The technical solutions of the application will be further described below in combination with the drawings and specific embodiments.

[0057] Reference Figure 1 It is a specific example of the machine vision-based pond intelligent feeding device, which comprises a PLC 1, a high-definition waterproof camera 2, a material throwing device 3, a material bin 4, an oxygenator 5, and a digital signal processor 6.

[0058] The high-definition waterproof camera 2 is installed at the middle of the pond bank, and the high-definition waterproof camera 2 is connected with the input end of the digital signal processor 6; the installation position of the camera ensures that the real-time picture of the entire culture water body can be shot;

[0059] The material throwing device 3 is fixed at the center position of the pond, and the position line of the material throwing device 3 and the high-definition waterproof camera 2 is parallel to one side of the pond bank;

[0060] The material bin 4 is installed at the right side outside the pond.

[0061] The oxygenator 5 is installed at a corner of the pond, and a PVC pipe is fixed at one side of the oxygenator 5 for reducing the water surface fluctuation interference caused by the oxygenator during feeding to disturb the water surface information quantification of the feeding area;

[0062] The output end of the digital signal processor 6 is connected with the input end of the PLC 1; the digital signal processor 6 receives the image information input by the high-definition waterproof camera 2 and makes corresponding processing, first analyzes the real-time feeding desire of the fish group through the image processing technology, and then the processor transmits the processing result to the PLC 1 to control the working state of the material throwing device 3.

[0063] The intelligent feeding device is applied to the intelligent feeding of the fish group, and the feeding method comprises the following steps:

[0064] 1) The PLC 1 controls the material throwing device 3 to work for 10 seconds, and feeds for the second time after an interval of 100 seconds; the high-definition waterproof camera 2 transmits the collected video data to the digital signal processor 6 in real time, and analyzes the feeding activity degree of the cultured organisms in the previous two feedings to determine the subsequent feeding state;

[0065] 2) Specifically, the digital signal processor 6 pre-processes the received video screen to extract the reflection area of the cultured water body: Wherein I s (x,y) and I v (x,y) represent the saturation and brightness of the image at (x,y), respectively, T s and T v are the saturation threshold and brightness threshold, respectively, and f(x,y) represents the value of the pixel point (x,y) after the binary processing;

[0066] 3) The light flow method is used to extract the change characteristics of the water surface reflection area caused by the movement of the cultured organisms, and the change amplitude of the feature point position of the target area is represented as F is the light flow between the two continuous frames of images, (x,y) represents the reflection area coordinates of the current frame, and M is the total number of non-zero motion vectors in the current frame;

[0067] The perspective transformation method is used to correct the light flow error caused by the installation position of the camera, and each point in the calculation result v in the motion vector field is corrected, and after the correction, v' = v a λ a ; v a is a point a in the motion vector field, λ a is the correction coefficient of the point; the vanishing point V = (B x C) x (D x E), and the vanishing point can be determined by calculating the intersection of two line segments; let l1 and l2 be two parallel line segments in the image, which are defined by points B and C on l1 and points D and E on l2; the correction coefficient λ a is determined by the distance from the point a to the vanishing point V where d(V, O) is the distance from the vanishing point to the point O in the image that is farthest from the vanishing point, d(V, P a ) is the distance from the vanishing point to the point a;

[0068] 4) The grid method is used to extract the motion blur of the cultured organisms. Grid-shaped particles are placed in each frame of image to simulate the individuals in the fish school. The grid size is 15*15, and the optical flow information of each individual is calculated as the average value of the optical flow of all pixels in the grid: i' ∈ [1, b / 15

[0069] c / 15], i ∈ [1, 15*15], where b and c are the image side lengths, v" is the optical flow of the i-th particle, N is the number of adjacent pixels belonging to each particle, represents the optical flow of the i'-th adjacent pixel belonging to the i-th particle.

[0070] 5) The influence weight of the moving target i on the particle j is calculated:

[0071] i, j ∈ [1, b*c],

[0072] d ij is the Euclidean distance between the particles i and j;

[0073] 6) The information entropy is used to measure the irregularity of the distribution probability of the change characteristics of the water body reflection area, so as to realize the analysis of the irregularity of the motion of the cultured organisms.

[0074]

[0075] z is the number of intervals into which v" is divided, and P(j) is the probability of the motion vector falling into the divided speed interval. The fish school feeding desire value is calculated by combining the information entropy and the influence weight: d = I D * ζ ij The greater D is, the more intense the current fish school feeding desire is. The feeding activity degree of the fish school is compared between two times of feeding to determine whether to continue feeding.

[0076] 7) The fish school feeding desire mean value in the first feeding time period t1 is calculated, and the fish school feeding desire mean value in the second feeding time period t2 is calculated and compared, where W γ is the feeding desire value at the γ-th second in the first feeding process, and D δ is the feeding desire value at the δ-th second in the second feeding process; if the processing result is input to the PLC 1 from the digital signal processor 6, and the PLC 1 controls the feeder to perform the next round of feeding.

[0077] 8) if then the digital signal processor 6 issues a stop feeding command to the PLC 1 and waits for the start of the next feeding operation.

[0078] The above disclosed is only specific embodiments of the present application, but the present application is not limited thereto, and the deformation made by the ordinary skilled in the art without departing from the present application should be considered as belonging to the protection scope of the present application.

Claims

1. A machine vision-based intelligent feeding method for a pond, characterized in that, The high-definition waterproof camera is installed on the bank of the pond, and the installation position ensures that the real-time picture of the entire aquaculture water body can be shot, and the position line of the throwing device of the feeding machine and the high-definition waterproof camera is parallel to the side bank of the pond; the method comprises: processing the image shot by the high-definition waterproof camera after feeding, determining the feeding desire value of the cultured organisms by combining information entropy and influence weight, quantifying the numerical relationship between the activity levels of the cultured organisms before and after feeding, and taking it as the basis for subsequent feeding, automatically controlling the working state of the throwing device, and specifically comprising the following: 1) First control the feeder to feed seconds, and feed for the second time after an interval of T seconds seconds, and the high-definition waterproof camera collects video data after feeding; 2) The video data is preprocessed frame by frame to extract the reflection area of the aquaculture water body: ， wherein and respectively represent the saturation and the brightness of the image at , and respectively are a saturation threshold and a brightness threshold, represents the value of the pixel point after the binarization processing. 3) Extract the change characteristics of the water surface reflection area generated by the movement of the cultured organisms using the optical flow method to obtain the change amplitude of the feature point position of the target area is expressed as: ， is the optical flow between the two consecutive images, represents the back light region coordinates of the current frame, is the total number of non-zero motion vectors in the current frame; Perspective transformation is used to correct optical flow errors caused by the camera's installation position, and the calculation results in the motion vector field are then analyzed. Each point in the data is corrected, resulting in: ； is the value of the motion vector field at any point in the image, is the value of the motion vector field at the point is the correction coefficient for this point, determined by the distance of the point to the vanishing point : ， For vanishing point, The point in the image that is farthest from the vanishing point. It is the vanishing point. Distance between points It is the vanishing point. Distance between points; set up and These are two parallel line segments in the image, respectively from... Take any point above and ,as well as any point on and Calculate the vanishing point based on the coordinates of the four points mentioned above. Coordinates: ， 4) Grid method is used to extract the motion blur of the cultured organisms. Each frame of picture is divided into grid regions, and the motion characteristics of each particle are estimated by averaging the optical flow of all pixels in each individual grid. Each individual optical flow information is calculated by the average value of the optical flow of all pixels in the grid. Thus, we have: ， ， ， Where b and c are the side lengths of the image. Let N be the optical flow of the i-th particle, and N be the number of pixels belonging to each grid cell. This indicates the i-th particle's... Optical flow of neighboring pixels; 5) Calculate the influence weight of moving object i on particle j : , , is the Euclidean distance between particles i and j; 6) Use of information entropy The irregularity of the distribution probability of the change characteristics of the reflective region of the water body is measured, so as to realize the analysis of the irregularity of the movement of the cultured organisms. The information entropy is calculated as: ； is the number of divided intervals, is the probability of the motion vector falling into the divided speed interval, the fish school feeding desire value is calculated by combining the information entropy and the influence weight: ， The larger D is, the stronger the feeding desire of the current fish group is, whether to continue feeding is determined by comparing the feeding activity of the fish group before and after feeding, and specifically comprising the following: the first feeding period the mean fish school feeding desire within the second feeding period the mean fish school feeding desire within and comparing wherein: ， ， the first feeding process, the first time point is 0 second, and the second time point is 30 seconds. the feeding desire value at the first time point in the first feeding process, the feeding desire value at the first time point in the second feeding process; and the feeding desire value at the second time point in the second feeding process. If then the feeder is controlled to perform the next round of feeding; If then stop the feeding instruction and wait for the start of the next feeding operation.

2. A pond intelligent feeding device based on machine vision, characterized in that, The high-definition waterproof camera, the oxygenator, the feeding machine, the PLC and the digital signal processor are included; the feeding machine comprises a bunker and a throwing device; The high-definition waterproof camera is installed on the bank of the pond, and the installation position needs to ensure that the real-time picture of the entire aquaculture water body is shot; The throwing device is fixed at the center position of the pond, and the position line of the throwing device and the high-definition waterproof camera is parallel to the side bank of the pond; The bunker is installed outside the pond; The oxygenator is installed in the pond, and the water surface disturbance caused by its work should be reduced as much as possible to interfere with the information quantization of the feeding area; The high-definition waterproof camera is connected with the digital signal processor, the digital signal processor is connected with the PLC, the PLC controls the throwing device, and the bunker supplies materials to the throwing device; The intelligent pond feeding method based on machine vision is realized.

3. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the intelligent pond feeding method based on machine vision.

4. An electronic device, comprising: The device comprises: One or more processors; Memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors realize the intelligent pond feeding method based on machine vision.

Images