Pond culture fish school growth monitoring method and feeding robot equipment

Abstract

The invention discloses a pond culture fish school growth monitoring method and a feeding robot device, relates to the technical field of fish school culture, and solves the problem that the existing fish school monitoring and recognition effect based on water surface and underwater images is poor, and the technical scheme is characterized in that a pond water surface image and a pond underwater image are acquired based on a binocular camera; de-noising processing is performed on the pond water surface image, and adaptive enhancement processing is performed on the pond underwater image; fish body identification is carried out based on the processed pond water surface and underwater images, a fish body surrounding frame is drawn, and the size and weight of a fish body are calculated based on the fish body surrounding frame and corresponding depth information; the fish body size and weight are calculated based on the pond water surface image and the pond underwater image, the growth monitoring result of the pond culture fish school is obtained through calculation, and the growth monitoring result comprises the fish school size and the fish school weight; and fish school growth prediction is carried out based on the processed pond water surface image and pond underwater image. The above problem is solved.

Description

Technical Field

[0001] The invention relates to the technical field of fish breeding, and more particularly to a method for monitoring the growth of fish in pond breeding and a feeding robot device. Background Art

[0002] Industrialized pond farming is one of the current development directions of fish farming technology. Monitoring the growth of pond-farmed fish based on industrialized technology is of great significance to improving the survival rate of fish. The current monitoring method mainly monitors the fish in the pond in real time by setting up monitoring cameras beside the pond, and some also monitor the fish underwater. However, the current monitoring method has shortcomings in monitoring the growth size and weight of fish. Specifically, fish monitoring based on water surface images is easily affected by bubbles in the aeration equipment, and a large amount of foam formed on the water surface affects fish monitoring; although fish monitoring based on underwater images can avoid the influence of foam, the underwater optical environment is dim, and the recognition effect of fish images is poor.

[0003] Therefore, the present application provides a pond culture fish growth monitoring method and feeding robot equipment to solve the above problems. Summary of the invention

[0004] The purpose of the present application is to provide a method for monitoring the growth of fish in pond culture and a feeding robot device, so as to solve the problem that fish monitoring based on water surface images is easily affected by bubbles in oxygenation equipment, and fish monitoring based on underwater images is limited by dim optical environment and the fish identification effect is poor; the present application combines pond surface images and pond underwater images to predict fish growth, denoises the pond surface images, and adaptively enhances the pond underwater images to solve the above problems.

[0005] The present application first provides a method for monitoring the growth of fish cultured in a pond, comprising: acquiring a pond surface image and a pond underwater image based on a binocular camera; calculating the depth information of the pond surface image and the pond underwater image based on the parameters of the binocular camera, and generating a pond surface depth image and a pond underwater depth image; aligning the pond surface image and the pond surface depth image, and aligning the pond underwater image and the pond underwater depth image, to obtain the depth information of the pixels in the pond surface image and the pond underwater image; denoising the pond surface image, and adaptively enhancing the pond underwater image; identifying fish bodies and drawing fish body bounding boxes based on the denoised pond surface image and the adaptively enhanced pond underwater image, respectively, and calculating the fish body size and weight based on the fish body bounding box and the corresponding depth information; calculating the growth monitoring results of the fish cultured in the pond based on the fish body size and weight calculated from the pond surface image and the fish body size and weight calculated from the pond underwater image, the growth monitoring results including: fish school size and fish school weight.

[0006] In one possible implementation, denoising is performed based on the pond water surface image; including: identifying foam in the image based on the pond water surface image and marking the foam area; setting a covering matrix based on the area size of the foam area; and covering each pixel point of the foam area from the outside to the inside based on the covering matrix to obtain a denoised pond water surface image.

[0007] In a possible implementation, adaptive enhancement processing is performed based on the underwater image of the pond; including: grayscale processing of the underwater image of the pond and dividing it into a plurality of non-overlapping small blocks; counting the grayscale histogram of each small block, cropping the grayscale histogram of each small block by a set contrast limit, and distributing the number of pixels exceeding the contrast limit to other gray levels; equalizing the cropped grayscale histogram; and splicing the processed small blocks based on an interpolation method to form an underwater image of the pond after adaptive enhancement processing.

[0008] In a possible implementation, fish are identified and fish bounding boxes are drawn based on the denoised pond surface image and the adaptively enhanced pond underwater image, and the size and weight of the fish are calculated based on the fish bounding box and the corresponding depth information; including: extracting features of different depths from the image, and splicing the features of different depths to obtain a feature image; detecting the feature map based on the trained YOLOv5 model to obtain an image containing a fish bounding box; correcting the perspective deformation in the fish bounding box based on the depth information of the pixels to obtain a standard fish bounding box; calculating the fish size based on the standard fish bounding box, and calculating the fish weight based on the fish size and the fish type.

[0009] In a possible implementation, features of different depths are extracted from an image, and features of different depths are concatenated to obtain a feature map; including: extracting features in the image based on a convolution layer, a batch normalization layer, and an activation function layer; inputting the features into multiple pooling layers of the same scale in sequence to obtain features of the same scale at different depths; concatenating features of the same scale at different depths to obtain multidimensional features; and inputting the multidimensional features into a convolution layer, a batch normalization layer, and an activation function layer to obtain a feature map.

[0010] In a possible implementation, a trained YOLOv5 model is obtained by the following steps: acquiring a plurality of pond surface images and pond underwater images; performing denoising and adaptive enhancement processing on the pond surface images and pond underwater images, respectively; annotating complete fish bodies and incomplete fish bodies in the processed pond surface images and pond underwater images; dividing the annotated fish body bounding boxes into a training set, a validation set, and a test set; training the YOLOv5 model based on the training set, adjusting the YOLOv5 model parameters based on the validation set, and evaluating the model performance based on the test set to obtain a trained YOLOv5 model.

[0011] In a possible implementation, perspective deformation in a fish body enclosing frame is corrected based on pixel depth information to obtain a standard fish body enclosing frame; including: obtaining four endpoints of the fish body enclosing frame; correcting the positions of the four endpoints based on the pixel depth information of the four endpoints, and drawing a standard fish body enclosing frame based on the corrected four endpoints; when the fish body in the standard fish body enclosing frame is an incomplete fish body, supplementing the fish body according to the fish body contour and updating the standard fish body enclosing frame.

[0012] In a possible implementation, the fish size is calculated based on a standard fish body bounding box, and the fish weight is calculated based on the fish body size and the fish body type; including: calculating the fish body size based on the maximum horizontal distance and the maximum vertical distance of the standard fish body bounding box; substituting the fish body size and the fish body type into the fish body weight estimation formula to obtain the fish body weight.

[0013] In a possible implementation, growth monitoring results of pond-farmed fish are calculated based on fish size and weight calculated from pond surface images and fish size and weight calculated from pond underwater images, and the growth monitoring results include: fish size and fish weight; including: solving the average fish size and average fish weight of pond surface images based on the fish size and weight calculated from pond surface images; solving the average fish size and average fish weight of pond underwater images based on the fish size and weight calculated from pond underwater images; solving the weighted average fish size and weighted average fish weight based on the average fish size and average fish weight of pond surface images, the average fish size and average fish weight of pond underwater images, and set pond surface image weight values ​​and pond underwater image weight values ​​as the growth monitoring results of pond-farmed fish.

[0014] The present application also provides a feeding robot device for executing the above-mentioned method for monitoring the growth of fish in pond culture, the device comprising: an image acquisition module for acquiring a pond surface image and a pond underwater image based on a binocular camera; a depth calculation module for calculating the depth information of the pond surface image and the pond underwater image based on the parameters of the binocular camera, and generating a pond surface depth image and a pond underwater depth image; A depth alignment module is used to align the pond surface image with the pond surface depth image, and to align the pond underwater image with the pond underwater depth image, so as to obtain the depth information of the pixels in the pond surface image and the pond underwater image; an image processing module is used to perform denoising on the pond surface image and adaptive enhancement on the pond underwater image; a sample fish school prediction module is used to perform fish body recognition and draw fish body bounding boxes based on the denoised pond surface image and the adaptively enhanced pond underwater image, respectively, and calculate the fish body size and weight based on the fish body bounding box and the corresponding depth information; a comprehensive fish school prediction module is used to calculate the growth monitoring results of the pond cultured fish based on the fish body size and weight calculated by the pond surface image and the fish body size and weight calculated by the pond underwater image, and the growth monitoring results include: fish school size and fish school weight; a cultured fish school feeding module is used to determine the type and weight of fish feed to be fed according to the growth monitoring results.

[0015] Compared with the prior art, the present application has the following beneficial effects: based on denoising processing, the influence of water surface foam in the pond surface image on fish body recognition is reduced; based on adaptive enhancement processing, the image quality of the pond underwater image is improved and the contour features are enhanced; based on depth information, the fish body bounding box is projected to the same plane to eliminate the influence of perspective distortion on the calculation of fish body size; based on the pond surface image and the pond underwater image, the size and weight of the pond farmed fish school are jointly calculated, and the advantages of the two observations are combined to more accurately calculate the size and weight of the pond farmed fish school. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 A flow chart of a method for monitoring the growth of fish in pond culture provided in an embodiment of the present application; Figure 2 A schematic diagram of multi-dimensional feature extraction provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of the feeding robot device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] Hereinafter, the term "include" or "may include" used in various embodiments of the present application indicates the presence of the function, operation or element applied for, and does not limit the increase of one or more functions, operations or elements. In addition, as used in various embodiments of the present application, the terms "include", "have" and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or the possibility of increasing one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items.

[0018] The terms used in the various embodiments of the present application are only used for the purpose of describing specific embodiments and are not intended to limit the various embodiments of the present application. As used herein, the singular form is intended to also include the plural form, unless the context clearly indicates otherwise. Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the meanings commonly understood by ordinary technicians in the field to which the various embodiments of the present application belong. The terms (such as those defined in a generally used dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized meaning or an overly formal meaning, unless clearly defined in the various embodiments of the present application.

[0019] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with examples and drawings. The illustrative implementation scheme of the present application and its description are only used to explain the present application and are not intended to limit the present application.

[0020] See also Figure 1 As shown, Figure 1A flow chart of a method for monitoring the growth of fish in a pond provided in an embodiment of the present application. The method includes: S1, obtaining a pond surface image and a pond underwater image based on a binocular camera; S2, calculating the depth information of the pond surface image and the pond underwater image based on the parameters of the binocular camera, and generating a pond surface depth image and a pond underwater depth image; S3, aligning the pond surface image and the pond surface depth image, aligning the pond underwater image and the pond underwater depth image, and obtaining the depth information of the pixels in the pond surface image and the pond underwater image; S4, denoising the pond surface image, and adaptively enhancing the pond underwater image; S5, respectively identifying fish bodies and drawing fish body bounding boxes based on the denoised pond surface image and the adaptively enhanced pond underwater image, and calculating the fish body size and weight based on the fish body bounding box and the corresponding depth information; S6, calculating the growth monitoring results of the fish in the pond based on the fish body size and weight calculated from the pond surface image and the pond underwater image, and the growth monitoring results include: fish school size and fish school weight.

[0021] Specifically, monitoring is performed according to the set monitoring frequency. During each monitoring, the binocular camera is moved to a set height to take two images of the pond surface, and the binocular camera is moved to a set depth to take two images of the pond underwater. Depth information is calculated based on the features of the binocular camera and a depth image is drawn. The depth image is aligned with the captured pond image (pond surface image, pond underwater image), and the pixels in the pond image have corresponding depth information. Considering that the foam generated by the aeration equipment on the pond surface will affect the fish body recognition, such as blocking the fish body, being mistakenly identified as the fish body, etc., the pond surface image is denoised to eliminate the influence of the surface foam. At the same time, considering that the light is dark and the image is blurred during underwater shooting, the pond underwater image is adaptively enhanced to improve the image quality by balancing the grayscale. Fish body recognition and bounding box drawing are performed based on the processed pond surface and underwater images, and the fish body size and weight are calculated based on the fish body bounding box. Finally, considering the respective advantages of the pond surface and underwater images, the size and weight of the pond fish school are comprehensively calculated, and the size and weight of the pond fish school can be used to guide the feeding strategy.

[0022] The improvements of the present application are: first, denoising is performed on the pond surface image to eliminate the influence of the foam generated by the oxygenation equipment on the water surface on the fish recognition; second, adaptive enhancement is performed on the pond underwater image to enhance the image quality of the underwater low-light image and enhance the contour features; third, the fish size and weight are calculated based on the fish body bounding box and the corresponding depth information, and the perspective distortion in the image is corrected based on the depth information to eliminate the influence of the perspective distortion on the fish size calculation; fourth, the size and weight of the pond farmed fish are calculated based on the pond surface image and the pond underwater image. The number of fish samples observed by the surface image is larger, and the fish samples observed by the underwater image are more accurate. The advantages of the two observations are combined to more accurately calculate the size and weight of the pond farmed fish.

[0023] In a possible implementation, S4 performs denoising based on the pond water surface image; including: S411, identifying foam in the image based on the pond water surface image and marking the foam area; S412, setting a coverage matrix based on the area size of the foam area; S413, covering each pixel point of the foam area from the outside to the inside based on the coverage matrix to obtain the pond water surface image after denoising.

[0024] Specifically, the foam on the surface of the pond is white, and the pond water surface is dark, so the foam floating on the water surface can be identified based on the color difference, thereby marking the foam area. For example, when converting the image from RGB space to HSV space, the foam usually appears white, so in HSV space, a higher brightness (V) threshold and a lower saturation (S) threshold are set to identify the foam area, and morphological operations such as corrosion and expansion are performed on the foam area to remove edge noise and internal gaps to obtain a more accurate foam area. Finally, the contour of the foam area is extracted based on the edge detection algorithm, and the foam area is marked in the image. The foam on the water surface generated by the aeration equipment appears in sheets, and some floats independently on the water surface in the form of dots. Therefore, different sizes of coverage matrices are set according to the size of the foam area. The larger the area of ​​the foam area, the larger the size of the coverage matrix. The coverage matrix is ​​used to cover the grayscale value of the target pixel (the pixel in the foam area) according to the grayscale mean of the surrounding pixels, reducing the boundary characteristics of the foam and avoiding its influence on fish body recognition. Based on the coverage matrix, the foam area is gradually covered from the periphery to the inside. First, the pixels on the outline of the foam area are traversed, and then the internal pixels are traversed, so that the foam area is merged with the surrounding area to obtain the denoised pond water surface image.

[0025] It should be noted that this scheme performs denoising on the pond water surface image, which can greatly eliminate the influence of water surface foam on fish body segmentation; and the size of the coverage matrix is ​​determined according to the size of the foam area, and the point-shaped foam areas and the sheet-shaped foam areas are processed with different precisions, so that foam areas of different sizes can be better integrated into the surrounding areas; each pixel point in the foam area is covered from the outside to the inside, which can reduce the influence of the pixels inside the foam area on the coverage.

[0026] In a possible implementation, the method of performing adaptive enhancement processing based on the underwater image of the pond in S4 includes: S421, graying the underwater image of the pond and dividing it into a plurality of non-overlapping small blocks; S422, counting the grayscale histogram of each small block, cropping the grayscale histogram of each small block by a set contrast limit, and distributing the number of pixels exceeding the contrast limit to other gray levels; S423, equalizing the cropped grayscale histogram; S424, splicing the processed small blocks based on an interpolation method to form an adaptively enhanced underwater image of the pond.

[0027] Specifically, the image is first divided into multiple non-overlapping small blocks, for example, (8, 8) means that the size of each small block is 8×8 pixels; the histogram of each small block is calculated separately, the number of pixels at each gray level is counted, and the histogram of each small block is cropped to evenly distribute the number of pixels exceeding the contrast limit to other gray levels to avoid excessive contrast enhancement and noise amplification; the cropped histogram is equalized and converted into a cumulative distribution function, which can also be stretched to increase the contrast; finally, the results of each small block are spliced ​​together using an interpolation method to eliminate the boundary effect between small blocks and obtain an image after adaptive enhancement.

[0028] In a possible implementation, S5, respectively, performs fish recognition and draws fish bounding boxes based on the pond surface image after denoising and the pond underwater image after adaptive enhancement, and calculates the fish size and weight based on the fish bounding box and the corresponding depth information; including: S51, extracts features of different depths from the image, and splices the features of different depths to obtain a feature image; S52, detects the feature map based on the trained YOLOv5 model to obtain an image containing a fish bounding box; S53, corrects the perspective deformation in the fish bounding box based on the depth information of the pixels to obtain a standard fish bounding box; S54, calculates the fish size based on the standard fish bounding box, and calculates the fish weight based on the fish size and fish type.

[0029] Specifically, feature extraction is performed on the processed pond water surface and underwater images, and feature images are obtained by splicing features of different depths; the feature image is detected based on the trained YOLOv5 model, the fish body is identified and a bounding box is drawn to obtain an image containing a fish body bounding box; the bounding box is scaled according to the depth information of the four endpoints of the bounding box, the distant fish body is enlarged and the close fish body is reduced to the same level to obtain a standard fish body bounding box; the length and width of the standard fish body bounding box are used as the fish body size, and the fish weight is calculated according to the length of the standard fish body bounding box and the fish type.

[0030] Furthermore, S51, extracting features of different depths from the image, and concatenating the features of different depths to obtain a feature image; including: S511, extracting features in the image based on a convolution layer, a batch normalization layer, and an activation function layer; S512, inputting the features into multiple pooling layers of the same scale in sequence to obtain features of the same scale at different depths; S513, concatenating features of the same scale at different depths to obtain multidimensional features; S514, inputting the multidimensional features into a convolution layer, a batch normalization layer, and an activation function layer to obtain a feature map.

[0031] Specifically, see Figure 2 As shown, Figure 2 Schematic diagram of multi-dimensional feature extraction provided for the embodiment of the present application. First, the convolution layer Conv, batch normalization layer BN and activation function layer ReLU are used to extract features in the image; then the features are sequentially input into multiple pooling layers of the same scale for downsampling to obtain scale features of different depths; multiple scale features are spliced ​​to obtain a multi-dimensional feature representation; finally, the multi-dimensional features are processed again through the convolution layer Conv, batch normalization layer BN and activation function layer ReLU to output a feature map. The feature map strengthens the texture and contour features in the image, so that YOLOv5 can more accurately identify the fish body.

[0032] Further, in S52, the trained YOLOv5 model is obtained through the following steps: S521, obtaining multiple pond surface images and pond underwater images; S522, performing denoising and adaptive enhancement processing on the pond surface images and pond underwater images respectively; S523, marking the complete fish bodies and incomplete fish bodies in the processed pond surface images and pond underwater images; S524, dividing the marked fish body bounding boxes into training set, validation set and test set; S525, training the YOLOv5 model based on the training set, adjusting the YOLOv5 model parameters based on the validation set, evaluating the model performance based on the test set, and obtaining the trained YOLOv5 model.

[0033] Specifically, YOLOv5, as a deep learning algorithm, is capable of object detection. The extracted feature map is input into the trained YOLOv5 model, and the model will identify the fish in the image and draw a bounding box on the image, thereby obtaining an image containing the bounding box of the fish. YOLOv5 provides models of different sizes: YOLOv5s: a small model suitable for scenarios with limited computing resources; YOLOv5m: a medium-sized model that balances speed and performance; YOLOv5l: a large model that provides higher accuracy but requires more computing resources; YOLOv5x: an extra-large model for scenarios that require the highest accuracy. Based on project requirements and available computing resources, this application selects YOLOv5m as the detection model.

[0034] When training YOLOv5, a large number of pond surface images and pond underwater images are collected. De-noising and adaptive enhancement processing are performed according to the image sources to eliminate the influence of water surface foam and dark bottom environment. Both complete and incomplete fish bodies in the images are labeled. The YOLOv5 model is trained based on the labeled images to obtain a trained YOLOv5 model.

[0035] Furthermore, S53, based on the pixel depth information, the perspective deformation in the fish body enclosing frame is corrected to obtain a standard fish body enclosing frame; including: S531, obtaining the four endpoints of the fish body enclosing frame; S532, based on the pixel depth information of the four endpoints, the positions of the four endpoints are corrected, and the standard fish body enclosing frame is drawn based on the corrected four endpoints; S533, when the fish body in the standard fish body enclosing frame is an incomplete fish body, the fish body is supplemented according to the fish body contour and the standard fish body enclosing frame is updated.

[0036] Specifically, the four endpoints of the fish body bounding box are obtained, and the pixel depth information provided by the binocular camera system is used to project the fish body bounding boxes at different depths onto the same plane, thereby obtaining a standard fish body bounding box. If the fish body detected in the standard fish body bounding box is not a complete fish body, the missing part of the fish body can be supplemented by the standard outline of the pond fish school and the standard fish body bounding box can be updated to ensure that the bounding box can completely cover the fish body.

[0037] Further, S54, calculating the fish size based on the standard fish body bounding box, and calculating the fish weight based on the fish body size and the fish body type; including: S541, calculating the fish body size based on the maximum horizontal distance and the maximum vertical distance of the standard fish body bounding box; S542, substituting the fish body size and the fish body type into the fish body weight estimation formula to obtain the fish body weight.

[0038] Specifically, the length and height of the actual fish body are calculated based on the length and height of the standard fish body bounding box based on the camera calibration parameters, and the fish weight is estimated by substituting the actual fish body length into the fish body weight estimation formula, such as W=α L b , where W is the fish weight, L The length of the fish ， α、 b is the proportional coefficient, which is related to the fish body type.

[0039] It should be noted that the present application extracts features of different depths from the image to obtain a feature image. This step can replace the spatial pyramid pooling part in the YOLOv5 model to retain feature information from multiple dimensions. Complete fish bodies and incomplete fish bodies can be identified based on the trained YOLOv5 model. Overlapping fish bodies can be identified and as many fish body samples as possible can be retained. The fish body bounding box is corrected to the same plane through the depth information of the pixels to avoid calculation deviations caused by the distance of the fish body.

[0040] In a possible implementation, S6, based on the fish size and weight calculated from the pond surface image and the fish size and weight calculated from the pond underwater image, calculate the growth monitoring result of the pond farmed fish, and the growth monitoring result includes: fish size and fish weight; including: S61, based on the fish size and weight calculated from the pond surface image, solve the average fish size and average fish weight of the pond surface image; S62, based on the fish size and weight calculated from the pond underwater image, solve the average fish size and average fish weight of the pond underwater image; S63, based on the average fish size and average fish weight of the pond surface image, the average fish size and average fish weight of the pond underwater image, and the set pond surface image weight value and pond underwater image weight value, solve the weighted average fish size and weighted average fish weight as the growth monitoring result of the pond farmed fish.

[0041] Specifically, after calculating the size and weight of a single fish sample, the average fish size and average fish weight of the pond water surface image can be obtained by averaging the size and weight of all fish samples in the image: Average fish size: ; Average fish weight: ; in, and Respectively represent The length and weight of the fish, is the total number of fish. Similarly, we can get the average fish size and average fish weight of the underwater image of the pond.

[0042] According to the weight value of the pond surface image and the weight value of the pond underwater image, the weighted average fish size and weighted average fish weight are calculated: Weighted average fish size:L’ = L 水面 ⋅ w 水面 + L 水下 ⋅ w 水下 ; Weighted average fish weight: W’ = W 水面 ⋅ w 水面 + W 水下 ⋅ w 水下 ; in, w 水面 is the pond surface image weight value, w 水下 is the weight value of the pond underwater image, which can be understood as the confidence of the data source. w 水面 + w 水下 =1; L 水面 , W 水面 The average fish size and average fish weight of the pond water surface image; L 水下 , W 水下 is the average fish size and average fish weight of the underwater image of the pond. The weighted average fish size obtained by solving L’ and weighted average fish weight W’ , which can be used as the growth monitoring result of pond-cultured fish and can be used to guide the feeding strategy of fish.

[0043] See also Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of the feeding robot device provided in the embodiment of the present application. The device is used to perform the following steps: Figure 1The growth monitoring method of fish swarms in pond culture shown in the figure includes: an image acquisition module, which is used to acquire a pond surface image and a pond underwater image based on a binocular camera; a depth calculation module, which is used to calculate the depth information of the pond surface image and the pond underwater image based on the parameters of the binocular camera, and generate a pond surface depth image and a pond underwater depth image; a depth alignment module, which is used to align the pond surface image and the pond surface depth image, and align the pond underwater image and the pond underwater depth image, and obtain the depth information of the pixel points in the pond surface image and the pond underwater image; an image processing module, which is used to denoise the pond surface image and align the pond underwater image a sample fish prediction module, for respectively identifying fish bodies and drawing fish body bounding boxes based on the pond water surface image after denoising and the pond underwater image after adaptive enhancement, and calculating the fish body size and weight based on the fish body bounding boxes and the corresponding depth information; a comprehensive fish prediction module, for calculating the growth monitoring results of pond cultured fish based on the fish body size and weight calculated from the pond water surface image and the fish body size and weight calculated from the pond underwater image, the growth monitoring results including: fish school size and fish school weight; a farmed fish feeding module, for determining the type and weight of fish feed to be fed according to the growth monitoring results.

[0044] It is understood that the feeding robot device is used to perform the following steps: Figure 1 The pond culture fish growth monitoring method shown corresponds to the method one by one and has corresponding technical effects, so it will not be elaborated on here.

[0045] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for monitoring the growth of fish in pond culture, characterized in that: include: Acquire pond surface images and pond underwater images based on a binocular camera; Calculating the depth information of the pond water surface image and the pond underwater image based on the parameters of the binocular camera to generate a pond water surface depth image and a pond underwater depth image; Aligning the pond water surface image with the pond water surface depth image, aligning the pond underwater image with the pond underwater depth image, and obtaining depth information of pixel points in the pond water surface image and the pond underwater image; Performing denoising processing on the pond surface image and performing adaptive enhancement processing on the pond underwater image; Based on the denoised pond surface image and the adaptively enhanced pond underwater image, fish recognition is performed and fish body bounding boxes are drawn, and fish body size and weight are calculated based on the fish body bounding boxes and corresponding depth information; Based on the fish size and weight calculated from the pond water surface image and the fish size and weight calculated from the pond underwater image, a growth monitoring result of the fish school cultured in the pond is calculated, and the growth monitoring result includes: fish school size and fish school weight.

2. A method for monitoring the growth of fish in pond culture according to claim 1, characterized in that: Performing denoising processing based on the pond water surface image; comprising: Based on the pond water surface image, foam in the image is identified and the foam area is marked; Set the coverage matrix based on the size of the foam area; Based on the covering matrix, each pixel point in the foam area is covered from the outside to the inside to obtain the denoised pond water surface image.

3. A method for monitoring the growth of fish in pond culture according to claim 1, characterized in that: Performing adaptive enhancement processing based on the underwater image of the pond; comprising: gray-scale the pond underwater image and divide it into a plurality of non-overlapping small blocks; Count the grayscale histogram of each small block, clip the grayscale histogram of each small block according to the set contrast limit, and distribute the number of pixels exceeding the contrast limit to other gray levels; Perform equalization on the cropped grayscale histogram; The processed small blocks are spliced ​​together based on the interpolation method to form an adaptively enhanced underwater image of the pond.

4. A method for monitoring the growth of fish in pond culture according to claim 1, characterized in that: Based on the denoised pond surface image and the adaptively enhanced pond underwater image, fish recognition is performed and a fish body bounding box is drawn, and the fish body size and weight are calculated based on the fish body bounding box and the corresponding depth information; comprising: Extract features of different depths from the image, and concatenate the features of different depths to obtain a feature image; The feature map is detected based on the trained YOLOv5 model to obtain an image containing a fish body bounding box; Correcting the perspective deformation in the fish body bounding box based on the depth information of the pixels to obtain a standard fish body bounding box; The fish body size is calculated based on the standard fish body bounding box, and the fish body weight is calculated based on the fish body size and the fish body type.

5. A method for monitoring the growth of fish in pond culture according to claim 4, characterized in that: The method extracts features of different depths from the image and concatenates the features of different depths to obtain a feature map; including: Extract features from the image based on convolutional layers, batch normalization layers, and activation function layers; Inputting the features into multiple same-scale pooling layers in sequence to obtain same-scale features of different depths; The same-scale features of different depths are concatenated to obtain multi-dimensional features; The multidimensional features are input into the convolution layer, the batch normalization layer and the activation function layer to obtain a feature map.

6. A method for monitoring the growth of fish in pond culture according to claim 4, characterized in that: The trained YOLOv5 model is obtained through the following steps: Acquire multiple pond surface images and pond underwater images; Performing denoising and adaptive enhancement processing on the pond surface image and the pond underwater image respectively; marking the complete fish bodies and the incomplete fish bodies in the processed pond water surface image and the pond underwater image; The labeled fish body bounding boxes are divided into training set, validation set and test set; Train the YOLOv5 model based on the training set, adjust the YOLOv5 model parameters based on the validation set, evaluate the model performance based on the test set, and obtain the trained YOLOv5 model.

7. A method for monitoring the growth of fish in pond culture according to claim 4, characterized in that: Correcting the perspective deformation in the fish body bounding box based on the depth information of the pixels to obtain a standard fish body bounding box; comprising: Get the four endpoints of the fish body bounding box; The positions of the four endpoints are corrected based on the pixel depth information of the four endpoints, and a standard fish body bounding box is drawn based on the corrected four endpoints; When the fish body within the standard fish body bounding box is an incomplete fish body, the fish body is supplemented according to the fish body contour and the standard fish body bounding box is updated.

8. A method for monitoring the growth of fish in pond culture according to claim 4, characterized in that: Calculate the size of the fish based on the standard fish body bounding box, and calculate the weight of the fish based on the fish body size and the fish body type; include: The fish body size is calculated based on the maximum horizontal distance and the maximum vertical distance of the standard fish body bounding box; Substitute the fish size and fish type into the fish weight estimation formula to get the fish weight.

9. A method for monitoring the growth of fish in pond culture according to claim 1, characterized in that: Based on the fish size and weight calculated from the pond water surface image and the fish size and weight calculated from the pond underwater image, the growth monitoring result of the fish school cultured in the pond is calculated, and the growth monitoring result includes: the fish school size and the fish school weight; including: Based on the fish size and weight calculated from the pond water surface image, solving for the average fish size and average fish weight of the pond water surface image; Based on the fish size and weight calculated from the underwater image of the pond, solving for the average fish size and average fish weight of the underwater image of the pond; Based on the average fish size and average fish weight of the pond surface image, the average fish size and average fish weight of the pond underwater image, and the set pond surface image weight value and pond underwater image weight value, the weighted average fish size and the weighted average fish weight are solved as the growth monitoring result of the pond cultured fish.

10. A feeding robot device, characterized in that: The device is used to implement a method for monitoring the growth of fish in pond culture as claimed in any one of claims 1 to 9, comprising: An image acquisition module, used for acquiring a pond surface image and a pond underwater image based on a binocular camera; A depth calculation module, used to calculate the depth information of the pond surface image and the pond underwater image based on the parameters of the binocular camera, and generate a pond surface depth image and a pond underwater depth image; A depth alignment module, used to align the pond water surface image with the pond water surface depth image, align the pond underwater image with the pond underwater depth image, and obtain depth information of pixel points in the pond water surface image and the pond underwater image; An image processing module, used for performing denoising processing on the pond surface image and performing adaptive enhancement processing on the pond underwater image; A sample fish school prediction module is used to identify fish bodies and draw fish body bounding boxes based on the pond surface image after denoising and the pond underwater image after adaptive enhancement, and calculate the size and weight of the fish body based on the fish body bounding box and corresponding depth information; A comprehensive fish school prediction module, used for calculating the growth monitoring results of the fish school cultured in the pond based on the fish body size and weight calculated by the pond water surface image and the fish body size and weight calculated by the pond underwater image, wherein the growth monitoring results include: fish school size and fish school weight; The farmed fish feeding module is used to determine the type and weight of fish feed to be fed according to the growth monitoring results.

Images